Author Archive

How to Debug Sleeping MySQL Processes

Tuesday, May 17th, 2011

A server in our environment was not working properly. This was a webserver running apache and a db server.

Everything was running well, and then we experienced lots of random slowdowns and seemingly queries that do not seem to run and execute properly. Another problem was that we experienced lots of sleeping processes when we checked the db server.

Restarting the db server didn’t work, nor did restarting the webserver. Checking the disk space on the db server showed that free disk space was more than adequate. So how do you debug sleeping mysql processes?

In the SHOW PROCESSLIST command, one of the fields returned is the Host field. For example:

xxx.xxx.xxx.xxx:39769

The most important part here is the port number (39769). The ip address here in my example showed that the originating ip was one of our webservers.

We now log into our webserver and type in the command:
netstat -ntp | grep 39769

This would now show (this is simulated output below):
tcp 0 0 192.168.1.20:39769 192.168.1.1:3306

We can filter out the pid (process id) using the netstat command and using the pid do a strace command:

strace -p [pid]

In my case here, it was found out that the webserver’s /var partition was full.

Hope these instructions help you guys when you have some weird stuff going on with mysql and your webserver.

A Lament on the State Of Linux

Sunday, May 15th, 2011

I’m getting old.

When I was young, I was fascinated with hardware and programming low-level stuff to do something nice as an experiment.

I got into programming DOS programs in assembly language (8086). In school, I was fascinated with 6502 assembly language (which wasn’t really that far from 8086, the concepts were mainly the same). As an Electronics Engineer student, my hardware thesis involved creating a hardware interface that detects acidity (pH) in soil with the added twist that it was plugged into the printer port (yes, PCs had parallel ports back in the day) and a program interprets the values that was in the printer port and records them. And yes, the program was in Visual Basic (shocker, right? An unusual use of the Evil Empire’s program environment).

Even friends from the Computer Engineering class asked my help in developing their computer interface projects (which included a voice recognition project for security and a vending machine – which detects what money was inserted into the machine).

One may ask… OK, you’re talking about the past, what has this got to do with Linux?

Let me give you an opinion that will be a shocker to you… Linux might never make it big into the desktop.

Let me start by telling you my experience with Linux. Don’t you get picky with me saying that Linux is the kernel. I am familiar with that.

I started delving into Linux when I was a sales agent of an isp. During that time, all of us sales agents have 8 hours of free internet (dial-up). I started into connecting my 33.6kbps modem through trumpet winsock and downloaded my first linux distribution: Monkey Linux. From that time on, I was hooked. Tried other hobby linux distributions (trilinux, looplinux, etc, settled for a few years with peanutlinux and gained quite a number of friends in the forums), and for a number of years, I was happy. I became an open source advocate. I remember downloading the 100Mb peanutlinux distribution for almost a day (dialup, whoooopeee). I brought linux into a company I worked for as an instructor (before that, I taught short courses for people like MS Office, and Basic computer instruction – I remember teaching a group of PLDT operators, yes operators, the ones that answer the line when you call 109 or 108 for long distance dialing). As I developed instruction modules for Visual Basic, I developed instruction modules for Linux (made even a mod of Peanutlinux, with Jay’s help in giving instructions, called Berkeley Linux, for internal use and passing on to students of the courses) – complete with Gnome environment (1.x at that point), and with WINE, to run some windows programs like winamp on Linux. As usual, I included Xitami and php in the mix of that mod, as standard.

I even got to create instruction modules for OpenOffice (then 1.x); but before that I used StarOffice 5 (when the license was not GPL, but rather free for non-commercial use).

Then reality set in. At first I didn’t mind it. Sound issues. Graphics issues. Wifi and network issues. I simply moved from one distribution to the next (PCLinux, Ubuntu, Mint, Fedora, and currently my laptop distribution, Meego).

And you know what? Even until today, these are still issues!

So, what’s keeping Linux from ever conquering the desktop? A big issue are drivers.

I love linux, don’t get me wrong. Don’t ever accuse me of being an troll – as you’ve seen on my blog, I dig linux.

However, one thing I’ve noticed is that the API for drivers is virtually non-existent. At least with OSX, they have a framework for extending or creating drivers (usually in userspace). Windows has also a framework for driver development (take your pick, WDM, vxD for older version of windows, etc)

When you update kernel versions, you have to recompile your drivers. Yes, I know, there is dkms. I’m not talking about that.

Linux simply has no formal frameworks for drivers. There was an attempt by Intel before (1998 to be exact), to create a framework named UDI, but was dissed upon by Richard Stallman and open source advocates. An excellent point for UDI was given by a former developer in this forum post.

I am a pragmatic person. I use whatever tools are necessary to get the job done. And that means using windows if that gets the job done – I would’ve preferred using open source, but if there is no software equivalent in linux (which is usually the case, unfortunately), I use windows or osx. Looking back, that decision in my opinion would fit in the category of boneheaded decisions of the open source community.

Linux advocates have a NIH syndrome. Instead of using stuff that works, they create a new one, usually REALLY buggy.

Case in point: pulseaudio. A big heaping piece of dog poo. Well, it has gotten better, but the latency issues are soooo annoying.

Yes, useability has gotten better, but when suspend/hibernate issues still exist even today, and wifi issues still exist, there is something definitely wrong. A friend of mine bought a laptop and installed Ubuntu, or Mint on it, and found out it didn’t work. Tried other distributions, didn’t work too. He almost gave up, but was lucky enough to get some instructions from friends who instructed him to get some firmware code through his driver cd and some command line stuff. I was even surprised by this even today, broadcom drivers are an issue (despite having broadcom-wl drivers and bcm driver stuff already).

Even my meego laptop, the lcd brightness control doesn’t work. And I had to compile OpenVPN for it myself (which reminds me, I’ll have to compile the newest OpenVPN and upload to the repository, when I get the time).

The list goes on and on. Enough of my rant. I’m an old guy. My experimentation and tweaking with the OS days are getting to an end (real life has come in – need money to pay my bills). I just need something that works.

Just to end (almost), I’ll give you some pseudo-code for a harmless virus I made as an experiment to learn how DOS works (this simple non-overwriting .com program became my thesis for a fifth-year paper on computer viruses – I created this experiment during my summer vacation in second year college).

Ordinary pseudo-code for a typical .com program:

jmp code
[data]
….
code:
….

My assembly code:
jmp [viral code]
[original data]

[original code]

viral code:
check if it is already resident (calls a modified int 22h hook with subfunction to see if it is installed)
if(not installed)
move viral code above DOS program memory
call int21h function to modify interrupt table to chain int 21h to viral code
edit 3 bytes at the beginning of the program to the original code
jmp [beginning of code]
[int 21 viral code]

Sounds unconventional, right? But this experiment I made in college helped me a lot in how OSes and computers work. DOS was excellent during that time because they provided a unified API – if I used purely assembly code and manipulated IO/Disk hardware registers directly, my code would have gone way over 1000 bytes (and wouldn’t have worked on all computers at that time) – even using an int13h bios code function would be more complicated to do. After all, it was all about developers. I would like to thank Ralph Brown for his excellent DOS Internals book (duh)! With this knowledge incidentally, I was able to make an anti-virus program in Turbo Pascal for the Tadpole virus (created by some students in another university) during my third year college.

If only linux had a unified driver interface, plus a unified media interface, and graphics interface with frameworks, then it would have leaped above windows and osx. Unfortunately, with the state linux is in, and the bickering and infighting of different groups in them (and misguided open source fanaticism), I really doubt linux will ever reach mainstream status (maybe android and chrome os).

Enough of my rant. What do you guys think? Am I a grumpy old man? Or do I talk sense?

A Possible Quick Fix to the Inconsistent “Invalid OAuth 2.0 Access Token” errors (facebook bug 15933)

Friday, April 29th, 2011

A few weeks ago, our recently converted FBML application to iFrame application (done last January), suddenly experienced endless redirection loops.

2 days into the problem and frantically searching for a possible solution, I found others experiencing the issue which is currently the most popular opened bug in facebook.

Since I’m using php for the sdk, I followed the advice in comment #119 by a certain Bret Stubbs, using the latest php sdk (although I use a particularly new sdk, a version before the latest) and changing this code:

// try loading session from signed_request in $_REQUEST
$signedRequest = $this->getSignedRequest();
if ($signedRequest) {
// sig is good, use the signedRequest
$session = $this->createSessionFromSignedRequest($signedRequest);
}

to this code here:

// try loading session from signed_request in $_REQUEST
$signedRequest = $this->getSignedRequest();
if ($signedRequest) {
if (!isset($signedRequest['user_id'])) {
return null;
}
// sig is good, use the signedRequest
$session = $this->createSessionFromSignedRequest($signedRequest);
}

And found out this didn’t work for me at first (looping redirects still occured). I found out that enabling the first option below fixed the looping problem:

This would fix most of the problems with the looping redirects that I encountered. There was other sections that still had the looping redirect problems and this was solved by removing all instances of the “session” parameter checks that facebook would add to the end of the url after authorization. What I’m finding odd though is that facebook even though I specifically enabled that deprecated option “Canvas Session Parameter”, facebook doesn’t already return this parameter (I would consider this already obsoleted).

Is this the end of the story for this bug? Not by a long shot. There are still errors when calling certain OLD REST APIs through the new SDK and calling the FQL API function (still not resolved yet). I have already migrated code to avoid these calls as much as possible(I use direct graph api calls in the new codebase as much as possible, and js SDK functions for certain functions that are in the new Graph API SDK but are inferior to the OLD REST APIs in customizability… and there are no more fql function calls in the codebase).

What do I think of all of these? Robert Turall, a UK based facebook developer sums it up nicely in his blog:

The problem with Facebook’s approach to their platform and to App developers is that we all have to work to a moving target, an inherently unstable platform. Such a platform is not really suitable for producing the kind of campaign-based Apps that I produce but Facebook market their platform as THE socially-aware marketing platform for campaign-based Apps.

It’s about time they started to run it like one – got some decent QA in place and thoroughly tested their releases before throwing bug-ridden code out into the World.

I agree with this completely. Facebook APIs have been moving in such a manner that it is difficult for a developer track all the changes by himself. Incomplete documentation especially with recommended new ways of doing facebook development, the defeaning silence of Facebook staff with regards to some bugs (for example I have been keeping track of a certain bug which is already almost a year open, which is important to an app which I’ve developed, until I’ve decided to do a workaround which results to a more clunky experience due to deadline constraints), all adds to tensions in the facebook developer community.

How to Set Up OpenVPN on Android

Wednesday, April 27th, 2011

I am simply placing these instructions which I made for some people asking how to use my tun.ko module (Based on Phantom King Skyfish) in a popular phone forum.  You can generally use these instructions to correctly install openvpn on android devices.

  1. Download OpenVPN Installer and OpenVPN Settings from the Market.  If you don’t have tun.ko yet download the module I compiled for kernel 2.6.32.9 ARMv6 devices from here.
  2. Run OpenVPN Installer (openvpn should be installed in /system/xbin, and ifconfig path in /system/xbin/bb).  /system/xbin/bb is not available yet, we’ll get to it later.  For roms like Bakpia Keju which do not have route and ifconfig busybox links properly installed, do the following in the terminal:

    adb shell
    bash
    cd /system/bin
    ln -s /system/xbin/busybox cp
    ln -s /system/xbin/busybox chmod
    ln -s /system/xbin/busybox route

  3. Use a barcode scanner (available in the market) to read the barcode below:
  4. Paste this code in SL4A application (also available in Skyfish).  If you’re not in Skyfish, make a script named openvpn.sh and paste the text from the barcode in there.  Save and run the script.  This should make the proper route, ifconfig and busybox binaries be symlinked to make the OpenVPN Settings run properly.  If you like to use SL4A, you can download the application from here.

Your openvpn is now configured properly and should now work as expected.

The… Android Dance?!!!

Sunday, March 27th, 2011

Just a short post…

I just happened to watch a youtube video of the Android mascot dancing to a spiffy beat. I don’t know about you, but I found this awesome!

Enjoy the vid below:

2.6.32.9 tun.ko armv6 module for Z71 variants (froyo)

Wednesday, March 23rd, 2011

Short post.

This is tested in the Bakpia mod kernel (2.6.32.9), module compiled from the rmcc github kernel 2.6.32.9 branch located here.

Compiled in a CentOS 5.3 environment with android ndk and sdk installed.

Anyways, copy the module to /system/lib/modules do an insmod tun.ko to run the module and tunneling is all set. You can now use Openvpn downloaded from the Market.

Download here.

How to compile tun.ko For Android (Commtiva Z71)

Monday, March 21st, 2011

I’ve been too busy these days to compile a kernel module for android.

For those who want to compile a tun.ko kernel for Android (a commtiva z71 variant), here are the general steps. You need a linux environment to compile this module.

1. Download the kernel and supporting files for commtiva z71 devices here: https://opensource.motorola.com/sf/frs/do/viewRelease/projects.quenchxt/frs.xt5.xt502 and extract it to a directory of your choice.
2. Download the SDK and NDK for android: http://developer.android.com/sdk/ndk/index.html and http://developer.android.com/sdk/index.html
3. Copy config.gz from your phone and place it into your kernel source folder. To get it from your phone, type:
adb pull /proc/config.gz .
gunzip config.gz
mv config .config
4. Edit .config so as TUN module will be included into the kernel:
CONFIG_TUN=m
5. Run uname to get running kernel version of your phone:
uname -a
Linux localhost 2.6.32.9 #7 PREEMPT Tue Jan 4 16:51:45 CST 2011 arv61 GNU/Linux
6. If there was some extra characters in your kernel version, like 2.6.32.9-063c4d24, you’d have to edit the Makefile to use the corresponding version:
EXTRAVERSION = -063c4d24
7. Set your environment variables:
export ARCH=arm
export CROSS_COMPILE=arm-eabi-
export PATH=$PATH:~[path of your arm-eabi compile toolkit binary in android ndk]
8. cd to your kernel folder and make modules
cd ~[path of your kernel source]
make modules

Resulting tun.ko should be in the drivers/net folder of your kernel source.

Just beware, it is not for the fainthearted.

Further information:
http://android-dls.com/wiki/index.php?title=Build_a_custom_kernel
http://forum.androidcentral.com/htc-hero-rooting-roms-hacks/8072-how-build-your-own-kernel-package-source.html

Froyo Mod on Commtiva Z71 Variant (Cherry Mobile Nova)

Monday, March 14th, 2011

Just a quickie video post. Since this reseller keeps quiet on whether they will update their commtiva z71 variant, I decided to upgrade my phone myself. Got much faster speed, and I’m very satisfied with the result! Been working with phonegap too – so android development, here I come!

Facebook Zero on Android

Wednesday, January 19th, 2011

I’ve been asked by some people on how to access facebook for free on their android mobile phone in the Philippines. Well, you can do this if you have a Sun cellular sim card, using a service called facebook zero.

I did this on my Cherry Mobile Nova, but basically all android phones have similar settings screen.

Facebook Zero is a service by Sun Cellular offered to its subscribers to access the popular social networking site for free (by now). Let us see how to configure this currently free service for android devices.

We need to make sure that the Sun wap APN is configured on the Settings->Wireless & Networks->Mobile network Settings->Access Point Names as shown below:

If its not created yet, let us create a new APN called “Sun GPRS” (screenshot below shows the important settings used for this wap service):

Select this APN for use and on the previous screen, enable Data:

This will let android connect to the wap service. Let us now open the default android browser and type in “0.facebook.com”:

If successful, you will now connect through the browser (through wap) to facebook for FREE. Happy surfing!

Short Post – Android Thoughts

Sunday, January 16th, 2011

Finally got myself a month ago an android phone, a cheapo commtiva z71 variant (cherry mobile nova)

Thoughts about it? Really cool, but android by itself is somewhat unpolished compared to iOS, or even Symbian for that matter.

Stay tuned for more.

Posted from android.

image

image

image

image