Changing Mysql data directory require change to AppArmor

After a bit of googling and hair-pulling, I realized that if I just changed the datadir directive in my.cnf will cause mysql start to fail on Ubuntu. The other thing is to add permissions to apparmor for mysql to access the new data directories.

Steps
1. sudo vi /etc/apparmor.d/usr.sbin.mysqld
2. Add
/newdir/ r,
/newdir/** rwk,
3. sudo /etc/init.d/apparmor restart
4. sudo /etc/init.d/mysql restart

If that still does not work, check the nix permissions to be sure mysql is owner and group for the new directory recursively.
chown -R mysql:mysql “new datadir path”

Update Delayed on Mysql and PHP

Wouldn’t it be nice if we could use UPDATE DELAYED in mysql just like we do use INSERT DELAYED? Unfortunately, the work around mysql created for this is to insert a record into the events table in order to separate UPDATE statement into another thread. There is a large overhead to inserting to then execute an update.

If you are using PHP with mysql, there is another work around.  PHP has a function called, register_shutdown_function, which will execute a function during the shutdown of processing a script file. So if we add an UPDATE LOW_PRIORITY sql statement wrapped in a function that is passed to register_shutdown_function, we can allow the user to receive the page without delay while the UPDATE statement waits until all locks on the table has been release before proceeding.

function update_delayed()
{
$sql = "UPDATE LOW_PRIORITY table_name SET col1 = 'something'";
mysql_query($sql, $conn);
}

register_shutdown_function('update_delayed');

I wonder if there are even better ways to handle this problem.

Mysql on 32bit processor vs 64bit processor

Here is the continuation of the experiment done previously on Mysql on Windows vs Linux. After doing more analysis and observation, it appeared that the difference in speed for the benchmark tests were not related to Windows and Linux but were related to the processor. The previous test just happened to have AMD 64s on the Linux machines and Intel Xeon 32bits on Windows. We hypothesized that the increased throughput of the 64bit processors resulted in about half the time required to return the same query run on a 32bit machine.

View the previous results

So we decided to test the same query again on a Windows machine with an AMD 64 X2. This time the result for the query took 7.35 second, almost the same as the other AMD 64s running Linux.

One more test to run the query on a Windows Intel 64bit chip machine would better solidify our hypothesis.

Mysql server has gone away

I have a spider crawling the web written in PHP running constantly to insert and update data in a Mysql database.  Today, it kept stopping on a records with the error message, “Mysql server has gone away”.  It certainly did not time out as just starting the process again would result in this message immediately.  The script did not close out the connection to the DB as it had worked just fine for over a year.

Finally, I found that the problem was max_allowed_packet setting in my.cnf was too small.  It was set to 2MB and when Mysql receives a query larger than that, it assumes that something must have gone wrong and closes the connection.  I increased that parameter to 4MB and everything is working fine now.

Mysql on Windows vs Linux

I was working on optimizing a Mysql database today and accidentally stumbled upon a benchmarking exercise. The original Mysql database is hosted on a Windows Server 2003. I develop on a Mysql database server running on Ubuntu Linux. The Linux server ran the same query twice as fast as the Windows server without using caching or anything. I know that Linux does I/O caching on its own as well so I even tried running the query after a fresh reboot to rule out that factor. Then I got help from a colleague and started tweaking with the my.cnf/my.ini to make sure they were the same and each time, the results came back about the same. Mysql on Windows was consistently slower than Mysql on Linux.

Then we decided to load the database onto other servers for more data points on this Mysql performance test. We ran the same query returning 429 rows of data with 13 table joins and a couple of sub-queries. All queries were run on the command line client on the servers themselves to avoid network lag.  All servers are running Mysql 5.0.x.

Results

1. Ubuntu Linux: 0.70 seconds
2. CentOS: 0.78 seconds
3. Windows 2003 Server: 1.40 seconds
4. Windows 2003 Server: 1.42 seconds

Server hardware

1. Ubuntu Linux
AMD ATHLON 64 X2 4200+
2GB DDR400
200GB 7200RPM SATA/150

2. CentOS
Dual Opteron 240
2GB DDR ECC
120GB 7200RPM SATA/300

3. Windows 2003 Server
Dual Xeon
2GB DDR ECC
7200RPM PATA

4. Windows 2003 Server
2 Dual Xeon (4 CPUs)
8GB DDR ECC
3 73GB 10,000RPM SCSI in RAID 5

A followup to the benchmark that clarifies the cause of the differences

Mysql SQL_NO_CACHE Benchmark Problem

I have been trying to benchmark the sql calls to mysql from an application without success. Even when I set the SELECT statement to SQL_NO_CACHE, the results seemed to be still cached. Apparently, Linux does its own caching of disk reads also. That was giving me incorrect benchmark results if I hit the same data twice.

For those using Linux kernel 2.6.16 or higher, you can clear Linux cache with the following command.
echo 3 > /proc/sys/vm/drop_caches

Unfortunately for me, I am stuck on 2.5.xx and will have to figure out another method.