Multithreading / parallel processing in PHP
I have a PHP script that will generate a report using PHPExcel from data requested from a MySQL database. It is currently linear in processing as it returns data from MySQL, reads into an Excel template, writes data to the template, and then outputs it. I have optimized the code to the point that the data is only repeated once and there is very little processing on the PHP side. The query returns hundreds of rows in less than 0.51 seconds, so it's fast enough. After some time, I found my bottlenecks (surprise, surprise) read the template and write the output. I would like to do this:
Spawn a thread/process to read the template
Spawn a thread/process to fetch the data
Return back to parent thread - Parent thread will wait until both are complete
Proceed on as normal
My main questions are: is it possible, is it worth it? If so, how would you handle this? Also, this is PHP 5 on CentOS
a source to share
It is generally not recommended to deploy the Apache process. This can lead to uncertain results. Instead, it is preferable to use some kind of queuing mechanism. Gearman is an open source queuing engine that you can use. I also have a blog post on Zend Server Job Queue that talks about running jobs asynchronously Are you queuing? Introduction to the Zend Server Run Queue .
You can also use something like Zend Framework's Queueing classes to implement some asynchronous work. Zend_Queue
@Swisstack, also I disagree with your statement that PHP is not built for high performance. Very rarely, language features are the reason for slow performance. Perhaps doing an original language test comparing $ a ++ among different languages will show you this, but this type of testing doesn't matter. I've consulted PHP for several years and I've never seen performance issues due to the language.
a source to share
You cannot multithread, but you can fork ( pcntl_fork , pcntl_wait ). As I'm sure you'll want to check your spawn time carefully to make sure it's even worth it for your situation.
$pid = pcntl_fork();
if ($pid == -1) {
// fork failed
} elseif ($pid > 0) {
// we're the parent! Wait for child to finish
pcntl_waitpid($pid);
} else {
// we're the child
}
a source to share
If both template reads and the db query were slow, then I would say there is a decent chance that useful performance can be achieved by running tasks in parallel. But you said it yourself, reading the template is slow and the db query is fast. So even ignoring any additional overhead created by the additions needed to run tasks in parallel, you are at best cost savings of 0.001 seconds (the time it takes to query the db).
Doing multiple tasks in parallel will always take the time of the slowest task. Completing tasks sequentially is the sum of all tasks. In your case templateTime + queryTime (0.001)
Should not be doing that.
Usually the database is the turtle in the equation. You can make this part async effortlessly. See the recently added mysqli_poll () and friend functions.
a source to share
You can definitely create processes on CentOS with PHP ( http://php.net/manual/en/function.pcntl-fork.php ). Before doing this, I would consider at least one thing ... If the bottleneck seems to be reading the template and writing the result, it might be an I / O constraint issue, and so accessing multiple processes might not help ... Personally I'll try to see if it's possible to do caching instead ...
a source to share