For a very long time, EWWW Image Optimizer has had the ability to optimize new images “in the background”. This is important for several reasons, but mostly because of the increased processing time from optimization. The average site generates 10+ additional images of various sizes (thumbnails) for every upload to your WordPress site, and some sites have 40 or 50 sizes being generated. With many sites limited to 30 seconds for processing, it’s a small miracle to even generate that many images without timeouts. Never mind trying to optimized all those images within 30 seconds…
With the release of version 7.4, EWWW Image Optimizer can now also optimize all existing images in the background. This means you can tell EWWW IO to optimize all your local images without waiting around or leaving the bulk optimization page open. It also opened the door for a host of improvements, one of which was increased reliability for thumbnail optimization.
More Reliable Thumbnail Optimization
This first showed up in version 7.3, as we laid the groundwork for the processing that would be needed to bulk optimize an entire media library in the background. Previously, there were two separate background optimization queues, one for media library uploads and one for non-media images. Unfortunately, due to aforementioned issues with too many thumbnails, one could end up with timeouts on the background process, and you’d never know it. To avoid this, we revamped the secondary queue to handle media thumbnails.
Now, when a media upload goes into the upload queue (#1), EWWW IO simply takes all the thumbnails and dumps them into the image optimization queue (#2), so that they can be processed individually. Each individual image file will thus get the full amount of time to process, usually 30 seconds or more, instead of trying to cram all the thumbnail (and full-size) optimization into that window of time. Additionally, any post processing by other plugins has been moved into a completely separate process in queue #3.
Besides all that, the bulk optimizer has had the ability to avoid timeouts by estimating how long it will take to optimize an individual image. This has now been applied to queue #2, so that EWWW IO will do as many images as possible in a single batch without going over the server time limits. If the plugin detects it can’t do all the thumbs for a given upload at one time, no problem! It simply spawns a new background process, and keeps going in the new process.
But How Does it Work?
Background or asynchronous (async) processing isn’t native to PHP. So to do anything “in the background”, EWWW IO sends an HTTP request from the server to itself. The request is done with a .01 second time limit, so that the original request doesn’t wait around for the secondary/background request to complete. Yet PHP won’t know that the original request has “given up” waiting, so it just keeps on going with the new async request. As we mentioned before, sometimes background processing can’t be done in a single request. To keep the background process alive, each async request simply repeats the process as needed until everything in the queue is complete.
There are some potential issues with telling servers to send HTTP requests to themselves, and if your site has any trouble with background mode, we detail some troubleshooting tips in our docs. Ultimately, a background process can still potentially timeout or the re-spawn process might get blocked. To work around that, EWWW IO hooks into the native WordPress cron system to reboot background optimization if it gets stalled out.
Timeout Avoidance for API Optimization
All of that is great, but there was one more thing needed to make the bulk process fully asynchronous: making the Compress API optimize images asynchronously. For better or worse, we’ve approached API-based optimization differently. When your site submits an image to our API, it sits there waiting until the image is done, and then sends back the optimized image.
No more! Or at least not always… We’ve built a new async process into the API, so that it will detect images that could be problematic, and automatically optimize them without making your site timeout while waiting. This is especially important for larger images like PDF files, but any of the traditional image formats can cause timeouts if the files are large enough. With async handling in EWWW IO, it simply keeps the background process alive until API-based optimization is complete.
Bulk Optimize in the Background
With the background optimization framework in place, the remaining task for 7.4 was to build the interface to make it possible to optimize ALL of your images this way. While that was plenty of fun, the biggest hurdle was showing the optimization results and the queue status. EWWW IO already had the ability to show the optimization results, but we moved it onto the settings page, so you can now see optimization results, and the current queue status all in one place.
This allows you to even see what is going on with new uploads, which is handy if you have a bulk import process that periodically pulls in a large number of images. Besides that, we took feedback from several users and improved the bulk tool to have a Scan-Only option (in Ludicrous Mode), so that you can review the images that are in the queue before starting the actual optimization phase. From there, you can remove images from the queue, search the queue, sort the queue by size, or even flat out exclude any image that you don’t want optimized.
In between 7.4 and 7.3, I found several ways to make background mode more efficient as well. Ultimately, all of this represents months of development, and we’re super excited about how it turned out. We hope it makes your life simpler, and makes EWWW IO more reliable along the way. Of course, we welcome your feedback, if you find any quirks or have any questions, just let us know!
