PageSpeed Insights goes Lossy

Since the PageSpeed Insights (PSI) testing tool was introduced, they have recommended lossless compression. This means that no quality is lost. In fact, if you change a single pixel, you can no longer call it lossless. On the flip side, the recommendations at webpagetest.org have always recommended a lossy compression with a quality level of 50, but why is that relevant? It matters because Google supports/sponsors webpagetest.org (WPT). I don’t know what exactly their involvement is, but they are listed prominently on the about page for WPT.

As such, I’ve long wondered when the time would come that Google would start changing their image compression recommendations to line up with WPT. A few weeks ago, on December 12, it finally happened, but I delayed writing about it, because some thought it was an accident, an oversight. Someone accidentally flipped a switch, and certainly Google would revert the changes in short order. That didn’t happen, and at this point it is unlikely Google will be reverting anything. Most likely, they are fine-tuning the quality detection used by the testing tool to give more consistent results.

So what does that mean for us, the website owners and developers, the content creators, and publishers?

First, it does NOT mean that you should just blindly follow their recommendations. The PageSpeed Insights tool is a computer after all, and it cannot determine the absolute best quality to compression ratio 100% of the time. It DOES mean that Google is recognizing a changed landscape in the world of image optimization tools. Not just “changing”, but the future is already here. In the past, lossy compression was seen as a way to get great compression and speed, but it was not for sites that needed great looking images. That time is no more, and there are many tools available that can give you the best of both worlds: high compression and high image quality. EWWW Image Optimizer is one of those, and we use the amazing TinyJPG and TinyPNG tools to do so.

So one would think, you can just compress your images using whatever lossy tool you want, even Photoshop’s “save for web”, and problem solved, right?

Well, yes… and no. You can achieve the savings that PSI recommends with several different methods. You can even achieve better quality than PSI when using an intelligent algorithm like TinyJPG, EWWW IO, or Imagify. You will achieve your goal, a faster website, and lower page loads, but there is something you need to watch out for when using PSI to confirm those results.

PSI’s switch to lossy compression means retesting has a fatal flaw. To get to that flaw, we have to know how they determine the optimal image compression level for your images. While we may never know their “secret sauce”, they have traditionally used jpegoptim for compression, and it is likely they use it (or the underlying libjpeg) for quality detection also. Even if they are using something more sophisticated, there are only a few methods that they could be using to determine quality (that I’m aware of, remember this is “secret” sauce we’re talking about).

So how do they know how much to compress my images?

The first, and most likely method, especially if they are still using jpegoptim, is to use the underlying compression library to determine what quality level was used on the image. Jpegoptim does not actually do the compression itself, it relies on one of the three most common JPEG libraries: stock libjpeg, mozjpeg, or libjpeg-turbo. If they use this method, it is likely they are using libjpeg-turbo for compression and to detect the existing quality of your images. Then, that number is compared to what PSI has established as the ideal quality level. While WPT uses a published level of 50, I don’t think PSI uses quite that low of a recommendation based on discussion from the PSI forums, but the exact level isn’t important, so long as we understand the method.

The second method is to establish an amount of acceptable quality loss and measure the resulting images with something like DSSIM. For example, a study published in 2014 stated that users found images with a DSSIM score of .015 or better to be acceptable. DSSIM is a measure of the amount of difference between two images. A score of .015 means that 1.5% of the pixels have been modified in the compression. So, using .015 for the sake of discussion, you can then compress an image at quality 80, and measure the DSSIM. If it is worse than .015, increase the quality and try again. If it is better than .015, you decrease the quality and try again until you find the quality level that gets you closest to your desired DSSIM.

The last method is similar to the second: PSI could have built an algorithm similar to Imagify or TinyJPG, which looks for patterns in the images, and reduces the quality in areas where the human eye is unlikely to notice. These algorithms don’t attempt to optimize for a specific quality level, or even a specific amount of DSSIM. Rather, they take a custom approach to every image to ensure maximum quality. You might make an image with large swaths of sky 90% smaller, whereas a super busy image, with lots of detail, might only be compressed 10%.

While all of these would be great the first time around, if you compress an image, and then retest it, we run into problems. The latter two methods are also much slower than the first, which is what leads me to believe that they use method #1, because it is generally the most efficient.

So, about those flaws, what’s wrong with these methods?

In the first method, there are several problems. First, the JPG quality level is an arbitrary measure. Every tool is free to implement their own quality levels, and interpret them however they want. While most tools use a 1-100 scale, Photoshop used to have a 1-12 slider, but no matter the numbers, the interpretation is arbitrary. Second, the quality level is not stored in the image, so the only way to find out what quality level was used in an image is to guess. The only way to have an educated guess, is to look for patterns in the image that tell you how much compression was used by that tool. Stock libjpeg is different than mozjpeg, and mozjpeg is very different from libjpeg-turbo, and the more sophisticated compression engines are a completely different ball of wax. Even if PSI built-in tests for every JPG compression tool out there, how would they know what tool was used to compress an image in the first place? They can’t, so they would fall back to the library they are using (most likely jpegoptim with libjpeg-turbo), and make an estimate of what compression was used, based on what libjpeg-turbo would do to an image at the ideal quality level. This works fine on relatively uncompressed images, but once you’ve compressed your images as much as they tell you to, the algorithm starts throwing false positives, especially if you are using something besides libjpeg-turbo.

The second method is also flawed, because you’re always shooting for 1.5% quality loss. Thus, every time you test an image, compress, and then retest, you’ve changed 1.5% of the pixels, and your algorithm doesn’t care what quality level was used before, it only tries to find the level that will change 1.5% more pixels. After the second time, you’ve changed 3%, after 10 times, you’ve changed 15%, and so on. Of course, with such a serious flaw, I can’t imagine Google actually using this method, and the test results don’t support this type of algorithm.

The last method is ideal, but as I’ve told many customers over the years, there is a potential that the algorithm, no matter how finely crafted, will remove pixels on every subsequent pass. It gets less and less after the first attempt, but because we are dealing with lossy compression, and computers attempting to approximate what the human eye will tolerate, there is always the potential for more pixel loss if you recompress an image with this method. This method also tends to be somewhat slower and more CPU intensive than #1, so while Google might have built such an algorithm, and they certainly have folks capable of doing so, it doesn’t seem likely that they are using such an algorithm.

So what’s the point of testing with PSI?

It gives you a target to aim for, and recommendations to make your site faster. If you achieve what their test says regarding image optimization, you will have a faster site, and that increases both your SEO, and your bottom-line. But unlike the days when they used lossless compression, retesting to verify that you’ve completed the steps will not always be accurate, and will sometimes be horribly inaccurate.

If you still want to do lossless compression, and you want a way to test that, there is good news. As of now, the PSI results that Gtmetrix.com reports are still lossless. They don’t appear to actually use PSI in their testing, they’ve just replicated the test, and they have not (yet) updated their results to match PSI’s new lossy recommendations.

So welcome to the future! Using EWWW IO’s API can give you amazing image quality and great compression. There’s no sacrifices here, just faster page loads. There’s really no reason not to use the new generation of lossy compression tools to make your site stand out from the rest.

Happy Optimizing!

3 thoughts on “PageSpeed Insights goes Lossy

  1. Whats the info in a nutshell? Is your plugin able to compress the images in a way, that PSI stops complaining about it? I used your free plugin and it did not work. Is your paid plugin able to do it?

    1. You’re right, you cannot use lossless compression to satisfy the demands of PSI, and the only lossy compression EWWW can do for free is with PNG images. Nutshell version is this, only by using EWWW’s lossy compression, which requires the paid API, can you achieve the results that PSI recommends for JPG images. Further, once you’ve compressed all your images on Maximum Lossy mode, retesting may give false positives. Hope that’s succinct enough 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *