Easily optimize images using PHP (and some binaries)

Our recently released image-optimizer package can shave off some kilobyes of PNGs, JPGs, SVGs and GIFs by running them through a chain of various image optimization tools. In this blog post I'll tell you all about it. First, here's a quick example on how you can use it:
use Spatie\ImageOptimizer\OptimizerChainFactory;

$optimizerChain = OptimizerChainFactory::create();

$optimizerChain->optimize($pathToImage);
The image at $pathToImage will be overwritten by an optimized version which should be smaller. Here are a few images that were optimized by this package.

Why we built this

On nearly every website we make, images are displayed. In fact, in most cases images make up the bulk of the size of the entire page. So to provide a fast page load it's best to make those images as small as they can be. The less bytes the browser needs to download the faster the page will be. Now, to make images as small as they can be (without sacrificing a lot of detail) there are a lot of tools available. These tools, such as jpegtran, pngquant, and gifsicle, work more or less by applying a little bit of compression, removing metadata and reducing the number of colors. In most cases these tools can make your images considerably smaller without you noticing, unless you have a trained eye for that. These tools are free to use and can be easily installed on any system. So far so good. What makes them a bit bothersome to use from PHP code is that you need to create a process to run them. You must make sure that you pass the right kind of image to the right optimizer. You also have to decide which optimization parameters you're going to use for each tool. None of this is rocket science, but I bet that the vast majority of small to medium sites don't bother writing this code or researching these optimizations. Our package aims to do all of this out of the box. It can find out which optimizers it should run, it can execute the binary and by default uses a set of optimizers with a sane configuration.

How to optimize images

Optimizing an image is very easy using our package.
use Spatie\ImageOptimizer\OptimizerChainFactory;

$optimizerChain = OptimizerChainFactory::create();

$optimizerChain->optimize($pathToImage);
The image at $pathToImage will be overwritten by an optimized version which should be smaller. The package will use these optimizers if they are present on your system: The readme of the package includes instructions on how to install these on Ubuntu and MacOS.

Which tools will do what?

Like already mentioned, the package will automatically pick the right tool for the right image.

JPGs

JPGs will be made smaller by running them through JpegOptim. These options are used: - --strip-all: this strips out all text information such as comments and EXIF data - --all-progressive: this will make sure the resulting image is a progressive one, meaning it can be downloading using multiple passes of progressively higher details.

PNGs

PNGs will be made smaller by running them through two tools. The first one is Pngquant 2, a lossy PNG comprossor. We set no extra options, their defaults are used. After that we run the image throug a second one: Optipng. These options are used: - -i0: this will result in a non-interlaced, progressive scanned image - -o2: this set the optimization level to two (multiple IDAT compression trials)

Customizing the optimization

If you want to customize the chain of optimizers you can do so by adding Optimizers manually to a OptimizerChain. Here's an example where we only want optipng and jpegoptim to be used:
use Spatie\ImageOptimizer\OptimizerChain;
use Spatie\ImageOptimizer\Optimizers\Jpegoptim;
use Spatie\ImageOptimizer\Optimizers\Pngquant;

$optimizerChain = (new OptimizerChain)
   ->addOptimizer(new Jpegoptim([
       '--strip-all',
       '--all-progressive',
   ]))

   ->addOptimizer(new Pngquant([
       '--force',
   ]))
If you want to use another tool than the package supports out of the box, you can easily write your own optimizer. An optimizer is any class that implements the Spatie\ImageOptimizer\Optimizers\Optimizer interface. If you want to view an example implementation take a look at the existing optimizers that ship with the package.

Integration in other packages

Our image-optimizer is not the first package we wrote the revolves around handling images. There also our image package which makes modifying images very easy. And we have laravel-medialibrary which can associate all kinds of files (including images) with Eloquent models. And lastly we have Browsershot, which can turn any webpage into an image. Let's take a look on how image-optimizer has been integrated in all of those.

Image

Using the image package you can manipulate an image like this:
Image::load('example.jpg')
    ->sepia()
    ->blur(50)
    ->save();
This creates a sepia version that is blurred. The package has many other available manipulations. Here's how you can create an optimized version.
Image::load('example.jpg')
    ->sepia()
    ->blur(50)
    ->optimize()
    ->save();
Yea, just add the optimize method in the chain and you're done. Easy right?

laravel-medialibrary

In laravel-medialibrary this is just as easy. Using that package you can define conversion profiles on your models. Whenever you associate a file with that model a derived file using that conversion profile is being generated. This is handy for creating thumbnails and such. Here's a quick example of such a conversion profile.
use Illuminate\Database\Eloquent\Model;
use Spatie\MediaLibrary\HasMedia\Interfaces\HasMediaConversions;
use Spatie\MediaLibrary\HasMedia\HasMediaTrait;

class NewsItem extends Model implements HasMediaConversions
{
    use HasMediaTrait;

    public function registerMediaConversions()
    {
        $this->addMediaConversion('thumb')
              ->width(368)
              ->height(232)
              ->sharpen(10);
    }
}
Let's add an image to the medialibrary:
$media = NewsItem::first()->addMedia($pathToImage)->toMediaCollection();
Besides storing the original item, the medialibrary will also create a derived image.
$media->getPath();  // the path to the where the original image is stored
$media->getPath('thumb') // the path to the converted image with dimensions 368x232

$media->getUrl();  // the url to the where the original image is stored
$media->getUrl('thumb') // the url to the converted image with dimensions 368x232
That's a crash course into using medialibrary. But wait, we didn't create an optimized version of that thumbnail. Let's do that now. Under the hood all conversions are done by the aforementioned image package. So you can just use optimize function in your conversion profile.
    public function registerMediaConversions()
    {
        $this->addMediaConversion('thumb')
              ->width(368)
              ->height(232)
              ->sharpen(10)
              ->optimize();
    }
Boom done. In the next major version of medialibrary we'll automatically call optimize behind the screens for all image conversions. So you'll get optimized conversion by default. We'll add a nonOptimized method if you want to opt out of that. We haven't introduced that behaviour in the current version because it's breaking change.

Browsershot

Browsershot is a package that leverages headless Chrome to turn any webpage into an image. Here's how to use it:
Browsershot::url('https://example.com')->save($pathToImage);
And here's how to save an optimized version:
Browsershot::url('https://example.com')->optimize()->save($pathToImage);

In closing

I should mention that our optimize package is based upon another one by Piotr Śliwa. All the basic principles on how the package should work are inspired by Piotr's work. The reason why we rewrote it is because his was not that easy to extend and did not use modern stuff such as Symfony's Process component or PSR-3 compatible logging. In this post I've mainly mentioned tools you can install locally, but there actually are a lot of SaaS solutions as well such as TinyPNG, Kraken.io, imgix.com and many many others. In this first release of our image-optimizer package I've mainly concentrated on supporting local optimizers. With remote optimizers you have to deal with slowness of the network and API keys and such. But I do recognize the value of those remote services. That's why you'll probably see some remote optimizers being referenced or included in package in the future. Here's an issue on the repo where the first thoughts on that were being exchanged. The package contains a few more features not covered by this blogpost. So check out image-optimizer on GitHub. I hope our tool can make your images smaller and your pages faster. If you haven't already done so, check out our previous work in the open source space. Please send us a postcard if any of our stuff makes in into your production environment.

A straightforward Vue component to filter and sort tables

Today we released our newest Vue component called vue-table-component. It aims to be a very easy to use component to make tables filterable and sortable. In this post I'd like to tell you all about it.

Why creating yet another table component?

Let's first touch upon why we created it. To make lists of models sortable and filterable we used to rely on the venerable DataTables.net component. It works great, but comes with a few caveats. First, it's built on jQuery. In our projects we avoid using jQuery nowadays and use Vue specific components. Secondly, there's a lot of stuff in there that we don't need, we probably are only using 10% of it's features. So for our use cases, it's a bit bloated. And lastly, because it contains so much features, using it can be a little unwieldy. When creating our own table component we had only our specific use case in mind: a small set of data that is passed via json.

Using vue-table-component

Here's an example of how you can use the component:
<table-component
     :data="[
     { firstName: 'John', lastName: 'Lennon', instrument: 'Guitar', editUrl: '<a href='#john'>Edit</a>', birthday: '04/10/1940', songs: 72 },
     { firstName: 'Paul', lastName: 'McCartney', instrument: 'Bass', editUrl: '<a href='#paul'>Edit</a>', birthday: '18/06/1942', songs: 70 },
     { firstName: 'George', lastName: 'Harrison', instrument: 'Guitar', editUrl: '<a href='#george'>Edit</a>', birthday: '25/02/1943', songs: 22 },
     { firstName: 'Ringo', lastName: 'Starr', instrument: 'Drums', editUrl: '<a href='#ringo'>Edit</a>', birthday: '07/07/1940', songs: 2 },
     ]"
     sort-by="songs"
     sort-order="asc"
     >
     <table-column show="firstName" label="First name"></table-column>
     <table-column show="lastName" label="Last name"></table-column>
     <table-column show="instrument" label="Instrument"></table-column>
     <table-column show="songs" label="Songs" data-type="numeric"></table-column>
     <table-column show="birthday" label="Birthday" data-type="date:DD/MM/YYYY"></table-column>
     <table-column show="editUrl" label="" :sortable="false" :filterable="false"></table-column>
 </table-component>
Let's take a look on how our component will render that piece of html. The component doesn't come with any styling, so you'll need to provide your own css. Head over to the demo page to play with the rendered output. You've probably noticed that a filter field is added to the top. By default all data will be used in to filter. If any columns contain html it will be filtered out before filtering. So even though that last column contains links, filtering on href won't yield any results. If you don't want a column of data to be filterable just add :filterable="false to table-column. Like expected, clicking a column header will sort the data. If the data contains a column with numerical values or dates you must set a data-type prop on table-column. Don't want a column to be sortable? No problem! Just pass a sortable property set to false to table-column. The component will also remember it's state for 15 minutes. So when you reload the page that filter and sorting you used previously will still be used. Head over to the readme on GitHub to learn all the props that can be passed.

In closing

Like mentioned before our table component is very much targeted to our use case. Our intent is to keep this component very clean and simple. If you need more features, take a look at the numerous other Vue components out there that can render table data. Because in our client projects we are using Vue more and more you can expect some more opensource Vue components from us soon. If you haven't already done so, be sure to take a look at this list of things we opensourced previously.

Our postcard collection

All our packages are MIT-licensed. But if you use our stuff and want to make us happy, we highly appreciate a postcard from your hometown. This suggestion is mentioned in all readme's of our packages We've been asking for postcards for quite some time now and have built up a nice collection. Today, we published a new page on our company website that lists all our received cards. If you like what we are doing in regards to opensource, please send us a postcard too. Our address: Spatie Samberstraat 69D 2060 Antwerp Belgium Thanks! https://spatie.be/en/opensource/postcards

Why I close PRs

Jeff Geerling, currently working as a technical architect at Aquina, wrote a good post on when and why he closes PRs to the packages he's maintaining. This paragraph resonated with me.
I don't cater to everyone. I usually cater to myself. And for 98% of my OSS projects, I'm actually using them, live, in production (often for dozens or hundreds of projects). So I'm generally happy with them as they are. I will not add something that increases my maintenance burden unless it's very compelling functionality or an obvious bugfix. I can't maintain a system I don't fully understand, so I like keeping things lighter and cutting off edge cases rather than adding technical debt I don't have time to pay off.
http://www.jeffgeerling.com/blog/2016/why-i-close-prs-oss-project-maintainer-notes

How I open sourced my way to my dream

Laravel employee #1, Mohammed Said, recently gave an interview at codeforaliving.io on his career and how he started with open source. Terrific story. A lot of what he says resonates with how I feel about working on open source.
Said believes many developers, in the Middle East and elsewhere, are interested in the open source community, but not sure how to get started: “They think they have to wait until they have something perfect.” That’s simply not the case, Said maintains. He encourages developers to dig deep into their favorite projects, especially into software they work with on a daily basis, and look for places they can offer “enhancements” to existing code. Open source, he believes, is the best experience a developer can show in an interview. “Tech interviewers want one thing,” he says: “Show me your code.” You can share code you’ve written for your day job, but it’s probably impersonal or boring or even proprietary and closed source, so you can’t share it at all. Some code at work is done as part of a team and you can’t pick out what you did and what Jane down the hall did. Open source, on the other hand, is all you. It’s your passion, it’s publicly accessible, and it has your name on it.
http://www.codeforaliving.io/how-i-open-sourced-my-way-to-my-dream-job-mohamed-said

🎉 Our packages have been downloaded one million times 🎉

I'm very happy to announce that, as of today, our Laravel and PHP packages have been downloaded a million times. We now have more than 80 packages registered on Packagist. All combined they are downloaded around 150 000 times a month, and that number seems to be growing. Our GitHub organisation has taken the number 13 spot on GitHub Awards. I'm very proud that we could achieve this in a company with only two back end developers. It's really nice to know that the Laravel and PHP community find our packages useful. In this post I'd like to share the story behind our packages. That one million number sounds impressive, but should be taken with a grain of salt. The amount of real users is probably much lower. Some of our packages require other ones of our own. For example, when downloading laravel-backup, our own db-dumper will get downloaded as well. And I’m pretty sure the total downloads get inflated by CI tools that download the various packages more than needed. I think the actual number of downloads isn’t that important. The monthly growth number is a more representative figure of how our packages are doing popularity-wise.

The first package

The company where I work, Spatie, exists from 2003. We've always been using PHP. In the early years we dared to write our own framework. Yeah, we were still inexperienced, so that seemed the way to go. It was very basic, but it served us well. We were only creating small sites at that time. But after several years we realised that writing and maintaining your own framework is very time-consuming and we switched to Zend Framework 1. In 2012 and 2013 we felt that the PHP ecosystem was pretty stale and we considered switching to Ruby and the Rail framework. We even made a couple of projects in Ruby to get our feet wet. But then we bumped into Laravel 4.0. I was immediately impressed by the expressive syntax and the big focus on developer happiness. I used it for a couple of projects and the more I learned about it, the more I loved using it. At that time a guy called Jeffrey Way started his next project Laracasts: a video tutorial site dedicated to Laravel. Realising that his videos could speed up my learning process immensely I immediately bought a lifetime subscription. On the 4th of March in 2014 Jeffrey published a video titled "Continuous Integration With Travis". My mind was blown. Travis and the integration with GitHub looked so cool that I wanted to use it. A couple of months before he made the Travis video Jeffrey made a miniseries on package development. Those videos sparked the thought in my mind that I could create a package of my own. In a project at the time we needed to automatically create screenshots of a website. With the things I learned in the Laracasts-videos I started working on my first package Browsershot. In essence it’s just a simple wrapper around PhantomJS. It was pretty exciting working in the open. I was thrilled every time the download counter got up and was incredibly happy that other people started writing about it.

Moarrrr packages!

As time went by we made more a more projects with Laravel. In our Zend Framework days we used a custom-built application template, called Blender, to kickstart all our projects. It was a sort of mini-cms. Beside the traditional CMS-functions Blender could do a lot of cool things: pull in data from Analytics to show graphs about the usage of the site, handle uploaded files, subscribe users to a MailChimp list, ... I started creating a Laravel version of Blender from scratch. I soon realised that some of the things I was porting to Laravel could be useful for other developers as well and decided to export these functionalities to external packages. In quick succession packages like laravel-newsletter, laravel-medialibrary and laravel-analytics were made. I started to enjoy package development more and more. The positive feedback that I got early on from package users was very encouraging. I always considered coding purely as work, but because it was so much fun creating code that others could use as well I started coding in my free time as well. Besides working on packages I also polished the aformentioned application template called Blender and open sourced that too. Since Spatie started I always was the only back-end developer. But that changed somewhere mid 2015 when Seb joined our team. He also likes working on our open source stuff. Because I blog and tweet a lot I’m the public face of our packages, but Seb does quite a lot too. He loves working on the more lower level stuff (the regex package is one of his). I myself enjoy working on the bigger infrastructure kind of packages like backup and medialibrary. Currently every new package that we create gets born inside a client project. In almost every project we find some functionality that can be extracted to it’s own package.

Making time to work on open source

People often ask me if it isn’t very time-consuming to create packages. The truth is that it indeed takes a lot of time. Creating the code for the package itself, writing tests, writing documentation and getting the word out all takes a fair amount of time. And that’s just the start. When a first stable release gets tagged the work is not over. Maintaining a package, responding to issues, reviewing PR’s takes a lot of time (and dedication) too. At the time of this writing we've responded to almost 1 000 issues and reviewed more than 750 PR's. We do long-term planning at Spatie, but we also have a weekly short-term planning meeting. When scheduling out to the coming week we only plan four days. So we have one work day we can be a bit flexible with. Do not imagine that day as like a fixed day, that time is mostly spread out in that week. Sometimes we do client work in that time, because an estimation was wrong, or we need to do some support things. But that time is also used to review/solve issues and review/merge pull requests. Like I’ve already mentioned I do enjoy working on packages in my free time as well. So a few times a week I spend some time in the evening to continue improving the released ones and creating new stuff.

The benefits of creating packages

There are many benefits of creating packages that makes time working on open source code well spent. You’ll learn a lot by working on packages. Each package needs to be carefully crafted. Laravel itself greatly emphasises developer happiness. It provides a clear and understandable syntax. I want that all our packages are easy to use too. Thinking about how other people will use your code will make you a better developer. We also learn a lot from the issues reported and the PR’s submitted by the users of our packages. A second benefit is, that sometimes, as a package gains some traction, you get quality code for free. Our laravel-fractal package is a developer friendly wrapper around The League’s Fractal package. I coded up the basic functionality to I needed myself and tagged that 1.0.0. In the next weeks I almost daily got a pull request adding another great feature to the package. And now it supports almost everything League’s Fractal can do. I think 90% of the code of that package was written by the community. Another benefit is a commercial one. When looking at our code I hope you’ll conclude that we do know our way around PHP and Laravel. Recently we’ve landed some projects because of our open source work. In most cases users of our packages were telling their bosses to hire us for a Laravel specific project. And last but not least we’re also dogfooding ourselves. Our own packages get used on most projects. If we discover a bug in a package used in a project we can very quickly fix that and distribute that fix through the power of Composer to our other projects. The only package that didn’t get used in any of our own projects is laravel-permission. That one was coded up just for fun.

Why we require the latest version of PHP

Before PHP 7 got released at the end of last year, we picked the oldest, still supported PHP version as a minimum requirement for a newly created package. When PHP 7 got released we changed our policy on that. Every new package now requires PHP 7. The most asked question in the issuetrackers of our packages is "Why can't you support PHP 5.6". I'd like to give some background on our stance. We create these packages primarily for our own future projects. The fact that they become popular is just a nice side effect. At our company PHP 5 is dead. So it makes zero sense to make our packages PHP 5 compatible. The latest new features such as the scalar type hints, return types, anonymous classes and the null coalescing operator can help create elegant code. I'm well aware that requiring PHP 7 will hurt the popularity of our packages in the short run. But, like already popularity is not our main goal. People who are using the latest and greatest version of PHP can benefit from our work. And I hope others will be nudged a bit towards PHP 7 by our decision. When PHP 7.1 gets released we'll probably set that as our minimum required version. Can't wait to use that fancy iterable typehint.

Top 10 of most popular packages

When taking amount of downloads into consideration these are our 10 most popular packages:
  1. laravel-backup
  2. pdf-to-image
  3. laravel-glide
  4. string
  5. laravel-analytics
  6. db-dumper
  7. laravel-medialibrary
  8. laravel-tail
  9. laravel-fractal
  10. laravel-permission
Like already mentioned in the intro some of these packages, like string and db-dumper, probably get a big download boost because there are required by other packages.

Not so popular packages

There are some packages, that we ourselves use in every single project, that I expected to be more popular. I hope by mentioning them here they'll get a little attention. If you decide to give the ones mentioned below a shot, I hope that you'll enjoy using them as much as we do. The first one in this category is laravel-menu. That one can generate html menus. It provides you with a beautiful, Laravel worthy, API to work with. It can even detect which one of the menu items should be considered active. Maybe I haven't search well enough for it, and I'm certainly biased, but I think there's no alternative that provides such an easy to work with API. Secondly, there's laravel-translatable. This one can make your Eloquent models hold translations. Unlike Dimitris Savvopoulos' excellent translatable package, ours does not need separate translation tables to store translations. Our package stores the translations as json in a column of the table. It's very easy to work with, there are no extra queries needed to fetch translations. We got the idea to store translations as json in a column from Mohamed Said (who now enjoys fame as Laravel employee #1 😎). And lastly I like to mention laravel-medialibrary. You saw this package at number 7 in the top 10 above, but I think this one deserves to be a few places higher. The package can associate all kind of media files (images, PDFs, ...) with Eloquent models. It can generate thumbnails, and can store files on external filesystem and generate URLs to the files. It's very easy to work with and it has excellent documentation. I think a lot of applications need to handle uploaded images and files. This package can save you countless hours by managing all of that for you.

The future

The past two years we’ve been regularly released packages. The most common problems in our projects are already solved with one of our packages. That’s why our package output will probably diminish a bit in the foreseeable future. Of course we’re dedicated to maintaining our code. We’ve recently created new major releases of laravel-backup, laravel-newsletter, laravel-analtyics and a few others. There’s no fixed roadmap for new packages. It really depends on the client projects we’re handling. If there is something there that can be solved in a generic way, we’ll create a package.

In closing

Creating and working on open source packages continues to be a truly fun experience. On a technical level I've learned lots of things that I wouldn't have learned when working solely on client projects. The positive feedback for our users also gave me enough confidence to start this blog and start speak publicly. The past year I've spoken at several local users groups (and I'm co-organizing the one in my home town), I got to speak at Laracon EU, and have two talks at [php]world to look forward to. At all those occasions I got to meet a lot of awesome fellow developers. That all probably wouldn't have happened if I stayed in my little bubble. So in short, working on open source helped getting to connect to the community. That sounds a bit corny, but that doesn't make it less true. If you haven't used our packages before, take a look at the list on our company website. Probably we've made some things that could be useful to you. If you already are using any of our stuff, and you like it, send us a postcard.