Optimize images in Laravel apps

A while ago we released image-optimizer. In short this package can make all kinds of images smaller by stripping out metadata and applying a little bit of compression. Read this blogpost to learn more about it. Although it's pretty easy to work with the package, we felt that we could deliver a more seamless experience in Laravel apps. That's why we created our newly released laravel-image-optimizer package. The package uses a bunch of binaries to optimize images. To learn which ones and how to install them, head over to the optimization tools section in the readme of the underlying image-optimizer package. That readme also contains info on what these tools will do to your images. Once the laravel-image-optimizer package is installed it's laughably easy to optimize images. If you like facades this is how you'd go about it:
use ImageOptimizer;

// the image will be replaced with an optimized version which should be smaller
ImageOptimizer::optimize($pathToImage);

// if you use a second parameter the package will not modify the original
ImageOptimizer::optimize($pathToImage, $pathToOptimizedImage);
You don't like facades you say? No problem! Just resolve a configured instance of Spatie\ImageOptimizer\OptimizerChain out of the container:
app(Spatie\ImageOptimizer\OptimizerChain::class)->optimize($pathToImage);
The package also contains a middleware to automatically optimize all images in a request.
Route::middleware('optimizeImages')->group(function () {
    // all images will be optimized automatically
    Route::post('upload-images', '[email protected]);
});
If you want to customize the chain of tools and the options being passed to them, you can do so by publishing and modifying the config file. This is the default contents:
use Spatie\ImageOptimizer\Optimizers\Svgo;
use Spatie\ImageOptimizer\Optimizers\Optipng;
use Spatie\ImageOptimizer\Optimizers\Gifsicle;
use Spatie\ImageOptimizer\Optimizers\Pngquant;
use Spatie\ImageOptimizer\Optimizers\Jpegoptim;

return [
    /*
     * When calling `optimize` the package will automatically determine which optimizers
     * should run for the given image.
     */
    'optimizers' => [

        Jpegoptim::class => [
            '--strip-all',  // this strips out all text information such as comments and EXIF data
            '--all-progressive',  // this will make sure the resulting image is a progressive one
        ],

        Pngquant::class => [
            '--force', // required parameter for this package
        ],

        Optipng::class => [
            '-i0', // this will result in a non-interlaced, progressive scanned image
            '-o2',  // this set the optimization level to two (multiple IDAT compression trials)
            '-quiet', // required parameter for this package
        ],

        Svgo::class => [
            '--disable=cleanupIDs', // disabling because it is know to cause troubles
        ],

        Gifsicle::class => [
            '-b', // required parameter for this package
            '-O3', // this produces the slowest but best results
        ],
    ],

    /*
     * The maximum time in seconds each optimizer is allowed to run separately.
     */
    'timeout' => 60,

    /*
     * If set to `true` all output of the optimizer binaries will be appended to the default log.
     * You can also set this to a class that implements `Psr\Log\LoggerInterface`.
     */
    'log_optimizer_activity' => false,
];
To learn more about laravel-image-optimzer head over to the readme on GitHub. And be sure to check out the other Laravel package we've previously made.

Easily optimize images using PHP (and some binaries)

Our recently released image-optimizer package can shave off some kilobyes of PNGs, JPGs, SVGs and GIFs by running them through a chain of various image optimization tools. In this blog post I'll tell you all about it. First, here's a quick example on how you can use it:
use Spatie\ImageOptimizer\OptimizerChainFactory;

$optimizerChain = OptimizerChainFactory::create();

$optimizerChain->optimize($pathToImage);
The image at $pathToImage will be overwritten by an optimized version which should be smaller. Here are a few images that were optimized by this package.

Why we built this

On nearly every website we make, images are displayed. In fact, in most cases images make up the bulk of the size of the entire page. So to provide a fast page load it's best to make those images as small as they can be. The less bytes the browser needs to download the faster the page will be. Now, to make images as small as they can be (without sacrificing a lot of detail) there are a lot of tools available. These tools, such as jpegtran, pngquant, and gifsicle, work more or less by applying a little bit of compression, removing metadata and reducing the number of colors. In most cases these tools can make your images considerably smaller without you noticing, unless you have a trained eye for that. These tools are free to use and can be easily installed on any system. So far so good. What makes them a bit bothersome to use from PHP code is that you need to create a process to run them. You must make sure that you pass the right kind of image to the right optimizer. You also have to decide which optimization parameters you're going to use for each tool. None of this is rocket science, but I bet that the vast majority of small to medium sites don't bother writing this code or researching these optimizations. Our package aims to do all of this out of the box. It can find out which optimizers it should run, it can execute the binary and by default uses a set of optimizers with a sane configuration.

How to optimize images

Optimizing an image is very easy using our package.
use Spatie\ImageOptimizer\OptimizerChainFactory;

$optimizerChain = OptimizerChainFactory::create();

$optimizerChain->optimize($pathToImage);
The image at $pathToImage will be overwritten by an optimized version which should be smaller. The package will use these optimizers if they are present on your system: The readme of the package includes instructions on how to install these on Ubuntu and MacOS.

Which tools will do what?

Like already mentioned, the package will automatically pick the right tool for the right image.

JPGs

JPGs will be made smaller by running them through JpegOptim. These options are used: - --strip-all: this strips out all text information such as comments and EXIF data - --all-progressive: this will make sure the resulting image is a progressive one, meaning it can be downloading using multiple passes of progressively higher details.

PNGs

PNGs will be made smaller by running them through two tools. The first one is Pngquant 2, a lossy PNG comprossor. We set no extra options, their defaults are used. After that we run the image throug a second one: Optipng. These options are used: - -i0: this will result in a non-interlaced, progressive scanned image - -o2: this set the optimization level to two (multiple IDAT compression trials)

Customizing the optimization

If you want to customize the chain of optimizers you can do so by adding Optimizers manually to a OptimizerChain. Here's an example where we only want optipng and jpegoptim to be used:
use Spatie\ImageOptimizer\OptimizerChain;
use Spatie\ImageOptimizer\Optimizers\Jpegoptim;
use Spatie\ImageOptimizer\Optimizers\Pngquant;

$optimizerChain = (new OptimizerChain)
   ->addOptimizer(new Jpegoptim([
       '--strip-all',
       '--all-progressive',
   ]))

   ->addOptimizer(new Pngquant([
       '--force',
   ]))
If you want to use another tool than the package supports out of the box, you can easily write your own optimizer. An optimizer is any class that implements the Spatie\ImageOptimizer\Optimizers\Optimizer interface. If you want to view an example implementation take a look at the existing optimizers that ship with the package.

Integration in other packages

Our image-optimizer is not the first package we wrote the revolves around handling images. There also our image package which makes modifying images very easy. And we have laravel-medialibrary which can associate all kinds of files (including images) with Eloquent models. And lastly we have Browsershot, which can turn any webpage into an image. Let's take a look on how image-optimizer has been integrated in all of those.

Image

Using the image package you can manipulate an image like this:
Image::load('example.jpg')
    ->sepia()
    ->blur(50)
    ->save();
This creates a sepia version that is blurred. The package has many other available manipulations. Here's how you can create an optimized version.
Image::load('example.jpg')
    ->sepia()
    ->blur(50)
    ->optimize()
    ->save();
Yea, just add the optimize method in the chain and you're done. Easy right?

laravel-medialibrary

In laravel-medialibrary this is just as easy. Using that package you can define conversion profiles on your models. Whenever you associate a file with that model a derived file using that conversion profile is being generated. This is handy for creating thumbnails and such. Here's a quick example of such a conversion profile.
use Illuminate\Database\Eloquent\Model;
use Spatie\MediaLibrary\HasMedia\Interfaces\HasMediaConversions;
use Spatie\MediaLibrary\HasMedia\HasMediaTrait;

class NewsItem extends Model implements HasMediaConversions
{
    use HasMediaTrait;

    public function registerMediaConversions()
    {
        $this->addMediaConversion('thumb')
              ->width(368)
              ->height(232)
              ->sharpen(10);
    }
}
Let's add an image to the medialibrary:
$media = NewsItem::first()->addMedia($pathToImage)->toMediaCollection();
Besides storing the original item, the medialibrary will also create a derived image.
$media->getPath();  // the path to the where the original image is stored
$media->getPath('thumb') // the path to the converted image with dimensions 368x232

$media->getUrl();  // the url to the where the original image is stored
$media->getUrl('thumb') // the url to the converted image with dimensions 368x232
That's a crash course into using medialibrary. But wait, we didn't create an optimized version of that thumbnail. Let's do that now. Under the hood all conversions are done by the aforementioned image package. So you can just use optimize function in your conversion profile.
    public function registerMediaConversions()
    {
        $this->addMediaConversion('thumb')
              ->width(368)
              ->height(232)
              ->sharpen(10)
              ->optimize();
    }
Boom done. In the next major version of medialibrary we'll automatically call optimize behind the screens for all image conversions. So you'll get optimized conversion by default. We'll add a nonOptimized method if you want to opt out of that. We haven't introduced that behaviour in the current version because it's breaking change.

Browsershot

Browsershot is a package that leverages headless Chrome to turn any webpage into an image. Here's how to use it:
Browsershot::url('https://example.com')->save($pathToImage);
And here's how to save an optimized version:
Browsershot::url('https://example.com')->optimize()->save($pathToImage);

In closing

I should mention that our optimize package is based upon another one by Piotr Śliwa. All the basic principles on how the package should work are inspired by Piotr's work. The reason why we rewrote it is because his was not that easy to extend and did not use modern stuff such as Symfony's Process component or PSR-3 compatible logging. In this post I've mainly mentioned tools you can install locally, but there actually are a lot of SaaS solutions as well such as TinyPNG, Kraken.io, imgix.com and many many others. In this first release of our image-optimizer package I've mainly concentrated on supporting local optimizers. With remote optimizers you have to deal with slowness of the network and API keys and such. But I do recognize the value of those remote services. That's why you'll probably see some remote optimizers being referenced or included in package in the future. Here's an issue on the repo where the first thoughts on that were being exchanged. The package contains a few more features not covered by this blogpost. So check out image-optimizer on GitHub. I hope our tool can make your images smaller and your pages faster. If you haven't already done so, check out our previous work in the open source space. Please send us a postcard if any of our stuff makes in into your production environment.

Easily convert webpages to images using PHP

Browsershot is a package that can easily convert any webpage into a image. Under the hood the conversion is made possible new headless options in Chrome 59. In this post I'd like to show you how you can use Browsershot v2. Here's a quick example of how it can be used:
Browsershot::url('https://example.com')->save($pathToImage);

History

About three years ago I started working on my first package, called Browsershot. It can convert any webpage to an image. Under the hood PhantomJS was used to perform the transformation. Since it got released it did it's job pretty well, but unfortunately PhantomJS recently has been abandoned. A few weeks ago Chrome 59 got released. In that version some pretty cool features were introduced. One of them is the ability to take screenshots of a webpage. A recent version of Chrome probably is better in rendering a correct page then and abandoned version of PhantomJS. That's why I decided to make a new major release of Browsershot. In this post I'd like to show you how you can use Browsershot v2.

Installing Chrome 59

In order to make use of Browsershot you must make sure that Chrome 59 or higher is installed on your system. On a Forge provisioned Ubuntu 16.04 server you can install the latest stable version of Chrome like this:
sudo wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add -
sudo sh -c 'echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list'
sudo apt-get update
sudo apt-get -f install
sudo apt-get install google-chrome-stable
Browsershot has been tested on MacOS and Ubuntu 16.04. If you use another OS your mileage may vary.

Using Browsershot

Here's the easiest way to create an image of a webpage:
Browsershot::url('https://example.com')->save($pathToImage);
Browsershot will make an education guess where Google Chrome is located. If on your system Chrome can not be found you can manually hint it's location:
Browsershot::url('https://example.com')
   ->setChromePath($pathToChrome)
   ->save($pathToImage);
By default the screenshot will be a png and it's size will match the resolution you use for your desktop. Want another size of screenshot? No problem!
Browsershot::url('https://example.com')
    ->windowSize(640, 480)
    ->save($pathToImage);
You can also set the size of the output image independently of the size of window. Here's how to resize a screenshot take with a resolution of 1920x1080 and scale that down to something that fits inside 200x200.
Browsershot::url('https://example.com')
    ->windowSize(1920, 1080)
    ->fit(Manipulations::FIT_CONTAIN, 200, 200)
    ->save($pathToImage);
In fact, you can use all the methods spatie/image provides. Here's an example where a greyscale image is created:
Browsershot::url('https://example.com')
    ->windowSize(640, 480)
    ->greyscale()
    ->save($pathToImage);
If, for some reason, you want to set the user agent Google Chrome should use when taking the screenshot you can do so:
Browsershot::url('https://example.com')
    ->userAgent('My Special Snowflake Browser 1.0')
    ->save($pathToImage);

In closing

The screenshot capabilities of Chrome are already quite good, but there's a lot of room for improvement still. Right now there's no way to specify where Chrome should save the screenshot (a problem that the package solves for you behind the scenes). In the DevTools there's an option to take a screenshot of the whole length of the page. Unfortunately this isn't possible with the current command line options. I'm pretty sure the headless and screenshotting options will improve in future versions of Chrome. I intend to update Browsershot as soon as new features become available in Chrome. Want to get started using Browsershot? Then head over to GitHub. Also take a look at the other packages our team has previously made.

A Vue component to display tabs

Last week my company released a vue-tabs-component, a quality Vue component to easily display tabs. You can view a demo of the component here. In this post I'd like to tell you all about it.

Why we created it

If you're just want to know what the component does, skip to the next section. Nearly all projects we deliver are powered by Blender, our Laravel application template. Blender has an admin section where users can manage content. For every content type it provides a crud interface. Nothing special there. But sometimes those content type can be pretty complex and have many many fields. Instead of displaying all the fields at once we put fields that belong in their own tap. So their might be a tab "Content" with some general fields, a tab "Media" with everything regarding images and uploads, ... This tab behaviour is provided by some custom JavaScript. Currently the JavaScript used in Blender is a bit of a patchwork. Because we nowadays use quite a bit of Vue in client projects we decided to make Vue a first class citizen in the admin section as well. We're currently searching an building Vue components to use in Blender. The tabs component is the first one that we finished building. Though it's a bit opinionated towards our use case, the component was generic enough to open source.

Introducing vue-tabs-component

If haven't already done so, head over to the demo page to get a feel of what the component can do. If you can registered the component globally like this:
//in your app.js or similar file
import Vue from 'vue';
import {Tabs, Tab} from 'vue-tabs-component';

Vue.component('tabs', Tabs);
Vue.component('tab', Tab);
you can use this piece of html to render the tabs like they are displayed on the demo page.
<tabs>
    <tab name="First tab">
        This is the content of the first tab
    </tab>
    <tab name="Second tab">
        This is the content of the second tab
    </tab>
    <tab id="oh-hi-mark" name="Custom fragment">
        The fragment that is appended to the url can be customised
    </tab>
    <tab prefix="<span class='glyphicon glyphicon-star'></span> " 
         name="Prefix and suffix" 
         suffix=" <span class='badge'>4</span>">
        A prefix and a suffix can be added
    </tab>
</tabs>
Easy right? The component doesn't come with any styling. If you like the pretty looks of our the demo, you can snatch the css used on GitHub.

Some nice features

The component will use the fragment of the url to choose which tab to open. So clicking navigating to https://example.com#second-tab will display the contents of the tab named Second tab. The component will also remember which tab was opened previously. If you reload without fragment the tab that is currently active will receive focus again. You'll find more about this feature on Github. The rendered output of the component adheres to the ARIA specification. Aria stand for "Accessible Rich Internet Applications" and describes which attributes html nodes should have in order to make the component accessible to people with disabilities

Testing the component

If you look at the source code of the projects you'll notice that has a test suite to make sure it works properly. Want to know more about how those tests are set up? Then head over the dedicated series of posts on those tests.

In closing

Head over the repo on GitHub to learn more about the component. Though our focus in open source is more creating on Laravel and PHP framework agnostic stuff, take a look at these other JavaScript packages we've previously made.

A conversation on laravel-html

Hi, how are you? I'm fine thanks! How are you? I saw you released another package last week. Yup yup, you mean laravel-html, right? It was actually coded up by my colleague Sebastian, who did an awesome job. But why put out yet another html generator? Html generation is a solved problem, right? Yes, it is but... I'm mean we already have laravelCollective/html... Yes, but... Why not just type html, ... come on, do we really need a package for this? It's easy to just type html. Let me say first that if you're happy with your current solution, just stick with it. There's no right or wrong here. Because everybody already has their own favourite solution, html generation is a pretty divisive subject. I believe that's why html generation was kicked out of the Laravel core. Stop rambling, just tell why you've created this. Well, like probably most fellow artisans we have been using the laravel collective package for quite some time and we were quite happy with it. Since html generation got kicked out of the Laravel core and has been put in the laravel collective package it has not evolved much. Take for instance this piece of code:
Form::email('element-name', null, ['class' => 'my-class']);
Seems like this is quite simple solution, but a few things bother me. That null parameter for instance when there is no value. And the fact that you need pass an array as the third parameter. In our package all html can be built up with a fluent interface. So you can do this:
html()->email('element-name')->class('my-class');
Yeah ok, that's certainly nice, but do you plan to code all of your html like this? No, certainly not. For simple things I prefer just writing the html. But when it comes to generating forms, I prefer using our package. Because when typing forms manually there's so much things to you need to take care of: the method field, the csrf token field etc... it's very error prone. Here's how you can create a form with laravel-html:
html()->form('PUT', '/post')->open();
html()->email('email')->placeholder('Your e-mail address');
html()->submit('Process this');
html()->form()->close();
That'll output:
<form method="POST" action="/post">
    <input type="hidden" name="_method" id="_method" value="PUT">
    <input type="hidden" name="_token" id="_token" value="csrf_token_will_be_here">
    <input type="email" name="email" id="email" placeholder="Your e-mail address">
    <input type="submit" value="Process this">"
</form>
As you can see those hidden _method and _token fields were added automatically. Cool! You can also use models or anything that implements ArrayAccess really to prefil the values of the form.
html()->model($user);

html()->input('name');
So if name is set on user the form element will receive it's value.
<input type="text" name="name" id="name" value="John">
This all sounds interesting, please continue. Well, if you want to know more, you really should really check out the documentation. It contains all you need to know. If you have a question about the package feel free to open up an issue at GitHub. Is there anything more that we should know about the package? It's PHP 7.1 only. Wut? Can you create a PHP 7.0 or PHP 5.6 version? Boring. Next question. But why did you make it PHP 7.1 only? My projects do not use that version yet. We mainly create packages to use them in our own projects. All our new client projects use PHP 7.1, so it makes no sense for us to create packages for an older version of PHP. But PHP 7.0 is still an active version... I really don't care. You're of course free to fork the package and maintain your own copy that's compatible with an older PHP version. Yay for open source. Ok, I'll go check it out! Please do, like already mentioned the docs are here. And be sure to take a look at the other packages we've created previously.

An easy to use server monitor written in PHP

We all dream of servers that need no maintenance at all. But unfortunately in reality this is not the case. Disks can get full, processes can crash, the server can run out of memory... Last week our team released a server monitor package written in PHP that keeps an eye on the health of all your servers. When it detects a problem it can, amongst others, notify you via slack. In this post I'd like to give some background why we created it and give you a run-through of what the package can do.

Why create another server health monitor?

The short answer: the available solutions were too complicated and / or too expensive for my company. If you want the long version, read on and otherwise skip to the introduction of laravel-server-monitor. In order to answer this question, let's first take a look at how my company has been doing things the last few years. We're what most people call a web agency. Our team is quite small: only a handful of developers with no dedicated operations team. We create a lot of web applications for clients. For the most part we also host all these applications. Up until a few years ago we created rather smallish sites and apps. We relied on traditional shared hosting. But as the complexity of our projects grew, shared hosting didn't cut it anymore. Thanks to excellent resources like serversforhackers.com and Laravel Forge we felt confident enough running our own servers. Each application is hosted on its own Digital Ocean server that was provisioned by Laravel Forge. Sure, running each application on a separate server is probably a bit more expensive than grouping some of them together on the same box. But using separate boxes has a lot of benefits:
  • for new projects you can just set up a new box with the latest versions of PHP, MySQL, etc...
  • When touching a project running on an older of PHP we can very easily upgrade the PHP version on that server. When running multiple applications on the same server you don't get this freedom without testing all the application running on it.
  • when a server is in trouble it only impacts one application
  • an application that is misbehaving in terms of memory and cpu usage can't impact other applications
  • each application has a lot of diskspace to play with (minimum 20 GB)
  • when Digital Ocean loses your server (yes, this can happen), it only impacts one application
Even though we are very happy with how we to things in regard to hosting, we don't see a lot of other companies of our size using this strategy. Most of them use managed / shared hosting. So as a small company with a lot of servers we're probably in a niche. The problem with paid server monitoring is that most services assume that if you have a lot of servers you're probably a large company that has a big budget for server monitoring. Pricing of paid plans is mostly per host. For single host this is mostly cheap (Datadog for example has a plan of $15 / per host / per month), but multiplied by a hundred hosts, this becomes too expensive. Also most of these services offer much more than we need. We don't need graphs of historical data or a lot of checks. We simply want to have a notification on our Slack channel when disk space is running low or when a service like memcached or beanstalk is down. There also are a lot of free open source solutions, like Zabbix, Nagios and Icinga. As a developer, the problem with these tools is that they don't target developers but people in an operations department. For developers these tools are quite complex to set up. Take a look at this guide to install Nagios. Sure, it's doable, but if you don't have much experience setting these kinds of things up, it can be quite daunting. There must be a better way.

Introducing laravel-server-monitor

To monitor our servers we built a Laravel package called laravel-server-monitor. This package can perform health checks on all your servers. If you're familiar with Laravel I'm sure you can install the package in a couple of mintutes. Not familiar with Laravel? No problem! We've also made a standalone version. More on that later. And to be clear: the package is able to monitor all kinds of servers, not only ones where Laravel is running. The package monitors your server by ssh'ing into them and performing certain commands. It'll interpret the output returned by the command to determine if the check failed or not. Let's illustrate this with the memcached check provided out of the box. This verifies if Memcached is running. The check runs service memcached status on your server and if it outputs a string that contains memcached is running the check will succeed. If not, the check will fail. When a check fails, and on other events, the package can send you a notification. Notifications look like this in Slack. You can specify which channels will send notifications in the config file. By default the package has support for Slack and mail notifications. Because the package leverages Laravel's native notifications you can use any of the community supported drivers or write your own. Hosts and checks can be added via the add-host artisan command or by manually adding them in the hosts and checks table. This package comes with a few built-in checks. But it's laughably easy to add your own checks.

Defining checks

The package will run checks on hosts. But what does such a check look like? A check actually is a very simple class that extends Spatie\ServerMonitor\CheckDefinitions\CheckDefinition. Let's take a look at the code of the built-in diskspace check.
namespace Spatie\ServerMonitor\CheckDefinitions;

use Spatie\Regex\Regex;
use Symfony\Component\Process\Process;

final class Diskspace extends CheckDefinition
{
    public $command = 'df -P .';

    public function resolve(Process $process)
    {
        $percentage = $this->getDiskUsagePercentage($process->getOutput());

        $message = "usage at {$percentage}%";

        if ($percentage >= 90) {
            $this->check->fail($message);

            return;
        }

        if ($percentage >= 80) {
            $this->check->warn($message);

            return;
        }

        $this->check->succeed($message);
    }

    protected function getDiskUsagePercentage(string $commandOutput): int
    {
        return (int) Regex::match('/(\d?\d)%/', $commandOutput)->group(1);
    }
}
This check will perform df -P . on the server. That will generate output much like this:
Filesystem                1024-blocks     Used Available Capacity Mounted on
/dev/disk/by-label/DOROOT    20511356 12378568   7067832      64% /
With a little bit of regex we extract the percentage listed in the Capacity column. If it's higher than 90% we'll call fail. This will mark the check as failed and will send out a notification. If it's higher than 80% it'll issue a warning. If it's below 80% the check succeeds. It's a simple as that. Let's take a look at another example: the memcached check.
namespace Spatie\ServerMonitor\CheckDefinitions;

use Symfony\Component\Process\Process;

final class Memcached extends CheckDefinition
{
    public $command = 'service memcached status';

    public function resolve(Process $process)
    {
        if (str_contains($process->getOutput(), 'memcached is running')) {
            $this->check->succeed('is running');

            return;
        }

        $this->check->fail('is not running');
    }
}
This check will run the command service memcached status on the server. If that commands outputs a string that contains memcached is running the check succeeds, otherwise it fails. Very simple.

Adding your own checks

Writing your own checks is very easy. Let's create a check that'll verify if nginx is running. Let's take a look at how to manually verify if Nginx is running. The easiest way is to run systemctl is-active nginx. This command outputs active if Nginx is running. Let's create an automatic check using that command. The first thing you must to do is create a class that extends from Spatie\ServerMonitor\CheckDefinitions\CheckDefinition. Here's an example implementation.
namespace App\MyChecks;

use Spatie\ServerMonitor\CheckDefinitions\CheckDefinition;
use Symfony\Component\Process\Process;

class Nginx extends CheckDefinition
{
    public $command = 'systemctl is-active nginx';

    public function resolve(Process $process)
    {
        if (str_contains($process->getOutput(), 'active')) {
            $this->check->succeed('is running');

            return;
        }

        $this->check->fail('is not running');
    }
}
Let's go over this code in detail. The command to be executed on the server is specified in the $command property of the class. The resolve function that accepts an instance of Symfony\Component\Process\Process. The output of that process can be inspected using $process->getOutput(). If the output contains active we'll call $this->check->succeed which will mark the check successful. If it does not contain that string $this->check->fail will be called and the check marked as failed. By default the package sends you a notification whenever a check fails. The string that is passed to $this->check->failed will be displayed in the notification. After creating this class you must register your class in the config file.
// config/server-monitor.php
'checks' => [
    ...
    'nginx' => App\MyChecks\Nginx::class,
],
And with that, you're done. A check definition can actually do a few more things like when it's supposed to be run the next time, setting timeouts and it has support for using custom properties. Take a look at the docs if you want to know more about this.

Using the stand alone version

If you're not familiar with Laravel, installing a package can be a bit daunting. That's why we also created a stand alone version called server-monitor-app. Under the hood it's simply a vanilla Laravel 5.4 application with the laravel-server-monitor package pre-installed into it. Using this app you can set up server monitoring in literally one minute. Here's a video that demonstrates the installation and using a check.

Under the hood

Let's take a look at a few cool pieces of source code. If you have a buch server than you can end up with al lot of checks that need to be run. Running all those checks one after the other can take a bit of time. That's why the package has support for running checks concurrently. In the config file you can configure how many ssh connections the package may use. This code is taken from CheckCollection which is responsable for running all the checks.
 public function runAll()
 {
     while ($this->pendingChecks->isNotEmpty() || $this->runningChecks->isNotEmpty()) {
         if ($this->runningChecks->count() < config('server-monitor.concurrent_ssh_connections')) {
             $this->startNextCheck();
         }

         $this->handleFinishedChecks();
     }
 }
This loop will run as long as there are pending checks or running checks. Whenever there are less checks as the amount configured in the config file, another new check is started. Let's take a look at what's happening inside the handleFinishedChecks function.
protected function handleFinishedChecks()
{
    [$this->runningChecks, $finishedChecks] = $this->runningChecks->partition(function (Check $check) {
        return $check->getProcess()->isRunning();
    });

    $finishedChecks->each->handleFinishedProcess();
}
This code leverages al lot of niceties offered by the latest versions of PHP and Laravel. It will filter out all the processes that are not running anymore (and are thus finished) and put them in the $finishedChecks collection. After that handleFinishedProcess will be called on each finished process. handleFinishedProcess will eventually call the resolve function seen in the CheckDefinition examples listed above.

Testing the code

Like all other packages we previously made, laravel-server-monitor contains a good suite of tests. This allows us to improve the code and accept PRs without fear of breaking the code. Because SSH connections are used in this package, testing all functionality provided some challenges. To easily code that relies on ssh connections the test suite contains a dummy SSH server written in JavaScript. When it runs it mimics all functionality of an SSH server. The SSH server itself is provided by the mscdex/ssh2. My colleague Seb wrote an easy to use abstraction around it. Using that abstraction we can let it respond whatever we want to when a command is sent to the server. This makes testing the package end to end a breeze. Here's how we test a succesful check.
 /** @test */
 public function it_can_run_a_successful_check()
 {
     $this->letSshServerRespondWithDiskspaceUsagePercentage(40);

     Artisan::call('server-monitor:run-checks');

     $check = Check::where('host_id', $this->host->id)->where('type', 'diskspace')->first();

     $this->assertEquals('usage at 40%', $check->last_run_message);
     $this->assertEquals(CheckStatus::SUCCESS, $check->status);
 }
Let's take a look at another example. The package will fire a CheckRestored event if a check succeeds again after it has failed previously.
 public function the_recovered_event_will_be_fired_when_an_check_succeeds_after_it_has_failed()
 {
     $this->letSshServerRespondWithDiskspaceUsagePercentage(99);

     Artisan::call('server-monitor:run-checks');

     $this->letSshServerRespondWithDiskspaceUsagePercentage(20);

     Event::assertNotDispatched(CheckRestored::class);

     Artisan::call('server-monitor:run-checks');

     Event::assertDispatched(CheckRestored::class, function (CheckRestored $event) {
         return $event->check->id === $this->check->id;
     })
 }

In closing

Laravel Server Monitor was a fun project to work on. We're quite happy with the results and now use it to monitor all our servers. If you're interested in learning more about the package head over to our documentation site or the package itself on GitHub. Keep in mind that this package was built specifically for teams without a dedicated ops member or department. So before using it, research the alternatives a bit yourself and make up your own mind what is a good solution for you. Our server monitor determines health by checking stuff inside your server. We also built another package called laravel-uptime-monitor that monitors your server from the outside. It'll regularly send http requests to verify if your server is up. It can even verify if the used ssl certificate is still valid for a certain amount of days. Take a look at the uptime monitor docs to know more. Also take a look at the other framework agnostic and laravel specific packages before. Maybe we've made something that can be of use in your next project.

An artisan command to easily test mailables

Most of the Laravel apps we create at Spatie will send mails. This can be a password reset mail, a welcome mail after registration, an order confirmation mail, ... One of the things we do is styling such mails so it has the same look and feel as the site it was sent from. When testing such mails our designers had to request a password reset or go through the entire checkout flow just to receive such an order confirmation mail. To make that testing process a lot easier we've created a package called laravel-mailable-test. This package provides an artisan command that can send a mailable to an mail-address. To send any mailable issue this artisan command:
php artisan mail:send-test "App\Mail\MyMailable" [email protected]
This will send the given mailable to the given email address. The to-, cc- and bcc-addresses that may be set in the given mailable will be cleared. The mail will only be sent to the email address given in the artisan command. The package will provide a value for any typehinted argument of the constructor of the mailable. If an argument is a int, string or bool the package will generated a value using Faker. Any argument that typehints an Eloquent model will receive the first record of that model. Image the constructor of your mailable looks like this:
public function __construct(string $title, Order $order) 
{
   ...
}
That constructor will receive a string generated by the sentence method of Faker and the first Order in your database. The values that are passed to the constructor of the mailable can be customized using the values option of the command.
php artisan mail:send-test "App\Mail\MyMailable" [email protected] --values="title:My title,order:5"
Using this command My title will be passed to $title and an Order with id 5 will be passed to $order. To learn more about the package head over to the readme on GitHub. Be sure take also take a look at this list of Laravel packages our team has previously made.

A Laravel package to rebuild the database

Out of the box Laravel comes with a few commands to migrate the database. One of them is migrate:refresh. That one will first run the down-steps for all your migrations and then run all the up steps. After that process your database should have the same structure as specified in your migrations. But what if your migrations don't have a down-step? Because I seldom have to revert a migration in my recent projects I haven't bothered with coding up the down steps. And without those down steps running migrate:refresh will result in errors, or worse, tears. I've created a little package that contains a command to quickly nuke all the tables in the database, run all migrations and all seeders. It will not use the down steps of the migrations but will simple use the output of the SHOW TABLES query to knock down all tables. Once the package is installed this is how you can build up your database again:
php artisan migrate:fresh
Need to run the seeds as well? No problem!
php artisan migrate:fresh --seed
The package works on MySQL, PostgreSQL and SQLite databases. If you need to perform some extra steps right before or right after the tables are dropped, you can hook into these events. This this all sound good to you? Then check out the package on GitHub. On our company site you'll find a list of Laravel packages we've previously made.

A package to fluently generate schema.org markup

Schema.org is a vocabulary of microdata markup that aims to make it easer for search crawlers to understand what's on a webpage. The vocabulary is very extensive: here's the official list of things that can be described with Schema.org. This article on Tutsplus explains schema.org and structured data in depth. It's kinda impossible to know the whole vocabulary by heart. That's why we created a package called spatie/schema-org that can greatly help you generate Schema.org markup. Let's take a look at some microdata that uses the Schema.org vocubulary. This an example (taken from the Google documentation on structured data) that describes an organization.
<script type="application/ld+json">
{
  "@context": "http://schema.org",
  "@type": "Organization",
  "url": "http://www.your-company-site.com",
  "contactPoint": [{
    "@type": "ContactPoint",
    "telephone": "+1-401-555-1212",
    "email": "[email protected]"
  }]
}
</script>
Here how that markup can be generated using our package:
Schema::organization()
    ->url('http://www.your-company-site.com')
    ->contactPoint(Schema::contactPoint()
        ->telephone(+1-401-555-1212)
        ->email('[email protected]')
    )
    ->toScript();
In our package every type in the vocabulary is it's own class, helping IDEs understand which types exist. So when you need to build up some microdata, code completion can help you immensely. Here's a demo recorded in PhpStorm. Converting the whole vocabulary manually to PHP classes would take a lot of time. That's why my colleague Sebastian built a generator that reads the specification and converts it to almost 600 separate classes. All classes contain helpful docblocks. When the schema.org vocubulary changes in the future, all we need to do is run the generator again to bring our package up to date. Take a look at the readme on GitHub to learn all the options of our schema-org package. Be sure to also check these framework agnostic and Laravel packages our team has made in the past.

A magic memoization function

Last friday Taylor Otwell tweeted an easy to use memoization function called once: Taylor was kind enough to share the source code behind the function. Because I'd like to use it in our projects I decided to make a small package out of it. I refactored Taylor's code for readability. His original code was a bit more powerful and could handle some more edge cases out of the box.

Usage

The spatie/once package provides you with a once function. It accepts a callable. Here's quick example:
class MyClass
{
    function getNumber()
    {
        return once(function () {
            return rand(1, 10000);
        });
    }
}
No matter how many times you run (new MyClass())->getNumber() inside the same request you'll always get the same number. The once function will only run once per combination of argument values the containing method receives.
class MyClass
{
    public function getNumberForLetter($letter)
    {
        return once(function () use ($letter) {
            return $letter . rand(1, 10000000);
        });
    }
}
So calling (new MyClass())->getNumberForLetter('A') will always return the same result, but calling (new MyClass())->getNumberForLetter('B') will return something else.

Behind the curtains

Let's go over the code of the once function to learn how all this magic works. In short: it will execute the given callable and save the result in a an array in the __memoized property of the instance once was called in. When we detect that once has already run before, we're just going to return the value stored inside the __memoized array instead of executing the callable again. The first thing it does is calling debug_backtrace. We'll use the output to determine in which function and class once is called and to get access to the object that function is running in. Yeah, we're already in voodoo-land. The output of the debug_backtrace is passed to a new instance of Backtrace. That class is just a simple wrapper so we can work more easily with the backtrace.
$trace = debug_backtrace(
    DEBUG_BACKTRACE_PROVIDE_OBJECT, 2
)[1];

$backtrace = new Backtrace($trace);
Next, we're going to check if once was called from within an object. If it was called from a static method or outside a class, we just bail out.
if (! $object = $backtrace->getObject()) {
   throw new Exception('Cannot use `once` outside a non-static method of a class');
}
Now that we're certain once is called within an instance of a class we're going to calculate a hash of the backtrace. This hash will be unique per function once was called in and takes into account the values of the arguments that function receives.
$hash = $backtrace->getArgumentHash();
Finally we will check if there's already a value stored for the given hash. If not, then execute the given $callback and store the result in the __memoized array on the object. In the other case just return the value in the __memoized array (the $callback isn't executed).
if (! isset($object->__memoized[$hash])) {
   $result = call_user_func($callback, $backtrace->getArguments());
   $object->__memoized[$hash] = $result;
}

Some things you need to be aware of

Because once will store results on the instance of the object it's called in, you cannot call once outside of on a object or inside a static method. Also, if you need to serialize an object that uses once be sure to unset the __memoized property when once returns objects. A perfect place for unsetting the __memoized would be the __sleep magic method.
If you like the package be sure to check out the framework agnostic and Laravel specific ones we've made before.

An opinionated tagging package for Laravel apps

There are a lot of quality tagging packages out there. Most of them offer the same thing: creating tags, associating them with models and some functions to easily retrieve models with certain tags. But in our projects at Spatie we need more functionality. Last week we released our own - very opinionated - tagging package aptly called laravel-tags. It comes with batteries included. Out of the box it has support for translating tags, multiple tag types and sorting capabilities. After the package is installed (be sure to check the requirements first) the only thing you have to do is to add the HasTags trait to an Eloquent model to make it taggable.
use Illuminate\Database\Eloquent\Model;
use Spatie\Tags\HasTags;

class NewsItem extends Model
{
    use HasTags;
    ...
}
Here are some code examples of what that trait allows you to do:
//create a model with some tags
$newsItem = NewsItem::create([
   'name' => 'testModel',
   'tags' => ['tag', 'tag2'], //tags will be created if they don't exist
]);

//attaching tags
$newsItem->attachTag('tag3');
$newsItem->attachTags(['tag4', 'tag5']);

//detaching tags
$newsItem->detachTag('tag3');
$newsItem->detachTags(['tag4', 'tag5']);

//syncing tags
$newsItem->syncTags(['tag1', 'tag2']); // all other tags on this model will be detached

//retrieving models that have any of the given tags
NewsItem::withAnyTags(['tag1', 'tag2']);

//retrieve models that have all of the given tags
NewsItem::withAllTags(['tag1', 'tag2']);
Under the hood those tags will be converted to Spatie\Tags\Tag models.

Adding translations

If you're creating a multilingual app it's really easy to translate the tags. Here's a quick example.
$tag = Tag::findOrCreate('my tag'); //store in the current locale of your app

//let's add some translation for other languages
$tag->setTranslation('name', 'fr', 'mon tag');
$tag->setTranslation('name', 'nl', 'mijn tag');

//don't forget to save the model
$tag->save();

$tag->getTranslation('name', 'fr'); // returns 'mon tag'

$tag->name // returns the name of the tag in current locale of your app.
The translations of the tags are stored in the name column of the tags table. It's a json column. To find a tag with a specific translation you can just use Laravel's query builder which has support for json columns.
 \Spatie\Tags\Tag
   ->where('name->fr', 'mon tag')
   ->first();
Behind the scenes spatie/laravel-translatable is used. You can use any method provided by that package.

Using tag types

In your application you might want to have multiple collections of tags. For example: you might want one group of tags for your News model and another group of tags for your BlogPost model. To create separate collections of tags you can use tag types.
//creating a tag with a certain type
$tagWithType = Tag::create('headline', 'newsTag'):
In addition to strings, all methods mentioned previously in this post can take instances of Tag as well.
$newsItem->attachTag($tagWithType);
$newsItem->detachTag($tagWithType);
...
The provided method scopes, withAnyTags and withAllTags, can take instances of Spatie\Tags\Tag too:
$tag = Tag::create('gossip', 'newsTag');
$tag2 = Tag::create('headline', 'newsTag');

NewsItem::withAnyTags([$tag, $tag2])->get();
To get all tags with a specific type use the getWithType method.
$tagA = Tag::findOrCreate('tagA', 'firstType');
$tagB = Tag::findOrCreate('tagB', 'firstType');
$tagC = Tag::findOrCreate('tagC', 'secondType');
$tagD = Tag::findOrCreate('tagD', 'secondType');

Tag::getWithType('firstType'); // returns a collection with $tagA and $tagB

//there's also a scoped version
Tag::withType('firstType')->get(); // returns the same result

Sorting tags

Whenever a tag is created it's order_column will be set the highest value in that column + 1.
$tag = Tag::findOrCreate('tagA');
$tag->order_column; // returns 1
$tag = Tag::findOrCreate('tagB');
$tag->order_column; // returns 2
Under the hood spatie/eloquent-sortable is used, so you can use any model provided by that package. Here are some examples:
//get all tags sorted on `order_column`
$orderedTags = Tags::ordered()->get(); 

//set a new order entirely
Tags::setNewOrder($arrayWithTagIds);

$myModel->moveOrderUp();
$myModel->moveOrderDown();

//move the tag to the first or last position
$myModel->moveToStart();
$myModel->moveToEnd();

$tag->swapOrder($anotherTag);
Of course you can also manually change the value of the order_column.
$tag->order_column = 10;
$tag->save();

If you want to know more about the package check out the documentation. Want to read at the source code (or ⭐ the project 😙)? Head over the repo to GitHub. This isn't the first Laravel package we've made, take a look at the list of Laravel packages on our company website to see if there's anything there that can be used on your projects.

A class to parse, build and manipulate URLs

For several projects and other packages we need to manipulate URL's. Instead of coding the same URL class over and over again, we extracted URL manipulation to it's own package. Here are some code examples on how you can use it.
$url = Url::fromString('https://spatie.be/opensource');

echo $url->withHost('github.com')->withPath('spatie');
// 'https://github.com/spatie'
Query parameters can be retrieved and transformed:
$url = Url::fromString('https://spatie.be/opensource?utm_source=github&utm_campaign=pacakges');

echo $url->getQuery(); // 'utm_source=github&utm_campaign=pacakges'
echo $url->getQueryParameter('utm_source'); // 'github'
echo $url->withoutQueryParameter('utm_campaign'); // 'https://spatie.be/opensource?utm_source=github'
It can retrieve path segments:
$url = Url::fromString('https://spatie.be/opensource/laravel');

echo $url->getSegment(1); // 'opensource'
echo $url->getSegment(2); // 'laravel'
It implements PSR-7's UriInterface interface:
class Url implements UriInterface { /* ... */ }
The league/uri is a more powerful package than this one. The main reason our package exists, is because the alternatives require non-standard php extensions. If you're dealing with special character encodings or need bulletproof validation, you're definitely better off using league/uri. That being said, if you need a lightweight solution, take a look at spatie/url on Github.

Managing opening hours with PHP

For several different clients we needed to display a schedule of opening hours on their websites. They also wanted to display if a department / store / ... is open on the moment you visit the site. My colleague Seb extracted all the functionality around opening hours to the newly released opening-hours package. You create an OpeningHours object that describes a business' opening hours. It can be queried for open or closed on days or specific dates, or use to present the times per day. A set of opening hours is created by passing in a regular schedule, and a list of exceptions.
$openingHours = OpeningHours::create([
    'monday' => ['09:00-12:00', '13:00-18:00'],
    'tuesday' => ['09:00-12:00', '13:00-18:00'],
    'wednesday' => ['09:00-12:00'],
    'thursday' => ['09:00-12:00', '13:00-18:00'],
    'friday' => ['09:00-12:00', '13:00-20:00'],
    'saturday' => ['09:00-12:00', '13:00-16:00'],
    'sunday' => [],
    'exceptions' => [
        '2016-11-11' => ['09:00-12:00'],
        '2016-12-25' => [],
    ],
]);
The object can be queried for a day in the week, which will return a result based on the regular schedule:
// Open on Mondays:
$openingHours->isOpenOn('monday'); // true

// Closed on Sundays:
$openingHours->isOpenOn('sunday'); // false
It can also be queried for a specific date and time:
// Closed because it's after hours:
$openingHours->isOpenAt(new DateTime('2016-09-26 19:00:00')); // false

// Closed because Christmas was set as an exception
$openingHours->isOpenAt(new DateTime('2016-12-25')); // false
It can also return arrays of opening hours for a week or a day:
// OpeningHoursForDay object for the regular schedule
$openingHours->forDay('monday');

// OpeningHoursForDay[] for the regular schedule, keyed by day name
$openingHours->forWeek();

// OpeningHoursForDay object for a specific day
$openingHours->forDate(new DateTime('2016-12-25'));

// OpeningHoursForDay[] of all exceptions, keyed by date
$openingHours->exceptions();
Take a look at the package on GitHub to learn the full api. Check out the list of PHP packages we've previously made to see if we've made something else that could be of use to you.

A little library to deal with color conversions

My colleague Seb needed to convert some color values in PHP. He looked around for some good packages, but there weren't any that fit the bill. So guess what he did? He just created a new package called spatie/color. Here's what it can do:
$rgb = Rgb::fromString('rgb(55,155,255)');

echo $rgb->red(); // 55
echo $rgb->green(); // 155
echo $rgb->blue(); // 255

echo $rgb; // rgb(55,155,255)

$rgba = $rgb->toRgba(); // `Spatie\Color\Rgba`
$rgba->alpha(); // 1
echo $rgba; // rgba(55,155,255,1)

$hex = $rgb->toHex(); // `Spatie\Color\Hex`
echo $hex; // #379bff
Currently the package only supports rgb, rgba and hex formats. If you need another format, submit a PR.

Taking care of backups with Laravel

A new major version of laravel-backup was recently tagged. This package can backup files and databases of your Laravel app (or any PHP application really). The backup consists of a zipfile containing a dump of the databases and all files that are selected for backup. The package will copy over the zipfile to external storage such as S3, Dropbox, SFTP, ... If something goes wrong during the package can notify you via, amongst others, mail or Slack. In this post I'd like to give some background on why and how it was built. Keep in mind that there is no one size fits all solution for backups. A small company will probably handle backups in another way than a bigger company. This post has small companies in mind with a couple of developers and no dedicated operations person or team.

Using modern hosting

Up until a few years ago the company where I work made small sites and applications. We used shared hosting for all our projects. When our client or we ourselves accidentally deleted files or some values in db, we asked our hosting provider make the necessary restores. We didn't do anything regarding backups ourselves. But as time went by we started to make bigger applications. Shared hosting wasn't cutting it for us anymore. We experimented with some of the newer cloud hosting providers and we liked the freedom that goes with using such solutions. Resources like serversforhackers.com and tools like Forge make it real easy to setup and administer your own servers. We settled on DigitalOcean. When I look around me I see many developers that are make the switch to DO, Linode, Rackspace, Amazon, etc... too.

The problem with modern hosting

But using such a provider can also bring a lot of sadness. At one fine morning I got a mail from DigitalOcean saying that they had lost our entire server. This kind of thing doesn't happen at DigitalOcean alone. Here's another horror story about a crashed server at Rackspace. If you google around a bit, you'll find similar stories for all major cloud hosting providers. Some of those providers provide a backup service. You might think that enabling such a backup service will save you from the problems described above. But unfortunately that's not the case. In the beginning of this year Taylor Otwell, the creator of Laravel, found that out the hard way. The hosting provider he was hosting his sites on was experiencing a denial of service attack. The whole datacenter was down for several days. Taylor had enabled the backups, but soon learned that if the entire datacenter is down, those backups can't be restored, because they can only be used in the datacenter where they were taken. Since that datacenter was down, the backups could not be restored. If you're on DigitalOcean you should be aware that their backup service only backs up your server once a week. So if your server crashes on a Friday, and your last backup was performed on a Monday, you've just lost data for an entire week. Your clients won't be happy.

Solutions

I should stress that I'm not trying to bash on DigitalOcean or any other specific provider. The point I want to make is that, in most cases, it's not enough to rely on the backups of your hosting provider alone. You shouldn't put your eggs in one basket. In addition to the backups performed by the hosting provider it's a good idea to take care of backups yourself as well. There are many options available, both free and paid. You could take matters in your own hands by running your own custom backup script. Here's a great one by Chris Fidao. This script will dump the database and will, together with the files that need to be backed up, be copied over to S3. The only downside that it will not notify you when something goes wrong with the backup. If you need notifications you will need to code that up yourself. Another option you have is to subscribe to Ottomatik.io, a paid service. I haven't used this myself but a lot of people seem to be happy with it. Take a look at their site to learn more about it. There are also free, open source, alternatives, such as BackupPC, Bacula and many others. They all work pretty good and have some great features, but the downside is that you need some system administration knowledge to set them up.

Introducing laravel-backup

My goal in creating laravel-backup was making a powerful backup solution that could be easily used by any developer that has some experience with Laravel. It can be installed into any Laravel app. These are the main functions it provides:

Backing up files and databases

Once installed the package can dump the databases used by the application and, together with the files selected in the config file, create a zipfile. The package will copy over the zipfile to one or more backup destinations. These backup destinations can be local, S3, Dropbox, ... Under the hood Laravel's cloud filesystem is used. When the package is installed, artisan will have gained some backup commands. A backup can be performed by running this command:
php artisan backup:run
Of course, this command can be scheduled in the console kernel of your application.

Cleaning up old backups

When using the package for a couple of weeks or months you will have hopefully taken many backups . Probably some of the older backups are not as important as the newer backups. When using S3 to store your backups you're going to pay for every byte you use. So it's probably a good idea to get rid of any excess backup that you don't need. Out of the box the package contains a sane strategy of cleaning up the backups that you don't need anymore. When running php artisan backup:clean the package will use the grandfather-father-son rotation scheme to remove backups. In short this means that all backups for a certain period are kept. After that, only one backup a day will be kept. After that only one backup a week will be kept. After that, only one a month. And, not surprisingly, after that only one backup a year will be kept.

Sending out notifications

The package can send out notifications when there was a problem taking a backup. Out of the box it can notify you via mail or Slack. The notifications are powered by Laravel 5.3's native notification capabilities. This means that, if you want to be notified using a another channel, you can easily use one of the community created notification drivers or create a notification driver yourself.

Monitoring the backups

Once you've setup and scheduled the backups you're probably never going to take another look at them again. You're going to assume that backup process will run frequently. But what if something unexpected goes silently wrong? The package comes with monitoring capabilities built in. Running php artisan backup:monitor will check if a backup was made in the last day and that the total storage used for the backups is not too high. Both criteria can be configured in the config file. You can monitor the backups from inside your app but I recommend setting up a separate server from which you do the monitoring. This will ensure that the monitoring will still work even if your app is completely down.

Stuck on PHP 5 or and older version of Laravel?

The new version of laravel-backup, v4, requires PHP 7 and Laravel 5.3 or higher. If you're stuck on PHP 5 or on Laravel 5.1 or 5.2 you can use v3 of the package. That version can do everything described above. Instead of relying on Laravel 5.3's notifications v3 of the package uses a custom notification system that has support for mail, Slack, Pushover and Telegram. There's also extensive documentation on v3 available. Though we won't introduce new features to v3 anymore we will still fix bugs.

Drawbacks

I recommend using this package only in small- to medium-sized apps. There should be enough free space on your local disk in order for the zip to be created. You should also be aware that, by using this package, your application holds the credentials to access your backups. In the case that your app is hacked, the hacker will gain access to your backups as well.

In closing

If you're interested in using this package, head over to our documentation site to learn in detail how to set it up and use it. If you like it be sure to take a look at the list of Laravel packages we've previously made.

A package to easily work with regex in PHP

PHP offers some functions to work with regular expressions, most notably preg_match, preg_match_all and preg_replace. Unfortunately those functions are a bit hard to use. Take preg_match_all for example, it requires you to pass in an array by reference to get all the matches. When something goes wrong some of those preg_ functions return false, while others return null. To get to the actual error message some truly horrible stuff needs to be done. To make working with regex a bit more developer-friendly my colleague Sebastian coded up a new package called regex. Here's how you can work with it:
use Spatie\Regex\Regex;

// Using `match`
Regex::match('/a/', 'abc'); // `MatchResult` object
Regex::match('/a/', 'abc')->hasMatch(); // true
Regex::match('/a/', 'abc')->result(); // 'a'

// Capturing groups with `match`
Regex::match('/a(b)/', 'abc')->result(); // 'ab'
Regex::match('/a(b)/', 'abc')->group(1); // 'a'

// Using `matchAll`
Regex::matchAll('/a/', 'abcabc')->hasMatch(); // true
Regex::matchAll('/a/', 'abcabc')->results(); // Array of `MatchResult` objects

// Using replace
Regex::replace('/a/', 'b', 'abc')->result(); // 'bbc';
Regex::replace('/a/', function (MatchResult $result) {
    return $result->result() . 'Hello!';
}, 'abc')->result(); // 'aHello!bc';
I'm pretty sure you agree this is much easier to work with (if you're not a regex-expert, but more of an occasional regex user). Take a look at the full documentation on GitHub to learn all the options. This isn't the first package our team has made, head over to our company website to view a list of all previously released packages.

Building a Laravel powered Slack bot

At Spatie we've recently introduced a bot to our Slack chat. We've named him Paolo (after the waiter in our favourite Italian restaurant in Antwerp: La Fontanella Da Enzo). Here's a demo of Paolo (the bot) in action. Behind the scenes Paolo is powered by a Laravel application that responds to all requests Slack is sending to it. In this post I'd like to explain how you can set up your own Laravel powered Slack bot.

General flow

A message in slack that starts with a slash is called a slash command. Whenever you type in a slash command in Slack channel, an http request will be sent to your Laravel app. You have to respond to that command within 3 seconds. Failing to do some will result in an error being displayed in the channel. After that initial response you're allowed to send multiple delayed responses. But there are some limitations for delayed responses. You may respond up to 5 times within 30 minutes after the user typed in the slash command on the slack channel. Want to know more about slash commands and how they work, then read this excellent article at Slack's API site. To make responding to Slack a breeze we're going to used the package our team released a few days ago called spatie/laravel-slack-slash-command.

Setting things up

Before you can get started building our Laravel app, you'll need to set up slash command at Slack.com. Head over to the custom integrations page at Slack.com to get started. There click "Slash commands" and on the next page click "Add configuration". On that screen you can choose a name for your Slack command. In our examples we'll use paolo but you can choose anything that Slack allows. You should now be on a screen that looks like this: paolo integration settings In the url field you should type the domain name of your Laravel app followed by one or more segments. In the screenshot we've added a slack segment. You can choose any segment you want. Normally the token field should already be filled. And that's all you need to do at Slack.com. The next thing you'll need to do is to install the spatie/laravel-slack-slash-command package. Let's pull it in via Composer.
composer require spatie/laravel-slack-slash-command
Next, you must install the service provider:
// config/app.php
'providers' => [
    ...
    Spatie\SlashCommand\SlashCommandServiceProvider::class,
];
The configuration file of the package can be published with:
php artisan vendor:publish --provider="Spatie\SlashCommand\SlashCommandServiceProvider" --tag="config"
This is the contents of the published config file:
return [

    /**
     * Over at Slack you can configure to which url the slack commands must be send.  
     * url here. You must specify that. Be sure to leave of the domain name.
     */
    'url' => 'slack',

    /**
     * The token generated by Slack with which to verify if a incoming slash command request is valid.
     */
    'token' => env('SLACK_SLASH_COMMAND_VERIFICATION_TOKEN'),

    /**
     * The handlers that will process the slash command. We'll call handlers from top to bottom
     * until the first one whose `canHandle` method returns true.
     */
    'handlers' => [
        //add your own handlers here


        //this handler will respond with a `Could not handle command` message.
        Spatie\SlashCommand\Handlers\CatchAll::class,
    ],
];
And with that you're ready to respond to http requests coming from Slack.

Setting up your first command handler

Whenever a user types in a slash command Slack will send an http request to the Laravel app. Next, our package will go over all classes in the handlers key of the config file from top to bottom until the first one whose canHandle method returns true. A handler is a class that is responsible for receiving a request from slack and sending a response back. Let's create our first handler. Handlers must extend Spatie\SlashCommand\Handlers\BaseHandler and implement the two abstract methods from that BaseHandler: canHandle and Handle. Here's an example.
namespace App\SlashCommandHandlers;

use Spatie\SlashCommand\Request;
use Spatie\SlashCommand\Response;
use Spatie\SlashCommand\Handlers\BaseHandler;

class Hodor extends BaseHandler
{
    /**
     * If this function returns true, the handle method will get called.
     *
     * @param \Spatie\SlashCommand\Request $request
     *
     * @return bool
     */
    public function canHandle(Request $request): bool
    {
        return true;
    }

    /**
     * Handle the given request.
     * 
     * @param \Spatie\SlashCommand\Request $request
     * 
     * @return \Spatie\SlashCommand\Response
     */
    public function handle(Request $request): Response
    {
        return $this->respondToSlack("Hodor, hodor...");
    }
}
This Hodor class will just respond with Hodor, hodor, ... to every request that is sent to it. You'll need to register this class in the config file.
// app/config/laravel-slack-slash-command
    'handlers' => [
        App\SlashCommandHandlers\Hodor::class,
        ...
    ], 
Let's see that in action.

A slightly more advanced handler

Let's create a slightly more interesting handler. This one that just repeats the command you've sent to it but only if the text after the command starts with repeat.
namespace App\SlashCommandHandlers;

use Spatie\SlashCommand\Request;
use Spatie\SlashCommand\Response;
use Spatie\SlashCommand\Handlers\BaseHandler;

class Repeat extends BaseHandler
{
    public function canHandle(Request $request): bool
    {
        return starts_with($request->text, 'repeat');
    }

    public function handle(Request $request): Response
    {   
        $textWithoutRepeat = substr($request->text, 7)

        return $this->respondToSlack("You said {$textWithoutRepeat}");
    }
}
Let's register this handler as well.
// app/config/laravel-slack-slash-command

    'handlers' => [
        App\SlashCommandHandlers\Repeat::class,
        App\SlashCommandHandlers\Hodor::class,
        ...
    ],    
If you type in /paolo repeat Hi, everybody in a slack channel now, you'll get a response Hi, everybody back. When you type in /poalo bla bla bla you'll get a response Hodor, hodor... because the Hodor handler is the first one which canHandle-method returns true. Notice that Spatie\SlashCommand\Request being past in canHandle and handle? It contains all data that's being passed by Slack to our Laravel app. These are it's most important properties:
  • command: the command name without the / that the user typed in. In our previous example this would be paolo.
  • text: all text text after the command. In our the example above this would be repeat Hi, everybody.
  • userName: the Slack username of the person that typed in the command
  • userId: the Slack user id of the person that typed in the command
  • channelName: the name of the channel where the user typed in the command
  • teamDomain: the name of the Slack subdomain. So if your team is on example.slack.com this would be example.

Customizing your response

By default the response will be sent to the user who typed in the original message. If you want the response to be visible to all users in the channel you can do this:
    public function handle(Request $request): Response
    {
        return $this
           ->respondToSlack("Hodor, hodor...")
           ->displayResponseToEveryoneOnChannel();
    }
There are also many formatting options. Take a look at this response on Slack: attachments
$this->respondToSlack()
    ->withAttachment(Attachment::create()
        ->setColor('good')
        ->setText('This is good!')
    )
    ->withAttachment(Attachment::create()
        ->setColor('warning')
        ->setText('Warning!')
    )
    ->withAttachment(Attachment::create()
        ->setColor('danger')
        ->setText('DANGER DANGER!')
    )
    ->withAttachment(Attachment::create()
        ->setColor('#439FE0')
        ->setText('This was a hex value')
    );
There are many more options to format a message. Take a look at Slacks documentation on attachments to learn what's possible.

Using signature handlers

A console command in Laravel can make use of a signature to set expectations on the input. A signature allows you to easily define arguments and options. If you let your handler extend Spatie\SlashCommand\Handlers\SignatureHandler you can make use of a $signature and the getArgument and getOption methods to get the values of arguments and options. Let's take a look at an example.
namespace App\SlashCommandHandlers;

use Spatie\SlashCommand\Request;
use Spatie\SlashCommand\Response;
use Spatie\SlashCommand\Handlers\SignatureHandler;

class SendEmail extends SignatureHandler
{
    public $signature = "paolo email:send {to} {message} {--queue}"

    public function handle(Request $request): Response
    {   
        $to = $this->getArgument('to');

        $message = $this->getArgument('message');

        $queue = $this->getOption('queue') ?? 'default';

        //send email message...
    }
}
Notice that there is no canHandle method present in that class. The package will automatically determine that a command /paolo email:send [email protected]l.com hello can be handled by this class.

Sending delayed responses

Remember that restriction mentioned above about the initial response to a slash command. Your Laravel app only has three seconds to respond otherwise an error message will be shown at Slack. After that initial fast response you're allowed to send 5 more responses in the next 30 minutes for the command. These responses are called "delayed responses". We're going to leverage Laravel's queued jobs to send those delayed responses. Please make sure that you've set up a real queue driver in your app, it needs to be something other than sync. Imagine you need to call a slow API to get a response for a slash command. Let's first create a handler that will send the initial fast response.
namespace App\SlashCommandHandlers;

use Spatie\SlashCommand\Request;
use Spatie\SlashCommand\Response;

class SlowApi extends BaseHandler
{
    public function canHandle(Request $request): bool
    {
        return starts_with($request->text, 'give me the info');
    }

    public function handle(Request $request): Response
    {
        $this->dispatch(new SlowApiJob());

        return $this->respondToSlack("Looking that up for you...");
    }
}
Notice that we're dispatching a job right before sending a response. Behind the scenes Laravel will queue that job. This is how that SlowApiJob would look like.
namespace App\SlashCommand\Jobs;

use Spatie\SlashCommand\Jobs\SlashCommandResponseJob;

class SlowApiJobJob extends SlashCommandResponseJob
{
    // notice here that Laravel will automatically inject dependencies here
    public function handle(MyApi $myApi)
    {
        $response = $myApi->fetchResponse();

        $this
           ->respondToSlack("Here is your response: {$response}")
           ->send();
    }
}
Notice that, unlike in the Handlers the response is not returned and that send() is called after the respondToSlack-method. With this in place a quick response Looking that info for you... will be displayed right after the user typed /your-command get me the info. After a little while, when MyApi has done it's job Here is your response: ... will be sent to the channel.

Some useful handlers

The previous examples of this post were quite silly. You'll probably never going to use to handlers in your bot. Let's review a real life example. Our Poalo bot can lookup dns records for a given domain. This is how that looks like in a Slack channel. This is the actual class that we use in our bot:
namespace App\SlashCommandHandlers;

use Spatie\SlashCommand\Attachment;
use Spatie\SlashCommand\AttachmentField;
use Spatie\SlashCommand\Handlers\SignatureHandler;
use Spatie\SlashCommand\Request;
use Spatie\SlashCommand\Response;

class Dns extends SignatureHandler
{
    protected $signature = 'paolo dns {domain}';

    /**
     * Handle the given request. Remember that Slack expects a response
     * within three seconds after the slash command was issued. If
     * there is more time needed, dispatch a job.
     *
     * @param Request $request
     *
     * @return Response
     */
    public function handle(Request $request): Response
    {
        $domain = $this->getArgument('domain');

        if (empty($domain)) {
            return $this->respondToSlack("You must provide a domain name.");
        }

        $sanitizedDomain = str_replace(['http://', 'https://'], '', strtolower($this->getArgument('domain')));

        $dnsRecords = dns_get_record($sanitizedDomain, DNS_ALL);

        if (!count($dnsRecords)) {
            return $this->respondToSlack("Could not get any dns records for domain {$domain}");
        }

        $attachmentFields = collect($dnsRecords)->reduce(function (array $attachmentFields, array $dnsRecord) {
            $value = $dnsRecord['ip'] ?? $dnsRecord['target'] ?? $dnsRecord['mname'] ?? $dnsRecord['txt'] ?? $dnsRecord['ipv6'] ?? '';

            $attachmentFields[] = AttachmentField::create('Type', $dnsRecord['type'])->displaySideBySide();
            $attachmentFields[] = AttachmentField::create('Value', $value)->displaySideBySide();

            return $attachmentFields;
        }, []);

        return $this->respondToSlack("Here are the dns records for domain {$domain}")
            ->withAttachment(Attachment::create()
                ->setColor('good')
                ->setFields($attachmentFields)
            );
    }
}
In order to get home every member of our team needs to bike a bit. That's why we've also created a command to display a rain forecast. This is what happens when /paolo rain is typed in our slack channels. This is the class responsible for creating that response.
namespace App\SlashCommandHandlers;

use Spatie\SlashCommand\Attachment;
use Spatie\SlashCommand\Handlers\SignatureHandler;
use Spatie\SlashCommand\Request;
use Spatie\SlashCommand\Response;

class Rain extends SignatureHandler
{
    protected $signature = 'paolo rain';

    /**
     * Handle the given request. Remember that Slack expects a response
     * within three seconds after the slash command was issued. If
     * there is more time needed, dispatch a job.
     *
     * @param Request $request
     *
     * @return Response
     */
    public function handle(Request $request): Response
    {
        return $this
            ->respondToSlack("Here you go!")
            ->withAttachment(
                Attachment::create()->setImageUrl('http://api.buienradar.nl/image/1.0/radarmapbe?width=550')
            );
    }
}

In closing

The spatie/laravel-slack-slash-command package makes is it easy to let a Laravel app respond to a slash command from Slack. If you start using the package, let me know in the comments below what your bot can do. And if you like our package, take a look at this list of Laravel packages we've previously released to see if we've made something that can be of use to you.

A package to log activity in a Laravel app

In your apps there's probably a lot going on. Users log in and out, they create, update and delete content, mails get sent and so on. For an administrator of an app these events provide useful insights. In almost every project we make at Spatie we log these events and show them in the admin-section of our site. Here's how that looks like in our Laravel template called Blender. log We made a new package called laravel-activitylog that makes logging activities in a Laravel app a cinch. In this blogpost I'd like to walk you through it.

Basic logging

This is the most basic way to log some activity:
activity()->log('Look mum, I logged something');
This will write a record in the activity_log table. You can retrieve activity using the Spatie\Activitylog\Models\Activity model that the package provides.
$lastActivity = Activity::all()->last(); //returns the last logged activity

$lastActivity->description; //returns 'Look mum, I logged something';
You can specify on which object the activity is performed by using performedOn:
activity()
   ->performedOn($someContentModel)
   ->log('edited');

$lastActivity = Activity::all()->last(); //returns the last logged activity

$lastActivity->subject; //returns the model that was passed to `performedOn`;
You can set who or what caused the activity by using causedBy:
activity()
   ->causedBy($userModel)
   ->performedOn($someContentModel)
   ->log('edited');

$lastActivity = Activity::all()->last(); //returns the last logged activity

$lastActivity->causer; //returns the model that was passed to `causedBy`;   
If you're not using causedBy the package will automatically use the logged in user.

Automatic model event logging

A neat feature of this package is that it can automatically log events such as when a model is created, updated and deleted. To make this work all you need to do is let your model use the Spatie\Activitylog\Traits\LogsActivity-trait. As a bonus the package will also log the changed attributes for all these events when setting $logAttributes property on the model. Here's an example:
use Illuminate\Database\Eloquent\Model;
use Spatie\Activitylog\Traits\LogsActivity;

class NewsItem extends Model
{
    use LogsActivity;

    protected $fillable = ['name', 'text'];

    protected static $logAttributes = ['name', 'text'];
}
Let's see what gets logged when creating an instance of that model.
$newsItem = NewsItem::create([
   'name' => 'original name',
   'text' => 'Lorum'
]);

//creating the newsItem will cause an activity being logged
$activity = Activity::all()->last();

$activity->description; //returns 'created'
$activity->subject; //returns the instance of NewsItem that was created
$activity->changes; //returns ['attributes' => ['name' => 'original name', 'text' => 'Lorum']];
Now let's update some that $newsItem.
$newsItem->name = 'updated name'
$newsItem->save();

//updating the newsItem will cause an activity being logged
$activity = Activity::all()->last();

$activity->description; //returns 'updated'
$activity->subject; //returns the instance of NewsItem that was created
Calling $activity->changes will return this array:
[
   'attributes' => [
        'name' => 'original name',
        'text' => 'Lorum',
    ],
    'old' => [
        'name' => 'updated name',
        'text' => 'Lorum',
    ],
];
Pretty cool, right? Now, what happens when you call delete?
$newsItem->delete();

//deleting the newsItem will cause an activity being logged
$activity = Activity::all()->last();

$activity->description; //returns 'deleted'
$activity->changes; //returns ['attributes' => ['name' => 'updated name', 'text' => 'Lorum']];
Read the documentation on event logging to learn how to choose the events that must be logged, how to customize the description for an event, and how to specify the attributes of which the changes must be logged.

Using multiple logs

When not specify a log name the activities will be logged on the default log.
activity()->log('hi');

$lastActivity = Spatie\Activitylog\Models\Activity::all()->last();

$lastActivity->log_name; //returns 'default', this value can be changed in the config file ;
You can specify the log on which an activity must be logged by passing the log name to the activity function:
activity('other-log')->log("hi");

Activity::all()->last()->log_name; //returns 'other-log';
Like mentioned before Activity model is just a regular Eloquent model that you know and love. So you can just use a where on it.
Activity::where('log_name' , 'other-log')->get(); //returns all activity from the 'other-log'
The package also provides an inLog scope you can use:
Activity::inLog('other-log')->get();

//you can pass multiple log names to the scope
Activity::inLog('default', 'other-log')->get();

//passing an array is just as good
Activity::inLog(['default', 'other-log'])->get();

This concludes the tour of laravel-activitylog. Want to know more about the package, then head over to the documentation to learn all the options. If you find this package useful, be sure to take a look at the Laravel packages we previous made.

Make your Laravel app comply with the crazy EU cookie law

All sites owned by EU citizens or targeted towards EU citizens must comply to a crazy EU law. This law requires a dialog to be displayed to inform the users of your websites how cookies are being used. You can read more info on the legislation on the site of the European Commission. The newest Laravel package made by my colleagues at Spatie and myself makes your app compliant with that law. Once installed the package will render the following dialog that, when styled, will look very much like this one: 68747470733a2f2f7370617469652e6769746875622e696f2f6c61726176656c2d636f6f6b69652d636f6e73656e742f696d616765732f6469616c6f672e706e67 When the user clicks "Allow cookies" a laravel_cookie_consent cookie will be set and the dialog will be removed from the DOM. On the next request Laravel will notice that the laravel_cookie_consent has been set and will not display the dialog again. We've made it easy to customize the texts shown by the dialog. You can also make changes to the dialog or JavaScript itself. The legislation is pretty very vague on how to display the warning, which texts are necessary, and what options you need to provide. This package will go a long way towards compliance, but if you want to be 100% sure that your website is ok, you should consult a legal expert. Take a look at the package on GitHub to learn how to install the package and which options it provides. If you like it, be sure to check out our full list of Laravel packages.

Laravel Analytics v2 has been released

One of our more popular packages is laravel-analytics. The package makes it easy to fetch information such as pageviews, top referrers, etc... from the Google Analytics API. In our Blender-based projects we use the fetched data to display a nice chart in the admin section: Screen Shot 2016-06-19 at 19.33.53 Laravel-analytics is one of the first packages we ever created. Since the first release the preferred way of authenticating with Google has changed. We also learned a lot on how to create readable code. That's why we did a cleanup of the code and published a new major version of the package.

Installation

Learning how to authenticate with a Google API can be quite a pain. A previous post already touched on this:
If you’ve ever worked with some API’s by Google then you know their documentation can be very confusing. It’s not that they don’t have documentation, but code examples of common use cases are simply not present. You must wade through a lot of pages to learn basic things such as how to make an authorized request.
To prevent potential tableflips on your part there's some documentation included to guide you through Google's authentication process.

Usage

Once the package is installed this is how to retrieve visitor and pageview data for the past week.
$analyticsData = Analytics::fetchVisitorsAndPageViews(Period::days(7));
$analyticsData is a collection in which each item is an array that holds keys date, visitors and pageViews. These are the other functions the package provides:
public function fetchMostVisitedPages(Period $period, int $maxResults = 20): Collection;

public function fetchTopReferrers(Period $period, int $maxResults = 20): Collection;

public function fetchTopBrowsers(Period $period, int $maxResults = 10): Collection;
To perform all other queries on the Google Analytics resource you can call performQuery. Google's Core Reporting API provides more information on on which metrics and dimensions might be used.
public function performQuery(Period $period, string $metrics, array $others = [])

Check out the documentation on GitHub to learn more. If you like laravel-analtyics, be sure to check out our other Laravel packages as well.