How we improved our PWA score by 53 points in 4 hours

On the madewithlove blog Frederick Vanbrabant wrote a post on how he and colleague improved the PWA score of their company site.

So the first thing you should know about PWA (or progressive web apps) is that it’s an adaptation of your current site or web app. This means that if you want to have all the features of a progressive web app, you are going to need to change your current site/application. The good news here is that they are all changes you would want to have anyway.

https://blog.madewithlove.be/post/improved_pwa_score

Visualizing PHPUnit Tests

In an article on the Hackster.io site explains in detail how he made a nice visualization of unit test results. Pretty cool what you can do with a raspberry pi, some LEDS and a couple of scripts.

I wanted to create a desktop gadget to visualize the progress of unit tests run via PHPUnit.I’ve named this project PHPUnicorn (by combining “PHPUnit” with “Unicorn pHAT”).

https://www.hackster.io/colinodell/phpunicorn-visualizing-phpunit-tests-896208

Working With PHPUnit and PhpStorm

On the JetBrains blog Gary Hockin explains how to easily run a single PHPUnit test.

To run all the tests in a single file, right-click the test in the Project Pane (the left-hand navigation pane), and select Run .

To run all the tests in a single class, right-click the class name in the editor, and select Run .

To run the tests in a single method, right-click the method name, and select Run .

https://blog.jetbrains.com/phpstorm/2017/01/working-with-phpunit-and-phpstorm/

Understanding Laravel’s HighOrder Collections

One of my favourite features that was introduced in Laravel 5.4 are the higher order collection functions. It allows you to rewrite

collect($models)->filter(function(Model $model) {
   $model->passesFilter();
});

to:

collect($models)->filter->passesFilter();

This works with the filter method an a bunch of other collection methods.

In a new post on his blog Nicola Malizia explains how these methods work under the hood.

A new version of Laravel is available from 24 January 2017 and, as usual, it comes with a lot of new features.
Among them, there is one that takes advantage of the dynamic nature of PHP. Some out of there will contempt this, but I find it awesome!

https://unnikked.ga/understanding-laravels-highorder-collections-ee4f65a3029e

Closure Binding as an alternative to “use” variables

On his blog Mark Baker shares some thoughts on how to use closure binding to avoid having import variables with the use keyword.

You’ll learn how to rewrite

$filteredArrayData = array_filter(
    $arrayData,
    function($value) use ($minimumPrice, $maximumPrice) {
        return $value->price >= $minimumPrice && $value->price < $maximumPrice;
    }
);

to

$filteredArrayData = array_filter(
    $bookData,
    $priceFilter->inRange(5.00, 15.00)
);

https://markbakeruk.net/2017/03/12/closure-binding-as-an-alternative-to-use-variables/

10 Lodash Features You Can Replace with ES6

In the JavaScript world Lodash is a pretty popular and awesome package with lots of handy array, collection and object methods. In this article Dan Prince explains that some of those methods do have a nice ES6 equivalent.

Lodash is the most depended on npm package right now, but if you’re using ES6, you might not actually need it. In this article, we’re going to look at using native collection methods with arrow functions and other new ES6 features to help us cut corners around many popular use cases.

https://www.sitepoint.com/lodash-features-replace-es6/

How agencies & freelancers should do web hosting

Andrew Welch of the New York based agency nystudio107 wrote a good overview of the options agencies & freelancers have regarding hosting.

Web hosting is something that many agencies and freelancers don’t give a whole lot of thought to. They just use whomever they’ve had a long-standing relationship with, and call it a day.

However, choosing the right host—and the right type of host—can be crucial to the success of a project. And the hosting world has changed a whole lot in the past few years, so let’s dive in.

https://nystudio107.com/blog/web-hosting-for-agencies-freelancers

At my company Spatie we’re pretty happy with our choice to hosts our client projects on VPSes. We provision them using Laravel Forge and some custom ansible scripts.

An easy way to validate front end forms using back end logic

Imagine you’re building an single page application that has a couple of forms. How are you going to validate those forms? Because we don’t want to refresh the page submitting the form as usual isn’t possible. Adding validation logic to the JavaScript side of things would just duplicate the validation logic already present on the server. Wouldn’t it be nice if we could validate our front end forms using back end logic?

A while ago Jeffrey Way published a lesson on Object-Oriented forms in the Vue 2.0 series that does just that. Because I wanted to use Jeffrey’s way of doing things in some real life projects I decided to put his code (and some extra niceties) in JavaScript package called spatie/form-backend-validation. Though it could work in other frameworks it is primarily built with a Laravel back end in mind.

To get started using the package it’s best to watch the aforementioned free video on Object-Oriented forms. That’ll explain the train of thought behind it. In short the package provides a Form class to submit all form values to the server. It will collect any errors the server sends back. Using a Vue component we can easily collect all values and display all errors. And this can all happen without having to refresh the page.

To help you understand it even better, we’ve created an example app in which the package is installed. These are the most important pieces:

If you set up the example app and hit it the / route of the app you’ll see a form for you to toy around with.

Hopefully this Form class can be of use in your projects as well. We’ve only made a couple of JavaScript packages before, but I can image a few more will pop up in the future.

An Animated Intro to RxJS

On css-tricks David Khourshid wrote a good introduction to RxJS.

According to ReactiveX.io: “ReactiveX is a library for composing asynchronous and event-based programs by using observable sequences.”

That’s a lot to digest in a single sentence. In this article, we’re going to take a different approach to learning about RxJS (the JavaScript implementation of ReactiveX) and Observables, by creating reactive animations.

https://css-tricks.com/animated-intro-rxjs/

If you want to toy around with the same ideas in PHP check out RxPHP

How I Got From 0 to 1 000 Stars on GitHub in Three Months With My Open Source Side Project

Ondřej Mirtes, author of PHPStan lists some great tips to make a side project succesful.

Most developers have side projects. That’s how we try out new things or make something that we miss on the market or in our dev stack. But most side projects end up unfinished and never actually see the light of day. And even if a developer builds up the courage to show his work to the public, he quickly finds out that just publishing a repository doesn’t actually bring the masses to his doorstep.

In this article, I’d like to share with you what I did to make sure that the project doesn’t end up in the dustbin of history. I will concentrate on open source software, but the following advice may as well apply to any creative endeavour.

https:[email protected][email protected]y-open-source-side-project-8ffe4725146#.211n6vihd

Why I’m frequently absent from open source

James Long gives some solid advice: always keep in mind that there are a lot of things that are more important than coding.

The goal of free open source development is empowerment: everyone can not only use code for free but also contribute to and influence it. This model allows people to teach and learn from each other, improves businesses by sharing work on similar ideas, and has given some people the chance to break out and become well-known leaders.

Unfortunately, in reality open source development is rife with problems and is ultimately unsustainable. Somebody has to pay the cost of maintaining a project.

http://jlongster.com/Why-Frequently-Absent-Open-Source

Symfony Routing performance considerations

On his blog Frank De Jonge explains how he solved a performance problem in one of his projects.

Last week I took a deep dive into Symfony’s Routing Component. A project I worked on suffered from a huge performance penalty caused by a routing mistake. This lead me on the path to discovering some interesting performance considerations. Some common practices align nicely with Symfony’s optimisations, let’s look into those.

https://blog.frankdejonge.nl/symfony-routing-performance-considerations/

An easy to use server monitor written in PHP

We all dream of servers that need no maintenance at all. But unfortunately in reality this is not the case. Disks can get full, processes can crash, the server can run out of memory…

Last week our team released a server monitor package written in PHP that keeps an eye on the health of all your servers. When it detects a problem it can, amongst others, notify you via slack.

In this post I’d like to give some background why we created it and give you a run-through of what the package can do.

Why create another server health monitor?

The short answer: the available solutions were too complicated and / or too expensive for my company. If you want the long version, read on and otherwise skip to the introduction of laravel-server-monitor.

In order to answer this question, let’s first take a look at how my company has been doing things the last few years. We’re what most people call a web agency. Our team is quite small: only a handful of developers with no dedicated operations team. We create a lot of web applications for clients. For the most part we also host all these applications.

Up until a few years ago we created rather smallish sites and apps. We relied on traditional shared hosting. But as the complexity of our projects grew, shared hosting didn’t cut it anymore. Thanks to excellent resources like serversforhackers.com and Laravel Forge we felt confident enough running our own servers.

Each application is hosted on its own Digital Ocean server that was provisioned by Laravel Forge. Sure, running each application on a separate server is probably a bit more expensive than grouping some of them together on the same box. But using separate boxes has a lot of benefits:

  • for new projects you can just set up a new box with the latest versions of PHP, MySQL, etc…
  • When touching a project running on an older of PHP we can very easily upgrade the PHP version on that server. When running multiple applications on the same server you don’t get this freedom without testing all the application running on it.
  • when a server is in trouble it only impacts one application
  • an application that is misbehaving in terms of memory and cpu usage can’t impact other applications
  • each application has a lot of diskspace to play with (minimum 20 GB)
  • when Digital Ocean loses your server (yes, this can happen), it only impacts one application

Even though we are very happy with how we to things in regard to hosting, we don’t see a lot of other companies of our size using this strategy. Most of them use managed / shared hosting. So as a small company with a lot of servers we’re probably in a niche.

The problem with paid server monitoring is that most services assume that if you have a lot of servers you’re probably a large company that has a big budget for server monitoring. Pricing of paid plans is mostly per host. For single host this is mostly cheap (Datadog for example has a plan of $15 / per host / per month), but multiplied by a hundred hosts, this becomes too expensive.

Also most of these services offer much more than we need. We don’t need graphs of historical data or a lot of checks. We simply want to have a notification on our Slack channel when disk space is running low or when a service like memcached or beanstalk is down.

There also are a lot of free open source solutions, like Zabbix, Nagios and Icinga. As a developer, the problem with these tools is that they don’t target developers but people in an operations department. For developers these tools are quite complex to set up. Take a look at this guide to install Nagios. Sure, it’s doable, but if you don’t have much experience setting these kinds of things up, it can be quite daunting.

There must be a better way.

Introducing laravel-server-monitor

To monitor our servers we built a Laravel package called laravel-server-monitor. This package can perform health checks on all your servers. If you’re familiar with Laravel I’m sure you can install the package in a couple of mintutes. Not familiar with Laravel? No problem! We’ve also made a standalone version. More on that later. And to be clear: the package is able to monitor all kinds of servers, not only ones where Laravel is running.

The package monitors your server by ssh’ing into them and performing certain commands. It’ll interpret the output returned by the command to determine if the check failed or not.

Let’s illustrate this with the memcached check provided out of the box. This verifies if Memcached is running. The check runs service memcached status on your server and if it outputs a string that contains memcached is running the check will succeed. If not, the check will fail.

When a check fails, and on other events, the package can send you a notification. Notifications look like this in Slack.

You can specify which channels will send notifications in the config file. By default the package has support for Slack and mail notifications. Because the package leverages Laravel’s native notifications you can use any of the community supported drivers or write your own.

Hosts and checks can be added via the add-host artisan command or by manually adding them in the hosts and checks table.

This package comes with a few built-in checks. But it’s laughably easy to add your own checks.

Defining checks

The package will run checks on hosts. But what does such a check look like? A check actually is a very simple class that extends Spatie\ServerMonitor\CheckDefinitions\CheckDefinition. Let’s take a look at the code of the built-in diskspace check.

namespace Spatie\ServerMonitor\CheckDefinitions;

use Spatie\Regex\Regex;
use Symfony\Component\Process\Process;

final class Diskspace extends CheckDefinition
{
    public $command = 'df -P .';

    public function resolve(Process $process)
    {
        $percentage = $this->getDiskUsagePercentage($process->getOutput());

        $message = "usage at {$percentage}%";

        if ($percentage >= 90) {
            $this->check->fail($message);

            return;
        }

        if ($percentage >= 80) {
            $this->check->warn($message);

            return;
        }

        $this->check->succeed($message);
    }

    protected function getDiskUsagePercentage(string $commandOutput): int
    {
        return (int) Regex::match('/(\d?\d)%/', $commandOutput)->group(1);
    }
}

This check will perform df -P . on the server. That will generate output much like this:

Filesystem                1024-blocks     Used Available Capacity Mounted on
/dev/disk/by-label/DOROOT    20511356 12378568   7067832      64% /

With a little bit of regex we extract the percentage listed in the Capacity column. If it’s higher than 90% we’ll call fail. This will mark the check as failed and will send out a notification. If it’s higher than 80% it’ll issue a warning. If it’s below 80% the check succeeds. It’s a simple as that.

Let’s take a look at another example: the memcached check.

namespace Spatie\ServerMonitor\CheckDefinitions;

use Symfony\Component\Process\Process;

final class Memcached extends CheckDefinition
{
    public $command = 'service memcached status';

    public function resolve(Process $process)
    {
        if (str_contains($process->getOutput(), 'memcached is running')) {
            $this->check->succeed('is running');

            return;
        }

        $this->check->fail('is not running');
    }
}

This check will run the command service memcached status on the server. If that commands outputs a string that contains memcached is running the check succeeds, otherwise it fails. Very simple.

Adding your own checks

Writing your own checks is very easy. Let’s create a check that’ll verify if nginx is running.

Let’s take a look at how to manually verify if Nginx is running. The easiest way is to run systemctl is-active nginx. This command outputs active if Nginx is running.

Let’s create an automatic check using that command.

The first thing you must to do is create a class that extends from Spatie\ServerMonitor\CheckDefinitions\CheckDefinition. Here’s an example implementation.

namespace App\MyChecks;

use Spatie\ServerMonitor\CheckDefinitions\CheckDefinition;
use Symfony\Component\Process\Process;

class Nginx extends CheckDefinition
{
    public $command = 'systemctl is-active nginx';

    public function resolve(Process $process)
    {
        if (str_contains($process->getOutput(), 'active')) {
            $this->check->succeed('is running');

            return;
        }

        $this->check->fail('is not running');
    }
}

Let’s go over this code in detail. The command to be executed on the server is specified in the $command property of the class.

The resolve function that accepts an instance of Symfony\Component\Process\Process. The output of that process can be inspected using $process->getOutput(). If the output contains active we’ll call $this->check->succeed which will mark the check successful. If it does not contain that string $this->check->fail will be called and the check marked as failed. By default the package sends you a notification whenever a check fails. The string that is passed to $this->check->failed will be displayed in the notification.

After creating this class you must register your class in the config file.

// config/server-monitor.php
'checks' => [
    ...
    'nginx' => App\MyChecks\Nginx::class,
],

And with that, you’re done. A check definition can actually do a few more things like when it’s supposed to be run the next time, setting timeouts and it has support for using custom properties. Take a look at the docs if you want to know more about this.

Using the stand alone version

If you’re not familiar with Laravel, installing a package can be a bit daunting. That’s why we also created a stand alone version called server-monitor-app. Under the hood it’s simply a vanilla Laravel 5.4 application with the laravel-server-monitor package pre-installed into it.

Using this app you can set up server monitoring in literally one minute. Here’s a video that demonstrates the installation and using a check.

Under the hood

Let’s take a look at a few cool pieces of source code.

If you have a buch server than you can end up with al lot of checks that need to be run. Running all those checks one after the other can take a bit of time. That’s why the package has support for running checks concurrently. In the config file you can configure how many ssh connections the package may use.

This code is taken from CheckCollection which is responsable for running all the checks.

 public function runAll()
 {
     while ($this->pendingChecks->isNotEmpty() || $this->runningChecks->isNotEmpty()) {
         if ($this->runningChecks->count() < config('server-monitor.concurrent_ssh_connections')) {
             $this->startNextCheck();
         }

         $this->handleFinishedChecks();
     }
 }

This loop will run as long as there are pending checks or running checks. Whenever there are less checks as the amount configured in the config file, another new check is started.

Let’s take a look at what’s happening inside the handleFinishedChecks function.

protected function handleFinishedChecks()
{
    [$this->runningChecks, $finishedChecks] = $this->runningChecks->partition(function (Check $check) {
        return $check->getProcess()->isRunning();
    });

    $finishedChecks->each->handleFinishedProcess();
}

This code leverages al lot of niceties offered by the latest versions of PHP and Laravel. It will filter out all the processes that are not running anymore (and are thus finished) and put them in the $finishedChecks collection. After that handleFinishedProcess will be called on each finished process.

handleFinishedProcess will eventually call the resolve function seen in the CheckDefinition examples listed above.

Testing the code

Like all other packages we previously made, laravel-server-monitor contains a good suite of tests. This allows us to improve the code and accept PRs without fear of breaking the code. Because SSH connections are used in this package, testing all functionality provided some challenges.

To easily code that relies on ssh connections the test suite contains a dummy SSH server written in JavaScript. When it runs it mimics all functionality of an SSH server. The SSH server itself is provided by the mscdex/ssh2. My colleague Seb wrote an easy to use abstraction around it. Using that abstraction we can let it respond whatever we want to when a command is sent to the server.

This makes testing the package end to end a breeze. Here’s how we test a succesful check.

 /** @test */
 public function it_can_run_a_successful_check()
 {
     $this->letSshServerRespondWithDiskspaceUsagePercentage(40);

     Artisan::call('server-monitor:run-checks');

     $check = Check::where('host_id', $this->host->id)->where('type', 'diskspace')->first();

     $this->assertEquals('usage at 40%', $check->last_run_message);
     $this->assertEquals(CheckStatus::SUCCESS, $check->status);
 }

Let’s take a look at another example. The package will fire a CheckRestored event if a check succeeds again after it has failed previously.

 public function the_recovered_event_will_be_fired_when_an_check_succeeds_after_it_has_failed()
 {
     $this->letSshServerRespondWithDiskspaceUsagePercentage(99);

     Artisan::call('server-monitor:run-checks');

     $this->letSshServerRespondWithDiskspaceUsagePercentage(20);

     Event::assertNotDispatched(CheckRestored::class);

     Artisan::call('server-monitor:run-checks');

     Event::assertDispatched(CheckRestored::class, function (CheckRestored $event) {
         return $event->check->id === $this->check->id;
     })
 }

In closing

Laravel Server Monitor was a fun project to work on. We’re quite happy with the results and now use it to monitor all our servers. If you’re interested in learning more about the package head over to our documentation site or the package itself on GitHub. Keep in mind that this package was built specifically for teams without a dedicated ops member or department. So before using it, research the alternatives a bit yourself and make up your own mind what is a good solution for you.

Our server monitor determines health by checking stuff inside your server. We also built another package called laravel-uptime-monitor that monitors your server from the outside. It’ll regularly send http requests to verify if your server is up. It can even verify if the used ssl certificate is still valid for a certain amount of days. Take a look at the uptime monitor docs to know more.

Also take a look at the other framework agnostic and laravel specific packages before. Maybe we’ve made something that can be of use in your next project.

Shipping Docker, a video tutorial series on Docker

Chris Fidao, the hero behind serversforhackers.com, and Deploy PHP!, created a new video course called Shipping Docker. It’s in early access right now and it covers everything from the basics to advanced topics like building a multi-server production environment.

I’m only a couple of videos deep in the course, and like expected, I’m liking a lot of what I’m seeing.

This is a comprehensive course in the many uses of Docker.
From playing to developing, testing to deploying, we’ll cover it all in a way that is easy to understand.

https://shippingdocker.com/

(In case you were wondering, I’m not being paid for posting this)

Our postcard collection

All our packages are MIT-licensed. But if you use our stuff and want to make us happy, we highly appreciate a postcard from your hometown. This suggestion is mentioned in all readme’s of our packages

We’ve been asking for postcards for quite some time now and have built up a nice collection. Today, we published a new page on our company website that lists all our received cards.

If you like what we are doing in regards to opensource, please send us a postcard too. Our address:

Spatie
Samberstraat 69D
2060 Antwerp
Belgium

Thanks!

https://spatie.be/en/opensource/postcards

Non-breaking, SEO Friendly Url’s in Laravel

Sebastian De Deyne, author of many Spatie packages, posted a new blog article on how to generate SEO Friendly Urls in Laravel.

When admins create or update an news item—or any other entity—in our homegrown CMS, a url slug is generated based on it’s title. The downside here is that when the title changes, the old url would break. On the other hand, if we wouldn’t regenerate the url on updates, titles that were edited later on would still have an old slug in the url, which isn’t an ideal situation either.

https://sebastiandedeyne.com/posts/2017/non-breaking-seo-friendly-urls-in-laravel

How to organize a meetup

On his blog Jef Claes, organiser of DDDBE, shares some good tips on how to organize a meetup.

I’ve organized a few DDDBE meetups in the past, and always succeed in forgetting something. Either someone points it out well in advance, or I end up stressing last minute. This post partly serves as a checklist for myself, but it would be a welcome side effect to also see it encourage others to help out organizing future meetups. Organizing a meetup is not rocket science, having a list of what to take care of is a good start.

http://www.jefclaes.be/2017/02/how-to-organize-meetup.html

Glossary of Modern JavaScript Concepts

Don’t know what the difference between stateful and stateless is, or what higher order functions are? On the auth0.com site Sebastián Peyrott explains these terms and other modern JavaScript concepts.

Modern JavaScript has experienced massive proliferation over recent years and shows no signs of slowing. Numerous concepts appearing in JS blogs and documentation are still unfamiliar to many front-end developers. In this post series, we’ll learn intermediate and advanced concepts in the current front-end programming landscape and explore how they apply to modern JavaScript.

https://auth0.com/blog/glossary-of-modern-javascript-concepts/