Write tests. Not too many. Mostly integration.

In a fantastic post on his blog, Kent C. Dodds makes the case for focusing on writing integration tests, and stop going for 100% code coverage. Amen to that!
You may also find yourself testing implementation details just so you can make sure you get that one line of code that’s hard to reproduce in a test environment. You really want to avoid testing implementation details because it doesn’t give you very much confidence that your application is working and it slows you down when refactoring. You should very rarely have to change tests when you refactor code.
https://blog.kentcdodds.com/write-tests-not-too-many-mostly-integration-5e8c7fff591c

A tool to automatically rerun PHPUnit tests when source code changes

In the JavaScript world Jest, built by Facebook, is an excellent tool to run tests. Not only can it automatically rerun your tests when source code changes, but there's also an interactive mode where you can set a filter on which tests to run while the tool is running. Would it be great if we could have these awesome features while working with PHPUnit? Our newly released phpunit-watcher tool provides a Jest like experience. You can simply install it by running composer global require spatie/phpunit-watcher. When you start it with phpunit-watcher watch this is what it looks like. By default it will watch all files in the src, app and tests subdirectories in the directory where it is started. If any of the files in those directories changes (or when a file in there gets created or deleted) the tests will rerun automatically. Cool, right? If you need to watch another directory just add a file named .phpunit-watcher.yml in your project directory. Here's some example content:
watch:
directories:
- myApplication
- tests
fileMask: '*.php'
Extra options can be passed to PHPUnit just by tacking them on the phpunit-watcher command. Here's an example where we only run tests whose name contains the word "includes": Now, if you want to get rid of that filter, there's no need to stop the tool. Like the little manual at the bottom will tell you, just press "a" to run all tests again. If you only want to run tests in a specific file or path, just type "p" to get to this screen: On that screen, just type the name of the file or directory that you want to filter on. And that's all there's really to it. The tool is fairly new, but the past days I've really enjoyed using our own tool myself and I wonder how I could ever have worked without it. Now I know that some IDE's, such as PHPStorm, ship with a PHPUnit watcher built-in, but our tool feels nicer (that's very subjective of course) and will work in combination with any editor or IDE (that's very objective). Some cool packages are powering our tool. The watching part is covered by the awesome Resource-watcher package. The interactivity is enabled by ReactPHP. I also want to give a shoutout to Colin O'Dell who helped fix a nasty performance problem and Christoper Pitt who wrote a nice blogpost that kickstarted the work on this package. If you want to know more about phpunit-watcher, be sure to check it out on GitHub. This is not the first time we've created something inspired by Jest. Take a look at this package which brings Jest's snapshot testing to PHPUnit. Want to see some mooarrrr packages we created previously, then check out the opensource pages on our company website.

How to write JavaScript-style test watchers in PHP

Christoper Pitt published another excellent piece over at Sitepoint. This time he describes how he built a watcher to automatically recompile his preprocessed code and rerun the tests.
In order to reduce the burden of invoking the transformation scripts, boilerplate projects have started to include scripts to automatically watch for file changes; and thereafter invoke these scripts. These projects I’ve worked on have used a similar approach to re-run unit tests. When I change the JavaScript files, these files are transformed and the unit tests are re-run. This way, I can immediately see if I’ve broken anything.
https://www.sitepoint.com/write-javascript-style-test-watchers-php/

What is snapshot testing, and is it viable in PHP?

In a new blogpost at Sitepoint Christopher Pitt talks about snapshot testing. He first explains how snapshot testing can help test React components. Then he demonstrates how you can use our home grown package to use snapshot testing in PHP. Finally he shows pretty cool use cases for snapshot tests.
Ah-ha moments are beautiful and rare in programming. Every so often, we’re fortunate enough to discover some trick or facet of a system that forever changes how we think of it. For me, that’s what snapshot testing is. You probably write a lot of PHP code, but today I want to talk about something I learned in JavaScript. We’ll learn about what snapshot testing is and then see how it can help us write better PHP applications.
https://www.sitepoint.com/snapshot-testing-viable-php/ If you want know more about snapshot testing, check out this blogpost by my colleague Sebastian, or watch this video where I refactor some tests to use snapshots.

Testing a Vue component part 4: more testing and faking time

Welcome to part 4 of a series on how to test a Vue component. Here's the table of contents of the series:
  1. Starting out with tests
  2. The first test and snapshot testing
  3. Creating an instance of a component
  4. More tests and faking time (you're here)
  5. Jest awesomeness and a skeleton to get started More tests and faking time

More tests

If you navigate to http://vue-tabs-component.spatie.be/#second-tab you'll notice that the tab named Second tab is automatically opened. Let's take a look at the test that makes sure this functionality works.
it('uses the fragment of the url to determine which tab to open', async () => {
    window.location.hash = '#second-tab';

    await createVm();

    expect(document.body.innerHTML).toMatchSnapshot();
});
Let's go through it in detail. First we set the hash of the current window.location (that's JavaScript speak for the current url) to #second-tab. Remember, Jest comes with jsdom out of the box. It's jsdom that provides us with that window instance. Next we create a new Vue instance and await until Vue has done it's thing. Finally we assert that the rendered output matches the saved snapshot. Of course I've manually verified, when that snapshot was first created, that it's content was correct.

Asserting against a component property

Instead of using snapshots to make assertion you can also just assert against a property in the state of a Vue component. Let's see how that works. Our vue-tabs-component has a neat feature where you can make a tab use a custom fragment. Here's the relevant test.
it('uses a custom fragment', async () => {
    document.body.innerHTML = `
        <div id="app">
            <tabs cache-lifetime="10">
                <tab id="my-fragment" name="First tab" >
                    First tab content
                </tab>
            </tabs>
        </div>
    `;

    const tabs = await createVm();

    expect(tabs.activeTabHash).toEqual('#my-fragment');
});
First we set up some html and we await until Vue has done it's thing. Like mentioned before the createVm method return an instance of the Tabs component. We implemented our component in such a way that the activeTabHash-property in it' state always contains the hash of the tab that's being displayed. In that last line of the test we simply assert that the property contains the expected value.

Faking time

By default our tabs component will remember which was the last open tab. If you for instance navigate to http://vue-tabs-component.spatie.be/#second-tab and then to the same url without the fragment you'll notice that the Second tab is being displayed. The component remembers, for a limited amount of time, the last opened tab by writing the hash of the selected tab to local storage. Here's the code that does that:
expiringStorage.set(this.storageKey, selectedTab.hash, this.cacheLifetime);
That expiringStorage class is something custom that is included in the project. It's a wrapper around local storage. It makes items in local storage expiring. expiringStorage has it's own tests. Let's take a look one of those tests.
it('remembers values by key', async () => {
    expiringStorage.set('my-key', 'my-value', 5);

    expect(expiringStorage.get('my-key')).toEqual('my-value');
});
Pretty simple right? We store my-value in the storage using the key my-key and then we assert that we get the same value when getting my-key. That last parameter is the lifetime in minutes. Of course we should not only test that the value is being saved, but that is expires after five minutes. Here's the test for that:
it('returns null if the value has expired ', () => {
    expiringStorage.set('my-key', 'my-value', 5);

    progressTime(5);

    expect(expiringStorage.get('my-key')).toEqual('my-value');

    progressTime(1);

    expect(expiringStorage.get('my-key')).toBeNull();
});
We assert that after exactly 5 minutes we still get the right value. If time progresses any further the value is forgotten and null is returned. That progressTime() function is not part of Jest. It's coded up in the tests themselves. Let's go take a look at it's implementation.
function progressTime(minutes) {

    const currentTime = (new Date()).getTime();

    const newTime = new Date(currentTime + (minutes * 60000));

    const originalDateClass = Date;

    Date = function (dateString) {
        return new originalDateClass(dateString || newTime.toISOString());
    };
}
We'll go over this line by line. In JavaScript a way of getting the time is the built-in Date class.
const currentTime = (new Date()).getTime();
We calculate the new time by adding the right amount of milliseconds to it.
const newTime = new Date(currentTime + (minutes * 60000));
Here comes the tricky part. We here going to hold a reference to the original date class. And then we are going to replace the built-in Date with our own function.
const originalDateClass = Date;

Date = function (dateString) {
    return new originalDateClass(dateString || newTime.toISOString());
};
This is called monkey patching. The original Date is replace by a function that, given no parameters, it won't return the actual time, but the time we gave it (newTime). Cool! Btw you can't do it this in PHP, which is a shame because it would make testing stuff a lot easier. Now that you understand how progressTime works let take a look at these two tests from inside tabs.test.js.
it('opens up the tabname found in storage', async () => {
    expiringStorage.set('vue-tabs-component.cache.blank', '#third-tab', 5);

    const tabs = await createVm();

    expect(tabs.activeTabHash).toEqual('#third-tab');
});

it('will not use the tab in storage if it has expired', async () => {
    expiringStorage.set('vue-tabs-component.cache.blank', '#third-tab', 5);

    progressTime(6);

    const tabs = await createVm();

    expect(tabs.activeTabHash).toEqual('#first-tab');
});
In the first test we set a value in the expiring storage with a key we know our component will use. Then we let Vue do it's thing. When it's ready we assert that the right tab is active. In the second test we set the same value in the expiring storage, but then we are going to progress time. Finally we assert that the item in expiringStorage is not being used, but that the first tab is being used. Now you might think that the that last test is a bit unnecessary. To so degree you're right: with the test of the expiringStorage class itself we already covered the items do in fact expire. But that extra extra component test gives me some extra confidence that the component behaves correctly.
You've reached the end of part four. Only one more part to go: Jest awesomeness and skeleton to get started

Testing a Vue component part 3: Creating an instance

Welcome to part 3 of a series on how to test a Vue component. Here's the table of contents of the series:
  1. Starting out with tests
  2. The first test and snapshot testing
  3. Creating an instance of a component (you're here)
  4. More tests and faking time
  5. Jest awesomeness and a skeleton to get started

Creating an instance of the Vue component

Now you know what snapshot testing is, let's take a fresh look at that first test:
it('can mount tabs', async () => {
    await createVm();

    expect(document.body.innerHTML).toMatchSnapshot();
});
The component itself is created in the createVm() function. We'll explain that await thingie a bit later. Let's take a look at createVm first. You can find it's implementation at the end of the file.
async function createVm() {
    const vm = new Vue({
        el: '#app',
    });

    await Vue.nextTick();

    return vm.$children[0];
}
Here we actually create a Vue instance. It will operate on the #app node. Remember we set the dom to some html earlier in the test. Vue will go do it's business and replace tabs and tab tags by the templates inside the Tabs and Tab components. It will not do this immediately but in an asynchronous way. We need to wait until Vue is ready. That's what that Vue.nextTick() line is all about. This will resolve after Vue's done with it's rendering work and all it's magic. If you read the documentation regarding nextTick you'll learn it returns a promise when you don't pass it any parameters. So you can use then() on it. If you're not familiar with promises, read this article about them.
Vue.nextTick().then(function() {
  //do stuff when Vue is ready
});
The problem if we were to use it that way is that only inside the callback we would know that Vue has done it's job. Luckily there is a new JavaScript feature that we can make use of: async/await. In short it lets you write asynchronous code in a synchronous way. Instead of having to pass a function to then when can put the await keyword before Vue.nextTick(). The code underneath that line will only execute when Vue.nextTick() is done.
await Vue.nextTick();

//do stuff when Vue is ready
Cool, so now we lose one level of indentation. You might think: "Freek, this is madness, now the code is blocking". But rest assured that behind the curtains JavaScript will still use promises and run asynchronously. Async/await is just syntactic sugar. You should also know that any function that uses await should be marked as async. That's why you see async appear in createVm()s function declaration. If you want to know more about async/await read this article on PonyFoo or this one on Hackernoon. This is the last line of the createVm function:
return vm.$children[0];
This line returns the first child of the Vue instance, which is the initialized Tabs component.

Back to the first test

Let's take a a third and final look at the our first test.
it('can mount tabs', async () => {
    await createVm();

    expect(document.body.innerHTML).toMatchSnapshot();
});
Now you should understand more or less everything that is going on. First we create a new Vue instance that will operate on the dom we set in the beforeEach each method. With that await keyword we will wait (in an asynchronous way) until Vue has done it's business. And finally we assert that the html matches the content of the snapshot. Congratulations for reading until here. I hope you learned a lot by diving into that first test. I promise that with this new knowledge the rest of the test are now easier to understand.
Understanding that first test was hard, but now you've taken that hurdle, grasping the next tests will be easier. If you're eager to know more head over to the fourth part of this series: More tests and faking time.

Testing a Vue component part 2: the first test and snapshots

Welcome to part 2 of a series on how to test a Vue component. Here's the table of contents of the series:
  1. Starting out with tests
  2. The first test and snapshot testing (you're here)
  3. Creating an instance of a component
  4. More tests and faking time
  5. Jest awesomeness and a skeleton to get started

Setting up the world

Let's take a further look at tabs.test.js. Near the top of the file you can see this line
beforeEach(() => {
The function that is passed to beforeEach will be run before each test. It's used to set up the 'world' you're testing in. It's equivalent to setUp method of PHPUnit if you're familiar with that. Let's take a look what's inside beforeEach. That first thing that's happening there is this:
document.body.innerHTML = `
    <div id="app">
        <tabs>
            <tab name="First tab">
                First tab content
            </tab>
            <tab name="Second tab">
                Second tab content
            </tab>
            <tab name="Third tab">
                Third tab content
            </tab>
        </tabs>
    </div>
`;
Here we see one of Jest's cool features. It comes with jsdom built in. Jsdom simulates an entire DOM as if you were in a browser. In that code snippet before we're making sure that each test starts out with that piece of html in the dom.

The first test

Before reviewing the rest of beforeEach, let's take a look at the very first test. The purpose of this test is to assert that when the component is mounted the right html is being used. "Mounted" is Vue speak for "when the component has fully started up and has been rendered".
it('can mount tabs', async () => {
    await createVm();

    expect(document.body.innerHTML).toMatchSnapshot();
});
Even though it's just a few lines there are a lot of things going on, and I can imagine it's a bit confusing if you're not familiar with modern Javascript. Let's go through it in detail. What happens here is that we first are creating an instance of our component (more on that later) and then assert that the html being displayed, so the value of document.body.innerHTML, is correct. That expect method is part of Jest itself. Jest contains various methods to assert things such as toEqual, toBeNull and much more. On this page in the Jest documentation you can see all available assertion methods. Instead of asserting the contents of document.body.innerHTML against a big piece of html, we use toMatchSnapshot here. Let's find out what that method does.

Snapshot testing

The basic idea of snapshot testing is that you compare the output of your code against a file that's written on disk. That file is called a snapshot. If the output of your test matches up with the contents of that snapshot, it succeeds, otherwise, it fails. It's as simple as that. In our project the test are stored in the __tests__/__snapshots__ directory. If you open up tabs.test.js.snap you'll see the html that is used in the can mount tabs tests. The cool thing is that you don't need to write those snapshots by hand. Jest can generate them for us. If you delete tabs.test.js.snap file and run the test again you might think that the can mount tabs will fail, but you're wrong. Let's see what happens. You can run the test separetly with this command
jest -t="can mount tabs"
You can see in the output of the command that there was one snapshot generated. The tabs.test.js.snap has been created. You could now verify in that snapshot if the content is equal to the output you were expecting. Run the test again and you'll see that it'll pass. Let's try and make the test fail. We're going to cheat a little here and mimick changed output by just editing the current snapshot (in real life you would never edit the snapshot manually). If I changed the html in the snapshot a bit and run the test again, it fails. If you want to know more about snapshot testing you can read this article my colleague Seb wrote about it, watch this video on how I used snapshot testing in PHP or just go over the relevant page in the Jest documentation.
We've learned some good things about that first test. But there's more. The story continues in part 3 of the series: Creating an instance of a component

Testing a Vue component part 1: getting started

Recently we released vue-tabs-component, a simple Vue component to render a tabular interface. If you want to know more about to the component itself I suggest you head to the introductory post on it. Like with the vast majority of our open source work this package comes with tests. In this series of blogposts I'd like to delve a little bit deeper in those tests. We'll touch upon Jest, jsdom, snapshot testing, async/await, monkey patching and a whole lot more. I hope you're ready for the ride. Here's the table of contents of the series:
  1. Starting out with tests (you're here)
  2. The first test and snapshot testing
  3. Creating an instance of a component
  4. More tests and faking time
  5. Jest awesomeness and a skeleton to get started

Starting out with tests

I'm not ashamed to say that starting out with writing tests for a Vue component was very daunting at first. The only thing I knew was that I wanted to use Jest, a modern testing platform for JavaScript, because I had used it before and was quite happy with it. More on Jest later. Sure, the official Vue docs explain how to do simple assertions But for a real life component that documentations doesn't cut it. The tests of our component should assert that it is rendered correctly in the dom, that is caches that name of the last opened tab](https://github.com/spatie/vue-tabs-component), that it opens the right tab when navigating to a url with a specific fragment, that the fragments can customized and much more. Luckily my colleague Sebastian helped a lot with setting up the right environment to write the tests in. If you want to write and test your own Vue component check out our skeleton-vue repo that contains all the right things to get started. With this solid setup I could move past the initial frustration and write some tests.

A first look at the tests

If you want to follow along with you should clone the vue-tabs-component repo with
git clone [email protected]:spatie/vue-tabs-component
Inside the cloned repo, run yarn (or npm install) to pull in all dependencies. Because it is listed in our package.json file that command will also pull in jest, our test runner. To run all the tests for the package issue this command ./node_modules/.bin/jest. You could opt to alias that to jest . You'll see this output. All tests are green, great! You might wonder where those tests are stored an how Jest finds them. Well, the tests themselves are stored in the __tests__ directory. You're not required to name that directory as such, but it seems the convention that most Jest users are following. Jest will just scan your entire project for files that end on test.js and execute the tests inside them. Let's take a look at our first real test. Open up __tests__/expiringStorageTest.js. This file contains all test that make sure our Vue component is working correctly. Near the top of the file you see this line:
describe('vue-tabs-component', () => {
That describe function is part of Jest. The first parameter is just the name of your test suite (it is displayed when you run Jest). The second parameter is a function that contains the tests you want to run. Inside that function you'll see lines starting with the it function such as:
it('can mount tabs', async () => {
it('displays the first tab by default', async () => {
...
Those it functions contain the actual tests. The second argument of it is function that is the test itself, the first argument is the name of that test. Notice that together with the it before it, the name of the test clearly says what is going to be tested: it can mount tabs. You haven't seen those labels in the output when running Jest before because there are multiple test files in our project. You can run all the test from a single file by just passing that file as a second argument of Jest.
jest __tests__/tabs.test.js
That'll output something this:
Our journey into the wicked world of Vue component testing continues in the second part of this series: The first test and snapshot testing.

A practical introduction to snapshot testing

While working a little bit on laravel-sitemap I decided to refactor the tests to make use of our own snapshot assertions package. The snapshot package enables you to generate snapshots with the output of your tests. On subsequent runs of those tests it can assert if the output still matches the contents of the snapshot. In this video I demonstrate how the refactor of the tests of laravel-sitemap was done. (double-click on it to view it full-screen) If you want to know more about snapshot testing, then read this blog post written by my colleague Sebastian. The refactor of the tests can be viewed in this commit on GitHub. (Little note on the video itself: this is my first tutorial video I ever made. After recording it I noticed that I used a wrong ratio resulting in those black bars on the side. Ah well, you live you learn. Next time it will be better. For those interested, I used a Blue Yeti Pro mic to record my voice and ScreenFlow to record and edit the video.)

A package for snapshot testing in PHPUnit

My colleague Sebastian recently released a new package called phpunit-snapshot-assertions. In a new post on his blog he tells all about it.
The gist of snapshot testing is asserting that a set of data hasn’t changed compared to a previous version, which is a snapshot of the data, to prevent regressions. The difference between a classic assertEquals and an assertMatchesSnapshot is that you don't write the expectation yourself when snapshot testing. When a snapshot assertion happens for the first time, it creates a snapshot file with the actual output, and marks the test as incomplete. Every subsequent run will compare the output with the existing snapshot file to check for regressions.
https://medium.com/@sebdedeyne/a-package-for-snapshot-testing-in-phpunit-2e4558c07fe3

Visualizing PHPUnit Tests

In an article on the Hackster.io site explains in detail how he made a nice visualization of unit test results. Pretty cool what you can do with a raspberry pi, some LEDS and a couple of scripts.
I wanted to create a desktop gadget to visualize the progress of unit tests run via PHPUnit.I've named this project PHPUnicorn (by combining "PHPUnit" with "Unicorn pHAT").
https://www.hackster.io/colinodell/phpunicorn-visualizing-phpunit-tests-896208

Working With PHPUnit and PhpStorm

On the JetBrains blog Gary Hockin explains how to easily run a single PHPUnit test.
To run all the tests in a single file, right-click the test in the Project Pane (the left-hand navigation pane), and select Run . To run all the tests in a single class, right-click the class name in the editor, and select Run . To run the tests in a single method, right-click the method name, and select Run .
https://blog.jetbrains.com/phpstorm/2017/01/working-with-phpunit-and-phpstorm/

The Magic Tricks of Testing

In a mail sent to all subscribers on the testdrivenlaravel.com-mailinglist Adam Wathan mentioned a talk Sandi Metz gave a couple of years ago at Rails Conf. It's a really good talk that explains in a clear way when and what you should test.
Tests are supposed to save us money. How is it, then, that many times they become millstones around our necks, gradually morphing into fragile, breakable things that raise the cost of change? We write too many tests and we test the wrong kinds of things. This talk strips away the veil and offers simple, practical guidelines for choosing what to test and how to test it. Finding the right testing balance isn't magic, it's a magic trick; come and learn the secret of writing stable tests that protect your application at the lowest possible cost.

Testing interactive Artisan commands

For a new package I'm working on I had to test some Artisan commands. The commands I want to test contain calls to ask and confirm to interactively get some input by the user. I had a little trouble finding a way to tests such commands, but luckily a blogpost by Mohammed Said pointed me in the right direction, which was to leverage partial mocks.
Here's the most interesting part, Artisan Commands can ask the user to provided specific pieces of information using a predefined methods that cover all the use cases an application might need. ... So we mock the command, register the mocked version in Kernel, add our expectations for method calls, and pretend the user response in the form of return values. ...
http://themsaid.com/building-testing-interactive-console-20160409/

Preventing API drift with contract tests

In an older post on his blog Adam Wathan shared an interesting approach on how to prevent API drift between a test mock and an actual implementation.
One of the risks of writing your own test doubles is that the API of the double can fall out of sync with the API of the real implementation. In this screencast I walk through an example that explains: - How interfaces can provide a false sense of security - Why tests make better contracts than interfaces - How to run multiple implementations against a shared set of contract tests If you’ve ever had issues with API drift, try this approach and let me know how it works for you.
https://adamwathan.me/2016/02/01/preventing-api-drift-with-contract-tests/