How to remove a big file wrongly committed to a Git repo

Today I accidentally committed a multi GB file to the git repo of the project I was working on and pushed it. Damn! Because of that big file cloning the repo again would take a long long time. Removing the file locally and pushing again would not solve the problem as that big file is in Git’s history.

So I took a few moments to Google around and learned that that there actually is a git command that can rewrite history: filter-branch. This is how I removed that big file from history:

git filter-branch --tree-filter 'rm path/to/your/bigfile' HEAD

git push origin master --force

Both commands took a while to complete, but after that I had a light repo again.

If you need to do this, be sure to keep a copy of your repo around in case something goes wrong.

Here’s an old, but still seemingly still correct blogpost by Dalibor Nasevic with some more info on the subject.

Few weeks ago I froze gems on my blog and ended up with a very big repository. So, I wanted to clean up the mess and remove permanently gems folder from the repository. git rm wasn’t doing the job well, it only removes the folder from the working tree and the repository still contains the objects of this folder. After a quick search, I found that git-filter-branch was the command I was looking for.

https://dalibornasevic.com/posts/2-permanently-remove-files-and-folders-from-a-git-repository

Dockerize your Laravel app with Vessel

Chris Fidao has created an easy to handle, well documented, Docker dev environment for Laravel projects. It’s an excellent starting point if you want to have a taste of what Docker can do.

For the introductory newsletter:

I like Vessel the best for the following reasons:
1. It’s installed per-project instead of globally. This lets me customize it per project if need be.
2. Docker lets me change out versions of software such as Nginx, MySQL, Redis, and others very easily. (I’ve often needed to use an older MySQL version at work)
3. Docker lets me add extra software (perhaps Beanstalkd for queues, or PgSQL for database) really easily
4. Docker containers are more like processes than VMs. They generally only use what resources they need (with some caveats, but even with those, they’re lighter than Vagrant virtual machines)
5. I can fill up my workstation with one technology (Docker!) instead of many (PHP, Redis, MySQL, etc) with all their configuration files and data strewn about all over my file system
6. You can expand on your dev workflow to build up to a production workflow using all the same technology (Docker!) – You can check out Shipping Docker for my full course on bringing Docker from dev into production.

https://vessel.shippingdocker.com

Write tests. Not too many. Mostly integration.

In a fantastic post on his blog, Kent C. Dodds makes the case for focusing on writing integration tests, and stop going for 100% code coverage. Amen to that!

You may also find yourself testing implementation details just so you can make sure you get that one line of code that’s hard to reproduce in a test environment. You really want to avoid testing implementation details because it doesn’t give you very much confidence that your application is working and it slows you down when refactoring. You should very rarely have to change tests when you refactor code.

https://blog.kentcdodds.com/write-tests-not-too-many-mostly-integration-5e8c7fff591c

The official Vue.js style guide

The maintainers of Vue.js have recently published their official style guide.

This is the official style guide for Vue-specific code. If you use Vue in a project, it’s a great reference to avoid errors, bikeshedding, and anti-patterns. However, we don’t believe that any style guide is ideal for all teams or projects, so mindful deviations are encouraged based on past experience, the surrounding tech stack, and personal values.

For the most part, we also avoid suggestions about JavaScript or HTML in general. We don’t mind whether you use semicolons or trailing commas. We don’t mind whether your HTML uses single-quotes or double-quotes for attribute values. Some exceptions will exist however, where we’ve found that a particular pattern is helpful in the context of Vue.

https://vuejs.org/v2/style-guide/

Want to see some more style guides? At Spatie we have a guidelines site containing styleguides for Laravel and JavaScript.

This site contains a set of guidelines we use to bring our projects to a good end. We decided to document our workflow because consistency is one of the most valuable traits of maintainable software.

The contents of this site exist for ourselves—more importantly, our future selves—and for giving future collegues a reference to our way of doing things and their quirks. The guidelines cover workflow, code style, and other little things we consider worth documenting.

https://guidelines.spatie.be

Deep dive into Electron’s main and renderer processes

In a post on his Medium blog Cameron Nokes explains the two most important processes in any Electron app: the main process and the renderer process.

Central to Electron is the concept of two or more operating system level processes running concurrently — the “main” and “renderer” processes. Dealing with multiple processes is new territory if you’re coming from browser Javascript land. It was definitely a paradigm shift for me initially, and working with multiple processes may mean you make different design choices in your app that you wouldn’t otherwise.

https://medium.com/@ccnokes/deep-dive-into-electrons-main-and-renderer-processes-7a9599d5c9e2

Asynchronous stack traces: why await beats .then()

On his blog Mathias Bynens explains the differences under the hood between async/await and vanilla promises.

The fundamental difference between await and vanilla promises is that await X() suspends execution of the current function, while promise.then(X) continues execution of the current function after adding the X call to the callback chain. In the context of stack traces, this difference is pretty significant.

Enable JavaScript engines to handle stack traces in a more performant and memory-efficient manner by following these recommendations:

– Prefer async/await over desugared promises.
– Use babel-preset-env to avoid transpiling async/await unnecessarily.

https://mathiasbynens.be/notes/async-stack-traces

Handling Stripe webhooks in a Laravel application

In the project I’m currently working on I had to integrate Stripe webhooks. Stripe has great documentation on how to handle webhooks, but it still took a fair amount of time to get the integration just right. My solution for handling webhooks is pretty generic and reusable by others. I decided to extract it to a package called laravel-stripe-webhooks, so nobody has to code this stuff up again. In this blogpost I’d like to share how the package works.

Requests will be validated

Whenever an event occurs on Stripe, the service will notify your app by sending a http request to your app. Here’s a list of all events Stripe sends out. Stripe will sign all requests it sends to your app. This package will automatically verify if the signature is valid. If it is not, the request was probably not sent be Stripe.

Webhooks are restartable

Before doing any other work, all calls will immediately be logged to the database. This is incredibly valueable when something goes wrong handling a webhook call. You can easily retry the processing of the webhook call, after you investigated and fixed the cause of failure, like this:

use Spatie\StripeWebhooks\StripeWebhookCall;

StripeWebhookCall::find($id)->process();

Handling valid requests

There are two ways this package enables you to handle webhook requests: you can opt to queue a job or listen to the events the package will fire. In both cases we highly recommend queuing the job or queueing the event listener that does some work. By doing that, you can minimize the response time of the webhook requests. This allows you to handle more stripe webhook requests and avoiding timeouts.

Let’s take a look at both methods.

Using jobs

Laravel has a handy make command to scaffold a Job.

php artisan make:job ChargeSource

After that you must let the job accept a WebhookCall instance in it’s handle method.

namespace App\Listeners;

use Illuminate\Contracts\Queue\ShouldQueue;
use Spatie\StripeWebhooks\StripeWebhookCall;

class ChargeSource implements ShouldQueue
{
    public function handle(StripeWebhookCall $call)
    {
        // do your work here

        // you can access the payload of the webhook call with `$webhookCall->payload`
    }   
}

After having created your job you must register it at the jobs array in the stripe-webhooks.php config file. The key should be the name of the stripe event type where but with the . replaced by _. The value should be the fully qualified name of the class.

// config/stripe-webhooks.php

'jobs' => [
    'source_chargeable' => \App\Jobs\StripeWebhooks\HandleChargeableSource::class,
],

And with that you job will fire when strips sends out the source.chargeable webhook.

Using events

Instead of queueing jobs to perform some work when a webhook request comes in, you can opt to listen to the events this package will fire. Whenever a valid request hits your app the package will fire a stripe-webhooks::<name-of-the-event>. For if a source.chargeable event hits your app, the stripe-webhooks::source.chareable event will fire.

Let’s take a look at how you can listen for such an event. In the EventServiceProvider you can register a listener.

/**
 * The event listener mappings for the application.
 *
 * @var array
 */
protected $listen = [
    'stripe-webhooks::source.chargeable' => [
        App\Listeners\ChargeSource::class,
    ],
];

Here’s an example of such a listener:

namespace App\Listeners;

use Illuminate\Contracts\Queue\ShouldQueue;
use Spatie\StripeWebhooks\StripeWebhookCall;

class ChargeSource implements ShouldQueue
{
    public function handle(StripeWebhookCall $call)
    {
        // do your work here

        // you can access the payload of the webhook call with `$webhookCall->payload`
    }   
}

Like already mentioned above, we highly recommend that you make the event listener queueable.

The above example is only one way to handle events in a Laravel. To learn the other options, read the Laravel documentation on handling events.

In closing

Head over to the readme of laravel-stripe-webhooks on GitHub to learn a few more tidbits about the package. Rest assured that the package contains a suite of tests to make sure everything works correctly.

Because Stripe is quite popular, I can image that a lot of artisans already have written their own implementation for handling webhooks. In my mind it’s kind of silly that everybody has to code up the same solution over and over again. If you tackle a problem concerning common stuff consider open sourcing your solution. You will benefit yourself from the public scrutinization of your code (in mean this in a positive way). Your future self will be thankful for the documentation you write for your package. The other members of the community will be saved from writing the same boring code. And we all can get to building the fun and interesting stuff.

If you’re in a position in your company to advocate or to actively work on open source, I highly recommend you do so. There are only winners in this story!

That being said, be sure to take a look at the collection of Laravel, PHP and JavaScript packages our team has created:

Pretty sure there will be something there that can be helpful in one of your future projects.

Essential image optimization

Addy Osmani, a Google Chrome engineer, recently published a e-book on image optimization. You can read it in it’s entirety here: https://images.guide/

Images take up massive amounts of internet bandwidth because they often have large file sizes. According to the HTTP Archive, 60% of the data transferred to fetch a web page is images composed of JPEGs, PNGs and GIFs. As of July 2017, images accounted for 1.7MB of the content loaded for the 3.0MB average site.

Per Tammy Everts, adding images to a page or making existing images larger have been proven to increase conversion rates. It’s unlikely that images will go away and so investing in an efficient compression strategy to minimize bloat becomes important.

If you need to optimize images in your PHP application take a look at our image optimization package. If you’re a Laravel user you’ll be happy to know that there’s also a Laravel version of the optimizer. Our Laravel Medialibrary also comes with built-in support for optimizing images.

Nordic.js 2017 videos have been released

Like the name implies Nordic.js is a conference that focuses on JavaScript. The organisers recently released all videos of the 2017 edition. Even if you’re not into JavaScript I can recommend this watch this video of Harriet Lawrence on how you can use language to make your community more welcoming.

Here’s a YouTube playlist with all videos of Nordic.js 2017.

My colleague Sebastian attended the conference, be sure to check out his recap.

Handling Stripe payments in Laravel

Povilas Korop, creator of Laravel Daily and Quick Admin Panel, wrote an extensive guide on how to integrate Stripe into a Laravel application.

Stripe is one of the most popular payment merchants for web, but information about Laravel integration is pretty fragmented, so I decided to write a really long tutorial about this topic.

We will cover:

– General logic how Stripe works
– Simple one-time payment integration
– Testing and production environment setup
– Saving transaction data for future reference
– Recurring payments with Laravel Cashier
– Getting invoices data and PDF download

https://quickadminpanel.com/blog/stripe-payments-in-laravel-the-ultimate-guide/

Partitioning for concurrency in synchronous business processes

On his blog Frank De Jonge, member of the PHP League and creator of Flysystem, explains a cool pragmatic solution how to ensure queued processes for the same user complete in the right order.

The use of multiple workers allows for much higher throughput, but it also allows for race conditions during processing. When messages for the same user are sent to different workers, handling order can no longer be guaranteed. Therefore we have failed to fulfil our business rule.

If we were able to ensure every message from the same user were sent to the same worker, the worker could ensure those messages are handled in order while the system as a whole would still benefit from the degree of parallelism. But how do we make this happen?

https://blog.frankdejonge.nl/parallelise-synchronous-business-processes/

BetterReflection v2 has been released

The awesome Roave team has recently released v2 of their BetterReflection package. It can do anything PHP’s native reflection API can, but without actually autoloading the code.

The operational concept is quite simple, really:

1. We scan your codebase for files matching the one containing your class. This is fully configurable, but by default we use some ugly autoloader hacks to find the file without wasting disk I/O.
2. We feed your PHP file to PHP-Parser
3. We analyse the produced AST and wrap it in a matching Roave\BetterReflection\Reflection\* class instance, ready for you to consume it.

Read all about it on Marco Pivetta‘s blog: https://ocramius.github.io/blog/roave-better-reflection-v2.0/

New features in our packages

Every time our team releases a package I have the habit of writing an introductory blogpost. But after the initial release most pages gain more features through PRs by the community and ourselves. Mostly these new feature go unnoticed. That’s why I plan on regularly writings posts on noteworthy changes across our repos.

Browsershot

This package makes it easy to convert any webpage (or html string) to an image or a pdf. Under the hood headless Chrome is leveraged. A while ago I took the time to add a new bodyHtml method to it. Here’s how that can be used:

Browsershot::url('https://spatie.be')->bodyHtml();

The method returns the html of the dom after JavaScript has been executed. Pretty cool.

Among our packages there’s also a powerful crawler. bodyHtml paved the way for making that crawler execute JavaScript as well. So now it can also find links that were injected by JavaScript scripts on your site.

Our laravel-sitemap package uses that crawler to automatically create a sitemap of your site. Now that the crawler can execute JavaScript, the sitemap generator can now add pages which are linked to by JavaScript injected links.

laravel-medialibrary

Our medialibrary is a pretty powerful package that can associate all sort of files with eloquent models. Here’s a video that demonstrates what the package is capable of.

A PR by Alex Vanderbist makes it possible to generate temporary URLs to files that are stored on S3.

Here’s how you can do that

$yourModel
   ->getMedia()
   ->first()
   ->getTemporaryUrl(Carbon::now()->addMinutes(5), 'thumb'); // Temporary S3 url

laravel-translatable

This package makes it easy make eloquent models translatable. Translations are stored as json. There is no extra table needed to hold them. Previously a translation should be set like this:

$yourModel->setTranslation('name', 'en', 'Name in English')

A PR by flofloflo adds an alternative way to go about this:

// assuming your the locale of the app is set to `en`
$yourModel->name = 'Name in English

Feels much more natural.

laravel-demo-mode

Our demo mode package provides a middleware that can protect your work in progress from prying eyes. Visitors should first visit a specific (secret) url before they can see that content of the project you’re building.

A PR by Alex Vanderbist added these two options to the config file that make it possible to make a whitelist on an IP address basis.

/**
 * The following IP addresses will automatically have access to the app
 * without having to pass the `demoAccess` route.
 */
'authorized_ips' => [

],

/**
 * If strict mode is enabled then only the IP addresses listed in `authorized_ips` are allowed.
 * Vistors won't be able to gain access by visiting the  `demoAccess` route anymore.
 */
'strict_mode' => false,