Every two weeks I send out a newsletter containing lots of interesting stuff for the modern PHP developer. You can expect quick tips, links to interesting tutorials, opinions and packages. Want to learn the cool stuff? Then sign up now!

Automatically generate a sitemap in Laravel

Today my company released a package called laravel-sitemap. There are already a lot of excellent sitemap packages out there. They all have in common that you have to manually add links that must appear in the sitemap. With our new package that isn’t required. It can automatically build up a sitemap by crawling a site. In this post I’d like to explain why we built it and how it works.

Is a sitemap really needed?

In theory sitemaps helps web crawlers from search engines discover all pages of your site. Google’s own documentation has this to say about them:

If the pages of your site are properly linked, web crawlers can usually discover all links. Even so, a sitemap can improve the crawling of your site, particularly if your site meets one of the following criteria:

– Your site is really large. As a result, it’s more likely Google web crawlers might overlook crawling some of your new or recently updated pages.
– Your site has a large archive of content pages that are isolated or well not linked to each other. If you site pages do not naturally reference each other, you can list them in a sitemap to ensure that Google does not overlook some of your pages.
– Your site is new and has few external links to it. Googlebot and other web crawlers crawl the web by following links from one page to another. As a result, Google might not discover your pages if no other sites link to them.
– Your site uses rich media content, is shown in Google News, or uses other sitemaps-compatible annotations. Google can take additional information from sitemaps into account for search, where appropriate.

For bigger sites, where not all pages are not linked (for example a webshop, where not all products are linked in the dom), a sitemap is definitely needed. But for small to medium-sized sites where all url’s are linked properly, I’d conclude, when reading Google’s documentation, that a sitemap is not needed per se. When asking to peers about this and Googling around it becomes clear that there is no consensus if a sitemap is really needed for such sites. If you have an opinion on this or a link to a good blogpost on the subject, let me know in the comments below.

What often gets mentioned however is that sites will be crawled a bit faster if it has a sitemap and you submit it to the various search engines. Also heard quite often as an advantage of having a sitemap is that, in Google’s Search Console you can compare the number of pages in your sitemap versus the number of pages Google has crawled. In this way you can detect if Google is somehow failing to crawl sections of sites that you expect to be crawled.

There seem to be no disadvantages of having a sitemap and you might get, it’s not guaranteed, to enjoy at least some of it’s advantages. Google has this to say about it in their docs: I

Using a sitemap doesn’t guarantee that all the items in your sitemap will be crawled and indexed, as Google processes rely on complex algorithms to schedule crawling. However, in most cases, your site will benefit from having a sitemap, and you’ll never be penalized for having one.

So having a sitemap doesn’t look that important to me for a smaller site, so I don’t want to spend a lot of time creating a sitemap.xml. That’s why we made a package that can, in most cases, generate a sitemap with just a few lines of code.

Creating a sitemap

Imagine you have a Laravel app, running at example.com, where every page is properly linked (aka all pages appear in the dom somewhere). The app has a homepage, a contact page, some project pages and some news items. Using our package this is how you could generate a sitemap:

use Spatie\Sitemap\Sitemap;
use Spatie\Tags\Url;

$sitemap = Sitemap::create()

NewsItem::all()->each(function (NewsItem $newsItem) use ($sitemap) {

Projects::all()->each(function (Project $project) use ($sitemap) {


That’ll work but it’s quite verbose. If you add another content type or another loose page like /contact, you mustn’t forget to add it the sitemap.

Generating a sitemap

To avoid having to manually add links to a sitemap, the package includes a SitemapGenerator. This class can automatically crawl your site and put all the links it discovers in a sitemap.

Using a SitemapGenerator all of the code from the previous example can be replaced by this:

use Spatie\Sitemap\SitemapGenerator;


You can easily create an artisan command to create a sitemap and schedule it to run frequently. This will ensure that new pages and content types will be automatically picked up. Here’s how such a command could look like:

namespace App\Console\Commands;

use Illuminate\Console\Command;
use Spatie\Sitemap\SitemapGenerator;

class GenerateSitemap extends Command
     * The console command name.
     * @var string
    protected $signature = 'sitemap:generate';

     * The console command description.
     * @var string
    protected $description = 'Generate the sitemap.';

     * Execute the console command.
     * @return mixed
    public function handle()
        // modify this to your own needs

It can be scheduled in the console kernel to be run daily.

// app/Console/Kernel.php
protected function schedule(Schedule $schedule)

The best of both worlds

You can also combine the two approaches. You can manually add links to a generated sitemap. Here’s an example on how to do that:



Our package is targeted at small to medium-sized apps. According to the specification a sitemap can hold up to 50 000 items (if have more links you’ll need a sitemap index). There are also specific link types for video’s, image’s, etc… The package currently does not have support for sitemap indexes and these other types of links because it’s not needed for any of our projects. I’d accept a PR that adds these things to our package.

Here are some alternatives that already support these features (but they don’t include the crawler from our package)

Further reading

If you want to know more about sitemaps in general, take a look at these posts (provided by my colleague Jef)

In conclusion

If you need a sitemap for your small to medium sized app, laravel-sitemap can probably help you. Take a look at the package on GitHub to learn all the features not mentioned in this blogpost:

Be sure to also take a look at the list of Laravel packages we’ve previously made.

Freek Van der Herten is a partner and developer at Spatie, an Antwerp based company that specializes in creating web apps with Laravel. After hours he writes about modern PHP and Laravel on this blog. When not coding he’s probably rehearsing with his kraut rock band.
  • Hey Freek,

    I’ve worked for the past 5 years in the SEO industry and using a crawler to generate the sitemap is usually a fundamentally wrong approach. Reason being: Google probably does a much better job a crawling than any other general purpose crawler out there, so there is no added value in generating a sitemap that way. As you’ve already mentioned, there is now ranking benefit and it also doesn’t help in indexing prioritization because you “simply crawl your site” (in contrast to “only selecting the important pages/urls”). In addition, most websites that are not completely static do a horrable job of internal linking (i.e. using unrestricted facetted navigation, wrong pagination, etc.) so that an “unsupervised” crawl usually yields a tremendous amount of irrelevant URLs.

    I feel very strongly about this matter, because sitemaps are a wonderful tool for a technical SEO to define the really important URLs, cluster them by topic, get a good overview of their indexation status, compare them to log files etc. But all of that only makes sense if you put in the effort to make sure only those URLs are in there, that are relevant to your business. That being said, sitemaps should always be generated by your application – never by using an external crawler.

    Although you are correct in “it doesn’t do any harm”, I feel that it leads people in the wrong direction.


  • Pingback: Building a Search Engine Friendly Sitemap XML with Laravel – Rifix Blog()