Published: October 28, 2019
As of last week (22nd October 19), Screaming Frog updated to version 12, aka ‘Element 115’. With the update, SF brought Pagespeed Insights integration which for me, (currently working in the travel industry) is a god send 🙏.
The API has existed for a while which if you’re savvy, you can plug in to google sheets but it’s quite limiting. SF pulls from the API in a way that doesn’t exceed maximum requests in a given minute making it much more efficient and workable.
Within a few hours of release I was crawling my site and getting bulk speed data to wheedle out pages that have performance issues that go beyond bulky images so I can get things fixed for high season.
If you’re not yet familiar, the insights API in Screaming Frog gives you all the data you would ordinarily get from Pagespeed Insights test but on mass across your website:
Then, in true Screaming Frog style you can export the whole shebang into a spreadsheet and go to town on getting things fixed with your developers:
It’s awesome, but it took me far too long to figure out which key code I needed to put in the ‘secret key’ box and whether it was even free to get an API key (being completely unfamiliar with google developer hub and API’s 😵):
Visit the Google developer hub page for pagespeed insights:
If you run a big site, visiting every page and noticing long load times isn’t viable. Google Analytics gives you some help but it’s quite top level.
Testing page speed often involves shoving your homepage into a tool and using that as a benchmark. Homepages are seen the most, I get it, but convenience is king everywhere and consumers hate friction wherever it lives on your website.
This integration really helps quickly isolate weird stuff going on across your site that you haven’t spotted or even thought about.
So far I’ve found:
Buckets of isolated CSS that isn’t being used
Unminified code in files I didn’t even know we were using
Outlying script inefficiencies digging into time to first byte (e.g. legacy page templates that have poor php script calling to databases, this one was massive)
Pages with inaugural amounts of requests on them (this led me down a long road but extremely valuable)
Unused and render blocking JavaScript files and scripts