Version 7.1

How to find insecure pages in your site before Google start penalising them

by Simon Cox



How to find insecure pages in your site before Google start penalising them

Google will be highlighting all non-secure sites from July 2018 meaning that the Insecure Content Report of Screaming Frog will be in use a lot over the next few months - I will show you what to do.

Embrace the security push

With the announce­ment that Google will be high­light­ing all non-secure sites, https, from July 2018 web­site own­ers need to think about how their sites are being deliv­ered to the pub­lic. For those not yet aware there are two pro­to­cols for web pages — http and https — Hyper­Text Trans­fer Pro­to­col and Hyper­Text Trans­fer Pro­to­col Secure. The Secure ver­sion needs a cer­tifi­cate that pro­vides a lev­el of secu­ri­ty for the user to prove that the site they are vis­it­ing is encrypt­ing the data between the site and the user’s brows­er — which in the­o­ry is safer than not encrypt­ing it — I will not go into the deep dark depths of that now. What you do need to know is that secure sites are already in the major­i­ty after a big push by Google over the past few years to get the dig­i­tal com­mu­ni­ty to embrace secu­ri­ty more. https is slow­er than http but there are now SEO advan­tages that out­weigh the speed disadvantage.

The keen eyed of you will notice as at this point in time (Feb­ru­ary 2018) this simon​cox​.com is not actu­al­ly run­ning on a secure serv­er- oth­er sites have tak­en prece­dence but I will be mov­ing it to a new serv­er as part of whats been a very long set of migra­tions to the fab­u­lous WESH UK host­ing where I will be enjoy­ing auto­mat­ed SSL cer­ti­fi­ca­tion! Once a site has been moved over to https you need to check every inter­nal link to ensure that they are also https. Often image paths are not rel­a­tive (start with a / or ../ ) but con­tain the full URL, such as:

A full site audit needs to be under­tak­en to catch all the URLs that were not migrat­ed. It is pos­si­ble that those URLs are being redi­rect­ed to the https URL but that adds a redi­rect into the jour­ney and will slight­ly com­pro­mise the worth of the inter­nal link — bet­ter to link directly.

I rec­om­mend you down­load the Scream­ing Frog SEO spi­der tool to run the audit on your site. It is free for 500 URLs so any­thing big­ger you will need to pay or you could hire some­one to do that for you — I am avail­able to do this kind of SEO audit at coxand​.co​.uk of course!

With Scream­ing Frog run a crawl against your website.

Once fin­ished go to the Pro­to­col tab and then you can fil­ter by http to show the un-secure URLs on your site. 

Or you can use the Inter­nal tab and click on the Address col­umn head­er to sort it. You should then be able to review all URLs and eas­i­ly spot any­thing that is not https. If you have a shed load of URLs then you can use the fil­ter on the top right — just add http:// to that and it will fil­ter the list to show you all non secure inter­nal URLs found.

List them and fix them!


Other articles in this topic

Pludgebanging words into Google

I was testing out different made up words to see what Google returned. Read more about Pludgebanging words into Google

Using Google Data Studio to review your http to https migration

Migration from http to https can be fraught with issues you need to track and check. I looked for a way to help track migrations and give me additional insights using Google Data Studio. Read more about Using Google Data Studio to review your http to https migration

A first look at Safecont content quality analysis SEO tool

Review of a new tool from Data elasticity S.L in Spain that analyses the quality of your websites content Read more about A first look at Safecont content quality analysis SEO tool

How I add canonicals into Perch CMS sites

The Canonical link in a page's header lets the search engines know where the original page resides. Originally conceived for situations where articles were duplicated they would reference the original. Google tends to choose the oldest version of a page that it can find (but not the only method it uses) and any other pages with the same or very similar content are considered duplicates and will not do a well on the Search Engine Results Pages - SERPs and we want our pages to do well there for the traffic. Read more about How I add canonicals into Perch CMS sites

Using a CDN to help migrate your sites hosting

Moving a site to new hosting can be fraught with issues. Here is one technique that I find useful when moving to a website to a new hosting server. Read more about Using a CDN to help migrate your sites hosting

EU cookie consent law largely ignored

The EU has give the UK until May 28th 2012 to comply with the cookie consent law - how many of the organisations pinpointed by the ICO have met the deadline? Read more about EU cookie consent law largely ignored

HTML Test Page for CSS Style Guide

This content is used to test that css has been covered for all tags used in content. Read more about HTML Test Page for CSS Style Guide

EEUK11 report

The UK's first ExpressionEngine conference has taken place in Manchester in August 2011. I provide a report on the event itself and the surrounding activities. Read more about EEUK11 report

Assets for ExpressionEngine

Assets is a new file management add-on for ExpressionEngine and brings a new way of managing assets in your website. I installed it and built a simple image Gallery. Read more about Assets for ExpressionEngine

Responsive web design

Responsive web design has been emerging as a way forward over the past two years. Following inspiration from the way Simon Collison's crafted his personal website I have now implemented a Responsive Web Design on this, my personal website. Read more about Responsive web design


With HTML5 finally emerging from the primordial soup I thought it was time to have a look at where we have got to with fonts on the web, what today's techniques are and what is new in the world of the digital hot metal. Read more about WebFonts


Online testing suites are a very useful way of checking your website to see if it's up to scratch. I take you through Nibbler from Silktide. Read more about Nibbler

New look for 2010

I have finally taken the plunge and updated the site and have switched over to ExpressionEngine. Read more about New look for 2010