Google has just announced they are supporting HTTP Strict Transport Security (HSTS), which forces browsers to redirect to HTTPS if anyone tries to access Google on an HTTP URL.
https://security.googleblog.com/2016/07/bringing-hsts-to-wwwgooglecom.html
HSTS is a useful tool to help your migration from HTTP to HTTPS, as crawlers will also treat this as a redirect from HTTP to HTTPS, at a domain level.
You can include an HSTS tag in your response headers, which indicates a max-age duration value, and an option ‘includeSubDomains’ value. e.g.
Strict-Transport-Security: max-age=16070400; includeSubDomains
If the HSTS tag is included, it tells any browser or crawler to request the same URL on HTTPS. If the tag is detected on any URL, then it applies to every URL on the entire domain. And if you include the optional includeSubdomain, then it will apply to every subdomain of your primary domain too.
It’s recommended to include the tag on every URL, to ensure it gets detected as quickly as possible.
The max-age value is a duration for which the tag should be honoured. After which, the browser or crawler may start to request URLs on HTTP.
If you have permanently migrated to HTTPS, then you should set this to a high value.
Using this tag before you have a site which is fully functional on HTTPS could cause problems.
DeepCrawl 2 is already set up to detect HSTS tags, and every URL with one will be included in the ‘Pages with HSTS’ report, and also on the page details view under All Metrics.
