finning.com
robots.txt

Robots Exclusion Standard data for finning.com

Resource Scan

Scan Details

Site Domain finning.com
Base Domain finning.com
Scan Status Ok
Last Scan2024-09-24T20:40:04+00:00
Next Scan 2024-10-08T20:40:04+00:00

Last Scan

Scanned2024-09-24T20:40:04+00:00
URL https://finning.com/robots.txt
Redirect https://www.finning.com/robots.txt
Redirect Domain www.finning.com
Redirect Base finning.com
Domain IPs 51.11.3.236
Redirect IPs 13.107.246.59, 2620:1ec:bdf::59
Response IP 13.107.246.59
Found Yes
Hash b807644629f956bbe7832292ddd12d39ee5982511512b09171a08fcd9a03d273
SimHash 641afa60e312

Groups

yandex

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

*

Rule Path
Disallow /*/search
Disallow /*product-comparison.html
Disallow /*request-a-rental-quote.html?
Disallow /*request-a-used-quote.html?
Disallow /*request-a-quote.html?
Disallow /internal*
Disallow */forms*
Disallow */formularios*
Disallow */tpl/*
Disallow *typeFilter%3D*
Disallow *?qt=

Other Records

Field Value
sitemap https://www.finning.com/sitemap.xml