comparecontracthire.com
robots.txt

Robots Exclusion Standard data for comparecontracthire.com

Resource Scan

Scan Details

Site Domain comparecontracthire.com
Base Domain comparecontracthire.com
Scan Status Ok
Last Scan2024-09-21T00:39:27+00:00
Next Scan 2024-09-28T00:39:27+00:00

Last Scan

Scanned2024-09-21T00:39:27+00:00
URL https://comparecontracthire.com/robots.txt
Redirect https://www.comparecontracthire.com/robots.txt
Redirect Domain www.comparecontracthire.com
Redirect Base comparecontracthire.com
Domain IPs 46.101.66.178, 67.207.69.172
Redirect IPs 46.101.66.178, 67.207.69.172
Response IP 46.101.66.178
Found Yes
Hash be68005b974c4df1a2bbdf54175832142ecc8f805856d0a4a44c663ae1f96838
SimHash 601cd4006295

Groups

dirbuster

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

slurp

Rule Path
Disallow

Other Records

Field Value
crawl-delay 30

yandex

Rule Path
Disallow

Other Records

Field Value
crawl-delay 30

bingbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 10

msnbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /*?*Filters=1
Disallow /newcar/companycar/editor/
Disallow /newcar/companycar/editor_old/
Disallow