comcar.co.uk
robots.txt

Robots Exclusion Standard data for comcar.co.uk

Resource Scan

Scan Details

Site Domain comcar.co.uk
Base Domain comcar.co.uk
Scan Status Ok
Last Scan2024-11-11T06:36:10+00:00
Next Scan 2024-11-18T06:36:10+00:00

Last Scan

Scanned2024-11-11T06:36:10+00:00
URL https://comcar.co.uk/robots.txt
Domain IPs 46.101.66.178, 67.207.69.172
Response IP 67.207.69.172
Found Yes
Hash be68005b974c4df1a2bbdf54175832142ecc8f805856d0a4a44c663ae1f96838
SimHash 601cd4006295

Groups

dirbuster

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

slurp

Rule Path
Disallow

Other Records

Field Value
crawl-delay 30

yandex

Rule Path
Disallow

Other Records

Field Value
crawl-delay 30

bingbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 10

msnbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /*?*Filters=1
Disallow /newcar/companycar/editor/
Disallow /newcar/companycar/editor_old/
Disallow