calcmaps.com
robots.txt

Robots Exclusion Standard data for calcmaps.com

Resource Scan

Scan Details

Site Domain calcmaps.com
Base Domain calcmaps.com
Scan Status Ok
Last Scan2024-10-04T19:37:11+00:00
Next Scan 2024-10-11T19:37:11+00:00

Last Scan

Scanned2024-10-04T19:37:11+00:00
URL https://calcmaps.com/robots.txt
Domain IPs 136.243.94.95
Response IP 136.243.94.95
Found Yes
Hash f3053204f15e67c08c49041a7e8ea9582cdaa574cc6420e4a3182f8674ac3227
SimHash d07a5f3846db

Groups

*

Rule Path
Disallow /ajax.php
Disallow /cgi-bin/

ia_archiver

Rule Path
Disallow /

yadirectbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

foobar

Rule Path
Disallow /

lcc

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

voilabot

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

facebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

facebookexternalhit

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

Warnings

  • 2 invalid lines.