jackyan.com
robots.txt

Robots Exclusion Standard data for jackyan.com

Resource Scan

Scan Details

Site Domain jackyan.com
Base Domain jackyan.com
Scan Status Ok
Last Scan2024-11-10T21:50:12+00:00
Next Scan 2024-11-17T21:50:12+00:00

Last Scan

Scanned2024-11-10T21:50:12+00:00
URL https://jackyan.com/robots.txt
Domain IPs 104.21.21.136, 172.67.199.22, 2606:4700:3031::6815:1588, 2606:4700:3032::ac43:c716
Response IP 172.67.199.22
Found Yes
Hash 8571d6a977ffde2891e84fce327dbb1ed9fb69de669597eba99bd97c43652d3a
SimHash b8189902fa28

Groups

*

Rule Path
Allow /

semrushbot

Rule Path
Disallow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3600

siteauditbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

semrushbot-ocob

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /