crabjang.com
robots.txt

Robots Exclusion Standard data for crabjang.com

Resource Scan

Scan Details

Site Domain crabjang.com
Base Domain crabjang.com
Scan Status Ok
Last Scan2025-04-26T12:24:14+00:00
Next Scan 2025-05-26T12:24:14+00:00

Last Scan

Scanned2025-04-26T12:24:14+00:00
URL https://crabjang.com/robots.txt
Domain IPs 183.111.232.72, 183.111.232.76
Response IP 183.111.232.72
Found Yes
Hash 1ded75448b00f6155b8c366c5b117dc147877b18534a16ad651a9ed733603052
SimHash 4000cc344371

Groups

*

Rule Path
Disallow /admin
Disallow /api
Allow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10