janitorai.com
robots.txt

Robots Exclusion Standard data for janitorai.com

Resource Scan

Scan Details

Site Domain janitorai.com
Base Domain janitorai.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-09-16T23:56:48+00:00
Next Scan 2025-11-15T23:56:48+00:00

Last Successful Scan

Scanned2024-10-23T01:12:17+00:00
URL https://janitorai.com/robots.txt
Domain IPs 104.18.36.27, 172.64.151.229, 2606:4700:4400::6812:241b, 2606:4700:4400::ac40:97e5
Response IP 104.18.36.27
Found Yes
Hash 56e7ab32fa1ceb3f832a62eac3f194b71a005ca5edf3a08bb2a19d27b10a51d5
SimHash 4144c450c511

Groups

*

Rule Path
Allow /

Other Records

Field Value
sitemap https://janitorai.com/sitemap.xml