jobs.waldorftoday.com
robots.txt

Robots Exclusion Standard data for jobs.waldorftoday.com

Resource Scan

Scan Details

Site Domain jobs.waldorftoday.com
Base Domain waldorftoday.com
Scan Status Ok
Last Scan4/15/2025, 12:33:27 PM
Next Scan 5/15/2025, 12:33:27 PM

Last Scan

Scanned4/15/2025, 12:33:27 PM
URL https://jobs.waldorftoday.com/robots.txt
Domain IPs 104.21.40.228, 172.67.157.144, 2606:4700:3034::ac43:9d90, 2606:4700:3035::6815:28e4
Response IP 172.67.157.144
Found Yes
Hash 0ecb191ce836504c83b1ec59012327852e03cb5fdde846291c46c5c70159dd37
SimHash 6918da72c731

Groups

*

Rule Path
Allow /
Disallow /api/

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://jobs.waldorftoday.com/sitemap.xml