jobsspotter.com
robots.txt

Robots Exclusion Standard data for jobsspotter.com

Resource Scan

Scan Details

Site Domain jobsspotter.com
Base Domain jobsspotter.com
Scan Status Ok
Last Scan2026-03-09T14:11:29+00:00
Next Scan 2026-03-16T14:11:29+00:00

Last Scan

Scanned2026-03-09T14:11:29+00:00
URL https://jobsspotter.com/robots.txt
Domain IPs 104.248.117.11
Response IP 104.248.117.11
Found Yes
Hash bcd393d26f6c6dd6aa6dccb94c7e65fd1b185b31f6bba5674ece9f118e5bf7fe
SimHash 6100df067397

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /privacy-policy/
Disallow /terms-and-conditions/
Disallow /privacy-policy
Disallow /terms-and-conditions