localjobster.com
robots.txt

Robots Exclusion Standard data for localjobster.com

Resource Scan

Scan Details

Site Domain localjobster.com
Base Domain localjobster.com
Scan Status Ok
Last Scan2025-11-26T19:34:55+00:00
Next Scan 2025-12-03T19:34:55+00:00

Last Scan

Scanned2025-11-26T19:34:55+00:00
URL https://localjobster.com/robots.txt
Domain IPs 207.207.26.195
Response IP 207.207.26.195
Found Yes
Hash ca393d716f43f91f82164c14e3d825f191000e0070956dbf6f689128f895cfd9
SimHash b6856d052570

Groups

*

Rule Path
Disallow /create
Disallow /step
Disallow /finish
Disallow /jump_to
Disallow /zip_s
Disallow /lead_s2
Disallow /index3
Disallow /assets

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-Agent: *
  • Disallow: /