Log In Sign Up

hiredtoday.com
robots.txt

Robots Exclusion Standard data for hiredtoday.com

Resource Scan

Scan Details

Site Domain hiredtoday.com
Base Domain hiredtoday.com
Scan Status Ok
Last Scan2024-11-11T12:46:05+00:00
Next Scan 2024-11-18T12:46:05+00:00

Last Scan

Scanned2024-11-11T12:46:05+00:00
URL https://hiredtoday.com/robots.txt
Redirect https://www.hiredtoday.com//robots.txt
Redirect Domain www.hiredtoday.com
Redirect Base hiredtoday.com
Domain IPs 158.69.66.115
Redirect IPs 158.69.66.115
Response IP 158.69.66.115
Found Yes
Hash 02803324f2e4d99cd1fbbdc95cb41c1b20ca27cba3fa5cd09649929f17c43fa9
SimHash ba842d8d7670

Groups

*

Rule Path
Disallow /en/admin/
Disallow /admin/
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /
  • Old site
Back to top