Log In Sign Up

getcalfresh.org
robots.txt

Robots Exclusion Standard data for getcalfresh.org

Resource Scan

Scan Details

Site Domain getcalfresh.org
Base Domain getcalfresh.org
Scan Status Ok
Last Scan2025-09-08T22:46:25+00:00
Next Scan 2025-09-22T22:46:25+00:00

Last Scan

Scanned2025-09-08T22:46:25+00:00
URL https://getcalfresh.org/robots.txt
Redirect https://www.getcalfresh.org/robots.txt
Redirect Domain www.getcalfresh.org
Redirect Base getcalfresh.org
Domain IPs 18.155.68.126, 18.155.68.13, 18.155.68.18, 18.155.68.23
Redirect IPs 204.236.149.99, 54.183.96.97
Response IP 204.236.149.99
Found Yes
Hash 196f4297d906403c879a592f70f5ae4f43ce36f386dfcb2750cf543506a94997
SimHash 82006c8565c4

Groups

*

Rule Path
Disallow /admin/*
Disallow /application
Disallow /application/*
Disallow /admin
Disallow /delayed_job
Disallow /health_check
Disallow /docs_upload
Disallow /rescheduled-interviews
Disallow /styleguide
Disallow /styleguide/*
Disallow /document_reroute*
Disallow /sar7info*
Disallow /ssa*
Disallow /immigrants-faq*
Back to top

Comments

  • See https://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
Back to top