icdurham.org
robots.txt

Robots Exclusion Standard data for icdurham.org

Resource Scan

Scan Details

Site Domain icdurham.org
Base Domain icdurham.org
Scan Status Ok
Last Scan2025-09-17T03:40:59+00:00
Next Scan 2025-10-17T03:40:59+00:00

Last Scan

Scanned2025-09-17T03:40:59+00:00
URL https://icdurham.org/robots.txt
Redirect https://www.icdurham.org/robots.txt
Redirect Domain www.icdurham.org
Redirect Base icdurham.org
Domain IPs 199.34.228.48
Redirect IPs 199.34.228.48
Response IP 199.34.228.48
Found Yes
Hash 6cb198b3bfe5503627511651343d37ffd9402a6c74ac63cc5dd7e5debaac98d0
SimHash 7100d86cc789

Groups

nerdybot

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /ajax/
Disallow /apps/
Disallow /joining-ic.html
Disallow /https%3A//www.osvhub.com/icc-durham/funds
Disallow /welcome.html
Disallow /pastoral-council2.html
Disallow /worship.html
Disallow /home-study.html
Disallow /http%3A//www.icdurham.org/adult-faith-formation.html
Disallow /terms-of-use.html
Disallow /councils.html
Disallow /news.html
Disallow /http%3A//www.immaculataschool.org/
Disallow /https%3A//icdurham-spanish.weebly.com/
Disallow /catechesis-of-the-good-shepherd.html
Disallow /rcia-elect--candidates.html
Disallow /christmas-mass-schedule-2024.html
Disallow /pastoral-council-us-only.html

Other Records

Field Value
sitemap https://www.icdurham.org/sitemap.xml