goodcatholicbooks.org
robots.txt
Robots Exclusion Standard data for goodcatholicbooks.org
Resource Scan
Scan Details
| Site Domain | goodcatholicbooks.org |
| Base Domain | goodcatholicbooks.org |
| Scan Status | Ok |
| Last Scan | 2024-06-13T04:18:22+00:00 |
| Next Scan | 2024-06-20T04:18:22+00:00 |
Last Scan
| Scanned | 2024-06-13T04:18:22+00:00 |
| URL | https://goodcatholicbooks.org/robots.txt |
| Domain IPs | 70.39.151.243 |
| Response IP | 70.39.151.243 |
| Found | Yes |
| Hash | 92f8caa2fbbacd4df9a2905477ec0a20a1c4313c7ac24218ff2171bad87777dd |
| SimHash | 630b74604c81 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /images/ |
| Disallow | /InteriorCastle.htm |
| Disallow | /books/* |
| Disallow | /pdf/* |
| Disallow | /pdf/on-the-inerrancy-of-scripture.pdf |
| Disallow | /francis-love-of-god.pdf |
| Disallow | /dekoninck/wiki/ |
| Disallow | /dekoninck/w/ |
| Disallow | /blog/labels |
| Disallow | /blogW/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
Other Records
| Field | Value |
|---|---|
| sitemap | //www.goodcatholicbooks.org/sitemap.xml |