jamesburgess.com
robots.txt

Robots Exclusion Standard data for jamesburgess.com

Resource Scan

Scan Details

Site Domain jamesburgess.com
Base Domain jamesburgess.com
Scan Status Ok
Last Scan2025-10-30T15:38:36+00:00
Next Scan 2025-11-29T15:38:36+00:00

Last Scan

Scanned2025-10-30T15:38:36+00:00
URL https://jamesburgess.com/robots.txt
Redirect https://www.jamesburgess.com/robots.txt
Redirect Domain www.jamesburgess.com
Redirect Base jamesburgess.com
Domain IPs 199.34.228.46
Redirect IPs 199.34.228.46
Response IP 199.34.228.46
Found Yes
Hash 88876e7a5f979871425f375ebd0b5d8ad1511a1ff882d89a9b3f7ad206aa167c
SimHash e970dc2a6b93

Groups

nerdybot

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /ajax/
Disallow /apps/
Disallow /study-material.html
Disallow /the-7-mysteries.html
Disallow /astrology.html
Disallow /sabians-by-signs.html
Disallow /other-sabian-studies.html

Other Records

Field Value
sitemap https://www.jamesburgess.com/sitemap.xml