astrology.net
robots.txt

Robots Exclusion Standard data for astrology.net

Resource Scan

Scan Details

Site Domain astrology.net
Base Domain astrology.net
Scan Status Ok
Last Scan2024-05-18T07:22:11+00:00
Next Scan 2024-05-25T07:22:11+00:00

Last Scan

Scanned2024-05-18T07:22:11+00:00
URL http://astrology.net/robots.txt
Redirect https://www.astrology.com/robots.txt
Redirect Domain www.astrology.com
Redirect Base astrology.com
Domain IPs 96.45.82.253, 96.45.82.37, 96.45.83.225, 96.45.83.64
Redirect IPs 104.18.98.44, 104.18.99.44, 2606:4700::6812:622c, 2606:4700::6812:632c
Response IP 104.18.98.44
Found Yes
Hash 45477929acb63bc525918dc929c5112d45370a6f26f3d79a319049b6b1220c6c
SimHash 084451354f33

Groups

*

Rule Path
Allow /
Disallow /us/search.aspx

grapeshot

Rule Path
Disallow

Other Records

Field Value
sitemap https://www.astrology.com/sitemap.xml
sitemap https://www.astrology.com/sitemap-news.xml