steynonline.com
robots.txt

Robots Exclusion Standard data for steynonline.com

Resource Scan

Scan Details

Site Domain steynonline.com
Base Domain steynonline.com
Scan Status Ok
Last Scan2025-03-27T05:34:05+00:00
Next Scan 2025-04-26T05:34:05+00:00

Last Scan

Scanned2025-03-27T05:34:05+00:00
URL https://steynonline.com/robots.txt
Redirect https://www.steynonline.com/robots.txt
Redirect Domain www.steynonline.com
Redirect Base steynonline.com
Domain IPs 104.26.4.75, 104.26.5.75, 172.67.68.40, 2606:4700:20::681a:44b, 2606:4700:20::681a:54b, 2606:4700:20::ac43:4428
Redirect IPs 104.26.4.75, 104.26.5.75, 172.67.68.40, 2606:4700:20::681a:44b, 2606:4700:20::681a:54b, 2606:4700:20::ac43:4428
Response IP 172.67.68.40
Found Yes
Hash 519a34912f8b08049af8fdbfc065636c9a7713a27f72a1f1e452ec21391f1a54
SimHash 29105496a393

Groups

*

Rule Path
Disallow /admin/
Disallow /list/

googlebot

Rule Path
Disallow /*.gif$
Disallow /*.jpg$
Disallow /*.png$

ia_archiver

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Disallow /