asnt.org
robots.txt

Robots Exclusion Standard data for asnt.org

Resource Scan

Scan Details

Site Domain asnt.org
Base Domain asnt.org
Scan Status Ok
Last Scan2024-06-02T11:11:39+00:00
Next Scan 2024-07-02T11:11:39+00:00

Last Scan

Scanned2024-06-02T11:11:39+00:00
URL https://asnt.org/robots.txt
Domain IPs 131.153.157.75
Response IP 131.153.157.75
Found Yes
Hash 257345b37394f77e5338c49a5f504c58b2962b620293fedb5dc6fe8dd269d48c
SimHash 0b4ad08648d5

Groups

seekport crawler

Rule Path
Disallow /

mozilla/5.0+(compatible;+seekport+crawler;+http://seekport.com/)

Rule Path
Disallow /

screaming frog

Rule Path
Disallow /

screaming+frog+seo+spider/13.2

Rule Path
Disallow /

*

Rule Path
Disallow /Buyers_Guide/
Disallow /en/Buyers_Guide/
Disallow /en/sitecore/media_library/Files/Events-Meetings/ASNT_Presentation_Tips.aspx
Disallow /en/sitecore/media_library/Files/Events-Meetings/commercialism.aspx

Other Records

Field Value
crawl-delay 20