horsan.tech
robots.txt

Robots Exclusion Standard data for horsan.tech

Resource Scan

Scan Details

Site Domain horsan.tech
Base Domain horsan.tech
Scan Status Ok
Last Scan2025-11-23T07:37:23+00:00
Next Scan 2025-12-23T07:37:23+00:00

Last Scan

Scanned2025-11-23T07:37:23+00:00
URL https://horsan.tech/robots.txt
Domain IPs 192.110.165.157
Response IP 192.110.165.157
Found Yes
Hash af56ef054c3b7e24b343dd63be480b5d436d324a08f8f5c54aadfdf6bb9b71b5
SimHash 6818d913d711

Groups

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

googlebot-mobile

Rule Path
Disallow

msnbot

Rule Path
Disallow

slurp

Rule Path
Disallow

teoma

Rule Path
Disallow

yahoo-mmcrawler

Rule Path
Disallow

siteauditbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

semrishbot-si

Rule Path
Allow /

yahoo-blogs/v3.9

Rule Path
Disallow

*

Rule Path
Disallow
Disallow /cgi-bin/
Disallow /horsanadmin/

Other Records

Field Value
sitemap https://horsan.tech/horsan-sitemap.xml