sextophd.net
robots.txt

Robots Exclusion Standard data for sextophd.net

Resource Scan

Scan Details

Site Domain sextophd.net
Base Domain sextophd.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2026-01-11T17:08:14+00:00
Next Scan 2026-04-11T17:08:14+00:00

Last Successful Scan

Scanned2025-09-13T15:02:13+00:00
URL https://sextophd.net/robots.txt
Domain IPs 104.21.40.71, 172.67.180.174, 2606:4700:3032::ac43:b4ae, 2606:4700:3035::6815:2847
Response IP 172.67.180.174
Found Yes
Hash d08355baa5e809bd9903030109883c375226a9bf410d6a0533d4dc9297612258
SimHash 4f0dc822c7b3

Groups

yandex

Rule Path
Disallow /video/*

Other Records

Field Value
crawl-delay 2.0

bingbot

Rule Path
Disallow /video/*

Other Records

Field Value
crawl-delay 5

*

Rule Path
Disallow /video/*

Other Records

Field Value
sitemap https://sextophd.net/sitemap.xml

Warnings

  • `host` is not a known field.