dongtaiwang.com
robots.txt

Robots Exclusion Standard data for dongtaiwang.com

Resource Scan

Scan Details

Site Domain dongtaiwang.com
Base Domain dongtaiwang.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-03-01T18:34:08+00:00
Next Scan 2025-05-30T18:34:08+00:00

Last Successful Scan

Scanned2024-01-14T18:31:49+00:00
URL https://dongtaiwang.com/robots.txt
Domain IPs 104.26.4.19, 104.26.5.19, 172.67.69.76, 2606:4700:20::681a:413, 2606:4700:20::681a:513, 2606:4700:20::ac43:454c
Response IP 104.26.4.19
Found Yes
Hash b21738ca4162dca7ec90fa3601e7aa79bce99522f7716cff900f864569b9ef9b
SimHash 6950c26247f1

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /do/
Allow /

Warnings

  • `host` is not a known field.