forums.uesp.net
robots.txt

Robots Exclusion Standard data for forums.uesp.net

Resource Scan

Scan Details

Site Domain forums.uesp.net
Base Domain uesp.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-08-10T16:37:16+00:00
Next Scan 2025-11-08T16:37:16+00:00

Last Successful Scan

Scanned2025-03-21T16:36:10+00:00
URL https://forums.uesp.net/robots.txt
Domain IPs 104.26.12.250, 104.26.13.250, 172.67.70.44, 2606:4700:20::681a:cfa, 2606:4700:20::681a:dfa, 2606:4700:20::ac43:462c
Response IP 104.26.13.250
Found Yes
Hash 77293837dfdc12c7a1c7b3d5db70b96a9bd7fc916a063a153935c3cf2c505f41
SimHash 68385d40e155

Groups

*

Rule Path
Disallow /viewtopic.php?p=
Disallow /viewtopic.php?=&p=
Disallow /viewtopic.php?t=
Disallow /viewtopic.php?start=
Disallow /*%26view%3Dprevious
Disallow /*%26view%3Dnext
Disallow /*%26sid%3D
Disallow /*%26p%3D
Disallow /*%26sd%3Da
Disallow /*%26start%3D0

Other Records

Field Value
crawl-delay 10

Comments

  • should limit bots to viewing links of form /viewtopic.php?f=X&t=X

Warnings

  • 18 invalid lines.