the-village.me
robots.txt

Robots Exclusion Standard data for the-village.me

Resource Scan

Scan Details

Site Domain the-village.me
Base Domain the-village.me
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't establish SSL connection.
Last Scan2024-05-26T19:53:59+00:00
Next Scan 2024-08-24T19:53:59+00:00

Last Successful Scan

Scanned2023-10-30T19:42:34+00:00
URL https://the-village.me/robots.txt
Redirect https://www.the-village.me/robots.txt
Redirect Domain www.the-village.me
Redirect Base the-village.me
Domain IPs 136.243.198.241
Redirect IPs 136.243.198.241
Response IP 136.243.198.241
Found Yes
Hash 6c5cc95c387b3d817b39c3897e9ace5b2a85dcdbf2ed77a286652cff0f5b1884
SimHash 2234bca1efa9

Groups

*

Rule Path
Disallow /comments
Disallow /search
Disallow /users/*/settings
Disallow */new$
Disallow */edit$
Disallow *.json$
Disallow /*?utm_*

Other Records

Field Value
crawl-delay 0.3

Other Records

Field Value
sitemap https://www.the-village.me/sitemap.xml

Warnings

  • `clean-param` is not a known field.
  • `host` is not a known field.