antarvasnavideos.club
robots.txt

Robots Exclusion Standard data for antarvasnavideos.club

Resource Scan

Scan Details

Site Domain antarvasnavideos.club
Base Domain antarvasnavideos.club
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan4/10/2025, 2:35:22 AM
Next Scan 7/9/2025, 2:35:22 AM

Last Successful Scan

Scanned5/23/2024, 2:15:58 AM
URL https://antarvasnavideos.club/robots.txt
Domain IPs 104.21.90.209, 172.67.161.103, 2606:4700:3035::6815:5ad1, 2606:4700:3036::ac43:a167
Response IP 104.21.90.209
Found Yes
Hash 02f79653aa9605d08f562bbf7bcc13e98d800766ad785faeddd1ea3e6d96a124
SimHash 6f09c064c7b0

Groups

yandex

Rule Path
Disallow /files/*

Other Records

Field Value
crawl-delay 2.0

bingbot

Rule Path
Disallow /files/*

Other Records

Field Value
crawl-delay 5

*

Rule Path
Disallow /files/*

Other Records

Field Value
sitemap https://antarvasnavideos.club/sitemap.xml

Warnings

  • `host` is not a known field.