Log In Sign Up

bestvideodl.com
robots.txt

Robots Exclusion Standard data for bestvideodl.com

Resource Scan

Scan Details

Site Domain bestvideodl.com
Base Domain bestvideodl.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-07-29T17:29:38+00:00
Next Scan 2025-10-27T17:29:38+00:00

Last Successful Scan

Scanned2025-01-01T17:27:52+00:00
URL https://bestvideodl.com/robots.txt
Domain IPs 104.21.89.82, 172.67.139.17, 2606:4700:3035::6815:5952, 2606:4700:3036::ac43:8b11
Response IP 104.21.89.82
Found Yes
Hash 99a361c3c20f0acd28687e1d4b700bc0c557995e7625ee4341c67c5b65122a86
SimHash 4204586be133

Groups

ia_archiver-web.archive.org
ia_archiver

Rule Path
Disallow /

yandex

Rule Path
Disallow

ccbot
google-extended
gptbot
chatgpt-user
omgilibot
omgili
facebookbot
anthropic-ai
cohere-ai

Rule Path
Disallow /

*

Rule Path
Disallow /404.html
Disallow /assets/*.js
Disallow /legal/
Disallow /app/
Disallow /api/
Disallow */?*
Allow *.js
Allow *.css
Allow *.png
Allow *.gif
Allow *.jpg
Allow *.jpeg
Back to top

Other Records

Field Value
sitemap https://bestvideodl.com/sitemap.xml
Back to top

Warnings

  • `clean-param` is not a known field.
  • `host` is not a known field.
Back to top