djav.org
robots.txt

Robots Exclusion Standard data for djav.org

Resource Scan

Scan Details

Site Domain djav.org
Base Domain djav.org
Scan Status Ok
Last Scan2026-02-28T20:56:45+00:00
Next Scan 2026-03-30T20:56:45+00:00

Last Scan

Scanned2026-02-28T20:56:45+00:00
URL https://djav.org/robots.txt
Domain IPs 104.26.14.210, 104.26.15.210, 172.67.74.138, 2606:4700:20::681a:ed2, 2606:4700:20::681a:fd2, 2606:4700:20::ac43:4a8a
Response IP 104.26.15.210
Found Yes
Hash 97335cf6a41459a6ca49f7fb89b691f0de60d7c76a1b012b3ad9c8890a76b181
SimHash d11a8804e213

Groups

*

Rule Path
Disallow /admin/
Allow /includes/videojs/
Disallow /controllers/
Disallow /api/
Disallow /content/
Disallow /csv_photos/
Disallow /ftp_content/
Disallow /ftp_photos/
Disallow /temp_users_uploads/
Disallow /cache/
Disallow /action.php?action=*
Disallow /filter-content/
Disallow /filters/
Disallow /filter/

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

tineye-bot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

pimeyesbot

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

amazonadbot

Rule Path
Disallow /

amazonproductdiscoverybot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://djav.org/sitemap.xml

Comments

  • Disallow: /includes/
  • Disallow: /templates/default_tube2019/template.ajax_comments.php
  • Google’s AI training bot