jav.directory
robots.txt

Robots Exclusion Standard data for jav.directory

Resource Scan

Scan Details

Site Domain jav.directory
Base Domain jav.directory
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-09-15T15:36:21+00:00
Next Scan 2025-10-15T15:36:21+00:00

Last Successful Scan

Scanned2025-07-24T17:08:28+00:00
URL https://jav.directory/robots.txt
Domain IPs 104.21.92.238, 172.67.200.30, 2606:4700:3034::6815:5cee, 2606:4700:3037::ac43:c81e
Response IP 104.21.92.238
Found Yes
Hash 187dbfdb4c49956500cea8db08105dd228bcdd3cb2dafa06abe89b5ed31ba55a
SimHash c80cccb8ec9a

Groups

ia_archiver

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

censysinspect

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

pimeyes

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://jav.directory/sitemap.xml