/.well-known/

Log In Sign Up

truyenhentai18.app
robots.txt

Robots Exclusion Standard data for truyenhentai18.app

Archived Snapshots

Resource Scan

Scan Details

Site Domain	truyenhentai18.app
Base Domain	truyenhentai18.app
Scan Status	Ok
Last Scan	2025-05-27T22:20:12+00:00
Next Scan	2025-06-10T22:20:12+00:00

Last Scan

Scanned	2025-05-27T22:20:12+00:00
URL	https://truyenhentai18.app/robots.txt
Domain IPs	104.21.29.140, 172.67.149.64, 2606:4700:3032::6815:1d8c, 2606:4700:3033::ac43:9540
Response IP	104.21.29.140
Found	Yes
Hash	b36362ffbfaffd9f055c34bb3a55c51717d557124f6d7f882b5354c40a6a5cc9
SimHash	36574981c4c7

Groups

ai2bot

Rule

Path

Disallow

/

amazonbot

Rule

Path

Disallow

/

amazon-kendra

Rule

Path

Disallow

/

anthropic-ai

Rule

Path

Disallow

/

applebot

Rule

Path

Disallow

/

applebot-extended

Rule

Path

Disallow

/

awariorssbot

Rule

Path

Disallow

/

awariosmartbot

Rule

Path

Disallow

/

brightbot

Rule

Path

Disallow

/

bytespider

Rule

Path

Disallow

/

chatgpt-user

Rule

Path

Disallow

/

claudebot

Rule

Path

Disallow

/

diffbot

Rule

Path

Disallow

/

duckassistbot

Rule

Path

Disallow

/

facebookbot

Rule

Path

Disallow

/

friendlycrawler

Rule

Path

Disallow

/

google-extended

Rule

Path

Disallow

/

gptbot

Rule

Path

Disallow

/

iaskspider/2.0

Rule

Path

Disallow

/

icc-crawler

Rule

Path

Disallow

/

img2dataset

Rule

Path

Disallow

/

kangaroo bot

Rule

Path

Disallow

/

linerbot

Rule

Path

Disallow

/

machinelearningforpeacebot

Rule

Path

Disallow

/

meltwater

Rule

Path

Disallow

/

meta-externalagent

Rule

Path

Disallow

/

meta-externalfetcher

Rule

Path

Disallow

/

nicecrawler

Rule

Path

Disallow

/

oai-searchbot

Rule

Path

Disallow

/

omgili

Rule

Path

Disallow

/

omgilibot

Rule

Path

Disallow

/

pangubot

Rule

Path

Disallow

/

perplexitybot

Rule

Path

Disallow

/

perplexity-user

Rule

Path

Disallow

/

petalbot

Rule

Path

Disallow

/

piplbot

Rule

Path

Disallow

/

qualifiedbot

Rule

Path

Disallow

/

scoop.it

Rule

Path

Disallow

/

seekr

Rule

Path

Disallow

/

semrushbot-ocob

Rule

Path

Disallow

/

sidetrade indexer bot

Rule

Path

Disallow

/

timpibot

Rule

Path

Disallow

/

velenpublicwebcrawler

Rule

Path

Disallow

/

webzio-extended

Rule

Path

Disallow

/

youbot

Rule

Path

Disallow

/

Back to top

Comments

Cloudflare Managed Robots.txt to block AI related bots.

Back to top