/.well-known/

Log In Sign Up

hentaicb.fun
robots.txt

Robots Exclusion Standard data for hentaicb.fun

Archived Snapshots

Resource Scan

Scan Details

Site Domain	hentaicb.fun
Base Domain	hentaicb.fun
Scan Status	Ok
Last Scan	2025-04-14T22:01:58+00:00
Next Scan	2025-05-14T22:01:58+00:00

Last Scan

Scanned	2025-04-14T22:01:58+00:00
URL	https://hentaicb.fun/robots.txt
Redirect	https://hentaicb.fit/robots.txt
Redirect Domain	hentaicb.fit
Redirect Base	hentaicb.fit
Domain IPs	104.21.84.140, 172.67.193.205, 2606:4700:3033::ac43:c1cd, 2606:4700:3037::6815:548c
Redirect IPs	104.21.25.251, 172.67.134.244, 2606:4700:3032::ac43:86f4, 2606:4700:3034::6815:19fb
Response IP	104.21.25.251
Found	Yes
Hash	b36362ffbfaffd9f055c34bb3a55c51717d557124f6d7f882b5354c40a6a5cc9
SimHash	36574981c4c7

Groups

ai2bot

Rule

Path

Disallow

/

amazonbot

Rule

Path

Disallow

/

amazon-kendra

Rule

Path

Disallow

/

anthropic-ai

Rule

Path

Disallow

/

applebot

Rule

Path

Disallow

/

applebot-extended

Rule

Path

Disallow

/

awariorssbot

Rule

Path

Disallow

/

awariosmartbot

Rule

Path

Disallow

/

brightbot

Rule

Path

Disallow

/

bytespider

Rule

Path

Disallow

/

chatgpt-user

Rule

Path

Disallow

/

claudebot

Rule

Path

Disallow

/

diffbot

Rule

Path

Disallow

/

duckassistbot

Rule

Path

Disallow

/

facebookbot

Rule

Path

Disallow

/

friendlycrawler

Rule

Path

Disallow

/

google-extended

Rule

Path

Disallow

/

gptbot

Rule

Path

Disallow

/

iaskspider/2.0

Rule

Path

Disallow

/

icc-crawler

Rule

Path

Disallow

/

img2dataset

Rule

Path

Disallow

/

kangaroo bot

Rule

Path

Disallow

/

linerbot

Rule

Path

Disallow

/

machinelearningforpeacebot

Rule

Path

Disallow

/

meltwater

Rule

Path

Disallow

/

meta-externalagent

Rule

Path

Disallow

/

meta-externalfetcher

Rule

Path

Disallow

/

nicecrawler

Rule

Path

Disallow

/

oai-searchbot

Rule

Path

Disallow

/

omgili

Rule

Path

Disallow

/

omgilibot

Rule

Path

Disallow

/

pangubot

Rule

Path

Disallow

/

perplexitybot

Rule

Path

Disallow

/

perplexity-user

Rule

Path

Disallow

/

petalbot

Rule

Path

Disallow

/

piplbot

Rule

Path

Disallow

/

qualifiedbot

Rule

Path

Disallow

/

scoop.it

Rule

Path

Disallow

/

seekr

Rule

Path

Disallow

/

semrushbot-ocob

Rule

Path

Disallow

/

sidetrade indexer bot

Rule

Path

Disallow

/

timpibot

Rule

Path

Disallow

/

velenpublicwebcrawler

Rule

Path

Disallow

/

webzio-extended

Rule

Path

Disallow

/

youbot

Rule

Path

Disallow

/

Back to top

Comments

Cloudflare Managed Robots.txt to block AI related bots.

Back to top