compat.io
robots.txt

Robots Exclusion Standard data for compat.io

Archived Snapshots

Resource Scan

Scan Details

Site Domain	compat.io
Base Domain	compat.io
Scan Status	Ok
Last Scan	2025-05-17T15:53:28+00:00
Next Scan	2025-06-16T15:53:28+00:00

Last Scan

Scanned	2025-05-17T15:53:28+00:00
URL	https://compat.io/robots.txt
Domain IPs	104.26.6.125, 104.26.7.125, 172.67.70.48, 2606:4700:20::681a:67d, 2606:4700:20::681a:77d, 2606:4700:20::ac43:4630
Response IP	172.67.70.48
Found	Yes
Hash	b36362ffbfaffd9f055c34bb3a55c51717d557124f6d7f882b5354c40a6a5cc9
SimHash	36574981c4c7

Groups

ai2bot

Rule	Path
Disallow	/

Rule

Path

Disallow

amazonbot

Rule	Path
Disallow	/

Rule

Path

Disallow

amazon-kendra

Rule	Path
Disallow	/

Rule

Path

Disallow

anthropic-ai

Rule	Path
Disallow	/

Rule

Path

Disallow

applebot

Rule	Path
Disallow	/

Rule

Path

Disallow

applebot-extended

Rule	Path
Disallow	/

Rule

Path

Disallow

awariorssbot

Rule	Path
Disallow	/

Rule

Path

Disallow

awariosmartbot

Rule	Path
Disallow	/

Rule

Path

Disallow

brightbot

Rule	Path
Disallow	/

Rule

Path

Disallow

bytespider

Rule	Path
Disallow	/

Rule

Path

Disallow

chatgpt-user

Rule	Path
Disallow	/

Rule

Path

Disallow

claudebot

Rule	Path
Disallow	/

Rule

Path

Disallow

diffbot

Rule	Path
Disallow	/

Rule

Path

Disallow

duckassistbot

Rule	Path
Disallow	/

Rule

Path

Disallow

facebookbot

Rule	Path
Disallow	/

Rule

Path

Disallow

friendlycrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

google-extended

Rule	Path
Disallow	/

Rule

Path

Disallow

gptbot

Rule	Path
Disallow	/

Rule

Path

Disallow

iaskspider/2.0

Rule	Path
Disallow	/

Rule

Path

Disallow

icc-crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

img2dataset

Rule	Path
Disallow	/

Rule

Path

Disallow

kangaroo bot

Rule	Path
Disallow	/

Rule

Path

Disallow

linerbot

Rule	Path
Disallow	/

Rule

Path

Disallow

machinelearningforpeacebot

Rule	Path
Disallow	/

Rule

Path

Disallow

meltwater

Rule	Path
Disallow	/

Rule

Path

Disallow

meta-externalagent

Rule	Path
Disallow	/

Rule

Path

Disallow

meta-externalfetcher

Rule	Path
Disallow	/

Rule

Path

Disallow

nicecrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

oai-searchbot

Rule	Path
Disallow	/

Rule

Path

Disallow

omgili

Rule	Path
Disallow	/

Rule

Path

Disallow

omgilibot

Rule	Path
Disallow	/

Rule

Path

Disallow

pangubot

Rule	Path
Disallow	/

Rule

Path

Disallow

perplexitybot

Rule	Path
Disallow	/

Rule

Path

Disallow

perplexity-user

Rule	Path
Disallow	/

Rule

Path

Disallow

petalbot

Rule	Path
Disallow	/

Rule

Path

Disallow

piplbot

Rule	Path
Disallow	/

Rule

Path

Disallow

qualifiedbot

Rule	Path
Disallow	/

Rule

Path

Disallow

scoop.it

Rule	Path
Disallow	/

Rule

Path

Disallow

seekr

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot-ocob

Rule	Path
Disallow	/

Rule

Path

Disallow

sidetrade indexer bot

Rule	Path
Disallow	/

Rule

Path

Disallow

timpibot

Rule	Path
Disallow	/

Rule

Path

Disallow

velenpublicwebcrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

webzio-extended

Rule	Path
Disallow	/

Rule

Path

Disallow

youbot

Rule	Path
Disallow	/

Rule

Path

Disallow

Comments

Cloudflare Managed Robots.txt to block AI related bots.

compat.iorobots.txt

Resource Scan

Scan Details

Last Scan

Groups

ai2bot

amazonbot

amazon-kendra

anthropic-ai

applebot

applebot-extended

awariorssbot

awariosmartbot

brightbot

bytespider

chatgpt-user

claudebot

diffbot

duckassistbot

facebookbot

friendlycrawler

google-extended

gptbot

iaskspider/2.0

icc-crawler

img2dataset

kangaroo bot

linerbot

machinelearningforpeacebot

meltwater

meta-externalagent

meta-externalfetcher

nicecrawler

oai-searchbot

omgili

omgilibot

pangubot

perplexitybot

perplexity-user

petalbot

piplbot

qualifiedbot

scoop.it

seekr

semrushbot-ocob

sidetrade indexer bot

timpibot

velenpublicwebcrawler

webzio-extended

youbot

Comments

compat.io
robots.txt