birkod.az
robots.txt

Robots Exclusion Standard data for birkod.az

Archived Snapshots

Resource Scan

Scan Details

Site Domain	birkod.az
Base Domain	birkod.az
Scan Status	Ok
Last Scan	5/19/2025, 9:27:50 PM
Next Scan	6/18/2025, 9:27:50 PM

Last Scan

Scanned	5/19/2025, 9:27:50 PM
URL	https://birkod.az/robots.txt
Domain IPs	104.21.55.101, 172.67.147.117, 2606:4700:3030::ac43:9375, 2606:4700:3032::6815:3765
Response IP	104.21.55.101
Found	Yes
Hash	b36362ffbfaffd9f055c34bb3a55c51717d557124f6d7f882b5354c40a6a5cc9
SimHash	36574981c4c7

Groups

ai2bot

Rule	Path
Disallow	/

Rule

Path

Disallow

amazonbot

Rule	Path
Disallow	/

Rule

Path

Disallow

amazon-kendra

Rule	Path
Disallow	/

Rule

Path

Disallow

anthropic-ai

Rule	Path
Disallow	/

Rule

Path

Disallow

applebot

Rule	Path
Disallow	/

Rule

Path

Disallow

applebot-extended

Rule	Path
Disallow	/

Rule

Path

Disallow

awariorssbot

Rule	Path
Disallow	/

Rule

Path

Disallow

awariosmartbot

Rule	Path
Disallow	/

Rule

Path

Disallow

brightbot

Rule	Path
Disallow	/

Rule

Path

Disallow

bytespider

Rule	Path
Disallow	/

Rule

Path

Disallow

chatgpt-user

Rule	Path
Disallow	/

Rule

Path

Disallow

claudebot

Rule	Path
Disallow	/

Rule

Path

Disallow

diffbot

Rule	Path
Disallow	/

Rule

Path

Disallow

duckassistbot

Rule	Path
Disallow	/

Rule

Path

Disallow

facebookbot

Rule	Path
Disallow	/

Rule

Path

Disallow

friendlycrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

google-extended

Rule	Path
Disallow	/

Rule

Path

Disallow

gptbot

Rule	Path
Disallow	/

Rule

Path

Disallow

iaskspider/2.0

Rule	Path
Disallow	/

Rule

Path

Disallow

icc-crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

img2dataset

Rule	Path
Disallow	/

Rule

Path

Disallow

kangaroo bot

Rule	Path
Disallow	/

Rule

Path

Disallow

linerbot

Rule	Path
Disallow	/

Rule

Path

Disallow

machinelearningforpeacebot

Rule	Path
Disallow	/

Rule

Path

Disallow

meltwater

Rule	Path
Disallow	/

Rule

Path

Disallow

meta-externalagent

Rule	Path
Disallow	/

Rule

Path

Disallow

meta-externalfetcher

Rule	Path
Disallow	/

Rule

Path

Disallow

nicecrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

oai-searchbot

Rule	Path
Disallow	/

Rule

Path

Disallow

omgili

Rule	Path
Disallow	/

Rule

Path

Disallow

omgilibot

Rule	Path
Disallow	/

Rule

Path

Disallow

pangubot

Rule	Path
Disallow	/

Rule

Path

Disallow

perplexitybot

Rule	Path
Disallow	/

Rule

Path

Disallow

perplexity-user

Rule	Path
Disallow	/

Rule

Path

Disallow

petalbot

Rule	Path
Disallow	/

Rule

Path

Disallow

piplbot

Rule	Path
Disallow	/

Rule

Path

Disallow

qualifiedbot

Rule	Path
Disallow	/

Rule

Path

Disallow

scoop.it

Rule	Path
Disallow	/

Rule

Path

Disallow

seekr

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot-ocob

Rule	Path
Disallow	/

Rule

Path

Disallow

sidetrade indexer bot

Rule	Path
Disallow	/

Rule

Path

Disallow

timpibot

Rule	Path
Disallow	/

Rule

Path

Disallow

velenpublicwebcrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

webzio-extended

Rule	Path
Disallow	/

Rule

Path

Disallow

youbot

Rule	Path
Disallow	/

Rule

Path

Disallow

Comments

Cloudflare Managed Robots.txt to block AI related bots.

birkod.azrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

ai2bot

amazonbot

amazon-kendra

anthropic-ai

applebot

applebot-extended

awariorssbot

awariosmartbot

brightbot

bytespider

chatgpt-user

claudebot

diffbot

duckassistbot

facebookbot

friendlycrawler

google-extended

gptbot

iaskspider/2.0

icc-crawler

img2dataset

kangaroo bot

linerbot

machinelearningforpeacebot

meltwater

meta-externalagent

meta-externalfetcher

nicecrawler

oai-searchbot

omgili

omgilibot

pangubot

perplexitybot

perplexity-user

petalbot

piplbot

qualifiedbot

scoop.it

seekr

semrushbot-ocob

sidetrade indexer bot

timpibot

velenpublicwebcrawler

webzio-extended

youbot

Comments

birkod.az
robots.txt