breedingwiki.com
robots.txt

Robots Exclusion Standard data for breedingwiki.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	breedingwiki.com
Base Domain	breedingwiki.com
Scan Status	Ok
Last Scan	2024-05-25T11:43:23+00:00
Next Scan	2024-06-01T11:43:23+00:00

Last Scan

Scanned	2024-05-25T11:43:23+00:00
URL	https://www.breedingwiki.com/robots.txt
Domain IPs	104.21.36.239, 172.67.201.49, 2606:4700:3032::ac43:c931, 2606:4700:3034::6815:24ef
Response IP	104.21.36.239
Found	Yes
Hash	31ecb9b9aee91023f64a6197aa2f302941119f515c31ccefd2caf415dad22b91
SimHash	e29c6e524c12

Groups

ahrefsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

alphabot

Rule	Path
Disallow	/

Rule

Path

Disallow

baiduspider

Rule	Path
Disallow	/

Rule

Path

Disallow

baiduspider-render

Rule	Path
Disallow	/

Rule

Path

Disallow

buck

Rule	Path
Disallow	/

Rule

Path

Disallow

changedetection

Rule	Path
Disallow	/

Rule

Path

Disallow

cliqzbot

Rule	Path
Disallow	/

Rule

Path

Disallow

exabot

Rule	Path
Disallow	/

Rule

Path

Disallow

flamingo_searchengine

Rule	Path
Disallow	/

Rule

Path

Disallow

grobbot

Rule	Path
Disallow	/

Rule

Path

Disallow

jobboersebot

Rule	Path
Disallow	/

Rule

Path

Disallow

jobs.de-robot

Rule	Path
Disallow	/

Rule

Path

Disallow

linkdexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

mail.ru_bot

Rule	Path
Disallow	/

Rule

Path

Disallow

mediatoolkitbot

Rule	Path
Disallow	/

Rule

Path

Disallow

megaindex.ru/2.0

Rule	Path
Disallow	/

Rule

Path

Disallow

mega-index

Rule	Path
Disallow	/

Rule

Path

Disallow

megaindex.ru

Rule	Path
Disallow	/

Rule

Path

Disallow

mj12bot

Rule	Path
Disallow	/

Rule

Path

Disallow

mojeekbot

Rule	Path
Disallow	/

Rule

Path

Disallow

pinterestbot

Rule	Path
Disallow	/

Rule

Path

Disallow

rogerbot

Rule	Path
Disallow	/

Rule

Path

Disallow

safednsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot/2~bl

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot-ba

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot-bm

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot-sa

Rule	Path
Disallow	/

Rule

Path

Disallow

seokicks-robot

Rule	Path
Disallow	/

Rule

Path

Disallow

seznambot

Rule	Path
Disallow	/

Rule

Path

Disallow

smtbot

Rule	Path
Disallow	/

Rule

Path

Disallow

spbot

Rule	Path
Disallow	/

Rule

Path

Disallow

trendictionbot

Rule	Path
Disallow	/

Rule

Path

Disallow

tweetmemebot

Rule	Path
Disallow	/

Rule

Path

Disallow

vebidoobot

Rule	Path
Disallow	/

Rule

Path

Disallow

wonderbot

Rule	Path
Disallow	/

Rule

Path

Disallow

wotbox

Rule	Path
Disallow	/

Rule

Path

Disallow

yacybot

Rule	Path
Disallow	/

Rule

Path

Disallow

yak

Rule	Path
Disallow	/

Rule

Path

Disallow

yandexantivirus

Rule	Path
Disallow	/

Rule

Path

Disallow

yandexbot/3.0

Rule	Path
Disallow	/

Rule

Path

Disallow

yandeximages

Rule	Path
Disallow	/

Rule

Path

Disallow

yandexmobilebot

Rule	Path
Disallow	/

Rule

Path

Disallow

yeti

Rule	Path
Disallow	/

Rule

Path

Disallow

Other Records

Field	Value
sitemap	https://www.breedingwiki.com/sitemap_index.xml

Field

Value

sitemap

https://www.breedingwiki.com/sitemap_index.xml

breedingwiki.comrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

ahrefsbot

alphabot

baiduspider

baiduspider-render

buck

changedetection

cliqzbot

exabot

flamingo_searchengine

grobbot

jobboersebot

jobs.de-robot

linkdexbot

mail.ru_bot

mediatoolkitbot

megaindex.ru/2.0

mega-index

megaindex.ru

mj12bot

mojeekbot

pinterestbot

rogerbot

safednsbot

semrushbot

semrushbot/2~bl

semrushbot-ba

semrushbot-bm

semrushbot-sa

seokicks-robot

seznambot

smtbot

spbot

trendictionbot

tweetmemebot

vebidoobot

wonderbot

wotbox

yacybot

yak

yandexantivirus

yandexbot/3.0

yandeximages

yandexmobilebot

yeti

Other Records

breedingwiki.com
robots.txt