Log In Sign Up

antenati-italiani.org
robots.txt

Robots Exclusion Standard data for antenati-italiani.org

Resource Scan

Scan Details

Site Domain antenati-italiani.org
Base Domain antenati-italiani.org
Scan Status Ok
Last Scan2024-11-18T10:08:18+00:00
Next Scan 2024-11-25T10:08:18+00:00

Last Scan

Scanned2024-11-18T10:08:18+00:00
URL https://antenati-italiani.org/robots.txt
Redirect https://www.antenati-italiani.org/robots.txt
Redirect Domain www.antenati-italiani.org
Redirect Base antenati-italiani.org
Domain IPs 2a02:4780:84:8db9:7409:fd1e:f555:b191, 93.127.187.162
Redirect IPs 2a02:4780:16:4ef5:e014:ff54:603f:4dd7, 93.127.196.70
Response IP 179.61.189.130
Found Yes
Hash 2c85fa5054c665b4427fc8d31bc095df553e7cfb5d061598ca266bb2410cf99e
SimHash 6936cfd3ebb1

Groups

*

Rule Path
Disallow /it/albero-genealogico/
Disallow /en/family-tree/
Disallow /pt/arvore-genealogica/
Disallow /fr/arbre-genealogique/
Disallow /es/arbol-genealogico/
Disallow /de/stammbaum/

Other Records

Field Value
crawl-delay 16

ahrefsbot

Rule Path
Disallow /

ahrefsbot/7.0

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

applebot/0.1

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

barkrowler/0.9

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bomborabot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

cincraw/1.0

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

crawlbot/1.0.0

Rule Path
Disallow /

criteobot

Rule Path
Disallow /

criteobot/0.1

Rule Path
Disallow /

deusu

Rule Path
Disallow /

discordbot/2.0

Rule Path
Disallow /

domaincrawler/3.0

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

dotbot/1.1

Rule Path
Disallow /

exabot/3.0

Rule Path
Disallow /

grapeshotcrawler/2.0

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

istellabot/t.1.13

Rule Path
Disallow /

linguee bot

Rule Path
Disallow /

mail.ru_bot/2.0

Rule Path
Disallow /

mail.ru_bot/fast/2.0

Rule Path
Disallow /

mail.ru_bot/img/2.0

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mj12bot/v1.4.8

Rule Path
Disallow /

mojeekbot/0.10

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

neevabot/1.0

Rule Path
Disallow /

obot/2.3.1

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot/7~bl

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-bm/1.0

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

serpstatbot/2.1

Rule Path
Disallow /

seznambot/3.2

Rule Path
Disallow /

spbot

Rule Path
Disallow /

vuhuvbot/1.0

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

weborama-fetcher

Rule Path
Disallow /

webprosbot/2.0

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandexbot/3.0

Rule Path
Disallow /

zoombot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /
Back to top

Warnings

  • `request-rate` is not a known field.
Back to top