Log In Sign Up

gedistatic.it
robots.txt

Robots Exclusion Standard data for gedistatic.it

Resource Scan

Scan Details

Site Domain gedistatic.it
Base Domain gedistatic.it
Scan Status Ok
Last Scan2025-05-14T02:27:38+00:00
Next Scan 2025-05-21T02:27:38+00:00

Last Scan

Scanned2025-05-14T02:27:38+00:00
URL https://www.gedistatic.it/robots.txt
Domain IPs 52.222.144.38, 52.222.144.45, 52.222.144.65, 52.222.144.66
Response IP 18.165.140.16
Found Yes
Hash 5066e58b1c9053d074e78b4e44e352bdb42e2f6bb13213f38f66db1eed2f7fac
SimHash 320c415089a7

Groups

*

Rule Path
Allow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

web-archive-net.com.bot

Rule Path
Disallow /

fetch

Rule Path
Disallow /

nextgensearchbot

Rule Path
Disallow /

url_spider_pro

Rule Path
Disallow /

europarchive.org

Rule Path
Disallow /

nerdbynature.bot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

crystalsemanticsbot

Rule Path
Disallow /

linkextractorpro

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

kbcrawl

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

bixocrawler

Rule Path
Disallow /

jyxobot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

awariorssbot

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

friendlycrawler

Rule Path
Disallow /

pangubot

Rule Path
Disallow /

wotbot

Rule Path
Disallow /

npbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

extractorpro

Rule Path
Disallow /

nnetseer crawler

Rule Path
Disallow /

umbot-ln

Rule Path
Disallow /

duckassistbot

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

true_robot

Rule Path
Disallow /

dloader(naverrobot)

Rule Path
Disallow /

moreoverbot

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

slurp

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

linkarchiver

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

kangaroo bot

Rule Path
Disallow /

searchpreview

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

quora-bot

Rule Path
Disallow /

arquivo-web-crawler

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

primalbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

gumgum bot

Rule Path
Disallow /

peer39_crawler/1.0

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

openbot

Rule Path
Disallow /

wesee:search

Rule Path
Disallow /

flamingo_searchengine

Rule Path
Disallow /

nicecrawler

Rule Path
Disallow /

verticalsearch

Rule Path
Disallow /

exabot

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

nabot

Rule Path
Disallow /

trovitbot

Rule Path
Disallow /

discoverybot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

livelapbot

Rule Path
Disallow /

sitebot/0.1

Rule Path
Disallow /

converacrawler

Rule Path
Disallow /

lexxebot/1.0

Rule Path
Disallow /

seoengbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

jetbot

Rule Path
Disallow /

queryseekerspider

Rule Path
Disallow /

archivebot

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Disallow /

pixray-seeker

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

peer39_crawler

Rule Path
Disallow /

youbot

Rule Path
Disallow /
Back to top