cilentonotizie.it
robots.txt

Robots Exclusion Standard data for cilentonotizie.it

Resource Scan

Scan Details

Site Domain cilentonotizie.it
Base Domain cilentonotizie.it
Scan Status Ok
Last Scan2024-09-24T07:01:59+00:00
Next Scan 2024-10-01T07:01:59+00:00

Last Scan

Scanned2024-09-24T07:01:59+00:00
URL https://cilentonotizie.it/robots.txt
Redirect https://www.cilentonotizie.it/robots.txt
Redirect Domain www.cilentonotizie.it
Redirect Base cilentonotizie.it
Domain IPs 31.11.35.125
Redirect IPs 31.11.35.125
Response IP 31.11.35.125
Found Yes
Hash 281ab2efda6201c01ed8cd5a974e226b9d1366eeb1bb1ca7341566e3a689553a
SimHash 22045301c5a5

Groups

*

Rule Path
Disallow /download/
Disallow /utility/
Disallow /chisiamo/

googlebot

Rule Path
Disallow

adsbot-google

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

mediapartners-google

Rule Path
Disallow

google-extended

Rule Path
Disallow /

bingbot

Rule Path
Disallow

bingpreview

Rule Path
Disallow

quora-bot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

viberbot

Rule Path
Disallow /

yak

Rule Path
Disallow /

yandex

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

applebot

Rule Path
Disallow /

discordbot

Rule Path
Disallow /

duckduckbot

Rule Path
Disallow

linkedinbot

Rule Path
Disallow

msnbot-media

Rule Path
Disallow

msnbot

Rule Path
Disallow

naverbot

Rule Path
Disallow

pinterestbot

Rule Path
Disallow

screaming frog seo spider

Rule Path
Disallow

seznambot

Rule Path
Disallow

slurp

Rule Path
Disallow

teoma

Rule Path
Disallow

telegrambot

Rule Path
Disallow

twitterbot

Rule Path
Disallow

yeti

Rule Path
Disallow

meta-externalagent

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

facebookbot

Rule Path
Disallow /

zoombot

Rule Path
Disallow /

friendlycrawler

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

*

Rule Path
Disallow /