Log In Sign Up

consulente-della-salute.it
robots.txt

Robots Exclusion Standard data for consulente-della-salute.it

Resource Scan

Scan Details

Site Domain consulente-della-salute.it
Base Domain consulente-della-salute.it
Scan Status Ok
Last Scan2024-09-29T23:35:23+00:00
Next Scan 2024-10-29T23:35:23+00:00

Last Scan

Scanned2024-09-29T23:35:23+00:00
URL https://consulente-della-salute.it/robots.txt
Domain IPs 104.26.2.231, 104.26.3.231, 172.67.71.163, 2606:4700:20::681a:2e7, 2606:4700:20::681a:3e7, 2606:4700:20::ac43:47a3
Response IP 104.26.2.231
Found Yes
Hash 9010ecb5923ce68586883b5edfd992ac4ec29c3f911c68c5f04f61a9e99ec278
SimHash 530dd7504682

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

adbeat_bot

Rule Path
Disallow /

adscanner

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

amazon-kendra

Rule Path
Disallow /

amazon-qbusiness

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

applebot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

arquivo-web-crawler

Rule Path
Disallow /

awariorssbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

backlink-check.de

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-render

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

buzzsumo

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

ccbot/1.0

Rule Path
Disallow /

ccbot/2.0

Rule Path
Disallow /

changedetection

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claritybot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

cognitiveseo

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

duckduckbot

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

extractorpro

Rule Path
Disallow /

fasterfox

Rule Path
Disallow /

flamingo_searchengine

Rule Path
Disallow /

friendlycrawler

Rule Path
Disallow /

gammaspider

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

http banner detection

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

img2dataset

Rule Path
Disallow /

ioncrawl

Rule Path
Disallow /

is_archiver

Rule Path
Disallow /

lcc

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

linkextractorpro

Rule Path
Disallow /

linkwalker

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

mega-index

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

newsnow

Rule Path
Disallow /

objectssearch

Rule Path
Disallow /

odin

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

outclicksbot

Rule Path
Disallow /

peer39_crawler

Rule Path
Disallow /

peer39_crawler/1.0

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

pimptrain

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

proxemic

Rule Path
Disallow /

raven

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

scoop.it

Rule Path
Disallow /

sebot-wa

Rule Path
Disallow /

seekr

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

seobility

Rule Path
Disallow /

seobilitybot

Rule Path
Disallow /

seodat

Rule Path
Disallow /

seoengbot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

seranking bot

Rule Path
Disallow /

seranking.com

Rule Path
Disallow /

similarweb

Rule Path
Disallow /

sirdatabot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

swiftbot

Rule Path
Disallow /

trendkite-akashic-crawler

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

um-ic

Rule Path
Disallow /

url_spider_pro

Rule Path
Disallow /

wapspider

Rule Path
Disallow /

webzinger

Rule Path
Disallow /

wonderbot

Rule Path
Disallow /

xovi

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

yandexmobilebot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

youbot

Rule Path
Disallow /

botify

Rule Path
Disallow /

oncrawl

Rule Path
Disallow /

gingercrawler

Rule Path
Disallow /

webmon

Rule Path
Disallow /

httrack

Rule Path
Disallow /

contextad bot

Rule Path
Disallow /

addsearchbot

Rule Path
Disallow /

admantx

Rule Path
Disallow /
Back to top

Comments

  • Last updated on: 24.07.2024
  • Block bots and crawler. Especially AI scraper
Back to top