soldionline.it
robots.txt

Robots Exclusion Standard data for soldionline.it

Resource Scan

Scan Details

Site Domain soldionline.it
Base Domain soldionline.it
Scan Status Ok
Last Scan2024-09-16T01:29:06+00:00
Next Scan 2024-09-23T01:29:06+00:00

Last Scan

Scanned2024-09-16T01:29:06+00:00
URL https://soldionline.it/robots.txt
Redirect https://www.soldionline.it/robots.txt
Redirect Domain www.soldionline.it
Redirect Base soldionline.it
Domain IPs 176.34.239.234, 34.248.24.67
Redirect IPs 23.50.90.17, 2600:1413:b000:68a::3198, 2600:1413:b000:68d::3198
Response IP 104.69.46.165
Found Yes
Hash a45c900d023bae2ee4311f6d68e54fecd5c7b02f18f16d0d548df64ce3576d0e
SimHash 4f1a331ac213

Groups

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /voto/*
Disallow /ricerca/*
Disallow /sondaggi/*
Disallow /sendFriend/*
Disallow /fake*
Disallow /cerca/*
Disallow /autore/
Disallow /notizie/blog
Disallow /quotazioni/*/sort/*
Disallow /SOL_Editoriale.nsf*
Disallow /SOL_Editoriale.nsf/*
Disallow /a.pic1?EL*
Disallow /isin-feed/*
Disallow /gallery/
Disallow /images/
Disallow /pictures/2008/
Disallow /pictures/2009/
Disallow /pictures/2010/
Disallow /pictures/2011/
Disallow /pictures/2012/
Disallow /pictures/2013/
Disallow /pictures/2014/
Disallow /pictures/2015/
Disallow /pictures/2016/
Disallow /pictures/2017/01/
Disallow /isin/
Disallow /network-blog_OLD/
Disallow /blog_OLD/
Disallow /infografiche/
Disallow /SOL_Editoriale.nsf/
Disallow /isin-feed/
Disallow /soldinews/
Disallow /network/
Disallow /ptr/
Disallow /rss/*
Disallow /feed/*
Disallow /related/*

gptbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

seekr

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

Comments

  • Robots.txt - soldionline.it
  • Blocco crawler BOT
  • 2019-02-21
  • ADD 2011.05.06 - by Pj e Bono.
  • Da testare
  • Disallow: /*?
  • 2016.10.25 by Vinci
  • 2017.02.24
  • issue SOL-19
  • issue SOL-65
  • issue SOL-86
  • 2019-01-10
  • issue SOL-206
  • FINE issue SOL-206

Warnings

  • 2 invalid lines.