scout.it
robots.txt

Robots Exclusion Standard data for scout.it

Resource Scan

Scan Details

Site Domain scout.it
Base Domain scout.it
Scan Status Ok
Last Scan2025-09-28T07:17:22+00:00
Next Scan 2025-10-28T07:17:22+00:00

Last Scan

Scanned2025-09-28T07:17:22+00:00
URL https://scout.it/robots.txt
Redirect https://www.scout.it/robots.txt
Redirect Domain www.scout.it
Redirect Base scout.it
Domain IPs 151.101.1.124, 151.101.129.124, 151.101.193.124, 151.101.65.124
Redirect IPs 151.101.1.124, 151.101.129.124, 151.101.193.124, 151.101.65.124
Response IP 146.75.45.124
Found Yes
Hash 64c5e45640dcb17220306239aea572de6f789dd7dc939a33fc0eed9577311776
SimHash 720cea025ff0

Groups

*

Rule Path Comment
Disallow /*product_list_order%3D* -
Disallow /*price%3D* -
Disallow /*product_list_dir%3D* ordinamento dei listati asc/dis
Disallow /*?dir* -
Disallow /*?dir=desc -
Disallow /*?dir=asc -
Disallow /*?limit=all -
Disallow /*?mode* -
Disallow /*?*=* -
Disallow /*?SID -
Disallow /catalogsearch/ -
Disallow *%5C%5C* -
Disallow /index.php/ -
Disallow /LICENSE.txt -
Disallow /LICENSE.html -
Disallow /brands/?* -
Disallow /404/ -
Disallow */customer/* -
Disallow */checkout/* -
Disallow */onestepcheckout/* -
Disallow */wishlist/* -
Disallow */sendfriend/* -
Disallow */nds/* -
Disallow /admin/ -
Disallow /review/product/ -
Disallow /skin/ -
Disallow /directory/ -
Disallow /catalog/product_compare/ -
Disallow /catalog/category/view/ -
Disallow /catalog/product/view/ -
Disallow /app/ -
Disallow /bin/ -
Disallow /dev/ -
Disallow /lib/ -
Disallow /phpserver/ -

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

meta-externalagent

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

claudebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.scout.it/media/it_sitemap_category.xml
sitemap https://www.scout.it/media/it_sitemap_product.xml
sitemap https://www.scout.it/media/it_sitemap_additional_link.xml
sitemap https://www.scout.it/media/it_sitemap_other.xml
sitemap https://www.scout.it/media/en_sitemap_category.xml
sitemap https://www.scout.it/media/en_sitemap_product.xml
sitemap https://www.scout.it/media/en_sitemap_additional_link.xml
sitemap https://www.scout.it/media/en_sitemap_other.xml

Comments

  • Filtri
  • Paths (no clean URLs)
  • Directories
  • Blocking CMS directories.
  • Disallow: /pub/
  • Allow: /pub/media/catalog/*
  • Allow: /pub/media/product/*
  • Allow: /pub/media/logo/*
  • Allow: /pub/media/wysiwyg/*
  • Allow: /pub/static/* #per abilitare il js responsabile del caricamento delle immagini
  • Indicazione del path della sitemap XML
  • Blocco bot meno importanti
  • Delay bot necessari