Log In Sign Up

spi-discount.net
robots.txt

Robots Exclusion Standard data for spi-discount.net

Resource Scan

Scan Details

Site Domain spi-discount.net
Base Domain spi-discount.net
Scan Status Ok
Last Scan2024-09-27T07:03:09+00:00
Next Scan 2024-10-04T07:03:09+00:00

Last Scan

Scanned2024-09-27T07:03:09+00:00
URL https://spi-discount.net/robots.txt
Domain IPs 163.172.73.188
Response IP 163.172.73.188
Found Yes
Hash 3cc20b00202712ac11db82a3361ec7f35cbb24cdf7c270136e42688dba5d731d
SimHash bc9303524ccd

Groups

*

Rule Path
Disallow /classes/
Disallow /config/
Disallow /download/
Disallow /mails/
Disallow /translations/
Disallow /tools/
Disallow /lang-fr/
Disallow /blog/cgi-bin
Disallow /blog/wp-login.php
Disallow /blog//wp-admin
Disallow /blog//wp-includes
Disallow /addresses.php
Disallow /address.php
Disallow /authentication.php
Disallow /cart.php
Disallow /discount.php
Disallow /footer.php
Disallow /get-file.php
Disallow /header.php
Disallow /history.php
Disallow /identity.php
Disallow /images.inc.php
Disallow /init.php
Disallow /my-account.php
Disallow /order.php
Disallow /order-opc.php
Disallow /order-slip.php
Disallow /order-detail.php
Disallow /order-follow.php
Disallow /order-return.php
Disallow /order-confirmation.php
Disallow /pagination.php
Disallow /password.php
Disallow /pdf-invoice.php
Disallow /pdf-order-return.php
Disallow /pdf-order-slip.php
Disallow /product-sort.php
Disallow /search.php
Disallow /statistics.php
Disallow /attachment.php
Disallow /guest-tracking
Disallow /*orderby%3D
Disallow /*orderway%3D
Disallow /*tag%3D
Disallow /*id_currency%3D
Disallow /*search_query%3D
Disallow /*id_lang%3D
Disallow /*back%3D
Disallow /*utm_source%3D
Disallow /*utm_medium%3D
Disallow /*utm_campaign%3D

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

webmeupbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

updownerbot

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

semrushbot-sa

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

mauibot (crawler.feedback+wc@gmail.com)

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

adequat

Rule Path
Disallow /

admantx bot

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

corporama

Rule Path
Disallow /

dotmic dotbot

Rule Path
Disallow /

ellisphere

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

fetch

Rule Path
Disallow /

grapeshot bot

Rule Path
Disallow /

houzzbot

Rule Path
Disallow /

infoseek

Rule Path
Disallow /

jetbot

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

leadbox

Rule Path
Disallow /

linkfluence

Rule Path
Disallow /

linko

Rule Path
Disallow /

mail.ru bot

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

npbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

opinion-tracker

Rule Path
Disallow /

proxem

Rule Path
Disallow /

proximic bot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

spotter

Rule Path
Disallow /

sputnikbot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

teleport

Rule Path
Disallow /

updownerbot

Rule Path
Disallow /

verticalsearch

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webzip

Rule Path
Disallow /

wget

Rule Path
Disallow /

winello

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

archive.org_bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

soso spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

sogou spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

net veille

Rule Path
Disallow /

paarly

Rule Path
Disallow /

spycommerce

Rule Path
Disallow /

pricing assistant

Rule Path
Disallow /

price observatory

Rule Path
Disallow /

workit

Rule Path
Disallow /

keox

Rule Path
Disallow /

pricejet

Rule Path
Disallow /

prixan

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.spi-discount.net/1_de_0_sitemap.xml
sitemap https://www.spi-discount.net/1_en_0_sitemap.xml
sitemap https://www.spi-discount.net/1_es_0_sitemap.xml
sitemap https://www.spi-discount.net/1_fr_0_sitemap.xml
sitemap https://www.spi-discount.net/1_it_0_sitemap.xml
sitemap https://www.spi-discount.net/1_nl_0_sitemap.xml
Back to top

Comments

  • robots.txt automaticaly generated by PrestaShop e-commerce open-source solution
  • http://www.prestashop.com - http://www.prestashop.com/forums
  • This file is to prevent the crawling and indexing of certain parts
  • of your site by web crawlers and spiders run by sites like Yahoo!
  • and Google. By telling these "robots" where not to go on your site,
  • you save bandwidth and server resources.
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/wc/robots.html
  • Directories
  • Blog
  • Files
  • Sitemap
  • Robots SEO
  • Bloquage Robots indesirables
  • Ralentissement de robots moins utiles
  • Bloquage des outils de veille de prix
Back to top