smoking.fr
robots.txt

Robots Exclusion Standard data for smoking.fr

Resource Scan

Scan Details

Site Domain smoking.fr
Base Domain smoking.fr
Scan Status Ok
Last Scan2024-09-13T03:52:52+00:00
Next Scan 2024-10-13T03:52:52+00:00

Last Scan

Scanned2024-09-13T03:52:52+00:00
URL https://smoking.fr/robots.txt
Redirect https://www.smoking.fr/robots.txt
Redirect Domain www.smoking.fr
Redirect Base smoking.fr
Domain IPs 51.83.14.54
Redirect IPs 51.83.14.54
Response IP 51.83.14.54
Found Yes
Hash 9efc87d9ba7ed547c3567dd73eaba17f71297e1ac7610fb5121206376b17a119
SimHash 401f13e2eecb

Groups

*

Rule Path
Disallow /advanced_search_result.php
Disallow /advanced_search.php
Disallow /-c-*
Disallow /-pr-*
Disallow /shop_by_price.php
Disallow /includes
Disallow /cgi-bin
Disallow /account.php
Disallow /account_edit.php
Disallow /account_history.php
Disallow /account_history_info.php
Disallow /account_password.php
Disallow /add_checkout_success.php
Disallow /address_book.php
Disallow /address_book_process.php
Disallow /checkout_confirmation.php
Disallow /checkout_payment.php
Disallow /checkout_payment_address.php
Disallow /checkout_process.php
Disallow /checkout_shipping.php
Disallow /checkout_shipping_address.php
Disallow /checkout_success.php
Disallow /contact_bean.php
Disallow /cookie_usage.php
Disallow /create_account.php
Disallow /create_account_success.php
Disallow /login.php
Disallow /password_forgotten.php
Disallow /popup_image.php
Disallow /shopping_cart.php
Disallow /product_reviews_write.php
Disallow /recherche-populaire.php
Disallow /newsletter
Disallow /popup_privacy.php
Disallow /news-perso/*

mauibot (crawler.feedback+wc@gmail.com)

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

textbulkerbot

Rule Path
Disallow /

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

semrushbot-sa

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

adequat

Rule Path
Disallow /

admantx bot

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

corporama

Rule Path
Disallow /

dotmic dotbot

Rule Path
Disallow /

ellisphere

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

fetch

Rule Path
Disallow /

grapeshot bot

Rule Path
Disallow /

houzzbot

Rule Path
Disallow /

infoseek

Rule Path
Disallow /

jetbot

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

leadbox

Rule Path
Disallow /

linkfluence

Rule Path
Disallow /

linko

Rule Path
Disallow /

mail.ru bot

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

npbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

opinion-tracker

Rule Path
Disallow /

proxem

Rule Path
Disallow /

proximic bot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

spotter

Rule Path
Disallow /

sputnikbot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

teleport

Rule Path
Disallow /

updownerbot

Rule Path
Disallow /

verticalsearch

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webzip

Rule Path
Disallow /

wget

Rule Path
Disallow /

winello

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

archive.org_bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

soso spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

sogou spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

net veille

Rule Path
Disallow /

paarly

Rule Path
Disallow /

spycommerce

Rule Path
Disallow /

pricing assistant

Rule Path
Disallow /

price observatory

Rule Path
Disallow /

workit

Rule Path
Disallow /

keox

Rule Path
Disallow /

pricejet

Rule Path
Disallow /

prixan

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.smoking.fr/sitemap-index.xml

Comments

  • Robots SEO
  • User-agent: GPTBot
  • Disallow: /
  • Bloquage Robots indesirables
  • Ralentissement de robots moins utiles
  • Bloquage des outils de veille de prix