itcpiercing.com
robots.txt

Robots Exclusion Standard data for itcpiercing.com

Resource Scan

Scan Details

Site Domain itcpiercing.com
Base Domain itcpiercing.com
Scan Status Ok
Last Scan2024-11-15T12:39:15+00:00
Next Scan 2024-11-29T12:39:15+00:00

Last Scan

Scanned2024-11-15T12:39:15+00:00
URL https://itcpiercing.com/robots.txt
Redirect https://www.itcpiercing.com/robots.txt
Redirect Domain www.itcpiercing.com
Redirect Base itcpiercing.com
Domain IPs 104.26.2.60, 104.26.3.60, 172.67.72.12, 2606:4700:20::681a:23c, 2606:4700:20::681a:33c, 2606:4700:20::ac43:480c
Redirect IPs 104.26.2.60, 104.26.3.60, 172.67.72.12, 2606:4700:20::681a:23c, 2606:4700:20::681a:33c, 2606:4700:20::ac43:480c
Response IP 104.26.2.60
Found Yes
Hash 238fc358dd3416458ab5c908c65877cad3769fe7d0272371288bddc2a20a75cf
SimHash a01453d20e86

Groups

*

Rule Path
Disallow /lib/
Disallow /*.php$
Disallow /pkginfo/
Disallow /report/
Disallow /var/
Disallow /catalog/
Disallow /customer/
Disallow /sendfriend/
Disallow /review/
Disallow /*SID%3D
Allow /*.css$
Allow /*.css?
Allow /*.js$
Allow /*.js?
Allow /*.gif
Allow /*.jpg
Allow /*.jpeg
Allow /*.png
Allow /*.svg
Allow /*.webp
Allow /*.mp4
Disallow /customer/*
Disallow /checkout/*
Disallow /productalert/*
Disallow /catalogsearch/*
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow *?q=*
Disallow *?xx_FiltreCouleur=*
Disallow /*?*product_list_mode=
Disallow /*?*product_list_order=
Disallow /*?*product_list_limit=
Disallow /*?*product_list_dir=
Disallow *?mc=*
Disallow *?start=*
Disallow /*?*xx_*
Allow /*?p=
Disallow /app/
Disallow /bin/
Disallow /dev/
Disallow /lib/
Disallow /phpserver/
Disallow /pub/
Disallow /tag/
Disallow /review/
Disallow /composer.json
Disallow /composer.lock
Disallow /CONTRIBUTING.md
Disallow /CONTRIBUTOR_LICENSE_AGREEMENT.html
Disallow /COPYING.txt
Disallow /Gruntfile.js
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /nginx.conf.sample
Disallow /package.json
Disallow /php.ini.sample
Disallow /RELEASE_NOTES.txt
Disallow /*.git
Disallow /*.CVS
Disallow /*.Zip$
Disallow /*.Svn$
Disallow /*.Idea$
Disallow /*.Sql$
Disallow /*.Tgz$

adequat

Rule Path
Disallow /

admantx bot

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

corporama

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

dotmic dotbot

Rule Path
Disallow /

ellisphere

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

fetch

Rule Path
Disallow /

grapeshot bot

Rule Path
Disallow /

houzzbot

Rule Path
Disallow /

infoseek

Rule Path
Disallow /

jetbot

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

leadbox

Rule Path
Disallow /

linkfluence

Rule Path
Disallow /

linko

Rule Path
Disallow /

mail.ru bot

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

npbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

opinion-tracker

Rule Path
Disallow /

proxem

Rule Path
Disallow /

proximic bot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

spotter

Rule Path
Disallow /

sputnikbot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

teleport

Rule Path
Disallow /

updownerbot

Rule Path
Disallow /

verticalsearch

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webzip

Rule Path
Disallow /

wget

Rule Path
Disallow /

winello

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

baiduspider

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

yandexbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

archive.org_bot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

mj12bot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

ahrefsbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

soso spider

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

sogou spider

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

net veille

Rule Path
Disallow /

paarly

Rule Path
Disallow /

spycommerce

Rule Path
Disallow /

pricing assistant

Rule Path
Disallow /

price observatory

Rule Path
Disallow /

workit

Rule Path
Disallow /

keox

Rule Path
Disallow /

pricejet

Rule Path
Disallow /

prixan

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.itcpiercing.com/media/sitemap/sitemap_itc.xml

Comments

  • Files & Scripts
  • Disable checkout & customer account
  • Disable Search pages
  • Disable useless parameters
  • Allow pagination
  • Disable common folders
  • Disable Tag & Review (Avoid duplicate content)
  • Common files
  • Disable version control folders and others
  • Bloquage Robots indesirables
  • Ralentissement de robots moins utiles
  • Bloquage des outils de veille de prix