son-video.com
robots.txt

Robots Exclusion Standard data for son-video.com

Resource Scan

Scan Details

Site Domain son-video.com
Base Domain son-video.com
Scan Status Ok
Last Scan2024-09-18T23:35:38+00:00
Next Scan 2024-10-18T23:35:38+00:00

Last Scan

Scanned2024-09-18T23:35:38+00:00
URL https://son-video.com/robots.txt
Redirect https://www.son-video.com/robots.txt
Redirect Domain www.son-video.com
Redirect Base son-video.com
Domain IPs 18.198.145.80, 3.123.74.196, 52.57.148.76
Redirect IPs 18.198.145.80, 3.123.74.196, 52.57.148.76
Response IP 52.57.148.76
Found Yes
Hash ee07b1d6f551796cbd1c61b46c77a351b97349c27dd8f57528f518668e0684d7
SimHash 6b3e78508ab2

Groups

*

Rule Path
Disallow /recherche
Disallow /api/
Disallow /autocompletion/api
Disallow /mon-panier/ajouter
Disallow /mon-panier/ajouter-garantie
Disallow /mon-panier/supprimer-garantie
Disallow /mon-panier/ajouter-accessoire
Disallow /mon-panier/supprimer-accessoire
Disallow /mon-panier/ajouter-promo
Disallow /mon-panier/retirer-promo
Disallow /mon-panier/mettre-a-jour-quantite-accessoire
Disallow /question-article/
Disallow /questions-details/
Disallow /avis-article/
Disallow /repondre-a-une-question/
Disallow /*?filtres=
Disallow /*%26filtres%3D

boitho.com-dc
cazoodlebot
converacrawler
discobot
dotbot
fasterfox
gigabot
heritrix
huaweisymantecspider
ichiro
irlbot
kalooga
lexxebot
mj12bot
mlbot
msiecrawler
naverbot
nutch
ocelli
penthesilea
phpdig
scoutjet
searchdaimon.com-dc
shopwiki
snapbot
turnitinbot
twiceler
twengabot
woozweb-monitoring
woriobot
yacybot
yacybit
yandexbot
yandex
yeti
yodaobot
zyborg
dataforseobot
ahrefsbot
blexbot
ccbot
chatgpt-user
google-extended
gptbot
claudebot

Rule Path
Disallow /

mojeekbot
petalbot
buck/*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://www.son-video.com/sitemap.xml