1001degustations.com
robots.txt

Robots Exclusion Standard data for 1001degustations.com

Resource Scan

Scan Details

Site Domain 1001degustations.com
Base Domain 1001degustations.com
Scan Status Ok
Last Scan2024-10-09T11:19:04+00:00
Next Scan 2024-11-08T11:19:04+00:00

Last Scan

Scanned2024-10-09T11:19:04+00:00
URL https://1001degustations.com/robots.txt
Redirect https://www.1001degustations.com/robots.txt
Redirect Domain www.1001degustations.com
Redirect Base 1001degustations.com
Domain IPs 109.234.166.160
Redirect IPs 109.234.166.160
Response IP 109.234.166.160
Found Yes
Hash 7ca67f3e34f20dcb7d95b5c4c52c21f00b448843684aad23e852f51ff1c54bb1
SimHash b25e940bc910

Groups

nerdybot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

linkdex.com

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

shopperreports

Rule Path
Disallow /

spinn3r

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

uptimerobot/2.0

Rule Path
Disallow /

perl lwp

Rule Path
Disallow /

wiseguys robot

Rule Path
Disallow /

turnitin robot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

*

Rule Path
Disallow /pro/my-account
Disallow /%5BLANG%5D/pro/my-account
Disallow /taster/my-account
Disallow /%5BLANG%5D/taster/my-account
Disallow /espace
Disallow /%5BLANG%5D/espace

Comments

  • Ceci est un fichier généré automatiquement
  • cf CreateRobotsTxtCommand pour en modifier le contenu