sudoku-land.com
robots.txt

Robots Exclusion Standard data for sudoku-land.com

Resource Scan

Scan Details

Site Domain sudoku-land.com
Base Domain sudoku-land.com
Scan Status Ok
Last Scan2025-05-27T16:55:14+00:00
Next Scan 2025-06-26T16:55:14+00:00

Last Scan

Scanned2025-05-27T16:55:14+00:00
URL http://sudoku-land.com/robots.txt
Redirect http://www.sudoku-land.com/robots.txt
Redirect Domain www.sudoku-land.com
Redirect Base sudoku-land.com
Domain IPs 217.160.0.48
Redirect IPs 217.160.0.48
Response IP 217.160.0.48
Found Yes
Hash 0f45ac46b3bb217513dd75456e4d620fae158fdc220e9e128a4dfcd3c0906af3
SimHash 917f94d64db8

Groups

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

webbandit

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

extractorpro

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

lnspiderguy

Rule Path
Disallow /

black hole

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

cherrypicker

Rule Path
Disallow /

omniexplorer_bot

Rule Path
Disallow /

biglotron (beta 2;gnu/linux)

Rule Path
Disallow /

*

Rule Path
Disallow /admin/
Disallow /Connections/

Other Records

Field Value
crawl-delay 20

Comments

  • Protection Robots
  • Robots à interdire en totalité
  • Règle valable pour tous les robots
  • Délai minimum entre chaque nouvelle visite de robot
  • Minimum (en secondes) entre chaque page crawlée
  • Interdiction de crawler les répertoires suivants
  • Fichier à compéter si besoin

Warnings

  • `revisit-after 60 mins` is not a known field.