waterwheel.com
robots.txt

Robots Exclusion Standard data for waterwheel.com

Resource Scan

Scan Details

Site Domain waterwheel.com
Base Domain waterwheel.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2025-12-21T16:09:49+00:00
Next Scan 2026-03-21T16:09:49+00:00

Last Successful Scan

Scanned2025-02-02T07:53:31+00:00
URL https://waterwheel.com/robots.txt
Domain IPs 104.21.83.248, 172.67.183.158, 2606:4700:3034::6815:53f8, 2606:4700:3034::ac43:b79e
Response IP 172.67.183.158
Found Yes
Hash 23bbb9cfe4841873465d5ce46ca4575517d6fede72a1cc287d8c7479ddfaf548
SimHash fd79d1b6a793

Groups

*

Rule Path
Disallow /_fpclass/
Disallow /_private/
Disallow /_themes/
Disallow /_derived/
Disallow /_vti_cnf/
Disallow /_vti_pvt/
Disallow /images/
Disallow /pages/
Disallow /cgi-bin/

mediapartners-google*

Rule Path
Disallow

atspider

Rule Path
Disallow /

cherrypicker

Rule Path
Disallow /

dsurf

Rule Path
Disallow /

elitesys entry

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

mail sweeper

Rule Path
Disallow /

munky

Rule Path
Disallow /

roverbot

Rule Path
Disallow /

webemailextrac

Rule Path
Disallow /

Comments

  • Disallow Collectors and Spam