Log In Sign Up

crm.capient.de
robots.txt

Robots Exclusion Standard data for crm.capient.de

Resource Scan

Scan Details

Site Domain crm.capient.de
Base Domain capient.de
Scan Status Ok
Last Scan2025-09-11T19:48:16+00:00
Next Scan 2025-09-25T19:48:16+00:00

Last Scan

Scanned2025-09-11T19:48:16+00:00
URL https://crm.capient.de/robots.txt
Domain IPs 94.130.131.41
Response IP 94.130.131.41
Found Yes
Hash 3e6ff632993737ca5799d3bac249f277f37f6ab036975f1166b37a932749c2e2
SimHash 8a4f66c461d2

Groups

teleport*

Rule Path
Disallow /

webwhacker

Rule Path
Disallow /

webdevil

Rule Path
Disallow /

webzip

Rule Path
Disallow /

net attache

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

wx_mail/2.000

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

whowhere

Rule Path
Disallow /

roverbot

Rule Path
Disallow /

activeagent

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

websitewiki

Rule Path
Disallow /
Back to top

Comments

  • Robot Exclusion File -- robots.txt
  • digital//m 2019
  • Keep Rover from grabbing Email addresses from our site
Back to top