Log In Sign Up

research.domaintools.com
robots.txt

Robots Exclusion Standard data for research.domaintools.com

Resource Scan

Scan Details

Site Domain research.domaintools.com
Base Domain domaintools.com
Scan Status Ok
Last Scan2024-05-24T07:29:18+00:00
Next Scan 2024-06-23T07:29:18+00:00

Last Scan

Scanned2024-05-24T07:29:18+00:00
URL https://research.domaintools.com/robots.txt
Domain IPs 199.30.228.113
Response IP 199.30.228.113
Found Yes
Hash 83231d90812cc72722f28194035ee27e43a3e2c2932fcfae2c2312be6c448462
SimHash 0e34e980d5a1

Groups

mediapartners-google

Rule Path
Disallow /go/
Disallow /dmoz/
Disallow /redirect/
Disallow /index.html?
Disallow /index.php?
Disallow /google-search/
Disallow /seo-browser/
Disallow /feed/
Disallow /-
Disallow /logout/
Disallow /corporate/
Disallow /research/traceroute/?query=
Disallow /research/ping/?query=
Disallow /research/dns/?query=

googlebot

Rule Path
Disallow /go/
Disallow /dmoz/
Disallow /redirect/
Disallow /index.html?
Disallow /index.php?
Disallow /google-search/
Disallow /seo-browser/
Disallow /feed/
Disallow /-
Disallow /logout/
Disallow /corporate/
Disallow /research/traceroute/?query=
Disallow /research/ping/?query=
Disallow /research/dns/?query=

bingbot

Rule Path
Disallow /go/
Disallow /dmoz/
Disallow /redirect/
Disallow /index.html?
Disallow /index.php?
Disallow /google-search/
Disallow /seo-browser/
Disallow /feed/
Disallow /-
Disallow /logout/
Disallow /corporate/
Disallow /research/traceroute/?query=
Disallow /research/ping/?query=
Disallow /research/dns/?query=

msnbot

Rule Path
Disallow /go/
Disallow /dmoz/
Disallow /redirect/
Disallow /index.html?
Disallow /index.php?
Disallow /google-search/
Disallow /seo-browser/
Disallow /feed/
Disallow /-
Disallow /logout/
Disallow /corporate/
Disallow /research/traceroute/?query=
Disallow /research/ping/?query=
Disallow /research/dns/?query=

slurp

Rule Path
Disallow /go/
Disallow /dmoz/
Disallow /redirect/
Disallow /index.html?
Disallow /index.php?
Disallow /google-search/
Disallow /seo-browser/
Disallow /feed/
Disallow /-
Disallow /logout/
Disallow /corporate/
Disallow /research/traceroute/?query=
Disallow /research/ping/?query=
Disallow /research/dns/?query=

teoma

Rule Path
Disallow /go/
Disallow /dmoz/
Disallow /redirect/
Disallow /index.html?
Disallow /index.php?
Disallow /google-search/
Disallow /seo-browser/
Disallow /feed/
Disallow /-
Disallow /logout/
Disallow /corporate/
Disallow /research/traceroute/?query=
Disallow /research/ping/?query=
Disallow /research/dns/?query=

baiduspider

Rule Path
Disallow /go/
Disallow /dmoz/
Disallow /redirect/
Disallow /index.html?
Disallow /index.php?
Disallow /google-search/
Disallow /seo-browser/
Disallow /feed/
Disallow /-
Disallow /logout/
Disallow /corporate/
Disallow /research/traceroute/?query=
Disallow /research/ping/?query=
Disallow /research/dns/?query=

facebookexternalhit/*

Rule Path
Disallow /go/
Disallow /dmoz/
Disallow /redirect/
Disallow /index.html?
Disallow /index.php?
Disallow /google-search/
Disallow /seo-browser/
Disallow /feed/
Disallow /-
Disallow /logout/
Disallow /corporate/
Disallow /research/traceroute/?query=
Disallow /research/ping/?query=
Disallow /research/dns/?query=

sogou

Rule Path
Disallow /go/
Disallow /dmoz/
Disallow /redirect/
Disallow /index.html?
Disallow /index.php?
Disallow /google-search/
Disallow /seo-browser/
Disallow /feed/
Disallow /-
Disallow /logout/
Disallow /corporate/
Disallow /research/traceroute/?query=
Disallow /research/ping/?query=
Disallow /research/dns/?query=

naverbot

Rule Path
Disallow /go/
Disallow /dmoz/
Disallow /redirect/
Disallow /index.html?
Disallow /index.php?
Disallow /google-search/
Disallow /seo-browser/
Disallow /feed/
Disallow /-
Disallow /logout/
Disallow /corporate/
Disallow /research/traceroute/?query=
Disallow /research/ping/?query=
Disallow /research/dns/?query=

seznambot

Rule Path
Disallow /go/
Disallow /dmoz/
Disallow /redirect/
Disallow /index.html?
Disallow /index.php?
Disallow /google-search/
Disallow /seo-browser/
Disallow /feed/
Disallow /-
Disallow /logout/
Disallow /corporate/
Disallow /research/traceroute/?query=
Disallow /research/ping/?query=
Disallow /research/dns/?query=

yandex

Rule Path
Disallow /go/
Disallow /dmoz/
Disallow /redirect/
Disallow /index.html?
Disallow /index.php?
Disallow /google-search/
Disallow /seo-browser/
Disallow /feed/
Disallow /-
Disallow /logout/
Disallow /corporate/
Disallow /research/traceroute/?query=
Disallow /research/ping/?query=
Disallow /research/dns/?query=

blekkobot

Rule Path
Disallow /go/
Disallow /dmoz/
Disallow /redirect/
Disallow /index.html?
Disallow /index.php?
Disallow /google-search/
Disallow /seo-browser/
Disallow /feed/
Disallow /-
Disallow /logout/
Disallow /corporate/
Disallow /research/traceroute/?query=
Disallow /research/ping/?query=
Disallow /research/dns/?query=

duckduckbot

Rule Path
Disallow /go/
Disallow /dmoz/
Disallow /redirect/
Disallow /index.html?
Disallow /index.php?
Disallow /google-search/
Disallow /seo-browser/
Disallow /feed/
Disallow /-
Disallow /logout/
Disallow /corporate/
Disallow /research/traceroute/?query=
Disallow /research/ping/?query=
Disallow /research/dns/?query=

yodaobot

Rule Path
Disallow /go/
Disallow /dmoz/
Disallow /redirect/
Disallow /index.html?
Disallow /index.php?
Disallow /google-search/
Disallow /seo-browser/
Disallow /feed/
Disallow /-
Disallow /logout/
Disallow /corporate/
Disallow /research/traceroute/?query=
Disallow /research/ping/?query=
Disallow /research/dns/?query=

surveybot

Rule Path
Disallow /go/
Disallow /dmoz/
Disallow /redirect/
Disallow /index.html?
Disallow /index.php?
Disallow /google-search/
Disallow /seo-browser/
Disallow /feed/
Disallow /-
Disallow /logout/
Disallow /corporate/
Disallow /research/traceroute/?query=
Disallow /research/ping/?query=
Disallow /research/dns/?query=

*

Rule Path
Disallow /
Back to top

Comments

  • Notice: if you would like to crawl DomainTools you can
  • contact us here: https://www.domaintools.com/contact
  • to apply for white listing.
Back to top