Log In Sign Up

drk7.jp
robots.txt

Robots Exclusion Standard data for drk7.jp

Resource Scan

Scan Details

Site Domain drk7.jp
Base Domain drk7.jp
Scan Status Ok
Last Scan2024-10-03T02:59:21+00:00
Next Scan 2024-10-10T02:59:21+00:00

Last Scan

Scanned2024-10-03T02:59:21+00:00
URL https://www.drk7.jp/robots.txt
Domain IPs 219.94.255.217
Response IP 219.94.255.217
Found Yes
Hash cb599b7c0fbba4ed4da2f3ab19550e8534a8b66a29eebcc5edaf376f38b349ba
SimHash 7022dee8c693

Groups

mediapartners-google

Rule Path
Disallow /admin/
Disallow /App/
Disallow /images/
Disallow /lib/
Disallow /logs/
Disallow /pub/
Disallow /MT/search/

*

Rule Path
Disallow /admin/
Disallow /App/
Disallow /images/
Disallow /lib/
Disallow /logs/
Disallow /cgi-bin/
Disallow /MT/search/
Disallow /MT/archives/mt-preview*

icc-crawler

Rule Path
Disallow /search/
Disallow /MT/search/
Back to top

Comments

  • robots, scram
  • Disallow: /search/
  • $ld: robots.txt v 1.2 1999/05/18 13:43:00 mreed Exp
  • Disallow: /pub/
  • Disallow: /search/
  • $ld: robots.txt v 1.2 1999/05/18 13:43:00 mreed Exp
  • User-agent: *
  • Crawl-delay: 120
  • User-agent: Slurp
  • Crawl-delay: 2
Back to top