Log In Sign Up

p3dhack.ru
robots.txt

Robots Exclusion Standard data for p3dhack.ru

Resource Scan

Scan Details

Site Domain p3dhack.ru
Base Domain p3dhack.ru
Scan Status Ok
Last Scan2025-09-17T19:11:58+00:00
Next Scan 2025-09-24T19:11:58+00:00

Last Scan

Scanned2025-09-17T19:11:58+00:00
URL https://p3dhack.ru/robots.txt
Domain IPs 104.21.3.56, 172.67.130.70, 2606:4700:3033::6815:338, 2606:4700:3036::ac43:8246
Response IP 172.67.130.70
Found Yes
Hash b7adcdc8f5e2bc49aad9f2304bd07df47018e09dda29bc1c7a028f9679ee266c
SimHash b61c514bc6f7

Groups

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

yandex

Rule Path
Allow /applications/core/interface/font/
Disallow /applications/
Disallow /datastore/
Disallow /plugins/
Disallow /_piwik/
Disallow /system/
Disallow /Credits.txt
Disallow /upgrading.html
Disallow /login/
Disallow /register/
Disallow /lostpassword/
Disallow /search/
Disallow /online/
Disallow /contact/
Disallow /activity/
Disallow /discover/
Disallow /?tab=*
Disallow /*?app=*
Disallow /*sortby%3D*
Disallow /profile/*/?do=*
Disallow /profile/*/content/
Disallow /clients/info/
Disallow /admin/

fast

Rule Path
Disallow /

wget

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

npbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

fasterfox

Rule Path
Disallow /

*

Rule Path
Disallow /cgi-bin/
Allow /applications/core/interface/font/
Disallow /applications/
Disallow /datastore/
Disallow /plugins/
Disallow /_piwik/
Disallow /system/
Disallow /error.html
Disallow /Credits.txt
Disallow /upgrading.html
Disallow /login/
Disallow /register/
Disallow /lostpassword/
Disallow /search/
Disallow /online/
Disallow /contact/
Disallow /activity/
Disallow /discover/
Disallow /admin/
Disallow /?tab=*
Disallow /*?app=*
Disallow /*sortby%3D*
Disallow /profile/*/?do=*
Disallow /profile/*/content/
Back to top

Other Records

Field Value
sitemap https://p3dhack.ru/sitemap.php
Back to top

Comments

  • robots.txt for https://p3dhack.ru/
  • By: Defaul Fox
  • Sitemap...
  • Host...
  • Crawlers that are kind enough to obey, but which we had rather not have
  • unless they are feeding search engines.
  • Some bots are known to be trouble, particularly those designed to copy
  • entire sites. Please obey robots.txt.
  • Misbehaving: requests much too fast:
  • Sorry, wget in its recursive mode is a frequent problem.
  • Please read the man page and use it properly; there is a
  • --wait option you can use to set the delay between hits,
  • for instance.
  • The Grub distributed client has been *very* poorly behaved.
  • no follow robots.txt anyway, but...
  • Hits many times per second, not acceptable
  • http://www.nameprotect.com/botinfo.html
  • A capture bot, downloads gazillions of pages with no public benefit
  • http://www.webreaper.net/
  • deny access to Wayback Machine
  • and also to Fasterfox prefetching
  • All others...
  • No cgi-bin
  • No ipb-file&directory
  • No also this...
  • ... And this!
Back to top

Warnings

  • `host` is not a known field.
Back to top