Log In Sign Up

achatpublic.com
robots.txt

Robots Exclusion Standard data for achatpublic.com

Resource Scan

Scan Details

Site Domain achatpublic.com
Base Domain achatpublic.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-06-01T04:00:36+00:00
Next Scan 2024-07-01T04:00:36+00:00

Last Successful Scan

Scanned2024-04-10T03:55:54+00:00
URL https://achatpublic.com/robots.txt
Redirect https://www.achatpublic.com/robots.txt
Redirect Domain www.achatpublic.com
Redirect Base achatpublic.com
Domain IPs 91.232.40.52
Redirect IPs 91.232.40.52
Response IP 91.232.40.52
Found Yes
Hash 4e68706e2a20ad104f78984926365ce3233af2619fbf89539548fe1a7e9ed8c9
SimHash 28945d884f74

Groups

*

Rule Path
Disallow /includes/
Disallow /misc/
Disallow /modules/
Disallow /profiles/
Disallow /scripts/
Disallow /sites/
Disallow /themes/
Disallow /CHANGELOG.txt
Disallow /cron.php
Disallow /INSTALL.mysql.txt
Disallow /INSTALL.pgsql.txt
Disallow /install.php
Disallow /INSTALL.txt
Disallow /LICENSE.txt
Disallow /MAINTAINERS.txt
Disallow /update.php
Disallow /UPGRADE.txt
Disallow /xmlrpc.php
Disallow /admin/
Disallow /comment/reply/
Disallow /contact/
Disallow /logout/
Disallow /node/add/
Disallow /search/
Disallow /user/register/
Disallow /user/password/
Disallow /user/login/
Disallow /?q=admin%2F
Disallow /?q=comment%2Freply%2F
Disallow /?q=contact%2F
Disallow /?q=logout%2F
Disallow /?q=node%2Fadd%2F
Disallow /?q=search%2F
Disallow /?q=user%2Fpassword%2F
Disallow /?q=user%2Fregister%2F
Disallow /?q=user%2Flogin%2F

Other Records

Field Value
crawl-delay 10

converacrawler

Rule Path
Disallow /

quepasacreep

Rule Path
Disallow /

jetbot

Rule Path
Disallow /

newsnow

Rule Path
Disallow /

tunitinbot

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

knowings d

Rule Path
Disallow /

kbcrawl

Rule Path
Disallow /

wget

Rule Path
Disallow /

newzbin

Rule Path
Disallow /

zite

Rule Path
Disallow /

kbcrawl

Rule Path
Disallow /

readability.com

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

libwww

Rule Path
Disallow /

wget

Rule Path
Disallow /

adequat

Rule Path
Disallow /

adequat-systems

Rule Path
Disallow /

moreover

Rule Path
Disallow /

verticalsearch

Rule Path
Disallow /

vsw

Rule Path
Disallow /

fetch

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

asknread.com

Rule Path
Disallow /
Back to top

Comments

  • $Id: robots.txt,v 1.9.2.1 2008/12/10 20:12:19 goba Exp $
  • robots.txt
  • This file is to prevent the crawling and indexing of certain parts
  • of your site by web crawlers and spiders run by sites like Yahoo!
  • and Google. By telling these "robots" where not to go on your site,
  • you save bandwidth and server resources.
  • This file will be ignored unless it is at the root of your host:
  • Used: http://example.com/robots.txt
  • Ignored: http://example.com/site/robots.txt
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/wc/robots.html
  • For syntax checking, see:
  • http://www.sxw.org.uk/computing/robots/check.html
  • Directories
  • Files
  • Paths (clean URLs)
  • Paths (no clean URLs)
  • Robots exclus de toute indexation.
Back to top

Warnings

  • 2 invalid lines.
Back to top