1awww.info
robots.txt

Robots Exclusion Standard data for 1awww.info

Resource Scan

Scan Details

Site Domain 1awww.info
Base Domain 1awww.info
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-09-06T19:14:00+00:00
Next Scan 2024-12-05T19:14:00+00:00

Last Successful Scan

Scanned2024-04-17T18:32:19+00:00
URL https://1awww.info/robots.txt
Redirect https://www.1awww.info/robots.txt
Redirect Domain www.1awww.info
Redirect Base 1awww.info
Domain IPs 176.31.12.52
Redirect IPs 176.31.12.52
Response IP 176.31.12.52
Found Yes
Hash 60ecbb98b2ff0ce168c75e01ba86c4f563bdf9e5c9b869779de8bfa699b4bdab
SimHash e81cd3c0f31d

Groups

*

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 300

applebot

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 60

baiduspider

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 60

bingbot

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 60

google-news

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 10

google-sitemaps

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 100

google-video

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 150

googlebot

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 5

googlebot-image

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 60

mediapartners-google

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 60

msnbot

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 100

msnbot-media/1.1

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 150

msnbot-media/2.0b

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 150

naverbot

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 300

seznambot

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 300

slurp

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 100

teoma

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 100

twitterbot

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 100

yandex

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 600

yeti

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 100

advanced email extractor

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

alphabot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

atspider

Rule Path
Disallow /

autoemailspider

Rule Path
Disallow /

biglotron

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bogospider

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

contactbot

Rule Path
Disallow /

contentsmartz

Rule Path
Disallow /

datacha0s

Rule Path
Disallow /

dbrowse

Rule Path
Disallow /

demo bot dot 16b

Rule Path
Disallow /

demo bot z 16b

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

dsurf15a

Rule Path
Disallow /

eapollobot

Rule Path
Disallow /

ebrowse

Rule Path
Disallow /

educate search vxb

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailspider

Rule Path
Disallow /

emeraldshield.com web spider

Rule Path
Disallow /

esurf15a

Rule Path
Disallow /

extractorpro

Rule Path
Disallow /

favorstarbot

Rule Path
Disallow /

findxbot

Rule Path
Disallow /

franklin locator

Rule Path
Disallow /

fsurf15a

Rule Path
Disallow /

full web bot

Rule Path
Disallow /

goforitbot

Rule Path
Disallow /

guestbook auto submitter

Rule Path
Disallow /

industry program

Rule Path
Disallow /

indy library

Rule Path
Disallow /

iplexx spider

Rule Path
Disallow /

isc systems irc search

Rule Path
Disallow /

iupui research bot

Rule Path
Disallow /

java

Rule Path
Disallow /

jyxobot

Rule Path
Disallow /

kbeebot

Rule Path
Disallow /

larbin-experimental

Rule Path
Disallow /

letscrawl.com

Rule Path
Disallow /

lincoln state web browser

Rule Path
Disallow /

lmqueuebot

Rule Path
Disallow /

mac finder

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

memorybot

Rule Path
Disallow /

microsoft url control

Rule Path
Disallow /

missauga locate

Rule Path
Disallow /

missigua locator

Rule Path
Disallow /

missouri college browse

Rule Path
Disallow /

mizzu labs

Rule Path
Disallow /

mfc foundation class library

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mo college

Rule Path
Disallow /

mvaclient

Rule Path
Disallow /

nameofagent

Rule Path
Disallow /

nasa search

Rule Path
Disallow /

netresearchserver

Rule Path
Disallow /

noxtrumbot

Rule Path
Disallow /

nsauditor

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

pbrowse

Rule Path
Disallow /

peval

Rule Path
Disallow /

php version tracker

Rule Path
Disallow /

php version tracker

Rule Path
Disallow /

pingdom

Rule Path
Disallow /

pingdom gigrib

Rule Path
Disallow /

poirot

Rule Path
Disallow /

port huron labs

Rule Path
Disallow /

privoxy

Rule Path
Disallow /

production bot

Rule Path
Disallow /

program shareware

Rule Path
Disallow /

psurf15a

Rule Path
Disallow /

psycheclone

Rule Path
Disallow /

pycurl

Rule Path
Disallow /

rsurf15a

Rule Path
Disallow /

scan4mail

Rule Path
Disallow /

searchbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

shablastbot

Rule Path
Disallow /

snapbot

Rule Path
Disallow /

snap.com

Rule Path
Disallow /

sogou develop spider

Rule Path
Disallow /

sogou orion spider

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

sogou music spider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sohu agent

Rule Path
Disallow /

spbot

Rule Path
Disallow /

ssurf15a

Rule Path
Disallow /

steeler

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

tools.ua.random

Rule Path
Disallow /

tsurf15a

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

under the rainbow

Rule Path
Disallow /

uptimebot

Rule Path
Disallow /

vadixbot

Rule Path
Disallow /

webvulncrawl

Rule Path
Disallow /

webbot

Rule Path
Disallow /

wells search

Rule Path
Disallow /

wep search

Rule Path
Disallow /

local231031

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

Comments

  • Robots.txt (c) by 1awww.com 2017 - Vor Anpassungen lesen Sie bitte die Hilfe-Dokumentation unter https://www.1awww.info/de/service-support/fragen-und-antworten/2-hosting/robots-txt-ein-zwingendes-muss.html
  • zu viele Disallows sind grundsaetzlich schadlos, aber verhindern das Scannen und Indizieren dieses Inhaltes!
  • Disallows, die zugefuegt werden, muessen fuer jede Suchmaschine / Bot zugefuegt werden !!!