Log In Sign Up

1awww.com
robots.txt

Robots Exclusion Standard data for 1awww.com

Resource Scan

Scan Details

Site Domain 1awww.com
Base Domain 1awww.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-09-05T18:16:27+00:00
Next Scan 2024-10-05T18:16:27+00:00

Last Successful Scan

Scanned2024-07-15T18:14:55+00:00
URL https://1awww.com/robots.txt
Redirect https://www.1awww.com/robots.txt
Redirect Domain www.1awww.com
Redirect Base 1awww.com
Domain IPs 176.31.12.52
Redirect IPs 176.31.12.52
Response IP 176.31.12.52
Found Yes
Hash e6ce70f1410fc5697d73aea7e7a5ff1925ad5e4f3fe9fca5199295310240a37e
SimHash e81cd1c0f35d

Groups

*

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 300

applebot

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 60

baiduspider

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 60

bingbot

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 60

google-news

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 10

google-sitemaps

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 100

google-video

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 150

googlebot

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 5

googlebot-image

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 60

mediapartners-google

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 60

msnbot

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 100

msnbot-media/1.1

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 150

msnbot-media/2.0b

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 150

naverbot

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 300

seznambot

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 300

slurp

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 100

teoma

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 100

twitterbot

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 100

yandex

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 600

yeti

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /cache/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cdi/
Disallow /components/
Disallow /libraries/
Disallow /includes/
Disallow /modules/
Disallow /templates/
Disallow /temp/
Disallow /tmp/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /cart/
Disallow /checkout/
Disallow /wp-content/
Disallow /my-account/
Disallow /wp-login.php*

Other Records

Field Value
crawl-delay 100

advanced email extractor

Rule Path
Disallow /

atspider

Rule Path
Disallow /

autoemailspider

Rule Path
Disallow /

biglotron

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bogospider

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

contactbot

Rule Path
Disallow /

contentsmartz

Rule Path
Disallow /

datacha0s

Rule Path
Disallow /

dbrowse

Rule Path
Disallow /

demo bot dot 16b

Rule Path
Disallow /

demo bot z 16b

Rule Path
Disallow /

dsurf15a

Rule Path
Disallow /

eapollobot

Rule Path
Disallow /

ebrowse

Rule Path
Disallow /

educate search vxb

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailspider

Rule Path
Disallow /

emeraldshield.com web spider

Rule Path
Disallow /

esurf15a

Rule Path
Disallow /

extractorpro

Rule Path
Disallow /

favorstarbot

Rule Path
Disallow /

findxbot

Rule Path
Disallow /

franklin locator

Rule Path
Disallow /

fsurf15a

Rule Path
Disallow /

full web bot

Rule Path
Disallow /

goforitbot

Rule Path
Disallow /

guestbook auto submitter

Rule Path
Disallow /

industry program

Rule Path
Disallow /

indy library

Rule Path
Disallow /

iplexx spider

Rule Path
Disallow /

isc systems irc search

Rule Path
Disallow /

iupui research bot

Rule Path
Disallow /

java

Rule Path
Disallow /

jyxobot

Rule Path
Disallow /

kbeebot

Rule Path
Disallow /

larbin-experimental

Rule Path
Disallow /

letscrawl.com

Rule Path
Disallow /

lincoln state web browser

Rule Path
Disallow /

lmqueuebot

Rule Path
Disallow /

mac finder

Rule Path
Disallow /

memorybot

Rule Path
Disallow /

microsoft url control

Rule Path
Disallow /

missauga locate

Rule Path
Disallow /

missigua locator

Rule Path
Disallow /

missouri college browse

Rule Path
Disallow /

mizzu labs

Rule Path
Disallow /

mfc foundation class library

Rule Path
Disallow /

mo college

Rule Path
Disallow /

mvaclient

Rule Path
Disallow /

nameofagent

Rule Path
Disallow /

nasa search

Rule Path
Disallow /

netresearchserver

Rule Path
Disallow /

noxtrumbot

Rule Path
Disallow /

nsauditor

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

pbrowse

Rule Path
Disallow /

peval

Rule Path
Disallow /

php version tracker

Rule Path
Disallow /

php version tracker

Rule Path
Disallow /

pingdom

Rule Path
Disallow /

pingdom gigrib

Rule Path
Disallow /

poirot

Rule Path
Disallow /

port huron labs

Rule Path
Disallow /

privoxy

Rule Path
Disallow /

production bot

Rule Path
Disallow /

program shareware

Rule Path
Disallow /

psurf15a

Rule Path
Disallow /

psycheclone

Rule Path
Disallow /

pycurl

Rule Path
Disallow /

rsurf15a

Rule Path
Disallow /

scan4mail

Rule Path
Disallow /

searchbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

shablastbot

Rule Path
Disallow /

snapbot

Rule Path
Disallow /

snap.com

Rule Path
Disallow /

sogou develop spider

Rule Path
Disallow /

sogou orion spider

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

sogou music spider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sohu agent

Rule Path
Disallow /

spbot

Rule Path
Disallow /

ssurf15a

Rule Path
Disallow /

steeler

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

tools.ua.random

Rule Path
Disallow /

tsurf15a

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

under the rainbow

Rule Path
Disallow /

uptimebot

Rule Path
Disallow /

vadixbot

Rule Path
Disallow /

webvulncrawl

Rule Path
Disallow /

webbot

Rule Path
Disallow /

wells search

Rule Path
Disallow /

wep search

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

alphabot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

local231031

Rule Path
Disallow /
Back to top

Comments

  • Robots.txt (c) by 1awww.com 2017 - Vor Anpassungen lesen Sie bitte die Hilfe-Dokumentation unter https://www.1awww.info/de/service-support/fragen-und-antworten/2-hosting/robots-txt-ein-zwingendes-muss.html
  • zu viele Disallows sind grundsaetzlich schadlos, aber verhindern das Scannen und Indizieren dieses Inhaltes!
  • Disallows, die zugefuegt werden, muessen fuer jede Suchmaschine / Bot zugefuegt werden !!!
Back to top