htae.net
robots.txt

Robots Exclusion Standard data for htae.net

Resource Scan

Scan Details

Site Domain htae.net
Base Domain htae.net
Scan Status Ok
Last Scan2024-11-16T17:59:33+00:00
Next Scan 2024-11-23T17:59:33+00:00

Last Scan

Scanned2024-11-16T17:59:33+00:00
URL https://www.htae.net/robots.txt
Domain IPs 142.251.10.121, 2404:6800:4003:c04::79
Response IP 74.125.130.121
Found Yes
Hash af4658bdcd487a2db4132e1564aaf92ac0075db648c3b389d03c5e2deaf45182
SimHash c9246de36698

Groups

megaindex.ru

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

coccocbot-image

Rule Path
Disallow /

coccocbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

quantify

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

dnyzbot/1.0

Rule Path
Disallow /

dnyzbot

Rule Path
Disallow /

linkwalker

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

lightspeedsystemscrawler

Rule Path
Disallow /

baiduspider-news

Rule Path
Disallow /

baiduspider-favo

Rule Path
Disallow /

baiduspider-cpro

Rule Path
Disallow /

baiduspider-ads

Rule Path
Disallow /

baidu

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

screenerbot

Rule Path
Disallow /

mrsputnik

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

urllib

Rule Path
Disallow /

nutch

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

vscooter

Rule Path
Disallow /

psbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

taptubot

Rule Path
Disallow /

twengabot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

infopath

Rule Path
Disallow /

infopath.2

Rule Path
Disallow /

swebot

Rule Path
Disallow /

ec2linkfinder

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

searchmetericsbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

ip-web-crawler.com

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

aboundex

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

spbot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

a6-indexer

Rule Path
Disallow /

riddler

Rule Path
Disallow /

loadtimebot

Rule Path
Disallow /

obot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

memorybot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

advbot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

lssrocketcrawler

Rule Path
Disallow /

gsa-crawler

Rule Path
Disallow /

tbot-nutch

Rule Path
Disallow /

thunderstone

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

ranksonicbot

Rule Path
Disallow /

betabot

Rule Path
Disallow /

parsijoo-bot

Rule Path
Disallow /

nextgensearchbot

Rule Path
Disallow /

gocrawl

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

applebot

Rule Path
Disallow /

lipperhey

Rule Path
Disallow /

safednsbot

Rule Path
Disallow /

rankactivelinkbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

uptimebot

Rule Path
Disallow /

seeker

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

yoozbot

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

findxbot

Rule Path
Disallow /

garlikcrawler

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

jamesbot

Rule Path
Disallow /

contacts-crawler

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

idbot

Rule Path
Disallow /

*

Rule Path
Disallow /ICALOutput/*
Disallow /CSVtask_all_list/
Disallow /requestemailverification/
Disallow /accounts/confirm/
Disallow /userprofile_list/
Disallow /userprofile/
Disallow /locked/
Disallow /pingit/
Disallow /sign_out/
Disallow /saveVote/
Disallow /campaign/
Disallow /onlinediary/
Disallow /stylesheet.css
Disallow /address_book/
Disallow /taskmanagementmethodology/
Disallow /htaeonlinetasklist/
Disallow /client/update/
Disallow /addressbook_list/
Disallow /task_mypriority_list/
Disallow /questionlizard/
Disallow /yesnogame/
Disallow /thingstodotoday/
Disallow /clientRelationshipCredits_add/
Disallow /relationship_invite/
Disallow /clientrelationship_create/
Disallow /invite_approval/
Disallow /globalprojects_list/
Disallow /task_event_list/
Disallow /relationship_approve/
Disallow /relationshipinvites_received/
Disallow /relationshipinvites_sent/
Disallow /relationshipinvites_awaitingapproval/
Disallow /publishedProformaReview_list/
Disallow /proformaApprove/
Disallow /proformaReject/
Disallow /personalprojects_list/
Disallow /clientprojects_list/
Disallow /globalprojects_list/
Disallow /taskbiscuit/
Disallow /contact.htm
Disallow /cgi-bin/
Disallow /question_create/
Disallow /static/img/glyphicons-halflings.png
Disallow /favicon.ico
Disallow /question_list/
Disallow /question_list
Disallow /in-the-news
Disallow /_/rsrc/1353947088000/system/app/css/camelot/allthemes-view.css
Disallow /photo1.jpg
Disallow /photo2.jpg
Disallow /photo3.jpg
Disallow /photo4.jpg
Disallow /photo5.jpg
Disallow /photo6.jpg
Disallow /photo7.jpg
Disallow /photo8.jpg
Disallow /photo9.jpg
Disallow /photo10.jpg
Disallow /photo1.html
Disallow /photo2.html
Disallow /photo3.html
Disallow /photo4.html
Disallow /photo5.html
Disallow /photo6.html
Disallow /photo7.html
Disallow /photo8.html
Disallow /photo9.html
Disallow /photo10.html
Disallow /answerslist/
Disallow /answerlist/
Disallow /person/
Disallow /assist/
Disallow /links.html
Disallow /privacy.html
Disallow /about.html
Disallow /services.html
Disallow /copyright.html
Disallow /index.html
Disallow /business/
Disallow /clients/
Disallow /system/
Disallow /terms/
Disallow /questionoranswer/
Disallow /questionoranswer
Disallow /guessanswerbool/
Disallow /getquestionandchoices/
Disallow /askquestions/
Disallow /answerlist/
Disallow /gettheword/
Disallow /refreshlist/
Disallow /endsessionatserver/
Disallow /getquestionandchoices/
Disallow /gettheword/
Disallow /processlistpost/
Disallow /refreshlist/
Disallow /processmcanswer/
Disallow /our-company/our-staff/person-s-name---cto
Disallow /wp-login.php
Disallow /in-the-news/personsnameonfacethenation
Disallow /%EC%BC%80%EC%9D%B4%EB%B8%94%ED%8B%B0%EB%B9%84%ED%95%B4%EC%A7%80__90949.gif
Disallow /a
Disallow /system/app/pages/removeAccess
Disallow /_/rsrc/1352499109611/our-company/our-staff/image-1.gif
Disallow /home
Disallow /task_list/
Disallow /userprofileamend/
Disallow /maintain_contact/
Disallow /add_contact/
Disallow /emailaddresscreate/
Disallow /emailaddressamend/
Disallow /addressamend/
Disallow /addresscreate/
Disallow /emailaddress_list/
Disallow /address_list/
Disallow /link_contact/
Disallow /share_task/
Disallow /task_mylist/
Disallow /task_listreset/
Disallow /subtask_mylist/
Disallow /task_personallist/
Disallow /subtask_list/
Disallow /tasklist_update/
Disallow /tasklist_update
Disallow /projectmanagementmethodologyscopedevelop/
Disallow /taskdepute/
Disallow /teamlist_create/
Disallow /teamlist_update/
Disallow /availableteams_list/
Disallow /availableteammembers_onaddevent/
Disallow /availableteammembers_onaddtask/
Disallow /availableteammembers_onaddsubtask/
Disallow /updateteammember/
Disallow /saveTasks/
Disallow /saveLastTask/
Disallow /flagtimerasstopped/
Disallow /flagTimerAsStopped/
Disallow /updateTeammemberInvite/
Disallow /searchresults/
Disallow /searchresultslocal/
Disallow /teammembersList_create/
Disallow /getsubcatagories/
Disallow /taskcomments/
Disallow /testimonials-1/personsnamesavedmybacon
Disallow /clients/resources
Disallow /world.gif
Disallow /accounts/login/?next=%2Fproject_list%2F
Disallow /accounts/login/
Disallow /about
Disallow /no_project_task_list_load_view/
Disallow /_/rsrc/1352499109611/our-company/our-staff/person-s-name---cto/HeadShot.jpg
Disallow /flagItemAsDone/
Disallow /get/

Warnings

  • 6 invalid lines.
  • `host` is not a known field.