lifeonai.com
robots.txt
Robots Exclusion Standard data for lifeonai.com
Resource Scan
Scan Details
Site Domain | lifeonai.com |
Base Domain | lifeonai.com |
Scan Status | Ok |
Last Scan | 2024-09-29T18:06:14+00:00 |
Next Scan | 2024-10-06T18:06:14+00:00 |
Last Scan
Scanned | 2024-09-29T18:06:14+00:00 |
URL | https://lifeonai.com/robots.txt |
Domain IPs | 107.172.103.161 |
Response IP | 107.172.103.161 |
Found | Yes |
Hash | 0a7715623935b00c6fc6ad9323588114cc1de0141495884c7f910b1f53cb50b7 |
SimHash | 53975263c9a5 |
Groups
*
Rule | Path |
---|---|
Disallow | /wp-admin/ |
Disallow | */feed/ |
Disallow | /?p=* |
Allow | /wp-admin/admin-ajax.php |
ahrefsbot
ahrefssiteaudit
screaming frog seo spider
semrushbot
semrushbot
semrushbot-sa
url_spider_pro
nimblecrawler
grapeshot
botrighthere
webzip
larbin
b2w/0.1
copernic
psbot
python-urllib
url_spider_pro
cherrypicker
emailcollector
emailsiphon
webbandit
emailwolf
extractorpro
copyrightcheck
crescent
sitesnagger
prowebwalker
cheesebot
lnspiderguy
alexibot
teleport
teleportpro
miixpc
telesoft
website quester
moget/2.1
webzip/4.0
webstripper
websauger
webcopier
netants
mister pix
webauto
thenomad
www-collector-e
rma
libweb/clshttp
asterias
httplib
turingos
spanner
infonavirobot
harvest/1.5
bullseye/1.0
mozilla/4.0 (compatible; bullseye; windows 95)
crescent internet toolpak http ole control v.1.0
cherrypickerse/1.0
cherrypickerelite/1.0
webbandit/3.50
nicerspro
microsoft url control - 5.01.4511
dittospyder
foobot
spankbot
botalot
lwp-trivial/1.34
lwp-trivial
bunnyslippers
microsoft url control - 6.00.8169
urly warning
wget/1.6
wget/1.5.3
wget
linkwalker
cosmos
moget
hloader
humanlinks
linkextractorpro
offline explorer
mata hari
lexibot
web image collector
the intraformant
true_robot/1.0
true_robot
blowfish/1.0
jennybot
miixpc/4.2
builtbottough
propowerbot/2.14
backdoorbot/1.0
tocrawl/urldispatcher
webenhancer
suzuran
tighttwatbot
vci webviewer vci webviewer win32
vci
szukacz/1.4
queryn metasearch
openfind data gatherer
openfind
zeus
repomonkey bait & tackle/v1.01
repomonkey
microsoft url control
openbot
url control
zeus link scout
zeus 32297 webster pro v2.9 win32
webster pro
erocrawler
linkscan/8.1a unix
keyword density/0.9
kenjin spider
iron33/1.0.2
bookmark search tool
getright/4.2
fairad client
gaisbot
aqua_products
radiation retriever 1.1
flaming attackbot
oracle ultra search
msiecrawler
perman
searchpreview
turnitinbot
webzip/4.21
webzip/5.0
httrack 3.0
turnitinbot/1.5
webcopier v3.2a
webcapture 2.0
webcopier v.2.2
callpod keeper
go-http-client
java browser
php
clickagy intelligence
uipbot
Rule | Path |
---|---|
Disallow | / |
Other Records
Field | Value |
---|---|
sitemap | https://lifeonai.com/sitemap_index.xml |