geo-location.org
robots.txt
Robots Exclusion Standard data for geo-location.org
Resource Scan
Scan Details
| Site Domain | geo-location.org |
| Base Domain | geo-location.org |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-11-13T06:49:35+00:00 |
| Next Scan | 2024-11-27T06:49:35+00:00 |
Last Successful Scan
| Scanned | 2024-10-29T06:49:18+00:00 |
| URL | https://geo-location.org/robots.txt |
| Domain IPs | 213.186.33.4 |
| Response IP | 213.186.33.4 |
| Found | Yes |
| Hash | 5e84a35098b134c91b8d2702adde5b5d8f8c65d8be78f78c194e744b2cbb935d |
| SimHash | 56a2326044ea |
Groups
*
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
adsbot
adscanner
ahrefsbot
ahrefsbot
aihitbot
alexibot
alvinetspider
antenne hatena
anthropic-ai
apocalxexplorerbot
asterias
backdoorbot/1.0
backlinkcrawler
barkrowler
becomebot
bizinformation
black hole
blexbot
blexbot
blowfish/1.0
botalot
bubing
buck
builtbottough
bullseye/1.0
bunnyslippers
careerbot
ccbot
ccbot
cegbfeieh
chatgpt-user
checkmarknetwork
cheesebot
cherrypicker
cherrypickerelite/1.0
cherrypickerse/1.0
cincraw
cliqzbot
cloudservermarketspider
coccoc
cohere-ai
copyrightcheck
cosmos
crawler4j
crescent
crescent internet toolpak http ole control v.1.0
dataforseobot
df
df bot
disco pump 3.1
dittospyder
domain re-animator bot
domaincrawler
domainstatsbot
dotbot
dotbot
dotbot
e-societyrobot
emailcollector
emailsiphon
emailwolf
erocrawler
exabot
exabot*
extractorpro
ezooms robot
facebookbot
flamingo_searchengine
foobot
fr-crawler
garlikcrawler
google-extended
google-extended
gosign-security-crawler
gptbot
gptbot
haosouspider
harvest/1.5
hloader
http://www.almaden.ibm.com/cs/crawler
httplib
httrack
httrack 3.0
humanlinks
i-market-bot
iccrawler
igentia
implisensebot
indeedbot
inetdex-bot
infonavirobot
jennybot
jikespider
jobboersebot
jobs.de-robot
kenjin spider
keywordspy.com
kraken
lexibot
libweb/clshttp
linguee
linkextractorpro
linkscan/8.1a unix
linkstats
linkwalker
lipperhey-kaus-australis
ltx71
lwp-trivial
magpie-crawler
mata hari
mauibot (crawler.feedback+dc@gmail.com)
meanpathbot
megaindex.com
megaindex.ru
metajobbot
microsoft url control - 5.01.4511
microsoft url control - 6.00.8169
miixpc
miixpc/4.2
mindupbot
mister pix
mj12bot
mlbot
moget
moget/2.1
ms search 4.0 robot
naverbot
netants
netattache
netestate
netmechanic
nextgensearchbot 1
nicerspro
obot
offline explorer
omgili
omgilibot
openfind
openhosebot
openindexspider
petalbot
plista
plukkie
propowerbot/2.14
prowebwalker
proximic
psbot
python/3.5 aiohttp
quepasacreep
queryn metasearch
qwantify
r6_commentreader
repomonkey
rma
rogerbot
sabsimbot
safednsbot
screaming frog seo spider
searchmetricsbot
seekportbot
semrushbot
semrushbot-sa
sentibot
sentibot
seodiver
seokicks-robot
seoscanners.net
serendeputybot
serpstatbot
seznambot
sg-orbiter
shopwiki
sidetrade
sightupbot
sistrix
sistrix
sistrix crawler
sitebot
siteliner
sitesnagger
sitesucker
sogou web spider
sosospider
spankbot
spanner
spbot
spbot
speedy
spiderbot
startmebot
suggybot
superbot
surveybot
suzuran
szukacz
t3versionsbot
teleport
telesoft
the intraformant
thenomad
thumbsniper
tighttwatbot
titan
tocrawl/urldispatcher
toscrawler
toweya.com
trendictionbot
true_robot
turingos
turnitinbot
um-ic
unisterbot
urlpouls
urly warning
vci
velenpublicwebcrawler
wbsearchbot
web image collector
webauto
webbandit
webbandit/3.50
webcopier
webcopy
webenhancer
webmasterworldforumbot
webmirror
webreaper
websauger
website extractor
website quester
webster pro
webstripper
webstripper/2.02
webzip
wget
wikiofeedbot
wikiwix-bot
winhttrack
wotbox
ws-bot-v1
www-collector-e
xenu link sleuth/1.3.8
xing
xing bot
yacy
yak
yandex
yrspider
zeus
zookabot
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Allow | / |
Warnings
- 4 invalid lines.
Comments