arkadasariyorum.gen.tr
robots.txt
Robots Exclusion Standard data for arkadasariyorum.gen.tr
Resource Scan
Scan Details
| Site Domain | arkadasariyorum.gen.tr |
| Base Domain | arkadasariyorum.gen.tr |
| Scan Status | Ok |
| Last Scan | 2024-09-27T09:34:10+00:00 |
| Next Scan | 2024-10-04T09:34:10+00:00 |
Last Scan
| Scanned | 2024-09-27T09:34:10+00:00 |
| URL | https://arkadasariyorum.gen.tr/robots.txt |
| Redirect | https://www.arkadasliksitesi.gen.tr/robots.txt |
| Redirect Domain | www.arkadasliksitesi.gen.tr |
| Redirect Base | arkadasliksitesi.gen.tr |
| Domain IPs | 185.45.66.150 |
| Redirect IPs | 185.45.66.150 |
| Response IP | 185.45.66.150 |
| Found | Yes |
| Hash | 15c149afd61720419a82ee67b3eda2664b372d8ecd0736c7b40599b9c51ba2c8 |
| SimHash | 339e73c74983 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /admin/ |
| Disallow | /temp/ |
| Disallow | /images/listings |
| Disallow | /beta/ |
| Disallow | /beta |
| Disallow | /db_backup/ |
| Disallow | /periodic.php |
| Disallow | /favorites.php |
| Disallow | /login.php |
| Disallow | /logout.php |
| Disallow | /reply.php |
| Disallow | /report.php |
| Disallow | /recommend.php |
| Disallow | /mailto.php |
| Disallow | /password_recovery.php |
| Disallow | /access_restricted.php |
| Disallow | /not_authorized.php |
| Disallow | /offline.php |
| Disallow | /include |
| Disallow | /f- |
| Disallow | /aj- |
| Disallow | /ah- |
| Disallow | /ai- |
| Disallow | /tl- |
| Disallow | /ms- |
| Disallow | /af- |
| Disallow | /yas- |
| Disallow | /2-bayan/yas- |
| Disallow | /2-bayan/aj- |
| Disallow | /2-bayan/ah- |
| Disallow | /2-bayan/ai- |
| Disallow | /2-bayan/tl- |
| Disallow | /2-bayan/ms- |
| Disallow | /2-bayan/af- |
| Disallow | /1-erkek/yas- |
| Disallow | /1-erkek/aj- |
| Disallow | /1-erkek/ah- |
| Disallow | /1-erkek/ai- |
| Disallow | /1-erkek/tl- |
| Disallow | /1-erkek/ms- |
| Disallow | /1-erkek/af- |
baiduspider/2.0
baiduspider
baiduspider
baiduspider+
sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm
sogou web spider/4.0
sogou web spider
mj12bot
ahrefsbot
ia_archiver
hotzonu/x.0
exabot
ezooms
discobot
speedy
gigabot
jooblebot
seznambot
python-urllib
voilabot
yeti
sbider
shopwiki
gidbot
turnitinbot
sitebot
speedy
httrack
grapeshot
grub-client
grub
looksmart
webzip
larbin
b2w/0.1
psbot
python-urllib
netmechanic
url_spider_pro
cherrypicker
emailcollector
emailsiphon
webbandit
emailwolf
extractorpro
copyrightcheck
crescent
sitesnagger
prowebwalker
cheesebot
lnspiderguy
teleport
teleportpro
miixpc
telesoft
website quester
moget/2.1
webzip/4.0
webstripper
websauger
webcopier
netants
mister pix
webauto
thenomad
www-collector-e
rma
libweb/clshttp
asterias
httplib
turingos
spanner
infonavirobot
harvest/1.5
bullseye/1.0
mozilla/4.0 (compatible; bullseye; windows 95)
crescent internet toolpak http ole control v.1.0
cherrypickerse/1.0
cherrypickerelite/1.0
webbandit/3.50
nicerspro
microsoft url control - 5.01.4511
dittospyder
foobot
webmasterworldforumbot
spankbot
botalot
lwp-trivial/1.34
lwp-trivial
bunnyslippers
microsoft url control - 6.00.8169
urly warning
wget/1.6
wget/1.5.3
wget
linkwalker
cosmos
moget
hloader
humanlinks
linkextractorpro
offline explorer
mata hari
lexibot
web image collector
the intraformant
true_robot/1.0
true_robot
blowfish/1.0
jennybot
miixpc/4.2
builtbottough
propowerbot/2.14
backdoorbot/1.0
tocrawl/urldispatcher
webenhancer
suzuran
vci webviewer vci webviewer win32
vci
szukacz/1.4
queryn metasearch
openfind data gathere
openfind
xenu's link sleuth 1.1c
xenu's
zeus
repomonkey bait & tackle/v1.01
repomonkey
microsoft url control
openbot
url control
zeus link scout
zeus 32297 webster pro v2.9 win32
webster pro
erocrawler
linkscan/8.1a unix
keyword density/0.9
kenjin spider
iron33/1.0.2
bookmark search tool
getright/4.2
fairad client
gaisbot
aqua_products
radiation retriever 1.1
flaming attackbot
oracle ultra search
msiecrawler
perman
searchpreview
panscient.com
vscooter
psbot
ia_archiver
mj12bot
twiceler
taptubot
twengabot
sitebot
ahrefsbot
ezooms
sistrix
aihitbot
infopath
infopath.2
swebot
ec2linkfinder
turnitinbot
searchmetericsbot
wbsearchbot
exabot
sosospider
ip-web-crawler.com
netestate ne crawler
aboundexbot
meanpathbot
mail.ru
spbot
archive.org_bot
linkpadbot
easouspider
seznambot
wotbox
yandex
yandexbot
semrushbot
semrushbot-sa
semrushbot-ba
semrushbot-si
semrushbot-swa
semrushbot-ct
| Product | Comment |
|---|---|
| sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm | 07) |
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.arkadasliksitesi.gen.tr/sitemap/sitemap.xml |
Warnings
- 2 invalid lines.