glossary.pp.ua
robots.txt
Robots Exclusion Standard data for glossary.pp.ua
Resource Scan
Scan Details
| Site Domain | glossary.pp.ua |
| Base Domain | glossary.pp.ua |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Couldn't connect to server. |
| Last Scan | 2024-08-06T23:55:50+00:00 |
| Next Scan | 2024-11-04T23:55:50+00:00 |
Last Successful Scan
| Scanned | 2024-01-10T22:23:15+00:00 |
| URL | https://glossary.pp.ua/robots.txt |
| Domain IPs | 159.69.125.74 |
| Response IP | 159.69.125.74 |
| Found | Yes |
| Hash | 97350d46b624a276b021763ef7446714b9c889251fa03ad7d6b4d593824d0471 |
| SimHash | cbd651abc2a1 |
Groups
mj12bot
israbot
orthogaffe
ubicrawler
doc
zao
zealbot
msiecrawler
sitesnagger
webstripper
webcopier
fetch
offline explorer
teleport
teleportpro
webzip
linko
httrack
microsoft.url.control
xenu
larbin
libwww
zyborg
download ninja
fast
wget
grub-client
k2spider
npbot
webreaper
ahrefs
seokicks-robot
sistrix crawler
uptimerobot/2.0
ezooms robot
perl lwp
blexbot
wiseguys robot
turnitin robot
turnitinbot
turnitin bot
heritrix
pimonster
pimonster
searchmetricsbot
megaindex.ru
twiceler
sitecheck.internetseer.com
nutch
spock
omniexplorer_bot
becomebot
geniebot
dotbot
mlbot
linguee bot
aihitbot
sbider/nutch
jyxobot
magent
speedy spider
shopwiki
huasai
datacha0s
atomic_email_hunter
mp3bot
winhttp
betabot
core-project
panscient.com
java
libwww-perl
bingpreview
adidxbot
semrushbot
semrushbot-sa
semrushbot-ba
semrushbot-si
semrushbot-swa
semrushbot-ct
semrushbot-bm
semrushbot-seoab
| Rule | Path |
|---|---|
| Disallow | / |
google
apis-google
mediapartners-google
adsbot-google-mobile
adsbot-google
googlebot-image
googlebot
googlebot-news
googlebot-video
adsbot-google-mobile-apps
| Rule | Path |
|---|---|
| Allow | / |
mail.ru
slurp
yahoo-slurp
bingbot
duckduckbot
baiduspider
msnbot
baiduspider-image
youdaobot
sogou web spider
sogou inst spider
sogou spider2
sogou blog
sogou news spider
sogou orion spider
chinasospider
sosospider
yisouspider
easouspider
exabot
facebot
ia_archiver
twitterbot
pinterest
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 3 |
*
| Rule | Path |
|---|---|
| Disallow | /search/* |
| Disallow | /translate/* |
| Disallow | /storage/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 50 |
Warnings
- 2 invalid lines.