forebears.co.uk
robots.txt
Robots Exclusion Standard data for forebears.co.uk
Resource Scan
Scan Details
| Site Domain | forebears.co.uk |
| Base Domain | forebears.co.uk |
| Scan Status | Ok |
| Last Scan | 2024-09-26T16:35:43+00:00 |
| Next Scan | 2024-10-03T16:35:43+00:00 |
Last Scan
| Scanned | 2024-09-26T16:35:43+00:00 |
| URL | https://forebears.co.uk/robots.txt |
| Redirect | https://forebears.io/robots.txt |
| Redirect Domain | forebears.io |
| Redirect Base | forebears.io |
| Domain IPs | 5.9.74.217 |
| Redirect IPs | 5.9.74.217 |
| Response IP | 5.9.74.217 |
| Found | Yes |
| Hash | 446675f2099704d91c95119008ed96b09ec630bbe007b2b1f2f620c0cc8126cd |
| SimHash | 46059a00c632 |
Groups
yandex
| Rule | Path |
|---|---|
| Allow | /ru$ |
| Allow | /ru/ |
| Allow | /$ |
| Allow | /surnames$ |
| Allow | /forenames$ |
| Allow | /resources$ |
| Allow | /baby-name-generator$ |
| Allow | /forenames/most-popular$ |
| Allow | /about |
| Allow | /guide |
| Allow | /russia$ |
| Allow | /ukraine$ |
| Allow | /belarus$ |
| Allow | /moldova$ |
| Allow | /transnistria$ |
| Allow | /abkhazia$ |
| Allow | /south-ossetia$ |
| Allow | /armenia$ |
| Allow | /artsakh$ |
| Allow | /kazakhstan$ |
| Allow | /turkmenistan$ |
| Allow | /estonia$ |
| Allow | /latvia$ |
| Allow | /lithuania$ |
| Allow | /georgia$ |
| Allow | /tajikistan$ |
| Allow | /uzbekistan$ |
| Allow | /kyrgyzstan$ |
| Allow | /united-states$ |
| Allow | /canada$ |
| Allow | /england$ |
| Allow | /scotland$ |
| Allow | /wales$ |
| Allow | /ireland$ |
| Allow | /northern-ireland$ |
| Allow | /france$ |
| Allow | /germany$ |
| Allow | /australia$ |
| Allow | /new-zealand$ |
| Allow | /china$ |
| Allow | /robots.txt |
| Allow | /data/sitemaps |
| Disallow | /*?* |
| Disallow | / |
| Disallow | /x/* |
*
| Rule | Path |
|---|---|
| Disallow | /account |
| Disallow | /contact |
| Disallow | /copyright |
| Disallow | /privacy |
| Disallow | /submit-surname-resource |
| Disallow | /*?* |
| Disallow | /sources/ |
| Disallow | /x/* |
| Disallow | /data/tpl/* |
| Disallow | /data/cache/* |
| Disallow | /translate-name* |
| Disallow | /account* |
| Disallow | /ru/ |
| Disallow | /ru$ |
cliqzbot
ntentbot
garlikcrawler
megaindex.ru
nutch
skimbot
yeti
vagabondo
coccoc
semrushbot
sistrix
mojeekbot
xovibot
spbot
domainappender
bubing
riddler
dotbot
tweetmemebot
ias_crawler
domainstatsbot
linguee
magpie-crawler
grapeshot
blexbot
infopath
wesee
smtbot
idmarch
larbin
maxpoint bot
maxpointcrawler
mj12bot
new zealand search
httrack
proximic
netseer
linkdexbot/2.0
genieo
java/1.7.0_11
infopath.2
grapeshot
ccbot
xenu's
backlink-check.de
backlinkcrawler
extractorpro
fasterfox
linkextractorpro
linkwalker
openbot
rogerbot
searchpreview
seodat
seoengbot
seokicks-robot
sistrix
tineye
true_robot
url control
url_spider_pro
xovi
meanpathbot
wikido
worldbrewbot
linkfluence
ahrefsbot
petalbot
| Rule | Path |
|---|---|
| Disallow | / |
Warnings
- 1 invalid line.