likearea.net
robots.txt
Robots Exclusion Standard data for likearea.net
Resource Scan
Scan Details
| Site Domain | likearea.net |
| Base Domain | likearea.net |
| Scan Status | Ok |
| Last Scan | 2024-06-01T22:45:40+00:00 |
| Next Scan | 2024-07-01T22:45:40+00:00 |
Last Scan
| Scanned | 2024-06-01T22:45:40+00:00 |
| URL | http://likearea.net/robots.txt |
| Domain IPs | 2a01:37:1000::53df:5b41:0, 83.223.91.65 |
| Response IP | 83.223.91.65 |
| Found | Yes |
| Hash | 8763f8b84ef6d89a0202f32d5c816ceb9e9c811acd473d0f3714eee812af3356 |
| SimHash | 2943e7b0e798 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | / |
baiduimagespider
baiduspider
baidumobaider
| Rule | Path |
|---|---|
| Disallow | /*?* |
| Allow | /en/?* |
| Allow | /es/?* |
| Allow | /fr/?* |
| Allow | /de/?* |
| Allow | /it/?* |
| Allow | /tr/?* |
| Disallow | /*.html |
| Allow | /en/*.html$ |
| Allow | /es/*.html$ |
| Allow | /fr/*.html$ |
| Allow | /de/*.html$ |
| Allow | /it/*.html$ |
| Allow | /tr/*.html$ |
| Disallow | /*.php$ |
| Disallow | /*.php?* |
bingpreview
bingbot
| Rule | Path |
|---|---|
| Disallow | /*?* |
| Allow | /en/?* |
| Allow | /es/?* |
| Allow | /fr/?* |
| Allow | /de/?* |
| Allow | /it/?* |
| Allow | /tr/?* |
| Disallow | /*.html |
| Allow | /en/*.html$ |
| Allow | /es/*.html$ |
| Allow | /fr/*.html$ |
| Allow | /de/*.html$ |
| Allow | /it/*.html$ |
| Allow | /tr/*.html$ |
| Disallow | /*.php$ |
| Disallow | /*.php?* |
googlebot
googlebot-image
googlebot-mobile
googleproducer
feedfetcher-google
kw-lp-suggest
pagefetcher-google-coop
google-sitemaps/1.0
googlebot-video
google web preview
google wireless transcoder
| Rule | Path |
|---|---|
| Disallow | /*?* |
| Allow | /en/?* |
| Allow | /es/?* |
| Allow | /fr/?* |
| Allow | /de/?* |
| Allow | /it/?* |
| Allow | /tr/?* |
| Disallow | /*.html |
| Allow | /en/*.html$ |
| Allow | /es/*.html$ |
| Allow | /fr/*.html$ |
| Allow | /de/*.html$ |
| Allow | /it/*.html$ |
| Allow | /tr/*.html$ |
| Disallow | /*.php$ |
| Disallow | /*.php?* |
msnbot
msnbot-media
msnbot-mobile
msnbot-newsblogs
msnbot-products
| Rule | Path |
|---|---|
| Disallow | /*?* |
| Allow | /en/?* |
| Allow | /es/?* |
| Allow | /fr/?* |
| Allow | /de/?* |
| Allow | /it/?* |
| Allow | /tr/?* |
| Disallow | /*.html |
| Allow | /en/*.html$ |
| Allow | /es/*.html$ |
| Allow | /fr/*.html$ |
| Allow | /de/*.html$ |
| Allow | /it/*.html$ |
| Allow | /tr/*.html$ |
| Disallow | /*.php$ |
| Disallow | /*.php?* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
voilabot
| Rule | Path |
|---|---|
| Disallow | /*?* |
| Allow | /en/?* |
| Allow | /es/?* |
| Allow | /fr/?* |
| Allow | /de/?* |
| Allow | /it/?* |
| Allow | /tr/?* |
| Disallow | /*.html |
| Allow | /en/*.html$ |
| Allow | /es/*.html$ |
| Allow | /fr/*.html$ |
| Allow | /de/*.html$ |
| Allow | /it/*.html$ |
| Allow | /tr/*.html$ |
| Disallow | /*.php$ |
| Disallow | /*.php?* |
yahooysmcm
yahoo-blogs
yahoofeedseeker
yahoo-mmcrawler
yahooseeker/m1a1-r2d2
yahoo! slurp
yahoo-verticalcrawler
| Rule | Path |
|---|---|
| Disallow | /*?* |
| Allow | /en/?* |
| Allow | /es/?* |
| Allow | /fr/?* |
| Allow | /de/?* |
| Allow | /it/?* |
| Allow | /tr/?* |
| Disallow | /*.html |
| Allow | /en/*.html$ |
| Allow | /es/*.html$ |
| Allow | /fr/*.html$ |
| Allow | /de/*.html$ |
| Allow | /it/*.html$ |
| Allow | /tr/*.html$ |
| Disallow | /*.php$ |
| Disallow | /*.php?* |
yandexantivirus
yandexblog
yandexbot
yandexcatalog
yandexdirect
yandexfavicon
yandeximageresizer
yandeximages
yandexmedia
yandexmetrika
yandexnews
yandexpagechecker
yandexvideo
yandexwebmaster
yandexzakladki
| Rule | Path |
|---|---|
| Disallow | /*?* |
| Allow | /en/?* |
| Allow | /es/?* |
| Allow | /fr/?* |
| Allow | /de/?* |
| Allow | /it/?* |
| Allow | /tr/?* |
| Disallow | /*.html |
| Allow | /en/*.html$ |
| Allow | /es/*.html$ |
| Allow | /fr/*.html$ |
| Allow | /de/*.html$ |
| Allow | /it/*.html$ |
| Allow | /tr/*.html$ |
| Disallow | /*.php$ |
| Disallow | /*.php?* |
Warnings
- `host` is not a known field.