b-21.com
robots.txt
Robots Exclusion Standard data for b-21.com
Resource Scan
Scan Details
| Site Domain | b-21.com |
| Base Domain | b-21.com |
| Scan Status | Ok |
| Last Scan | 2025-12-24T05:10:59+00:00 |
| Next Scan | 2026-01-23T05:10:59+00:00 |
Last Scan
| Scanned | 2025-12-24T05:10:59+00:00 |
| URL | https://b-21.com/robots.txt |
| Redirect | https://www.b-21.com/robots.txt |
| Redirect Domain | www.b-21.com |
| Redirect Base | b-21.com |
| Domain IPs | 45.223.64.217, 45.223.69.217 |
| Redirect IPs | 45.223.66.217 |
| Response IP | 45.223.66.217 |
| Found | Yes |
| Hash | acdee22b702508f339592a9d2d9ace1277a7958eac0de0e052059478d4219b9d |
| SimHash | 6c19de408693 |
Groups
*
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
*
| Rule | Path |
|---|---|
| Disallow | /feed/ |
| Disallow | /ipad/ |
| Disallow | /text/ |
| Disallow | /events/ |
| Disallow | /basket.asp |
| Disallow | /login.asp |
| Disallow | /statuslogin.asp |
| Disallow | /searchprods.asp* |
| Disallow | /searchwine.asp* |
| Disallow | /prodinfo.asp* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.b-21.com/sitemap.xml |