mafia.ua
robots.txt
Robots Exclusion Standard data for mafia.ua
Resource Scan
Scan Details
| Site Domain | mafia.ua |
| Base Domain | mafia.ua |
| Scan Status | Ok |
| Last Scan | 2024-08-29T16:46:20+00:00 |
| Next Scan | 2024-09-28T16:46:20+00:00 |
Last Scan
| Scanned | 2024-08-29T16:46:20+00:00 |
| URL | https://mafia.ua/robots.txt |
| Domain IPs | 104.26.0.21, 104.26.1.21, 172.67.68.231, 2606:4700:20::681a:115, 2606:4700:20::681a:15, 2606:4700:20::ac43:44e7 |
| Response IP | 172.67.68.231 |
| Found | Yes |
| Hash | 455c3204397ecd97e4639907d2d94521829dc33189471d5e6cc278d9000073fb |
| SimHash | 6815487aa7b9 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /kiev/menu-dostavki/add_to_cart/ |
| Disallow | /kiev/menu-dostavki/business-lunch/add_to_cart/ |
| Disallow | /kharkov/menu-dostavki/add_to_cart/ |
| Disallow | /kharkov/menu-dostavki/business-lunch/add_to_cart/ |
| Disallow | /admin/ |
| Disallow | /admin_tools/ |
| Disallow | /auth/ |
| Disallow | /cabinet/ |
| Disallow | /form/ |
| Disallow | /gallery/ |
| Disallow | /leave_contacts/ |
| Disallow | /leave_feedback/ |
| Disallow | /leave_your_number/ |
| Disallow | /mafia_misc/ |
| Disallow | /mobile_api/ |
| Disallow | /map/ |
| Disallow | /payment/ |
| Disallow | /rate_restaurant/ |
| Disallow | /shop/ |
| Disallow | /static/ |
| Disallow | /streets/ |
| Disallow | /switch-mode/ |
| Disallow | /*/add_to_cart/ |
| Disallow | /*/choose_gift/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://mafia.ua/sitemap.xml |
Warnings
- `host` is not a known field.