prosto-top.com
robots.txt
Robots Exclusion Standard data for prosto-top.com
Resource Scan
Scan Details
| Site Domain | prosto-top.com |
| Base Domain | prosto-top.com |
| Scan Status | Ok |
| Last Scan | 2024-11-13T05:05:11+00:00 |
| Next Scan | 2024-11-20T05:05:11+00:00 |
Last Scan
| Scanned | 2024-11-13T05:05:11+00:00 |
| URL | https://prosto-top.com/robots.txt |
| Domain IPs | 91.106.207.40 |
| Response IP | 91.106.207.40 |
| Found | Yes |
| Hash | acad2170c3f6d279325bf9ee1720a7cac92cd9cd5516634c5c78e544f6da7043 |
| SimHash | 3d58847acc33 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /2014/ |
| Disallow | /2008/ |
| Disallow | /2009/ |
| Disallow | /2010/ |
| Disallow | /2011/ |
| Disallow | /2012/ |
| Disallow | /2013/ |
| Disallow | /2014/ |
| Disallow | /tags/ |
| Disallow | /user/ |
| Disallow | /favorites/ |
| Disallow | /lastnews/ |
| Disallow | /newposts/ |
| Disallow | /catalog/ |
| Disallow | /*page/ |
| Disallow | |
| Disallow | /addnews.html |
| Disallow | /statistics.html |
| Disallow | /rules.html |
| Disallow | /dle-rules-page.html |
| Disallow | /engine/ajax/ |
| Disallow | /engine/go.php |
| Disallow | /engine/download.php |
| Disallow | /*?* |
| Disallow | /admin |
Other Records
| Field | Value |
|---|---|
| sitemap | https://prosto-top.com/sitemap.xml |
Warnings
- `host` is not a known field.