oned.net
robots.txt
Robots Exclusion Standard data for oned.net
Resource Scan
Scan Details
| Site Domain | oned.net |
| Base Domain | oned.net |
| Scan Status | Ok |
| Last Scan | 2024-05-31T19:59:03+00:00 |
| Next Scan | 2024-06-07T19:59:03+00:00 |
Last Scan
| Scanned | 2024-05-31T19:59:03+00:00 |
| URL | https://oned.net/robots.txt |
| Domain IPs | 108.156.133.11, 108.156.133.127, 108.156.133.13, 108.156.133.46 |
| Response IP | 108.156.133.46 |
| Found | Yes |
| Hash | b1775d9be27e90b395d5c1155c999f4f10ff5c82e4c50a63289f7316cceaa913 |
| SimHash | ecb052f2ef43 |
Groups
*
| Rule | Path | Comment |
|---|---|---|
| Disallow | /admin/ | Disallow crawling of the admin section |
| Disallow | /profile/ | Disallow crawling of user account pages |
| Disallow | /api/ | Disallow crawling of API endpoints |
| Disallow | /search/ | Disallow crawling of search result pages |
| Allow | /live-tv | - |
| Allow | /content/* | - |
| Allow | /activities | - |
| Allow | /faq/ | - |
Other Records
| Field | Value | Comment |
|---|---|---|
| crawl-delay | 5 | Add a crawl delay of 5 seconds (adjust as needed) |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.oned.net/sitemap.xml |
Comments