trader-oil.ru
robots.txt
Robots Exclusion Standard data for trader-oil.ru
Resource Scan
Scan Details
| Site Domain | trader-oil.ru |
| Base Domain | trader-oil.ru |
| Scan Status | Ok |
| Last Scan | 2025-10-19T08:01:11+00:00 |
| Next Scan | 2025-11-18T08:01:11+00:00 |
Last Scan
| Scanned | 2025-10-19T08:01:11+00:00 |
| URL | https://trader-oil.ru/robots.txt |
| Domain IPs | 104.21.1.226, 172.67.152.103, 2606:4700:3030::6815:1e2, 2606:4700:3037::ac43:9867 |
| Response IP | 104.21.1.226 |
| Found | Yes |
| Hash | 6d80bfb46797a16d9a60b3af7892aabb46e4e0e9063ad4102f5bb5f0794f867d |
| SimHash | 27155ce314b1 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | *SECTION%3D |
| Disallow | *CODE%3D |
| Disallow | /outlinks/ |
| Disallow | /bitrix/admin/ |
| Disallow | */index.php |
| Disallow | |
| Disallow | /catalog/*?PAGEN |
| Disallow | *.doc |
| Disallow | /search |
| Disallow | /subscribe/ |
| Disallow | /news/*/*/?PAGEN_ |
| Disallow | *bxajaxid* |
| Disallow | */popup.php |
| Disallow | /catalog/filter/type-is-* |
| Disallow | /local/media/js/links.js |
| Disallow | /privacy-policy/ |
| Disallow | /news/market/kanadskiy-energeticheskiy-gigant-nameren/ |
| Disallow | /news/company/cefc-stala-vladeltsem-aktsiy-rosnefti/ |
| Disallow | /news/company/v-ramkakh-soglasheniya-s-opek-rossiya-snizhaet-tempy-dobychi-nefti/ |
| Disallow | /?ref=xranks |
| Disallow | /catalog/sbt/ |
| Disallow | /catalog/neft/%C2%A0/ |
| Disallow | /o-kompanii/forum/ |
| Disallow | /kotirovki/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.trader-oil.ru/sitemap.xml |
| sitemap | https://www.trader-oil.ru/sitemap-catalog.xml |
Warnings
- `clean-param` is not a known field.