openwrt.org
robots.txt
Robots Exclusion Standard data for openwrt.org
Resource Scan
Scan Details
| Site Domain | openwrt.org |
| Base Domain | openwrt.org |
| Scan Status | Ok |
| Last Scan | 2024-06-07T05:21:10+00:00 |
| Next Scan | 2024-07-07T05:21:10+00:00 |
Last Scan
| Scanned | 2024-06-07T05:21:10+00:00 |
| URL | https://openwrt.org/robots.txt |
| Domain IPs | 2a03:b0c0:3:d0::1a51:c001, 64.226.122.113 |
| Response IP | 64.226.122.113 |
| Found | Yes |
| Hash | 69e8efe2ed73f39f337612b18b659d23a5cad34f47e0ed41e5a88845a27085ea |
| SimHash | 8113f20b4dc1 |
Groups
baiduspider
baiduspider-render
| Rule | Path |
|---|---|
| Disallow | /feed.php?mode=list&ns= |
| Disallow | /lib/plugins/datatables/ |
| Disallow | /*id%3D* |
| Disallow | /*datasrt%3D* |
| Disallow | /*dataflt* |
go-http-client
| Rule | Path |
|---|---|
| Disallow | /*datasrt%3D* |
| Disallow | /*dataflt* |
| Disallow | /*do%3Ddiff* |
| Disallow | /*do%3Dmedia* |
mauibot
| Rule | Path |
|---|---|
| Disallow | /toh/views/toh_admin* |
| Disallow | /toh/views/toh_available_864* |
| Disallow | /toh/views/toh_discontinued_864* |
| Disallow | /toh/views/toh_modem_supported* |
| Disallow | /*datasrt%3D* |
| Disallow | /*dataflt* |
sogou
| Rule | Path |
|---|---|
| Disallow | / |
| Disallow | /toh/views/toh_admin_modem_wlan* |
| Disallow | /toh/views/toh_admin_serial* |
| Disallow | /zh/toh/views/toh_available_864 |
| Disallow | /*datasrt%3D* |
| Disallow | /*dataflt* |
semrushbot
| Rule | Path |
|---|---|
| Disallow | /feed.php?* |
| Disallow | /*idx%3D* |
| Disallow | /*rev%3D* |
| Disallow | /*do%3Ddiff* |
| Disallow | /*do%3Dedit$ |
| Disallow | /*do%3Dindex$ |
| Disallow | /*do%3Dlogin* |
| Disallow | /*do%3Dmedia* |
| Disallow | /*do%3Drecent$ |
| Disallow | /*do%3Dregister$ |
| Disallow | /*do%3Dresendpwd* |
| Disallow | /*do%3Drevisions$ |
| Disallow | /*tab_details%3Dedit* |
| Disallow | /*tab_details%3Dhistory* |
| Disallow | /*tab_details%3Dview* |
| Disallow | /*tab_files%3Dfiles* |
| Disallow | /*tab_files%3Dsearch* |
| Disallow | /*tab_files%3Dupload* |
| Disallow | /*datasrt%3D* |
| Disallow | /*dataflt* |
| Disallow | /de/en/ |
| Disallow | /de/kr/ |
| Disallow | /en/kr/ |
| Disallow | /es/en/ |
| Disallow | /es/kr/ |
| Disallow | /fr/kr/ |
| Disallow | /hu/kr/ |
| Disallow | /ja/en/ |
| Disallow | /ja/kr/ |
| Disallow | /jp/en/ |
| Disallow | /ko/cz/ |
| Disallow | /ko/en/ |
| Disallow | /ko/kr/ |
| Disallow | /pl/cz/ |
| Disallow | /pl/ko/ |
| Disallow | /pl/kr/ |
| Disallow | /pt/en/ |
| Disallow | /pt/kr/ |
| Disallow | /ru/en/ |
| Disallow | /ru/kr/ |
| Disallow | /tr/en/ |
| Disallow | /zh/en/ |
| Disallow | /zh/kr/ |
| Disallow | /playground/ |
| Disallow | /*/playground/ |
| Disallow | /*/packages/pkgdata/ |
| Disallow | /*/toh/hwdata/ |
seokicks
scrapy
gigabot
httrack
wget
yeti
| Rule | Path |
|---|---|
| Disallow | /*datasrt%3D* |
| Disallow | /*dataflt* |
| Disallow | /*do%3Dbacklink$ |
| Disallow | /*do%3Dedit$ |
| Disallow | /*do%3Dindex$ |
| Disallow | /*do%3Dlogin* |
| Disallow | /*do%3Dmedia$ |
| Disallow | /*do%3Dregister$ |
| Disallow | /*do%3Drecent$ |
| Disallow | /*do%3Drevisions$ |
| Disallow | /*id%3D* |
| Disallow | /_export/ |
yacybot
| Rule | Path |
|---|---|
| Disallow | /*datasrt%3D* |
| Disallow | /*dataflt* |
| Disallow | /*do%3Dbacklink$ |
| Disallow | /*do%3Dedit$ |
| Disallow | /*do%3Dindex$ |
| Disallow | /*do%3Dlogin* |
| Disallow | /*do%3Dmedia$ |
| Disallow | /*do%3Dregister$ |
| Disallow | /*do%3Drecent$ |
| Disallow | /*do%3Drevisions$ |
| Disallow | /*id%3D* |
yisouspider
| Rule | Path |
|---|---|
| Disallow | /lib/* |
| Disallow | /lib/plugins/datatables/ |
| Disallow | /lib/plugins/datatables/* |
| Disallow | *jquery.dataTables.min.js$ |
| Disallow | *buttons.dataTables.min.css$ |
| Disallow | *buttons.html5.min.js$ |
| Disallow | *dataTables.buttons.min.js$ |
| Disallow | *dataTables.fixedColumns.min.js$ |
| Disallow | *dataTables.responsive.min.js$ |
| Disallow | /*idx%3D* |
| Disallow | /*datasrt%3D* |
| Disallow | /*dataflt* |
| Disallow | /*do%3Ddiff* |
| Disallow | /*do%3Dedit$ |
| Disallow | /*do%3Dlogin* |
| Disallow | /*do%3Dmedia* |
| Disallow | /*do%3Drecent$ |
| Disallow | /*do%3Dregister$ |
| Disallow | /*do%3Drevisions$ |
| Disallow | /*rev%3D* |
*
| Rule | Path |
|---|---|
| Disallow | /doku.php?id=* |
| Disallow | /feed.php?* |
| Disallow | /feed.php?mode=list&ns= |
| Disallow | /start?id=* |
| Disallow | /*idx%3D* |
| Disallow | /*rev%3D* |
| Disallow | /*do%3Ddiff* |
| Disallow | /*do%3Dedit$ |
| Disallow | /*do%3Dindex$ |
| Disallow | /*do%3Dlogin* |
| Disallow | /*do%3Dmedia$ |
| Disallow | /*do%3Dprofile$ |
| Disallow | /*do%3Drecent$ |
| Disallow | /*do%3Dregister$ |
| Disallow | /*do%3Dresendpwd* |
| Disallow | /*do%3Drevisions$ |
| Disallow | /*tab_details%3Dedit* |
| Disallow | /*tab_details%3Dhistory* |
| Disallow | /*tab_details%3Dview* |
| Disallow | /*tab_files%3Dfiles* |
| Disallow | /*tab_files%3Dsearch* |
| Disallow | /*tab_files%3Dupload* |
| Disallow | /*datasrt%3D* |
| Disallow | /*dataflt* |
| Disallow | /*tseed%3D018d61fc1fa2f9abdf54c6f201c65c0b |
| Disallow | /*tseed%3Dceddb583a0a603952a527c2bb95e91be |
| Disallow | /*tseed%3D93ef14dc4babe4f518a0f1af6cd6be73 |
| Disallow | /meta/fwfiles* |
| Disallow | /*/meta/fwfiles* |
| Disallow | /_export/raw/* |
| Disallow | /toh/views/dataclouds |
| Disallow | /toh/views/toh_admin_modem_wlan* |
| Disallow | /toh/views/toh_admin_serial* |
| Disallow | /toh/views/toh_dev_arch-target-cpu* |
| Disallow | /toh/views/toh_performance_details* |
| Disallow | /toh/views/toh_available_864* |
| Disallow | /es/toh/views/toh_available_864* |
| Disallow | /zh/toh/views/toh_available_864* |
| Disallow | /zh/toh/views/toh_fwdownload* |
| Disallow | /zh/toh_fwdownload* |
| Disallow | /zh/toh/views/toh_packagedownload* |
| Disallow | /zh/toh_packagedownload* |
| Disallow | /es/packages/start |
| Disallow | /ko/packages/start |
| Disallow | */oldwiki/* |
| Disallow | /packages/logfile |
| Disallow | /packages/maintainer-view |
| Disallow | /packages/table/start |
| Disallow | /zh/packages/table/start |
| Disallow | /packages/table_lede17_1/start |
| Disallow | /playground/ |
| Disallow | /*/playground/ |
| Disallow | /cs/cz/ |
| Disallow | /cs/de/ |
| Disallow | /cs/en/ |
| Disallow | /cs/kr/ |
| Disallow | /cs/cs/ |
| Disallow | /de/cz/ |
| Disallow | /de/en/ |
| Disallow | /de/kr/ |
| Disallow | /en/jp/ |
| Disallow | /en/kr/ |
| Disallow | /es/cz/ |
| Disallow | /es/en/ |
| Disallow | /es/kr/ |
| Disallow | /fr/cz/ |
| Disallow | /fr/en/ |
| Disallow | /fr/kr/ |
| Disallow | /hu/cz/ |
| Disallow | /hu/en/ |
| Disallow | /hu/kr/ |
| Disallow | /it/cz/ |
| Disallow | /it/en/ |
| Disallow | /ja/en/ |
| Disallow | /ja/kr/ |
| Disallow | /jp/cz/ |
| Disallow | /jp/en/ |
| Disallow | /jp/kr/ |
| Disallow | /ko/cz/ |
| Disallow | /ko/en/ |
| Disallow | /ko/kr/ |
| Disallow | /pl/cz/ |
| Disallow | /pl/ko/ |
| Disallow | /pl/kr/ |
| Disallow | /pt/cz/ |
| Disallow | /pt/en/ |
| Disallow | /pt/kr/ |
| Disallow | /ru/cz/ |
| Disallow | /ru/en/ |
| Disallow | /ru/kr/ |
| Disallow | /ru/ru/ |
| Disallow | /ru/tr/ |
| Disallow | /ru/zh/ |
| Disallow | /tr/cs/ |
| Disallow | /tr/cz/ |
| Disallow | /tr/de/ |
| Disallow | /tr/en/ |
| Disallow | /tr/es/ |
| Disallow | /tr/fr/ |
| Disallow | /tr/hu/ |
| Disallow | /tr/it/ |
| Disallow | /tr/ja/ |
| Disallow | /tr/ko/ |
| Disallow | /tr/kr/ |
| Disallow | /tr/pl/ |
| Disallow | /tr/pt/ |
| Disallow | /zh/cz/ |
| Disallow | /zh/en/ |
| Disallow | /zh/kr/ |
| Disallow | /zh-tw/cz/ |
| Disallow | /docs/user-guide/ |
| Disallow | /*/docs/user-guide/ |
| Disallow | /*/hwdata_lede/ |
| Disallow | /*/packages/pkgdata/ |
| Disallow | /*/toh/hwdata/ |
| Disallow | /*/toh/views/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://openwrt.org/doku.php?do=sitemap |
Warnings
- 1 invalid line.