the-warsaw.com
robots.txt
Robots Exclusion Standard data for the-warsaw.com
Resource Scan
Scan Details
| Site Domain | the-warsaw.com |
| Base Domain | the-warsaw.com |
| Scan Status | Ok |
| Last Scan | 2024-09-25T23:29:12+00:00 |
| Next Scan | 2024-10-02T23:29:12+00:00 |
Last Scan
| Scanned | 2024-09-25T23:29:12+00:00 |
| URL | https://the-warsaw.com/robots.txt |
| Redirect | https://www.the-warsaw.com/robots.txt |
| Redirect Domain | www.the-warsaw.com |
| Redirect Base | the-warsaw.com |
| Domain IPs | 104.26.4.141, 104.26.5.141, 172.67.72.180, 2606:4700:20::681a:48d, 2606:4700:20::681a:58d, 2606:4700:20::ac43:48b4 |
| Redirect IPs | 104.26.4.141, 104.26.5.141, 172.67.72.180, 2606:4700:20::681a:48d, 2606:4700:20::681a:58d, 2606:4700:20::ac43:48b4 |
| Response IP | 104.26.5.141 |
| Found | Yes |
| Hash | 74b4dec84b919712682fcb4cab850443783e181be1db09215a04fa2498e5a60f |
| SimHash | 3608d052ac35 |
Groups
*
| Product | Comment |
|---|---|
| * | обÑие пÑавила Ð´Ð»Ñ ÑобоÑов, кÑоме ЯндекÑа и Google, |
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin | папка на Ñ |
| Disallow | /? | вÑе паÑамеÑÑÑ Ð·Ð°Ð¿ÑоÑа на главной |
| Disallow | /wp- | вÑе ÑÐ°Ð¹Ð»Ñ WP: /wp-json/, /wp-includes, /wp-content/plugins |
| Disallow | /wp/ | еÑли еÑÑÑ Ð¿Ð¾Ð´ÐºÐ°Ñалог /wp/, где ÑÑÑановлена CMS (еÑли неÑ, |
| Disallow | *?s= | поиÑк |
| Disallow | *%26s%3D | поиÑк |
| Disallow | /search/ | поиÑк |
| Disallow | /users/ | аÑÑ |
| Disallow | */trackback | ÑÑекбеки, ÑÐ²ÐµÐ´Ð¾Ð¼Ð»ÐµÐ½Ð¸Ñ Ð² комменÑаÑиÑÑ |
| Disallow | */rss | rss Ñид |
| Disallow | */embed | вÑе вÑÑÑÐ°Ð¸Ð²Ð°Ð½Ð¸Ñ |
| Disallow | */wlwmanifest.xml | xml-Ñайл маниÑеÑÑа Windows Live Writer (еÑли не иÑполÑзÑеÑе, |
| Disallow | /xmlrpc.php | Ñайл WordPress API |
| Disallow | *utm*%3D | ÑÑÑлки Ñ utm-меÑками |
| Disallow | *openstat%3D | ÑÑÑлки Ñ Ð¼ÐµÑками openstat |
| Allow | */uploads | оÑкÑÑваем Ð¿Ð°Ð¿ÐºÑ Ñ Ñайлами uploads |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
googlebot
| Product | Comment |
|---|---|
| googlebot | пÑавила Ð´Ð»Ñ Google (комменÑаÑии не дÑблиÑÑÑ) |
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin | - |
| Disallow | /? | - |
| Disallow | /wp- | - |
| Disallow | /wp/ | - |
| Disallow | *?s= | - |
| Disallow | *%26s%3D | - |
| Disallow | /search/ | - |
| Disallow | /author/ | - |
| Disallow | /users/ | - |
| Disallow | */trackback | - |
| Disallow | */feed | - |
| Disallow | */rss | - |
| Disallow | */embed | - |
| Disallow | */wlwmanifest.xml | - |
| Disallow | /xmlrpc.php | - |
| Disallow | *utm*%3D | - |
| Disallow | *openstat%3D | - |
| Allow | */uploads | - |
| Allow | /*/*.js | оÑкÑÑваем js-ÑкÑипÑÑ Ð²Ð½ÑÑÑи /wp- (/*/ - Ð´Ð»Ñ Ð¿ÑиоÑиÑеÑа) |
| Allow | /*/*.css | оÑкÑÑваем css-ÑÐ°Ð¹Ð»Ñ Ð²Ð½ÑÑÑи /wp- (/*/ - Ð´Ð»Ñ Ð¿ÑиоÑиÑеÑа) |
| Allow | /wp-*.png | каÑÑинки в Ð¿Ð»Ð°Ð³Ð¸Ð½Ð°Ñ |
| Allow | /wp-*.jpg | каÑÑинки в Ð¿Ð»Ð°Ð³Ð¸Ð½Ð°Ñ |
| Allow | /wp-*.jpeg | каÑÑинки в Ð¿Ð»Ð°Ð³Ð¸Ð½Ð°Ñ |
| Allow | /wp-*.gif | каÑÑинки в Ð¿Ð»Ð°Ð³Ð¸Ð½Ð°Ñ |
| Allow | /wp-admin/admin-ajax.php | иÑполÑзÑеÑÑÑ Ð¿Ð»Ð°Ð³Ð¸Ð½Ð°Ð¼Ð¸, ÑÑÐ¾Ð±Ñ Ð½Ðµ блокиÑоваÑÑ JS и CSS |
yandex
| Product | Comment |
|---|---|
| yandex | пÑавила Ð´Ð»Ñ Ð¯Ð½Ð´ÐµÐºÑа (комменÑаÑии не дÑблиÑÑÑ) |
| Rule | Path |
|---|---|
| Disallow | /cgi-bin |
| Disallow | /? |
| Disallow | /wp- |
| Disallow | /wp/ |
| Disallow | *?s= |
| Disallow | *%26s%3D |
| Disallow | /search/ |
| Disallow | /author/ |
| Disallow | /users/ |
| Disallow | */trackback |
| Disallow | */feed |
| Disallow | */rss |
| Disallow | */embed |
| Disallow | */wlwmanifest.xml |
| Disallow | /xmlrpc.php |
| Allow | */uploads |
| Allow | /*/*.js |
| Allow | /*/*.css |
| Allow | /wp-*.png |
| Allow | /wp-*.jpg |
| Allow | /wp-*.jpeg |
| Allow | /wp-*.gif |
| Allow | /wp-admin/admin-ajax.php |
Other Records
| Field | Value |
|---|---|
| sitemap | https://the-warsaw.com/sitemap.xml |
| sitemap | https://the-warsaw.com/sitemap.xml.gz |
Warnings
- 8 invalid lines.
- `clean-param` is not a known field.
Comments