window.unian.net
robots.txt
Robots Exclusion Standard data for window.unian.net
Resource Scan
Scan Details
Site Domain | window.unian.net |
Base Domain | unian.net |
Scan Status | Failed |
Failure Stage | Fetching resource. |
Failure Reason | Server returned a client error. |
Last Scan | 2024-10-01T16:26:07+00:00 |
Next Scan | 2024-12-30T16:26:07+00:00 |
Last Successful Scan
Scanned | 2023-06-06T08:59:23+00:00 |
URL | https://window.unian.net/robots.txt |
Domain IPs | 104.18.24.128, 104.18.25.128 |
Response IP | 104.18.24.128 |
Found | Yes |
Hash | 18c3b45e6959aecfd57e6fca5b664b64faaf651d74677cf344bd2ab40468967f |
SimHash | 781dcac3b217 |
Groups
*
Rule | Path |
---|---|
Allow | *.jpg |
Allow | *.gif |
Allow | *.svg |
Allow | *.css |
Allow | *.png |
Allow | *.js |
Disallow | /cdn-cgi/ |
Disallow | /comments/ |
Disallow | /search/ |
Disallow | /content/ |
Disallow | /profile/ |
Disallow | /redirect/ |
Disallow | /pogoda/widget/ |
Disallow | /m/comments/ |
Disallow | /m/search/ |
Disallow | /m/content/ |
Disallow | /m/profile/ |
Disallow | /m/redirect/ |
Disallow | /m/pogoda/widget/ |
Disallow | *utm_source%3D* |
Disallow | *utm_medium%3D* |
Disallow | *utm_campaign%3D* |
Disallow | /?url=* |
Disallow | /*_sm_byp |
Disallow | /*CFID%3D* |
Disallow | /div_location%3D |
Disallow | /*?refresh |
Disallow | /?quot |
Disallow | /?_ga |
Disallow | /m/?_ga |
Disallow | /?desktop |
Disallow | /m/?desktop |
Disallow | /?device_view |
Disallow | /m/?device_view |
Disallow | /?desktop_pogoda |
Disallow | /m/?desktop_pogoda |
Disallow | /?news_date |
Disallow | /m/?news_date |
Disallow | /?can_infinite_scroll |
Disallow | /m/?can_infinite_scroll |
googlebot
Rule | Path |
---|---|
Allow | /*.jpg |
Allow | /*.gif |
Allow | /*.svg |
Allow | /*.css |
Allow | /*.png |
Allow | /*.js |
Disallow | /comments/ |
Disallow | /search/ |
Disallow | /profile/ |
yandex
Rule | Path |
---|---|
Allow | /*.jpg |
Allow | /*.gif |
Allow | /*.svg |
Allow | /*.css |
Allow | /*.png |
Allow | /*.js |
Disallow | /comments/ |
Disallow | /search/ |
Disallow | /profile/ |
Other Records
Field | Value |
---|---|
sitemap | https://window.unian.net/sitemap.xml |
sitemap | https://window.unian.net/sitemaps/sitemaps/sitemap.rubrics.all_rubrics.xml |
sitemap | https://window.unian.net/sitemap_google.last.news.xml |
sitemap | https://window.unian.net/sitemap.last.news.xml |