religions.unian.net
robots.txt

Robots Exclusion Standard data for religions.unian.net

Resource Scan

Scan Details

Site Domain religions.unian.net
Base Domain unian.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-11-14T19:32:30+00:00
Next Scan 2024-12-14T19:32:30+00:00

Last Successful Scan

Scanned2024-10-16T18:31:11+00:00
URL https://religions.unian.net/robots.txt
Domain IPs 195.137.240.103, 195.137.240.83
Response IP 195.137.240.83
Found Yes
Hash 487c8d4ae729c9d3553fd5c836c4bb537335031b97b54833185fe1dda8c42fc7
SimHash 781dcac3b21f

Groups

*

Rule Path
Allow *.jpg
Allow *.gif
Allow *.svg
Allow *.css
Allow *.png
Allow *.js
Disallow /cdn-cgi/
Disallow /comments/
Disallow /search/
Disallow /content/
Disallow /profile/
Disallow /redirect/
Disallow /pogoda/widget/
Disallow /m/comments/
Disallow /m/search/
Disallow /m/content/
Disallow /m/profile/
Disallow /m/redirect/
Disallow /m/pogoda/widget/
Disallow *utm_source%3D*
Disallow *utm_medium%3D*
Disallow *utm_campaign%3D*
Disallow /?url=*
Disallow /*_sm_byp
Disallow /*CFID%3D*
Disallow /div_location%3D
Disallow /*?refresh
Disallow /?quot
Disallow /?_ga
Disallow /m/?_ga
Disallow /?desktop
Disallow /m/?desktop
Disallow /?device_view
Disallow /m/?device_view
Disallow /?desktop_pogoda
Disallow /m/?desktop_pogoda
Disallow /?news_date
Disallow /m/?news_date
Disallow /?can_infinite_scroll
Disallow /m/?can_infinite_scroll

googlebot

Rule Path
Allow /*.jpg
Allow /*.gif
Allow /*.svg
Allow /*.css
Allow /*.png
Allow /*.js
Disallow /comments/
Disallow /search/
Disallow /profile/

yandex

Rule Path
Allow /*.jpg
Allow /*.gif
Allow /*.svg
Allow /*.css
Allow /*.png
Allow /*.js
Disallow /comments/
Disallow /search/
Disallow /profile/

googlebot-image

Rule Path
Allow *.jpg
Allow *.gif
Allow *.svg
Allow *.png

googlebot-video

Rule Path
Allow /*.jpg
Allow /*.gif
Allow /*.svg
Allow /*.css
Allow /*.png
Allow /*.js

yandexvideo

Rule Path
Allow /*.jpg
Allow /*.gif
Allow /*.svg
Allow /*.css
Allow /*.png
Allow /*.js
Allow /*?page=

Other Records

Field Value
sitemap https://religions.unian.net/sitemap.xml
sitemap https://religions.unian.net/sitemaps/sitemaps/sitemap.rubrics.all_rubrics.xml
sitemap https://religions.unian.net/sitemap_google.last.news.xml
sitemap https://religions.unian.net/sitemap.last.news.xml