superjob.ru
robots.txt

Robots Exclusion Standard data for superjob.ru

Resource Scan

Scan Details

Site Domain superjob.ru
Base Domain superjob.ru
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-11-12T16:42:01+00:00
Next Scan 2024-11-26T16:42:01+00:00

Last Successful Scan

Scanned2024-10-28T09:33:20+00:00
URL https://superjob.ru/robots.txt
Redirect https://www.superjob.ru/robots.txt
Redirect Domain www.superjob.ru
Redirect Base superjob.ru
Domain IPs 185.71.67.17
Redirect IPs 185.71.67.17
Response IP 185.71.67.17
Found Yes
Hash 9a0ea8b105390428d331d37f17bafa0c7ce04562961908c24f6327c432e0f70e
SimHash 6d92cd70862b

Groups

*

Rule Path
Allow /cdn/min/file/sj/css/
Allow /hr/export_vacs.html$
Allow /hr/mobile/$
Allow /hr/bill.html$
Allow /resume/create/$
Allow /vacancy/search_form.html$
Allow /*.html?page*
Allow /*.html?geo*
Allow /*.html?geo*&page*
Allow /*.html?t%5B*
Allow /*.html?t%5B*&page*
Allow /*.html?r%5B*
Allow /*.html?r%5B*&page*
Allow /*.html?o%5B*
Allow /*.html?o%5B*&page*
Allow /*?best=*
Allow /vacancy/search/$
Allow /resume/search_resume.html$
Allow /*?tab=reviews
Disallow /*curtain
Disallow /*gk-akup*
Disallow */message/
Disallow */print/
Disallow /albums/
Disallow /auth/
Disallow /authorize/
Disallow /app/
Disallow /athens/
Disallow /banners/
Disallow /cgi-bin/
Disallow /cito/
Disallow /contextadvert/
Disallow /cv/
Disallow /feedback
Disallow /files/
Disallow /functions/
Disallow /groups*
Disallow /hobby/
Disallow /hr/
Disallow /hr/.html?page
Disallow /hrom/
Disallow /images/
Disallow /includes/
Disallow /info/checkup/
Disallow /info/read.html
Disallow /info/set_simple_auth_cookie.html
Disallow /info/show_mail_return_reason.html
Disallow /no_mobile/
Disallow /oldversion
Disallow /partners/
Disallow /pohojie-resume/
Disallow /profile/
Disallow /reklama/
Disallow /request-call/
Disallow /resume/*/uploadPortfolioPhoto/
Disallow /resume/create/
Disallow /resume/download/
Disallow /resume/edit/
Disallow /resume/mailing_history-*.html
Disallow /resume/received_messages-
Disallow /resume/search_resume.html?
Disallow /resume/send_resume
Disallow /resume/send/*
Disallow /resume/update_datepub.html
Disallow /resume/views-
Disallow /rus/
Disallow /salary-review.html
Disallow /sn/
Disallow /sso/
Disallow /static/
Disallow /use_mobile/
Disallow /user/
Disallow /unsubscribe/
Disallow /vacancies/
Disallow /vacancy/
Disallow /vakansii/*.html?$
Disallow /w3timages/
Disallow /ws/
Disallow /z/
Disallow /zarplatomer/
Disallow /*type_of_work
Disallow /*citizenship
Disallow /*?experience
Disallow /*?period
Disallow /*?_openstat
Disallow /*?keywords
Disallow /*?showRestrictionBlock
Disallow /*escaped_fragments
Disallow /*drugoe%26page
Disallow /*payment_defined
Disallow /*utm_source
Disallow /*subs_msg_id
Disallow /*tab%3Dquestion%26modal
Disallow /*userType
Disallow /clients/?*key=
Disallow /*?*
Disallow /*.html?geo*&*
Disallow /*.html?geo*&*&page*
Disallow /*.html?t*&*
Disallow /*.html?t*&*&page*
Disallow /*.html?r*&*
Disallow /*.html?r*&*&page*
Disallow /*.html?o*&*
Disallow /*.html?o*&*&page*
Disallow /company_industry/*?*geo=
Disallow /resume/search_resume.html*
Disallow /vacancy/search/*
Disallow /community/resume/
Disallow /show-contacts-form-page/
Disallow /hr/*.html?*page*
Disallow /hr/inbox*
Disallow /hr/inbox.html?*page*
Disallow /hr/users.html?*page*
Disallow /notifications/*
Disallow /pohojie-resume/*
Disallow *complaint*
Disallow /i/*
Disallow *apply*
Disallow *cover_letter*
Disallow /ml/*
Disallow *review-published*
Disallow /vakansii/contacts/*
Disallow /mini-resume/*
Disallow /clients/*/about.html

googlebot

Rule Path
Allow /cdn/min/file/sj/css/
Allow /hr/export_vacs.html$
Allow /hr/mobile/$
Allow /hr/bill.html$
Allow /resume/create/$
Allow /vacancy/search_form.html$
Allow /*.html?page*
Allow /*.html?geo*
Allow /*.html?geo*&page*
Allow /*.html?t%5B*
Allow /*.html?t%5B*&page*
Allow /*.html?r%5B*
Allow /*.html?r%5B*&page*
Allow /*.html?o%5B*
Allow /*.html?o%5B*&page*
Allow /*?best=*
Allow /vacancy/search/$
Allow /resume/search_resume.html$
Allow /*?tab=reviews
Disallow /*curtain
Disallow */message/
Disallow */print/
Disallow /albums/
Disallow /app/
Disallow /athens/
Disallow /auth/
Disallow /authorize/
Disallow /cgi-bin/
Disallow /cito/
Disallow /contextadvert/
Disallow /cv/
Disallow /export/
Disallow /feedback
Disallow /files/
Disallow /functions/
Disallow /groups*
Disallow /hobby/
Disallow /hr/
Disallow /hr/.html?page
Disallow /hrom/
Disallow /includes/
Disallow /info/checkup/
Disallow /info/read.html
Disallow /info/set_simple_auth_cookie.html
Disallow /info/show_mail_return_reason.html
Disallow /no_mobile/
Disallow /oldversion
Disallow /partners/
Disallow /pohojie-resume/
Disallow /profile/
Disallow /reklama/
Disallow /request-call/
Disallow /resume/*/uploadPortfolioPhoto/
Disallow /resume/download/
Disallow /resume/edit/
Disallow /resume/mailing_history-*.html
Disallow /resume/received_messages-
Disallow /resume/search_resume.html?
Disallow /resume/send_resume
Disallow /resume/send/*
Disallow /resume/update_datepub.html
Disallow /resume/views-
Disallow /rus/
Disallow /salary-review.html
Disallow /sn/
Disallow /sso/
Disallow /static/
Disallow /unsubscribe/
Disallow /use_mobile/
Disallow /user/
Disallow /vacancies/
Disallow /vacancy/
Disallow /w3timages/
Disallow /ws/
Disallow /z/
Disallow /zarplatomer/
Disallow /*type_of_work
Disallow /*citizenship
Disallow /*?experience
Disallow /*?period
Disallow /*?_openstat
Disallow /*?keywords
Disallow /*?showRestrictionBlock
Disallow /*escaped_fragments
Disallow /*drugoe%26page
Disallow /*payment_defined
Disallow /*utm_source
Disallow /*subs_msg_id
Disallow /*tab%3Dquestion%26modal
Disallow /*userType
Disallow /clients/?*key=
Disallow /*?*
Disallow /*.html?geo*&*
Disallow /*.html?geo*&*&page*
Disallow /*.html?t*&*
Disallow /*.html?t*&*&page*
Disallow /*.html?r*&*
Disallow /*.html?r*&*&page*
Disallow /*.html?o*&*
Disallow /*.html?o*&*&page*
Disallow /company_industry/*?*geo=
Disallow /resume/search_resume.html*
Disallow /vacancy/search/*
Disallow /community/resume/
Disallow /show-contacts-form-page/
Disallow /hr/*.html?*page*
Disallow /hr/inbox*
Disallow /hr/inbox.html?*page*
Disallow /hr/users.html?*page*
Disallow /notifications/*
Disallow /pohojie-resume/*
Disallow *complaint*
Disallow /i/*
Disallow *apply*
Disallow *cover_letter*
Disallow /ml/*
Disallow *review-published*
Disallow /vakansii/contacts/*
Disallow /mini-resume/*
Disallow /clients/*/about.html

yandex

Rule Path
Allow /cdn/min/file/sj/css/
Allow /hr/export_vacs.html$
Allow /hr/mobile/$
Allow /hr/bill.html$
Allow /resume/create/$
Allow /vacancy/search_form.html$
Allow /*.html?page*
Allow /*.html?geo*
Allow /*.html?geo*&page*
Allow /*.html?t%5B*
Allow /*.html?t%5B*&page*
Allow /*.html?r%5B*
Allow /*.html?r%5B*&page*
Allow /*.html?o%5B*
Allow /*.html?o%5B*&page*
Allow /*?best=*
Allow /vacancy/search/$
Allow /resume/search_resume.html$
Allow /*?tab=reviews
Allow /proizvodstvennyj_kalendar/2024/
Allow /proizvodstvennyj_kalendar/2025/
Disallow /*curtain
Disallow /*gk-akup*
Disallow */message/
Disallow */print/
Disallow /albums/
Disallow /auth/
Disallow /authorize/
Disallow /app/
Disallow /athens/
Disallow /banners/
Disallow /cgi-bin/
Disallow /cito/
Disallow /contextadvert/
Disallow /cv/
Disallow /feedback
Disallow /files/
Disallow /functions/
Disallow /groups*
Disallow /hobby/
Disallow /hr/
Disallow /hr/.html?page
Disallow /hrom/
Disallow /images/
Disallow /includes/
Disallow /info/checkup/
Disallow /info/read.html
Disallow /info/set_simple_auth_cookie.html
Disallow /info/show_mail_return_reason.html
Disallow /no_mobile/
Disallow /oldversion
Disallow /partners/
Disallow /pohojie-resume/
Disallow /profile/
Disallow /reklama/
Disallow /request-call/
Disallow /resume/*/uploadPortfolioPhoto/
Disallow /resume/create/
Disallow /resume/download/
Disallow /resume/edit/
Disallow /resume/mailing_history-*.html
Disallow /resume/received_messages-
Disallow /resume/search_resume.html?
Disallow /resume/send_resume
Disallow /resume/send/*
Disallow /resume/update_datepub.html
Disallow /resume/views-
Disallow /rus/
Disallow /salary-review.html
Disallow /sn/
Disallow /sso/
Disallow /static/
Disallow /use_mobile/
Disallow /user/
Disallow /unsubscribe/
Disallow /vacancies/
Disallow /vacancy/
Disallow /w3timages/
Disallow /ws/
Disallow /z/
Disallow /zarplatomer/
Disallow /*type_of_work
Disallow /*citizenship
Disallow /*?experience
Disallow /*?period
Disallow /*?_openstat
Disallow /*?keywords
Disallow /*?showRestrictionBlock
Disallow /*escaped_fragments
Disallow /*drugoe%26page
Disallow /*payment_defined
Disallow /*utm_source
Disallow /*subs_msg_id
Disallow /*tab%3Dquestion%26modal
Disallow /*userType
Disallow /clients/?*key=
Disallow /*?*
Disallow /*.html?geo*&*
Disallow /*.html?geo*&*&page*
Disallow /*.html?t*&*
Disallow /*.html?t*&*&page*
Disallow /*.html?r*&*
Disallow /*.html?r*&*&page*
Disallow /*.html?o*&*
Disallow /*.html?o*&*&page*
Disallow /company_industry/*?*geo=
Disallow /resume/search_resume.html*
Disallow /vacancy/search/*
Disallow /community/resume/
Disallow /proizvodstvennyj_kalendar/20*
Disallow /show-contacts-form-page/
Disallow /hr/*.html?*page*
Disallow /hr/inbox*
Disallow /hr/inbox.html?*page*
Disallow /hr/users.html?*page*
Disallow /notifications/*
Disallow /pohojie-resume/*
Disallow *complaint*
Disallow /i/*
Disallow *apply*
Disallow *cover_letter*
Disallow /ml/*
Disallow *review-published*
Disallow /vakansii/contacts/*
Disallow /mini-resume/*
Disallow /clients/*/about.html

yandexdirect

Rule Path
Allow *?*
Disallow /community/*

twiceler

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

webinject

Rule Path
Disallow /

irlbot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

mail.ru

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

Other Records

Field Value
sitemap https://www.superjob.ru/sitemap_index.xml
sitemap https://www.superjob.ru/sitemap_index.xml
sitemap https://www.superjob.ru/sitemap_index.xml

Warnings

  • `clean-param` is not a known field.
  • `host` is not a known field.