sefan.ru
robots.txt

Robots Exclusion Standard data for sefan.ru

Resource Scan

Scan Details

Site Domain sefan.ru
Base Domain sefan.ru
Scan Status Ok
Last Scan2024-05-19T04:59:35+00:00
Next Scan 2024-06-18T04:59:35+00:00

Last Scan

Scanned2024-05-19T04:59:35+00:00
URL https://sefan.ru/robots.txt
Domain IPs 77.222.61.60
Response IP 77.222.61.60
Found Yes
Hash 84ff42ab8f042aa52d75d34ae0b286c6cc4f34f396965f7753c34fc0e86b663c
SimHash e117dc448213

Groups

*

Rule Path
Disallow /data/loads/*
Disallow /*-104/*
Disallow /1*
Disallow /immortal/k*
Allow /*lan%3Dru*style%3D2*
Allow /*lan%3Den*style%3D2*
Disallow /*lan%3D*style%3D*
Allow /*style%3D2*lan%3Dru*
Allow /*style%3D2*lan%3Den*
Disallow /*style%3D*lan%3D*
Allow /stories/ru/2*
Allow /stories/ru/2/search/*/body.html
Allow /stories/ru/2/search/top*.html
Disallow /stories/ru/2/search/*.html
Disallow /stories/ru/2/*_print.html
Disallow /stories/ru/2/random*
Disallow /stories/*
Allow /music/ru/2*
Disallow /music/*
Disallow /*/uk/*
Disallow /*/download-*
Disallow /pictures/
Disallow /i/*
Disallow /*?*no_js_rsz=*
Disallow /*?*sfn_stp=*
Disallow /*?*no_js_rds=*
Disallow /*?*PHPSESSID=*

ahrefsbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

coccocbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

linguee

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

nimbostratus

Rule Path
Disallow /

obot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

Warnings

  • `clean-param` is not a known field.