csdb-samara.ru
robots.txt

Robots Exclusion Standard data for csdb-samara.ru

Resource Scan

Scan Details

Site Domain csdb-samara.ru
Base Domain csdb-samara.ru
Scan Status Ok
Last Scan2025-08-17T05:42:56+00:00
Next Scan 2025-09-16T05:42:56+00:00

Last Scan

Scanned2025-08-17T05:42:56+00:00
URL https://csdb-samara.ru/robots.txt
Domain IPs 195.62.71.100
Response IP 195.62.71.100
Found Yes
Hash 431500fa1901015ab6827121d129ecf0573417927f2a29d5563659480c19f6cf
SimHash 6947d2e1c014

Groups

semrushbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

shodan

Rule Path
Disallow /

adidxbot

Rule Path
Disallow /

adsbot-google

Rule Path
Disallow /

aolbuild

Rule Path
Disallow /

applebot

Rule Path
Disallow /

bing

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

bingpreview

Rule Path
Disallow /

developers.facebook.com

Rule Path
Disallow /

docomo

Rule Path
Disallow /

duckduckgo

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

facebookplatform

Rule Path
Disallow /

feedfetcher-google

Rule Path
Disallow /

google-http-java-client

Rule Path
Disallow /

googlebot test

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /

googlebot-video

Rule Path
Disallow /

gsa-crawler

Rule Path
Disallow /

linkedinbot

Rule Path
Disallow /

mediapartners-google

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

msnbot-media

Rule Path
Disallow /

slackbot

Rule Path
Disallow /

slackbot-linkexpanding

Rule Path
Disallow /

slurp

Rule Path
Disallow /

twitterbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /user/
Disallow /backup/
Disallow /engine/
Disallow /language/
Disallow /templates/
Disallow /upgrade/
Disallow /uploads/
Disallow /autobackup.php
Disallow /admin.php
Disallow /index.php?do=addnews
Disallow /index.php?do=feedback
Disallow /index.php?do=lostpassword
Disallow /index.php?do=pm
Disallow /index.php?do=register
Disallow /index.php?do=stats
Disallow /index.php?do=search
Disallow /index.php?subaction=newposts
Disallow /?do=lastcomments
Disallow /statistics.html

*

Rule Path
Disallow /backup/
Disallow /user/
Disallow /engine/
Disallow /language/
Disallow /upgrade/
Disallow /autobackup.php
Disallow /admin.php
Disallow /index.php?do=addnews
Disallow /index.php?do=feedback
Disallow /index.php?do=lostpassword
Disallow /index.php?do=pm
Disallow /index.php?do=register
Disallow /index.php?do=stats
Disallow /index.php?do=search
Disallow /index.php?subaction=newposts
Disallow /?do=lastcomments
Disallow /statistics.html

Other Records

Field Value
sitemap http://csdb-samara.ru/sitemap.xml

Warnings

  • `host` is not a known field.