biznet.ru
robots.txt

Robots Exclusion Standard data for biznet.ru

Resource Scan

Scan Details

Site Domain biznet.ru
Base Domain biznet.ru
Scan Status Ok
Last Scan5/19/2025, 3:50:59 PM
Next Scan 5/26/2025, 3:50:59 PM

Last Scan

Scanned5/19/2025, 3:50:59 PM
URL https://biznet.ru/robots.txt
Domain IPs 104.21.36.67, 172.67.186.197, 2606:4700:3030::6815:2443, 2606:4700:3036::ac43:bac5
Response IP 172.67.186.197
Found Yes
Hash b43d3e8f4b1daa16b99e4b1532f8bccffc69a4d7e081df586cb0d6843505f0a9
SimHash 5f0d9d7dfeb2

Groups

amazonbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

criteobot/0.1

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

yandexdirect

Rule Path
Disallow

mediapartners-google*

Rule Path
Disallow

*

Rule Path
Disallow /api/
Disallow /admin/*
Disallow /cgi-bin/
Disallow /datastore/
Disallow /plugins/
Disallow /system/
Disallow /themes/
Disallow /go/
Disallow /403error.php
Disallow /404error.php
Disallow /500error.php
Disallow /Credits.txt
Disallow /error.php
Disallow /upgrading.html
Disallow /*tab%3D*
Disallow /*?app=*
Disallow /*sortby%3D*
Disallow /profile/*/content/
Disallow /*?do=*
Allow /applications/core/interface/imageproxy/imageproxy.php?img=*
Allow /applications/core/interface/font/
Disallow /applications/
Disallow /_piwik/
Disallow /login
Disallow /register
Disallow /lostpassword
Disallow /search
Disallow /online
Disallow /staff
Disallow /unsubscribe
Disallow /contact
Disallow /notifications
Disallow /messenger
Disallow /*follow*
Disallow /activity
Disallow /*csrfKey*
Disallow /*member
Disallow /*view%3D
Disallow /*content?type
Disallow /*page%3D0
Disallow /*page%3D1
Disallow /ptopic
Disallow /*lofiversion
Disallow /*mode%3D*
Disallow /*settings*
Disallow /*?ct=*
Disallow /*custom_date*
Disallow /*fromLogout
Disallow /*type%3Dnotification
Disallow /*%26key%3D*
Disallow /*submit*
Disallow /*ignore*
Disallow /*create*
Disallow /uploads/emoticons
Disallow /uploads/logs
Disallow /uploads/reactions

Other Records

Field Value
sitemap https://www.biznet.ru/sitemap.php

Comments

  • Sitemap...
  • Disallow directory
  • Disallow files
  • Querystring
  • Disallow: /index.php?*
  • Allow specific parts
  • additional parts
  • Disallow: /cache
  • Disallow: /discover
  • Disallow: /attachments
  • Disallow: /*user
  • Disallow: /*profile
  • Disallow: /*pass*

Warnings

  • `host` is not a known field.