klerk.ru
robots.txt

Robots Exclusion Standard data for klerk.ru

Resource Scan

Scan Details

Site Domain klerk.ru
Base Domain klerk.ru
Scan Status Ok
Last Scan2024-11-10T22:18:23+00:00
Next Scan 2024-11-17T22:18:23+00:00

Last Scan

Scanned2024-11-10T22:18:23+00:00
URL https://klerk.ru/robots.txt
Redirect https://www.klerk.ru/robots.txt
Redirect Domain www.klerk.ru
Redirect Base klerk.ru
Domain IPs 185.229.9.134
Redirect IPs 185.229.9.134
Response IP 185.229.9.134
Found Yes
Hash 1abee49bae2e1ff5a11addac40b7e1b63a2334e23a3a711629c8d3f6ecbfd9d7
SimHash b019584bcc34

Groups

*

Rule Path
Allow /admin/res.xml
Allow *.js*
Allow *.css*
Allow /jobs/resume/$
Allow /consultations/?page=*
Disallow /jobs/resume/*
Disallow /*?
Disallow /print/
Disallow /*print
Disallow /search/
Disallow */attachload/
Disallow /ajax/
Disallow /learn/offer/*
Disallow /yindex.php
Disallow /yindex.php/*
Disallow /is/
Disallow /kadmin/
Disallow /editpost/
Disallow /user/*/resume/
Disallow /user/create/chat/*
Disallow /user/createchat/*
Disallow /user/*/community/
Disallow /user/*/comments/
Disallow /user/*/subscriptions/
Disallow /user/*/subscribers/
Disallow /user/*/courses/
Disallow /user/cabinet/*
Disallow */ocompany/*
Disallow */sort/*
Disallow */rubric/*

grapeshot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

riddler (http://riddler.io/about)

Rule Path
Disallow /

python-urllib/2.6

Rule Path
Disallow /

disco pump

Rule Path
Disallow /

wget

Rule Path
Disallow /

teleport pro

Rule Path
Disallow /

psbot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

netinfobot

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

npbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.klerk.ru/sitemap.xml