inet-shop.su
robots.txt

Robots Exclusion Standard data for inet-shop.su

Resource Scan

Scan Details

Site Domain inet-shop.su
Base Domain inet-shop.su
Scan Status Ok
Last Scan2024-09-19T12:17:43+00:00
Next Scan 2024-10-19T12:17:43+00:00

Last Scan

Scanned2024-09-19T12:17:43+00:00
URL http://inet-shop.su/robots.txt
Domain IPs 185.26.122.82
Response IP 185.26.122.82
Found Yes
Hash fd2e8e8ea59a7134a9ecd9cdcc9837cc323c195cb4a52f8d88b716bb34378752
SimHash 03345f03da39

Groups

mediapartners-google

Rule Path
Disallow

*

Rule Path
Disallow /cat/
Disallow /ru.html
Disallow /en.html
Disallow /city.html
Disallow /login.html
Disallow /register.html
Disallow /new.html
Disallow /addnews.html
Disallow /*search*
Disallow /goto*
Disallow /cpanel.html
Disallow /profile.html
Disallow /logout.html
Disallow /print*
Disallow /vip*
Disallow /note.html
Disallow /noteprint.html
Disallow /contacts.html
Disallow /advert/ru.html
Disallow /advert/en.html
Disallow /advert/city.html
Disallow /advert/login.html
Disallow /advert/register.html
Disallow /advert/new.html
Disallow /advert/addnews.html
Disallow /advert/*search*
Disallow /advert/goto*
Disallow /advert/cpanel.html
Disallow /advert/profile.html
Disallow /advert/logout.html
Disallow /advert/print*
Disallow /advert/vip*
Disallow /advert/note.html
Disallow /advert/noteprint.html
Disallow /advert/contacts.html
Disallow /go/

ahrefsbot

Rule Path
Disallow /

ahrefsbot/5.1

Rule Path
Disallow /

ahrefsbot/5.2

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot/1.2~bl

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mj12bot/v1.4.7

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

blexbot/1.0

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10