gl5.ru
robots.txt

Robots Exclusion Standard data for gl5.ru

Resource Scan

Scan Details

Site Domain gl5.ru
Base Domain gl5.ru
Scan Status Ok
Last Scan2024-06-22T00:50:57+00:00
Next Scan 2024-06-29T00:50:57+00:00

Last Scan

Scanned2024-06-22T00:50:57+00:00
URL https://gl5.ru/robots.txt
Redirect https://www.gl5.ru/robots.txt
Redirect Domain www.gl5.ru
Redirect Base gl5.ru
Domain IPs 104.21.42.164, 172.67.163.139, 2606:4700:3036::6815:2aa4, 2606:4700:3037::ac43:a38b
Redirect IPs 104.21.42.164, 172.67.163.139, 2606:4700:3036::6815:2aa4, 2606:4700:3037::ac43:a38b
Response IP 172.67.163.139
Found Yes
Hash d73c8d0a35bcb261f5f6a2885fc13d776d26d11ffbcc301638951d0158295eb2
SimHash 9a1cce434b90

Groups

yandex

Rule Path
Disallow

mediapartners-google

Rule Path
Disallow

dotbot

Rule Path
Disallow /

hybridbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

ntentbot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

*

Rule Path
Disallow

Warnings

  • `host` is not a known field.