upsihologa.com.ua
robots.txt

Robots Exclusion Standard data for upsihologa.com.ua

Resource Scan

Scan Details

Site Domain upsihologa.com.ua
Base Domain upsihologa.com.ua
Scan Status Ok
Last Scan2024-10-10T16:10:27+00:00
Next Scan 2024-10-17T16:10:27+00:00

Last Scan

Scanned2024-10-10T16:10:27+00:00
URL https://upsihologa.com.ua/robots.txt
Domain IPs 104.21.65.234, 172.67.194.67, 2606:4700:3031::ac43:c243, 2606:4700:3034::6815:41ea
Response IP 104.21.65.234
Found Yes
Hash 8995157dd65ea6c47be0f1b450adbbacc4f20e99da88d4933dff8cf31a7fab75
SimHash 6a40d620d531

Groups

*

Rule Path
Disallow /admin/
Disallow /books/
Disallow /link_to_us.html
Disallow /adtraining.html
Disallow /inc/
Disallow /fetch*

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

yandex

Rule Path
Disallow /users/
Disallow /ask/ask.php?id=27612
Disallow /ask/ask.php?id=16653
Disallow /ask/ask.php?id=22297
Disallow /ask/ask.php?id=27062
Disallow /ask/ask.php?id=23002
Disallow /ask/ask.php?id=7560
Disallow /ask/ask.php?id=5078
Disallow /ask/ask.php?id=27247

Other Records

Field Value
crawl-delay 5

mail.ru_bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

bingbot

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

blexbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

obot

Rule Path
Disallow /

lcc

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

Warnings

  • `host` is not a known field.