myprivatetutor.qa
robots.txt

Robots Exclusion Standard data for myprivatetutor.qa

Resource Scan

Scan Details

Site Domain myprivatetutor.qa
Base Domain myprivatetutor.qa
Scan Status Ok
Last Scan2025-09-27T16:24:01+00:00
Next Scan 2025-10-27T16:24:01+00:00

Last Scan

Scanned2025-09-27T16:24:01+00:00
URL https://www.myprivatetutor.qa/robots.txt
Domain IPs 104.26.2.90, 104.26.3.90, 172.67.68.14, 2606:4700:20::681a:25a, 2606:4700:20::681a:35a, 2606:4700:20::ac43:440e
Response IP 104.26.3.90
Found Yes
Hash 945600dcfd39db1d80af2901cd773004236c6f492f8c2fbe094acb611994beca
SimHash 495cdd60e21b

Groups

*

Rule Path
Disallow *?pg=*
Disallow /blog/search?q=*

gptbot

Rule Path
Allow /

anthropic-ai

Rule Path
Allow /

claudebot

Rule Path
Allow /

claude

Rule Path
Allow /

yandexbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

gaisbot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

bomborabot

Rule Path
Disallow /

Comments

  • Disallow: /terms
  • Disallow: /privacy
  • Disallow: /blognew
  • tkt: 160101-06245