Log In Sign Up

indexergo.com
robots.txt

Robots Exclusion Standard data for indexergo.com

Resource Scan

Scan Details

Site Domain indexergo.com
Base Domain indexergo.com
Scan Status Ok
Last Scan2026-01-06T22:53:10+00:00
Next Scan 2026-01-13T22:53:10+00:00

Last Scan

Scanned2026-01-06T22:53:10+00:00
URL https://indexergo.com/robots.txt
Domain IPs 104.21.57.99, 172.67.162.238, 2606:4700:3036::6815:3963, 2606:4700:3036::ac43:a2ee
Response IP 172.67.162.238
Found Yes
Hash 5c2f74dd8a0b8ff4c5cd12bc507ee5ebc8c006a36f575061027350a7647fb60e
SimHash 400443c2d315

Groups

the knowledge ai

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semanticscholarbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

*

Rule Path
Allow /
Back to top

Comments

  • User-agent: Baiduspider
  • Disallow: /
  • User-agent: Yandex
  • Disallow: /
Back to top