Log In Sign Up

ia.loopus.tech
robots.txt

Robots Exclusion Standard data for ia.loopus.tech

Resource Scan

Scan Details

Site Domain ia.loopus.tech
Base Domain loopus.tech
Scan Status Ok
Last Scan2025-11-06T04:49:39+00:00
Next Scan 2025-12-06T04:49:39+00:00

Last Scan

Scanned2025-11-06T04:49:39+00:00
URL http://ia.loopus.tech/robots.txt
Redirect https://dev.loopus.tech/robots.txt
Redirect Domain dev.loopus.tech
Redirect Base loopus.tech
Domain IPs 79.137.33.21
Redirect IPs 79.137.33.21
Response IP 79.137.33.21
Found Yes
Hash c03aa4e68aa8543ae27f389ef8c973f70f13d212b3dda62112d5b2dd537e6e45
SimHash 4d188e82e793

Groups

*

Rule Path
Allow /
Disallow /api/
Disallow /admin/
Disallow /*.json$
Disallow /*.xml$

Other Records

Field Value
crawl-delay 1

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 0

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

slurp

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

duckduckbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://loopus.tech/sitemap.xml
Back to top

Comments

  • Robots.txt for LoopusTech
  • Professional Web Development Services
  • Allow all search engines
  • Sitemap location
  • Crawl-delay for politeness (in seconds)
  • Specific rules for major search engines
  • Block bad bots
Back to top