Log In Sign Up

returnit.online
robots.txt

Robots Exclusion Standard data for returnit.online

Resource Scan

Scan Details

Site Domain returnit.online
Base Domain returnit.online
Scan Status Ok
Last Scan2025-10-06T08:18:40+00:00
Next Scan 2025-10-13T08:18:40+00:00

Last Scan

Scanned2025-10-06T08:18:40+00:00
URL https://returnit.online/robots.txt
Domain IPs 34.111.179.208
Response IP 34.111.179.208
Found Yes
Hash 998923f2563422d69dc52ea73e4ee691d6cf4172347f92573f5048d86bc9bdd9
SimHash 7218dac265f2

Groups

*

Rule Path
Allow /
Allow /customer-mobile-app
Allow /about
Allow /pricing
Allow /contact
Allow /companies
Disallow /admin*
Disallow /dashboard*
Disallow /retailer*
Disallow /api/*
Disallow /driver*
Disallow /login
Disallow /register
Disallow /*.replit.dev$
Disallow /*.replit.app$

Other Records

Field Value
crawl-delay 1

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 0

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://returnit.online/sitemap.xml
Back to top

Comments

  • ReturnIt - Reverse Logistics Platform
  • robots.txt for https://returnit.online
  • Disallow admin and private pages
  • Disallow dev/staging environments
  • Crawl-delay
  • Sitemap
  • Special rules for Google
  • Special rules for Bing
  • Block bad bots
Back to top