rexlabs.io
robots.txt

Robots Exclusion Standard data for rexlabs.io

Resource Scan

Scan Details

Site Domain rexlabs.io
Base Domain rexlabs.io
Scan Status Ok
Last Scan2025-10-20T17:25:59+00:00
Next Scan 2025-11-19T17:25:59+00:00

Last Scan

Scanned2025-10-20T17:25:59+00:00
URL https://rexlabs.io/robots.txt
Redirect https://www.rexsoftware.com/robots.txt
Redirect Domain www.rexsoftware.com
Redirect Base rexsoftware.com
Domain IPs 75.2.70.75, 99.83.190.102
Redirect IPs 13.203.125.58, 13.233.175.166, 3.109.243.18
Response IP 13.233.175.166
Found Yes
Hash e84f4102d4c5ee3d202237a4ca31a6fb3b60a23fef566094c82f030809d9443d
SimHash d5d29d658634

Groups

*

Rule Path
Disallow /success/
Disallow /l/
Disallow /internal-tools/
Disallow /careers/
Disallow /c/
Disallow /e/
Disallow /logos/
Disallow /faqs/
Disallow /testimonials/
Disallow /content-call-to-actions/
Disallow /content-seo-blurbs/
Disallow /inside/
Disallow /agency-type/
Disallow /region/
Disallow /faq-tags/
Disallow /tags/
Disallow /s/
Disallow /admin-menu-ad-areas/
Disallow /menu-ad-area/
Disallow /v/
Disallow /nz/success/
Disallow /nz/l/
Disallow /nz/internal-tools/
Disallow /nz/careers/
Disallow /nz/c/
Disallow /nz/e/
Disallow /nz/logos/
Disallow /nz/faqs/
Disallow /nz/testimonials/
Disallow /nz/content-call-to-actions/
Disallow /nz/content-seo-blurbs/
Disallow /nz/inside/
Disallow /nz/agency-type/
Disallow /nz/region/
Disallow /nz/faq-tags/
Disallow /nz/tags/
Disallow /nz/s/
Disallow /nz/admin-menu-ad-areas/
Disallow /nz/menu-ad-area/
Disallow /nz/v/
Disallow /uk/success/
Disallow /uk/l/
Disallow /uk/internal-tools/
Disallow /uk/careers/
Disallow /uk/c/
Disallow /uk/e/
Disallow /uk/logos/
Disallow /uk/faqs/
Disallow /uk/testimonials/
Disallow /uk/content-call-to-actions/
Disallow /uk/content-seo-blurbs/
Disallow /uk/inside/
Disallow /uk/agency-type/
Disallow /uk/region/
Disallow /uk/faq-tags/
Disallow /uk/tags/
Disallow /uk/s/
Disallow /uk/admin-menu-ad-areas/
Disallow /uk/menu-ad-area/
Disallow /uk/v/

semrushbot-sa

Rule Path
Allow /

siteauditbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

semrushbot-si

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.rexsoftware.com/sitemap.xml