allbetterapp.com
robots.txt

Robots Exclusion Standard data for allbetterapp.com

Resource Scan

Scan Details

Site Domain allbetterapp.com
Base Domain allbetterapp.com
Scan Status Ok
Last Scan2026-03-20T17:57:39+00:00
Next Scan 2026-03-27T17:57:39+00:00

Last Scan

Scanned2026-03-20T17:57:39+00:00
URL https://allbetterapp.com/robots.txt
Domain IPs 34.149.120.3, 34.160.17.71, 34.160.81.203, 35.227.194.51
Response IP 34.149.36.179
Found Yes
Hash 36b1382f7c97d52fca1d10b2b413af5e53d17368dedb259baa285e3bc2db5c9d
SimHash 5fbcc943eeaa

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Allow /llms.txt
Disallow /*?__hstc=
Disallow /*%26__hstc%3D
Disallow /*?__hssc=
Disallow /*%26__hssc%3D
Disallow /*?__hsfp=
Disallow /*%26__hsfp%3D
Disallow /*?currency=
Disallow /*?language=
Disallow /*%26currency%3D
Disallow /*%26language%3D
Disallow /*?gclid=
Disallow /*%26gclid%3D
Disallow /*?utm_content=
Disallow /*?utm_medium=
Disallow /*?utm_source=
Disallow /*?utm_campaign=
Disallow /*?ref=
Disallow /*?from=
Disallow /*?gl=
Disallow /*?_gl=
Disallow /*%26_gl%3D
Disallow /*/1000
Disallow /*//1000
Disallow /*?elementor_font=
Disallow /*?elementor_library=
Disallow /wp-content/cache/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-json/
Disallow /*?replytocom=
Disallow /repair-software/
Disallow /help/
Disallow /find__trashed/
Disallow /cleaning-5/
Disallow /electrician-3/
Disallow /plumber-4/
Disallow /snow-removal-3/
Disallow /handyman-3/
Disallow /*/feed/
Disallow /feed/
Disallow /*/2/
Disallow /*/3/
Disallow /*/4/
Disallow /*/5/
Disallow /blog/2/
Disallow /blog/3/
Disallow /blog/4/
Disallow /blog/5/
Disallow /blog/6/
Disallow /blog/7/
Disallow /blog/8/
Disallow /blog/9/
Disallow /blog/1*/
Disallow /blog/2*/
Disallow /faq/page/

ahrefssiteaudit

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

gptbot

Rule Path
Allow /

claudebot

Rule Path
Allow /

anthropic-ai

Rule Path
Allow /

google-extended

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

applebot-extended

Rule Path
Allow /

cohere-ai

Rule Path
Allow /

facebookbot

Rule Path
Allow /

Other Records

Field Value
sitemap https://allbetterapp.com/sitemap_index.xml
sitemap https://allbetterapp.com/llms.txt

Comments

  • Block HubSpot tracking parameters (legacy - no longer using HubSpot)
  • Block other tracking/junk parameters
  • Block /1000 spam suffix
  • Block Elementor internal URLs
  • Block WordPress internal/asset crawling
  • Block legacy paths that no longer exist
  • Block old service slug patterns
  • Block feed URLs
  • Block pagination beyond page 1
  • Throttle aggressive crawlers
  • AI Crawlers — explicitly allowed for AEO / Answer Engine Optimization