Log In Sign Up

therapist.com
robots.txt

Robots Exclusion Standard data for therapist.com

Resource Scan

Scan Details

Site Domain therapist.com
Base Domain therapist.com
Scan Status Ok
Last Scan2024-11-09T07:59:10+00:00
Next Scan 2024-11-16T07:59:10+00:00

Last Scan

Scanned2024-11-09T07:59:10+00:00
URL https://therapist.com/robots.txt
Domain IPs 13.107.246.46, 2620:1ec:bdf::46
Response IP 13.107.246.46
Found Yes
Hash a38f10e9d64af5375b926ce3f0ee4543f7b9d0be20200f00edf583b4702f95a1
SimHash 992080386ab6

Groups

*

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-login.php
Disallow /*?filter
Disallow /*?f=
Disallow /*%26filter
Disallow /*%26f%3D
Disallow /21600225
Disallow /avatars/
Disallow /api/abtasty/
Disallow /*?category=*%2Fus%2F
Disallow /*/category%3D
Disallow /*/f%3D
Disallow /us/*/?category=*&provider_results_page=
Disallow /us/*/*/?category=*&provider_results_page=
Disallow /us/*/*/*/
Disallow /us/*/*/*/?category=
Disallow /us/*/*/*/?provider_results_page=
Disallow /us/*/*/*/*/
Disallow /us/*/*/*/*/?category=
Disallow /us/*/*/*/*/?provider_results_page=
Disallow /*?provider_results_page=5*0
Disallow /*?provider_results_page=5*1
Disallow /*?provider_results_page=5*2
Disallow /*?provider_results_page=5*3
Disallow /*?provider_results_page=5*4
Disallow /*?provider_results_page=5*5
Disallow /*?provider_results_page=5*6
Disallow /*?provider_results_page=5*7
Disallow /*?provider_results_page=5*8
Disallow /*?provider_results_page=5*9
Disallow /*?provider_results_page=6*0
Disallow /*?provider_results_page=6*1
Disallow /*?provider_results_page=6*2
Disallow /*?provider_results_page=6*3
Disallow /*?provider_results_page=6*4
Disallow /*?provider_results_page=6*5
Disallow /*?provider_results_page=6*6
Disallow /*?provider_results_page=6*7
Disallow /*?provider_results_page=6*8
Disallow /*?provider_results_page=6*9
Disallow /*?provider_results_page=7*0
Disallow /*?provider_results_page=7*1
Disallow /*?provider_results_page=7*2
Disallow /*?provider_results_page=7*3
Disallow /*?provider_results_page=7*4
Disallow /*?provider_results_page=7*5
Disallow /*?provider_results_page=7*6
Disallow /*?provider_results_page=7*7
Disallow /*?provider_results_page=7*8
Disallow /*?provider_results_page=7*9
Disallow /*?provider_results_page=8*0
Disallow /*?provider_results_page=8*1
Disallow /*?provider_results_page=8*2
Disallow /*?provider_results_page=8*3
Disallow /*?provider_results_page=8*4
Disallow /*?provider_results_page=8*5
Disallow /*?provider_results_page=8*6
Disallow /*?provider_results_page=8*7
Disallow /*?provider_results_page=8*8
Disallow /*?provider_results_page=8*9
Disallow /*?provider_results_page=9*0
Disallow /*?provider_results_page=9*1
Disallow /*?provider_results_page=9*2
Disallow /*?provider_results_page=9*3
Disallow /*?provider_results_page=9*4
Disallow /*?provider_results_page=9*5
Disallow /*?provider_results_page=9*6
Disallow /*?provider_results_page=9*7
Disallow /*?provider_results_page=9*8
Disallow /*?provider_results_page=9*9
Disallow /us/Alabama/
Disallow /us/Alaska/
Disallow /us/Arizona/
Disallow /us/Arkansas/
Disallow /us/California/
Disallow /us/Colorado/
Disallow /us/Connecticut/
Disallow /us/Delaware/
Disallow /us/Florida/
Disallow /us/Georgia/
Disallow /us/Hawaii/
Disallow /us/Idaho/
Disallow /us/Illinois/
Disallow /us/Indiana/
Disallow /us/Iowa/
Disallow /us/Kansas/
Disallow /us/Kentucky/
Disallow /us/Louisiana/
Disallow /us/Maine/
Disallow /us/Maryland/
Disallow /us/Massachusetts/
Disallow /us/Michigan/
Disallow /us/Minnesota/
Disallow /us/Mississippi/
Disallow /us/Missouri/
Disallow /us/Montana/
Disallow /us/Nebraska/
Disallow /us/Nevada/
Disallow /us/New
Disallow /us/North
Disallow /us/Ohio/
Disallow /us/Oklahoma/
Disallow /us/Oregon/
Disallow /us/Pennsylvania/
Disallow /us/Rhode
Disallow /us/South
Disallow /us/Tennessee/
Disallow /us/Texas/
Disallow /us/Utah/
Disallow /us/Vermont/
Disallow /us/Virginia/
Disallow /us/Washington/
Disallow /us/West
Disallow /us/Wisconsin/
Disallow /us/Wyoming/
Disallow /*?order=
Disallow /*%26order%3D
Disallow /*?__hstc=
Disallow /*?__hssc=
Disallow /*?__hsfp=
Disallow /*?_hsenc=
Disallow /*?_hsmi=
Disallow /*%26__hstc%3D
Disallow /*%26__hssc%3D
Disallow /*%26__hsfp%3D
Disallow /*%26_hsenc%3D
Disallow /*%26_hsmi%3D

adbeat

Rule Path
Disallow /

admantx

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bomborabot

Rule Path
Disallow /

bvbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

cincraw

Rule Path
Disallow /

clickagy

Rule Path
Disallow /

cocolyzebot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

domainstatsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

domcopbot

Rule Path
Disallow /

embed.ly

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

gulperbot

Rule Path
Disallow /

httrack

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

internet-structure-research-project-bot

Rule Path
Disallow /

linguee

Rule Path
Disallow /

linkfluence

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

nimbostratus

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

onalyticabot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

pubmatic crawler bot

Rule Path
Disallow /

riddlerbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

sbl-bot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

seekport

Rule Path
Disallow /

semanticbot

Rule Path
Disallow /

semantic-visions

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

sqlmap

Rule Path
Disallow /

traackr

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

voluumdsp

Rule Path
Disallow /

wc-test-dev-bot

Rule Path
Disallow /

whatcms

Rule Path
Disallow /

webtechbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://therapist.com/sitemap_index.xml
Back to top

Comments

  • I hope you brought the tacos
  • Disallow search results filter parameters
  • Disallow Google Ad Manager Path
  • Disallow BetterHelp Path
  • Disallow AB Tasty Path
  • Disallow invalid pagination links
  • Disallow invalid profile page results links
  • Disallow state and city category pagination
  • Disallow zip code level pages
  • Disallow address level pages
  • Disallow search results pagination after page 50
  • Disallow state pages that contain proper casing
  • Disallow sort parameters
  • Disallow HubSpot parameters
  • Disallow They forgot to bring tacos
Back to top