linktree.com
robots.txt

Robots Exclusion Standard data for linktree.com

Resource Scan

Scan Details

Site Domain linktree.com
Base Domain linktree.com
Scan Status Ok
Last Scan2024-05-30T03:43:08+00:00
Next Scan 2024-06-06T03:43:08+00:00

Last Scan

Scanned2024-05-30T03:43:08+00:00
URL https://linktree.com/robots.txt
Redirect https://linktr.ee/robots.txt
Redirect Domain linktr.ee
Redirect Base linktr.ee
Domain IPs 151.101.130.133, 151.101.194.133, 151.101.2.133, 151.101.66.133
Redirect IPs 151.101.130.133, 151.101.194.133, 151.101.2.133, 151.101.66.133
Response IP 151.101.66.133
Found Yes
Hash 8ba4693c2ed1b94ba44b52d9038d122664ccaf0955ee921b2f86b4c6301cea37
SimHash 80555b01cdb1

Groups

applebot

Rule Path
Disallow /admin

baiduspider

Rule Path
Disallow /admin

bingbot

Rule Path
Disallow /admin

discordbot

Rule Path
Disallow /admin

facebookexternalhit

Rule Path
Disallow /admin

googlebot

Rule Path
Disallow /admin

googlebot-image

Rule Path
Disallow /admin

google-inspectiontool

Rule Path
Disallow /admin

ia_archiver

Rule Path
Disallow /admin

linkedinbot

Rule Path
Disallow /admin

msnbot

Rule Path
Disallow /admin

naverbot

Rule Path
Disallow /admin

pinterestbot

Rule Path
Disallow /admin

screaming frog seo spider

Rule Path
Disallow /admin

seznambot

Rule Path
Disallow /admin

slurp

Rule Path
Disallow /admin

teoma

Rule Path
Disallow /admin

telegrambot

Rule Path
Disallow /admin

twitterbot

Rule Path
Disallow /admin

yandex

Rule Path
Disallow /admin

yeti

Rule Path
Disallow /admin

snapchatadsbot

Rule Path
Disallow /admin

semrushbot

Rule Path
Disallow /admin

pinterestbot

Rule Path
Disallow /

*

Rule Path
Disallow /

Other Records

Field Value
sitemap https://linktr.ee/discover/profile-directory/sitemapindex.xml
sitemap https://linktr.ee/s/sitemap-index.xml
sitemap https://linktr.ee/s/sitemap-0.xml
sitemap https://linktr.ee/marketplace/sitemap.xml

Comments

  • We allow everything from the root
  • and then set noindex on our backends for paths that we want to Disallow
  • to prevent this list from being too complex and long