lexitalkai.com
robots.txt

Robots Exclusion Standard data for lexitalkai.com

Resource Scan

Scan Details

Site Domain lexitalkai.com
Base Domain lexitalkai.com
Scan Status Ok
Last Scan2026-04-14T23:42:16+00:00
Next Scan 2026-04-21T23:42:16+00:00

Last Scan

Scanned2026-04-14T23:42:16+00:00
URL https://lexitalkai.com/robots.txt
Redirect https://www.lexitalkai.com/robots.txt
Redirect Domain www.lexitalkai.com
Redirect Base lexitalkai.com
Domain IPs 104.26.4.35, 104.26.5.35, 172.67.70.114, 2606:4700:20::681a:423, 2606:4700:20::681a:523, 2606:4700:20::ac43:4672
Redirect IPs 104.26.4.35, 104.26.5.35, 172.67.70.114, 2606:4700:20::681a:423, 2606:4700:20::681a:523, 2606:4700:20::ac43:4672
Response IP 172.67.70.114
Found Yes
Hash 7b93f8a94ded60e0267d7d0d64f38ab4608bbe378bc12112bea75ad3554ea458
SimHash b108c041c4b0

Groups

facebookexternalhit

Rule Path
Allow /

facebookbot

Rule Path
Allow /

meta-externalagent

Rule Path
Allow /

twitterbot

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

slackbot

Rule Path
Allow /

discordbot

Rule Path
Allow /

google-adstxt

Rule Path
Allow /

coccocbot-web

Rule Path
Allow /

barkrowler

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ahrefsbotseo

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

iboubot

Rule Path
Disallow /

*

Rule Path
Allow /
Disallow /admin/
Disallow /private/
Disallow /api/internal/
Allow /web/
Allow /download
Allow /hub

Other Records

Field Value
sitemap https://www.lexitalkai.com/sitemap.xml

Comments

  • Sitemap
  • Disallow admin and private areas
  • Allow important pages

Warnings

  • `content-signal` is not a known field.