meetext.us
robots.txt

Robots Exclusion Standard data for meetext.us

Resource Scan

Scan Details

Site Domain meetext.us
Base Domain meetext.us
Scan Status Ok
Last Scan2025-08-26T08:11:49+00:00
Next Scan 2025-09-02T08:11:49+00:00

Last Scan

Scanned2025-08-26T08:11:49+00:00
URL https://meetext.us/robots.txt
Redirect https://www.meetext.us/robots.txt
Redirect Domain www.meetext.us
Redirect Base meetext.us
Domain IPs 76.76.21.21
Redirect IPs 216.198.79.1, 64.29.17.65
Response IP 64.29.17.1
Found Yes
Hash 1f9cfc50a7d2e052d69f3c5d880db8d9dbf9c993f50b6742c774bcdbbb0fe24c
SimHash 239492f0e603

Groups

*

Rule Path
Allow /
Disallow /dashboard/
Disallow /api/
Disallow /_next/
Disallow /admin/
Allow /opengraph-image.png
Allow /twitter-image.png
Allow /favicon.ico
Allow /ai-models/
Allow /meeting-summarization/
Allow /custom-agents/
Allow /business-intelligence/

gptbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

chatgpt-user

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

google-extended

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

anthropic-ai

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

oai-searchbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://meetext.app/sitemap.xml
sitemap https://meetext.app/ai-sitemap.xml

Comments

  • Allow important resources
  • AI Search Optimization
  • AI Crawler Support
  • Sitemaps