Log In Sign Up

belle.ai
robots.txt

Robots Exclusion Standard data for belle.ai

Resource Scan

Scan Details

Site Domain belle.ai
Base Domain belle.ai
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2026-01-14T02:53:53+00:00
Next Scan 2026-04-14T02:53:53+00:00

Last Successful Scan

Scanned2025-08-24T09:11:38+00:00
URL https://belle.ai/robots.txt
Redirect https://www.belle.ai/robots.txt
Redirect Domain www.belle.ai
Redirect Base belle.ai
Domain IPs 104.21.112.1, 104.21.16.1, 104.21.32.1, 104.21.48.1, 104.21.64.1, 104.21.80.1, 104.21.96.1, 2606:4700:3030::6815:1001, 2606:4700:3030::6815:2001, 2606:4700:3030::6815:3001, 2606:4700:3030::6815:4001, 2606:4700:3030::6815:5001, 2606:4700:3030::6815:6001, 2606:4700:3030::6815:7001
Redirect IPs 2600:9000:28c2:2600:0:4a43:4b80:93a1, 2600:9000:28c2:5400:0:4a43:4b80:93a1, 2600:9000:28c2:6000:0:4a43:4b80:93a1, 2600:9000:28c2:b400:0:4a43:4b80:93a1, 2600:9000:28c2:c00:0:4a43:4b80:93a1, 2600:9000:28c2:c200:0:4a43:4b80:93a1, 2600:9000:28c2:cc00:0:4a43:4b80:93a1, 2600:9000:28c2:f800:0:4a43:4b80:93a1, 3.171.198.101, 3.171.198.119, 3.171.198.69, 3.171.198.95
Response IP 3.171.198.101
Found Yes
Hash 687c8855e472ad1245fd5c6c111ba720967855bf41f33b609e1ab4abb0401cb1
SimHash a52a1ab2f7a0

Groups

*

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

slurp

Rule Path
Allow /
Disallow /admin/
Disallow /private/
Disallow /internal/
Disallow /temp/
Disallow /tmp/
Disallow /*.tmp$
Disallow /*.temp$
Disallow /dev/
Disallow /test/
Disallow /staging/
Allow /assets/
Allow /images/
Allow /css/
Allow /js/
Allow /documents/
Disallow /api/
Disallow /rest/
Disallow /graphql/
Disallow /*.json$
Disallow /*.xml$
Disallow /*.txt$
Disallow /*.log$
Allow /documents/*.pdf
Allow /documents/*.doc
Allow /documents/*.docx

Other Records

Field Value
crawl-delay 1
Back to top

Other Records

Field Value
sitemap https://belle.ai/sitemap.xml
Back to top

Comments

  • BelleTorus Corporation - Robots.txt
  • https://www.robotstxt.org/robotstxt.html
  • Allow all crawlers
  • Allow crawling of all content
  • Sitemap location
  • Crawl-delay for respectful crawling
  • Specific rules for major search engines
  • Block access to admin areas (if any)
  • Block access to temporary files
  • Block access to development files
  • Allow access to important directories
  • Block access to API endpoints (if they exist)
  • Block access to configuration files
  • Allow access to public documents
Back to top