Log In Sign Up

bitbyte24.com
robots.txt

Robots Exclusion Standard data for bitbyte24.com

Resource Scan

Scan Details

Site Domain bitbyte24.com
Base Domain bitbyte24.com
Scan Status Ok
Last Scan2026-03-25T20:27:51+00:00
Next Scan 2026-04-01T20:27:51+00:00

Last Scan

Scanned2026-03-25T20:27:51+00:00
URL https://bitbyte24.com/robots.txt
Domain IPs 104.21.84.150, 172.67.194.46, 2606:4700:3031::ac43:c22e, 2606:4700:3037::6815:5496
Response IP 172.67.194.46
Found Yes
Hash 92dff98f925f714309cc7e32cfef0c359796819cf250de1b07f7a1220a0c90d8
SimHash 2834fb6140b7

Groups

*

Rule Path
Allow /
Disallow /api/
Disallow /_next/
Disallow /dashboard
Disallow /config.js
Allow /manifest.json
Allow /icon.svg
Allow /og-image.png
Allow /robots.txt
Allow /sitemap.xml

googlebot

Rule Path
Allow /
Disallow /dashboard
Disallow /config.js

googlebot-image

Rule Path
Allow /

bingbot

Rule Path
Allow /
Disallow /dashboard

Other Records

Field Value
crawl-delay 1

twitterbot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

slackbot

Rule Path
Allow /

discordbot

Rule Path
Allow /

applebot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://bitbyte24.com/sitemap.xml
sitemap https://stock-scan.bitbyte24.com/screener/news-sitemap.xml
Back to top

Comments

  • BitByte24 - Real-Time Stock Screener
  • https://bitbyte24.com
  • Allow all crawlers
  • Sitemap locations
  • Allow important static files
  • Google specific
  • Google Image Bot — allow crawling images for Google Images
  • Bing specific
  • Allow social media crawlers for rich previews
  • Allow Apple / iMessage link previews
Back to top