stockbit.com
robots.txt

Robots Exclusion Standard data for stockbit.com

Resource Scan

Scan Details

Site Domain stockbit.com
Base Domain stockbit.com
Scan Status Ok
Last Scan2024-09-25T15:46:19+00:00
Next Scan 2024-10-02T15:46:19+00:00

Last Scan

Scanned2024-09-25T15:46:19+00:00
URL https://stockbit.com/robots.txt
Domain IPs 18.155.68.121, 18.155.68.16, 18.155.68.55, 18.155.68.64
Response IP 18.155.68.16
Found Yes
Hash 463a650df0c8bc0c82b3ec1d1e25ea6763125e53cf897f53701fbbca8f4d2790
SimHash bc943070d9f5

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /tmp/
Disallow /assets/
Disallow /charting/
Disallow /build/
Disallow /mockups/
Disallow /system/
Disallow /node_modules/
Disallow /upload/
Disallow /company/chartTimeSpan/
Disallow /company/indexes/
Disallow /fundachart.html

twitterbot

Rule Path
Allow /assets/img/
Allow /
Allow /symbol/*
Allow /post/*

Other Records

Field Value
sitemap http://sitemap.stockbit.com/stockbit.symbol.xml
sitemap http://sitemap.stockbit.com/stockbit.stream.xml

Comments

  • Certain social media sites are whitelisted to allow crawlers to access page markup when links to /assets/img are shared.