brokerforum.com
robots.txt
Robots Exclusion Standard data for brokerforum.com
Resource Scan
Scan Details
| Site Domain | brokerforum.com |
| Base Domain | brokerforum.com |
| Scan Status | Ok |
| Last Scan | 2024-08-31T20:41:14+00:00 |
| Next Scan | 2024-09-30T20:41:14+00:00 |
Last Scan
| Scanned | 2024-08-31T20:41:14+00:00 |
| URL | https://brokerforum.com/robots.txt |
| Redirect | https://www.brokerforum.com/robots.txt |
| Redirect Domain | www.brokerforum.com |
| Redirect Base | brokerforum.com |
| Domain IPs | 45.60.151.100, 45.60.171.100 |
| Redirect IPs | 45.60.159.100 |
| Response IP | 45.60.159.100 |
| Found | Yes |
| Hash | d605a2794b90d6e4b6bc8fb97d608da10c70307156807a5f37e571414cabfd1a |
| SimHash | 3430dddae691 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /bf?%2FSESSION%2FLOGIN |
| Disallow | /bf?%2FCO%2FJOIN_NOW |
| Disallow | /bf?%2Fpassword%2FPassword%24ValidateIdentityPrep |
| Disallow | /bf?%2FLINK%2FBANNER_PREP |
googlebot
| Rule | Path |
|---|---|
| Disallow | /*cgi/zh* |
| Disallow | /*cgi/ko* |
| Disallow | /*cgi/en* |
| Disallow | /components-parts/popular-electronic-parts-*-kR-en.jsa* |
| Disallow | /tbf/free-trial-ru.jsp* |
| Disallow | /tbf/free-trial-zh.jsp* |
| Disallow | /tbf/free-trial-ko.jsp* |
| Disallow | /components-parts/*-kRS-en.jsa* |
| Disallow | /components-parts/*-ekR-en.jsa* |
| Disallow | /electronic-components-search-*-ck-en.jsa* |
| Disallow | /bf-geolocation-californie-en.jsa* |
slurp
| Rule | Path |
|---|---|
| Disallow | /*cgi/zh* |
| Disallow | /*cgi/ko* |
| Disallow | /*cgi/en* |
| Disallow | /bf-geolocation-californie-en.jsa* |
msnbot
| Rule | Path |
|---|---|
| Disallow | /*cgi/zh* |
| Disallow | /*cgi/ko* |
| Disallow | /*cgi/en* |
| Disallow | /bf-geolocation-californie-en.jsa* |
baiduspider
| Rule | Path |
|---|---|
| Disallow | /*cgi/zh* |
| Disallow | /*cgi/ko* |
| Disallow | /*cgi/en* |
| Disallow | /bf-geolocation-californie-en.jsa* |
sogou develop spider
| Rule | Path |
|---|---|
| Disallow | /*cgi/zh* |
| Disallow | /*cgi/ko* |
| Disallow | /*cgi/en* |
| Disallow | /bf-geolocation-californie-en.jsa* |
sogou orion spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm
| Product | Comment |
|---|---|
| sogou orion spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm | 07) |
| Rule | Path |
|---|---|
| Disallow | /*cgi/zh* |
| Disallow | /*cgi/ko* |
| Disallow | /*cgi/en* |
| Disallow | /bf-geolocation-californie-en.jsa* |
sogou spider
| Rule | Path |
|---|---|
| Disallow | /*cgi/zh* |
| Disallow | /*cgi/ko* |
| Disallow | /*cgi/en* |
| Disallow | /bf-geolocation-californie-en.jsa* |
sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm
| Product | Comment |
|---|---|
| sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm | 07) |
| Rule | Path |
|---|---|
| Disallow | /*cgi/zh* |
| Disallow | /*cgi/ko* |
| Disallow | /*cgi/en* |
| Disallow | /bf-geolocation-californie-en.jsa* |