financenews.com.br
robots.txt
Robots Exclusion Standard data for financenews.com.br
Resource Scan
Scan Details
Site Domain | financenews.com.br |
Base Domain | financenews.com.br |
Scan Status | Ok |
Last Scan | 2024-09-25T11:06:14+00:00 |
Next Scan | 2024-10-02T11:06:14+00:00 |
Last Scan
Scanned | 2024-09-25T11:06:14+00:00 |
URL | https://financenews.com.br/robots.txt |
Domain IPs | 104.21.43.129, 172.67.179.137, 2606:4700:3032::ac43:b389, 2606:4700:3037::6815:2b81 |
Response IP | 104.21.43.129 |
Found | Yes |
Hash | 237790cd6ac01cbe46d51490421e9e8a8cef1b1b0fab7f90ab120da1d6d9b738 |
SimHash | 5354d1d36b31 |
Groups
*
Rule | Path |
---|---|
Disallow | /*pmin |
Disallow | /*pmax |
Disallow | /*cart |
Disallow | /*srule |
Disallow | /*account |
Disallow | /*checkout |
Disallow | /*view%3Dgrid |
Disallow | /*view%3Dlist |
Disallow | /*quickview |
Disallow | /*search |
Disallow | /*wishlist |
Disallow | /*shipping-tracking |
Disallow | /*contentsearch |
Disallow | /*search-ajax |
Disallow | /*payment-details |
Disallow | /*add-payment |
Disallow | /*order |
Disallow | /*newsletter |
Disallow | /*login |
Disallow | /*logout |
Disallow | /*404 |
Disallow | /*giftregistry |
Disallow | /*error |
Disallow | /*offline |
Disallow | /*help |
Disallow | /*compare |
Disallow | /*cart |
Disallow | /*revieworder |
Disallow | /*billing |
Disallow | /*addressbook |
Disallow | /*add-address |
Disallow | /*register |
Disallow | /*profile |
baiduspider
baiduspider-video
baiduspider-image
mozilla/5.0 (compatible; baiduspider/2.0; +http://www.baidu.com/search/spider.html)
mozilla/5.0 (compatible; baiduspider/3.0; +http://www.baidu.com/search/spider.html)
mozilla/5.0 (compatible; baiduspider/4.0; +http://www.baidu.com/search/spider.html)
mozilla/5.0 (compatible; baiduspider/5.0; +http://www.baidu.com/search/spider.html)
baiduspider/2.0
baiduspider/3.0
baiduspider/4.0
baiduspider/5.0
Rule | Path |
---|---|
Disallow | / |
Warnings
- 5 invalid lines.
- `noindex` is not a known field.
Comments