agro-consultant.net
robots.txt
Robots Exclusion Standard data for agro-consultant.net
Resource Scan
Scan Details
| Site Domain | agro-consultant.net |
| Base Domain | agro-consultant.net |
| Scan Status | Ok |
| Last Scan | 2024-11-09T16:35:35+00:00 |
| Next Scan | 2024-11-16T16:35:35+00:00 |
Last Scan
| Scanned | 2024-11-09T16:35:35+00:00 |
| URL | https://agro-consultant.net/robots.txt |
| Redirect | https://www.agro-consultant.net/robots.txt |
| Redirect Domain | www.agro-consultant.net |
| Redirect Base | agro-consultant.net |
| Domain IPs | 212.5.51.67 |
| Redirect IPs | 212.5.51.67 |
| Response IP | 212.5.51.67 |
| Found | Yes |
| Hash | fb7281423844aaadb94b8192a592fe3a6528638dc335497778acc0fc81f0f98e |
| SimHash | a80cd5406a18 |
Groups
admantx platform semantic analyzer - admantx inc. - www.admantx.com - support@admantx.com
| Rule | Path |
|---|---|
| Disallow | / |
attribot/1.1 (compatible; attribot-site; http://static.attribyte.com/robotreadme.txt)
| Rule | Path |
|---|---|
| Disallow | / |
mozilla/5.0 (compatible; domainappender /1.0; +http://www.profound.net/domainappender)
| Rule | Path |
|---|---|
| Disallow | / |
mozilla/5.0 (compatible; domainsigmacrawler/0.1; +http://domainsigma.com/robot)
| Rule | Path |
|---|---|
| Disallow | / |
mozilla/5.0 (compatible; grapeshotcrawler/2.0; +http://www.grapeshot.co.uk/crawler.php)
| Rule | Path |
|---|---|
| Disallow | / |
mozilla/5.0 (compatible; meanpathbot/1.0; +http://www.meanpath.com/meanpathbot.html)
| Rule | Path |
|---|---|
| Disallow | / |
mozilla/5.0 (compatible; mj12bot/v1.4.5; http://www.majestic12.co.uk/bot.php?+)
| Rule | Path |
|---|---|
| Disallow | / |
mozilla/5.0 (compatible; pad-bot/9.0; +http://www.descargarprogramagratis.com/)
| Rule | Path |
|---|---|
| Disallow | / |
mozilla/5.0 (compatible; paperlibot/2.1; http://support.paper.li/entries/20023257-what-is-paper-li)
| Rule | Path |
|---|---|
| Disallow | / |
mozilla/5.0 (compatible; umbot-ln/1.0; mailto: crawling@ubermetrics-technologies.com)
| Rule | Path |
|---|---|
| Disallow | / |
mozilla/5.0 (macintosh; intel mac os x 10.9; rv:28.0) gecko/20100101 firefox/28.0 (flipboardproxy/1.1; +http://flipboard.com/browserproxy)
| Rule | Path |
|---|---|
| Disallow | / |
mozilla/5.0 (windows; u; windows nt 5.1; en-us; rv:1.8.1.6) gecko/20070725 firefox/2.0.0.6 - james bot - webcrawler http://cognitiveseo.com/bot.html
| Rule | Path |
|---|---|
| Disallow | / |
mozilla/5.0 (windows; u; windows nt 5.1; en-us; rv:1.9.1.2) gecko/20090729 firefox/3.5.2 (.net clr 3.5.30729; diffbot/0.1; +http://www.diffbot.com)
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /administrator/ |
| Disallow | /bin/ |
| Disallow | /cache/ |
| Disallow | /cli/ |
| Disallow | /components/ |
| Disallow | /includes/ |
| Disallow | /installation/ |
| Disallow | /language/ |
| Disallow | /layouts/ |
| Disallow | /libraries/ |
| Disallow | /logs/ |
| Disallow | /modules/ |
| Disallow | /plugins/ |
| Disallow | /tmp/ |
| Disallow | /testovi/ |
| Disallow | /agromrezhata/ |
| Disallow | /bg/ |
| Disallow | /joomla/ |
| Disallow | /blog |
| Allow | /plugins/system/jch_optimise/assets2/ |
| Allow | /plugins/system/jch_optimise/assets/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.agro-consultant.net/component/osmap/?view=xml&id=2&format=xml |
Warnings
- 6 invalid lines.