agribegri.com
robots.txt
Robots Exclusion Standard data for agribegri.com
Resource Scan
Scan Details
| Site Domain | agribegri.com |
| Base Domain | agribegri.com |
| Scan Status | Ok |
| Last Scan | 2024-10-03T18:09:00+00:00 |
| Next Scan | 2024-10-17T18:09:00+00:00 |
Last Scan
| Scanned | 2024-10-03T18:09:00+00:00 |
| URL | https://agribegri.com/robots.txt |
| Domain IPs | 194.61.31.149 |
| Response IP | 194.61.31.149 |
| Found | Yes |
| Hash | 635e5c4d16412044093929c0aff2d224c6942b25d13510baf5b1c147539b7f7e |
| SimHash | d11fde70450b |
Groups
dotbot
dotbot
mj12bot
semrushbot
dalvik
dalvik/2.1.0 (linux; u; android 9; sm-g955f build/ppr1.180610.011)
dalvik/2.1.0
bidswitchbot/1.0
blexbot
yandexbot
seznambot
ahrefsbot
mail.ru bot
yandexmobilebot
yisouspider
grapeshotcrawler
surdotlybot
mojeekbot
sogou web spider
zoominfobot
coccocbot-web/1.0
trident
proximic
grapeshotcrawler/2.0
bidswitchbot/1.0
yandexbot/3.0
lwp::simple/6.22
uptimerobot/2.0
empty user agent string
baiduspider
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Allow | /admin/images/prod_image/ |
| Allow | /assets |
| Allow | /admin/css/ |
| Allow | /admin/js/ |
| Disallow | /admin |
| Disallow | /design |
| Disallow | /font |
| Disallow | /phpmailer |
| Disallow | /sharetastic |
| Disallow | /userfile |
| Disallow | /betalive |
| Disallow | /API |
| Disallow | /staging.agribegri.com |
| Disallow | /staging |
| Disallow | /captcha1.php |
| Disallow | /captcha.php |
| Disallow | /set_session_search_cat.php |
Other Records
| Field | Value |
|---|---|
| sitemap | https://agribegri.com/sitemap.xml |