agro-sells.com
robots.txt

Robots Exclusion Standard data for agro-sells.com

Resource Scan

Scan Details

Site Domain agro-sells.com
Base Domain agro-sells.com
Scan Status Ok
Last Scan2024-11-12T00:07:12+00:00
Next Scan 2024-11-19T00:07:12+00:00

Last Scan

Scanned2024-11-12T00:07:12+00:00
URL https://agro-sells.com/robots.txt
Domain IPs 45.154.116.158
Response IP 45.154.116.158
Found Yes
Hash edb9b4a6e9fbf576ef159e109b7e451b4b15b05f597ac7e07a8aad2ee6a1a886
SimHash 2d5cbdcccbc1

Groups

googlebot

Rule Path
Disallow /ru/cabinet/
Disallow /cabinet/
Disallow /company/open
Disallow /ru/company/open
Disallow /ru/payments/liqpay/services
Disallow /payments/liqpay/services
Disallow /ru/terms
Disallow /terms
Disallow /ru/items-company
Disallow /items-company
Disallow /ru/items
Disallow /items
Disallow /ru/faq
Disallow /faq
Disallow /ru/privacy-policy
Disallow /privacy-policy
Disallow /ru/search/all
Disallow /search/all
Disallow /ru/user/login
Disallow /user/login
Disallow /password/reset
Disallow /ru/password/reset
Disallow /user/register
Disallow /ru/user/register
Disallow /contacts
Disallow /item/add
Disallow /ru/item/add
Disallow /ru/contacts
Disallow /*?
Disallow /*?*

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

alphaseobot-sa

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://agro-sells.com/sitemap.xml
sitemap https://agro-sells.com/sitemap_categories.xml
sitemap https://agro-sells.com/sitemap_categories_regions.xml
sitemap https://agro-sells.com/sitemap_companies.xml
sitemap https://agro-sells.com/sitemap_items.xml
sitemap https://agro-sells.com/sitemap_items_1.xml
sitemap https://agro-sells.com/sitemap_blog.xml