agriline-sa.com
robots.txt

Robots Exclusion Standard data for agriline-sa.com

Resource Scan

Scan Details

Site Domain agriline-sa.com
Base Domain agriline-sa.com
Scan Status Ok
Last Scan2024-11-14T16:28:21+00:00
Next Scan 2024-11-21T16:28:21+00:00

Last Scan

Scanned2024-11-14T16:28:21+00:00
URL https://agriline-sa.com/robots.txt
Redirect https://agronetto-sa.com/robots.txt
Redirect Domain agronetto-sa.com
Redirect Base agronetto-sa.com
Domain IPs 144.76.229.22
Redirect IPs 144.76.229.22
Response IP 144.76.229.22
Found Yes
Hash b8e890000303689eb3fa8855ed7300ffe8e5f2e0bec12bd6dba758b1bdafbd9d
SimHash e2af4f41e4c7

Groups

*

Rule Path
Disallow */version
Disallow */resizable
Disallow */focusable
Disallow */controlgroup
Disallow */checkboxradio
Disallow */mouse
Disallow */draggable
Disallow */button
Disallow /big-photos/
Disallow /api/
Disallow /autopl_map.php
Disallow /ban_stat.php
Disallow /banner_preview.php
Disallow /captcha/
Disallow /change-locale/
Disallow /change-measurements/
Disallow /charts/
Disallow /compare/
Disallow /components/
Disallow /contact-with-dealer/
Disallow /dealer_registration.php
Disallow /dealers-pages/
Disallow /dealers/map/
Disallow /delivery/tw/
Disallow /dev/
Disallow /e/
Disallow /email_form.php
Disallow /export/
Disallow /header-menu/
Disallow /image_val.php
Disallow /includes/
Disallow /index-menu/
Disallow /login/
Disallow /message-backup-controller/
Disallow /ni/
Disallow /order/
Disallow /orig_pages/
Disallow /payment/
Disallow /performatives.php
Disallow /performatives.php
Disallow /place_ad.php
Disallow /prices_example.php
Disallow /print-pdf/
Disallow /registration
Disallow /registration/
Disallow /rss/
Disallow /sales-history.php
Disallow /sales-history/
Disallow /sales-search-history/
Disallow /sales/
Disallow /search.php
Disallow /search/
Disallow /search/suggest/
Disallow /search_text.php
Disallow /secure_code.php
Disallow /share/
Disallow /stat/
Disallow /subscribe.php
Disallow /test-work/
Disallow /track-mail/
Disallow /translate.php
Disallow /translations/
Disallow /u.php
Disallow /ubl/index.php?adv_code=
Disallow /user_question/
Disallow /switchLocale/
Disallow /rchat/
Disallow /start
Disallow /user/messages/
Disallow /vehicle-inspection/
Disallow /vehicle-leasing/
Disallow /vehicle-service-request/
Disallow /place-for-free/
Disallow /feedback/
Disallow /impressum-d/
Disallow /vehicles-reviews/create/
Disallow /ni/user-menu-links/
Disallow /send-to-messenger/
Disallow /my/
Disallow /stock.php

baiduspider
baiduspider-image
baiduspider-video
baiduspider-news
baiduspider-favo
baiduspider-cpro
baiduspider-ads

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

petabot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

claudebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

gptbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

facebookexternalhit

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

blexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

imagesiftbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

Other Records

Field Value
sitemap https://agronetto-sa.com/sitemap.xml