Log In Sign Up

makserv.ee
robots.txt

Robots Exclusion Standard data for makserv.ee

Resource Scan

Scan Details

Site Domain makserv.ee
Base Domain makserv.ee
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2025-08-28T04:29:41+00:00
Next Scan 2025-09-27T04:29:41+00:00

Last Successful Scan

Scanned2025-07-07T03:46:47+00:00
URL https://makserv.ee/robots.txt
Domain IPs 217.146.69.14
Response IP 217.146.69.14
Found Yes
Hash 78f94a35394399e168dc851d1520af6bd6d2d6579479b0374a3262078fb9d194
SimHash 4008cf026311

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /cart/
Disallow /my-account/
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
crawl-delay 10

googlebot

Rule Path
Disallow /wp-admin/
Disallow /cart/
Disallow /my-account/

googlebot-image

Rule Path
Disallow /*?*

ahrefsbot
ahrefssiteaudit
semrushbot
semrushbot
semrushbot-sa
mj12bot

Rule Path
Disallow /
Disallow /*?add-to-cart=
Disallow /*?orderby=
Disallow /*?stock_status=
Disallow /*?remove_item=
Allow /*?page=1&
Allow /*?page=2&
Allow /*?page=1$
Allow /*?page=2$
Disallow /*?*page=

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

linguee

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top

Comments

  • block irrelevant filters (forever)
  • limit pagination
Back to top