Log In Sign Up

my.cosme.net
robots.txt

Robots Exclusion Standard data for my.cosme.net

Resource Scan

Scan Details

Site Domain my.cosme.net
Base Domain cosme.net
Scan Status Ok
Last Scan2024-09-22T19:40:55+00:00
Next Scan 2024-10-22T19:40:55+00:00

Last Scan

Scanned2024-09-22T19:40:55+00:00
URL https://my.cosme.net/robots.txt
Domain IPs 210.129.109.21
Response IP 210.129.109.21
Found Yes
Hash f440c8fc15247b02255aa7c71d88c3b95f35ed5db745a665c41ff2b549838d9f
SimHash 7410d9104834

Groups

*

Rule Path
Disallow /mylist/add/target/product/chid/
Disallow /config_entry_junction/show
Disallow /config/entry/review/add/product_id/
Disallow /*?_ga=
Disallow /*?*&_ga=
Disallow /*?_gl=
Disallow /*?*&_gl=

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

baidu

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

yeti

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15
Back to top

Comments

  • Block indexing of login request screen.
  • Block indexing of GA cross-domain tracking parameters
Back to top