cosme.net
robots.txt

Robots Exclusion Standard data for cosme.net

Resource Scan

Scan Details

Site Domain cosme.net
Base Domain cosme.net
Scan Status Ok
Last Scan2024-10-07T13:16:04+00:00
Next Scan 2024-10-14T13:16:04+00:00

Last Scan

Scanned2024-10-07T13:16:04+00:00
URL https://cosme.net/robots.txt
Redirect https://www.cosme.net/robots.txt
Redirect Domain www.cosme.net
Redirect Base cosme.net
Domain IPs 210.129.109.21
Redirect IPs 13.35.186.125, 13.35.186.34, 13.35.186.48, 13.35.186.91
Response IP 3.165.102.30
Found Yes
Hash bdea29af8c0de5f5b60b8a6862681b981deb3e2e642b5ffabaf2e2960f3cfa93
SimHash 64826632a890

Groups

*

Rule Path
Disallow /isauth/login/
Disallow /isauth/signup/
Disallow /contact/report-board/board_id/
Disallow /contact/report-board/board_reply_id/
Disallow /contact/report-article/article_id/
Disallow /contact/report-review/review_id/
Disallow /chieco/question/entry
Disallow /chieco/auth/login
Disallow /beautist/article-entry/entry
Disallow /beautist/login/
Disallow /login
Disallow /bloombox/auth/
Disallow /bloombox/m/auth/
Disallow /user/login
Disallow /specialist/login
Disallow /forward-login/
Disallow /*preview
Disallow /*?ref=faceted-nav
Disallow /*?*&ref=faceted-nav
Disallow /*?fbclid=
Disallow /*?*&fbclid=
Disallow /*?_ga=
Disallow /*?*&_ga=
Disallow /*?_gl=
Disallow /*?*&_gl=
Disallow /api/
Disallow /categories/api/
Disallow /tags/api/
Disallow /brand/gadget*
Disallow /html/spc/mail/
Disallow /biche/biche/beacon
Disallow /products/*context*schema.org*
Disallow /uid-app/

baidu

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

yeti

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://www.cosme.net/sitemap.xml
sitemap https://www.cosme.net/sitemap-index.xml
sitemap https://www.cosme.net/biche/biche/sitemap.xml
sitemap https://www.cosme.net/chieco/sitemap.xml
sitemap https://www.cosme.net/sitemapindex-products.sitemap
sitemap https://www.cosme.net/sitemapindex-variations.sitemap
sitemap https://www.cosme.net/sitemapindex-reviews.sitemap
sitemap https://www.cosme.net/sitemapindex-reviews-photo.sitemap
sitemap https://www.cosme.net/sitemapindex-content.sitemap
sitemap https://www.cosme.net/sitemapindex-blog.sitemap
sitemap https://www.cosme.net/sitemapindex-qa.sitemap
sitemap https://www.cosme.net/sitemapindex-bestcosme.sitemap
sitemap https://www.cosme.net/sitemapindex-other.sitemap

Comments

  • Block indexing of login request screen.
  • Block indexing of duplicate pages.
  • Block indexing of GA cross-domain tracking parameters
  • Block indexing of API.
  • Block indexing of others.
  • Block indexing of for app pages.
  • Control crawl requests.
  • Sitemaps