knoji.com
robots.txt

Robots Exclusion Standard data for knoji.com

Resource Scan

Scan Details

Site Domain knoji.com
Base Domain knoji.com
Scan Status Ok
Last Scan2024-10-05T20:22:49+00:00
Next Scan 2024-10-12T20:22:49+00:00

Last Scan

Scanned2024-10-05T20:22:49+00:00
URL https://knoji.com/robots.txt
Domain IPs 34.110.134.84
Response IP 34.110.134.84
Found Yes
Hash 21ac939c8807bd6a1b4ad9a72e4195e50a6971e8b74da1eb069ef0999c450b90
SimHash 48a59e13cdb7

Groups

*

Rule Path
Disallow /admin/
Disallow /admintools/
Disallow /analytics/
Disallow /coupon/
Disallow /cron/
Disallow /details/
Disallow /editor/
Disallow /getcoupon/
Disallow /getcoupons/
Disallow /gostore/
Disallow /hyperlink/
Disallow /tracker/
Disallow /info/
Disallow /link/
Disallow /open/
Disallow /out/
Disallow /redir/
Disallow /pricecut/
Disallow /promotion/
Disallow /reports/
Disallow /t/
Disallow /tools/
Disallow /use-code/
Disallow /visit/
Disallow /www/
Disallow /admin-scripts/
Disallow /ask/edit/
Disallow /questions/*/addrating/
Disallow /rate/
Disallow /product-out/
Disallow /product/
Disallow /rankings/
Disallow /compare/
Disallow /setup-brands/

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

ccbot

Rule Path
Disallow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

googlebot-news

Rule Path
Allow /blog
Allow /article/

Other Records

Field Value
sitemap https://knoji.com/sitemap/index.xml

Comments

  • User-agent: GPTBot
  • Disallow: /