Log In Sign Up

kennco.ie
robots.txt

Robots Exclusion Standard data for kennco.ie

Resource Scan

Scan Details

Site Domain kennco.ie
Base Domain kennco.ie
Scan Status Ok
Last Scan2026-03-18T23:33:36+00:00
Next Scan 2026-04-17T23:33:36+00:00

Last Scan

Scanned2026-03-18T23:33:36+00:00
URL https://kennco.ie/robots.txt
Redirect https://www.kennco.ie/robots.txt
Redirect Domain www.kennco.ie
Redirect Base kennco.ie
Domain IPs 104.26.14.66, 104.26.15.66, 172.67.75.174, 2606:4700:20::681a:e42, 2606:4700:20::681a:f42, 2606:4700:20::ac43:4bae
Redirect IPs 104.26.14.66, 104.26.15.66, 172.67.75.174, 2606:4700:20::681a:e42, 2606:4700:20::681a:f42, 2606:4700:20::ac43:4bae
Response IP 172.67.75.174
Found Yes
Hash 59531ef8d539ad2c60d7eba0492f7519db8ea707f2bc3526f6ebab5bc4f4de29
SimHash e43988427c5e

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-login.php
Disallow /wp-cron.php
Disallow /wp-json/
Disallow /xmlrpc.php
Disallow /?s=
Disallow /*?s=
Disallow /search
Disallow /*preview%3Dtrue
Disallow /embed
Disallow /api/
Disallow /ajax
Disallow /oauth
Disallow /cart/
Disallow /checkout/
Disallow /my-account/
Disallow /my-account/
Disallow /cdn-cgi/l/email-protection
Disallow /cdn-cgi/l
Disallow /cdn-cgi/
Disallow /page/
Disallow /category/*/page/

Other Records

Field Value
crawl-delay 10

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

petalbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

bytespider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20
Back to top

Other Records

Field Value
sitemap https://www.kennco.ie/sitemap_index.xml
Back to top

Comments

  • robots.txt generated to reduce load from aggressive crawlers on dynamic/uncached endpoints
  • Note: Googlebot ignores Crawl-delay, but Bing/Yandex/etc. respect it
  • Optional: limit indexation of paginated archives (reduces deep crawl)
  • Slow-down or rein in heavy SEO crawlers
  • User-agent: SemrushBot
  • Crawl-delay: 15
  • If needed, you can fully block particularly abusive agents by uncommenting:
  • User-agent: BadBotName
  • Disallow: /
  • WP Rocket enabled:
  • - Keep admin-ajax.php allowed (used by some front-end features and cache purges).
  • - Consider excluding logged-in users and previews from cache in WP Rocket (already standard).
Back to top