biggestkaka.co.ke
robots.txt

Robots Exclusion Standard data for biggestkaka.co.ke

Resource Scan

Scan Details

Site Domain biggestkaka.co.ke
Base Domain biggestkaka.co.ke
Scan Status Ok
Last Scan2024-06-16T12:22:27+00:00
Next Scan 2024-06-23T12:22:27+00:00

Last Scan

Scanned2024-06-16T12:22:27+00:00
URL https://biggestkaka.co.ke/robots.txt
Domain IPs 104.21.4.2, 172.67.131.104, 2606:4700:3033::6815:402, 2606:4700:3037::ac43:8368
Response IP 172.67.131.104
Found Yes
Hash bd1efd3e8232e37e12aad7940ae16640458165172e4097da1e7145ea72a6df87
SimHash 293b5ed0862f

Groups

msnbot

Rule Path
Disallow /*.xml$
Disallow /category/*.xml$
Disallow /mobile/
Disallow *?s=mobile
Disallow *?s=bpage-next
Disallow *?s=lightbox
Disallow /contest
Disallow /contests
Disallow /plugin/
Disallow /embed/
Disallow /_comments/
Disallow /bookmarks/
Disallow /drafts/

Other Records

Field Value
crawl-delay 120

*

Rule Path
Disallow /category/*.xml$
Disallow /mobile/
Disallow *?s=bpage-next
Disallow *?s=lightbox
Disallow *?s=feedpager
Disallow /_ga/
Disallow /static/
Disallow /dashboard/
Disallow /plugin/
Disallow /api/
Disallow /embed/
Disallow /_comments/
Disallow /bookmarks/
Disallow /drafts/
Disallow /bpage-preview/
Disallow /cms_preview_ui/
Disallow /cms_preview_ui_static/
Disallow /search/
Disallow /cgi-bin
Disallow /wp-admin
Disallow /wp-includes
Disallow /wp-content/plugins/
Disallow /wp-content/cache/
Disallow /wp-content/themes/
Disallow */trackback/
Disallow */feed/
Disallow /*/feed/rss/$

discobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 4

yacybot

Rule Path
Disallow /

googlebot-news

Rule Path
Disallow /politics/

Other Records

Field Value
sitemap https://biggestkaka.co.ke/sitemap_index.xml
sitemap https://biggestkaka.co.ke/category-sitemap.xml
sitemap https://biggestkaka.co.ke/post-sitemap.xml
sitemap https://biggestkaka.co.ke/page-sitemap.xml