Log In Sign Up

kibsons.com
robots.txt

Robots Exclusion Standard data for kibsons.com

Resource Scan

Scan Details

Site Domain kibsons.com
Base Domain kibsons.com
Scan Status Ok
Last Scan2025-09-29T14:51:07+00:00
Next Scan 2025-10-06T14:51:07+00:00

Last Scan

Scanned2025-09-29T14:51:07+00:00
URL https://kibsons.com/robots.txt
Redirect https://www.kibsons.com/robots.txt
Redirect Domain www.kibsons.com
Redirect Base kibsons.com
Domain IPs 104.20.31.173, 172.66.167.48, 2606:4700:10::6814:1fad, 2606:4700:10::ac42:a730
Redirect IPs 104.20.31.173, 172.66.167.48, 2606:4700:10::6814:1fad, 2606:4700:10::ac42:a730
Response IP 172.66.167.48
Found Yes
Hash 72595462272701ac5c857e0897f4b4e1089043632bba419aacfa09dc09bd702d
SimHash d046f5544cf8

Groups

*

Rule Path
Disallow /myshopcart*
Disallow /myshopcart*
Disallow /Multicart*
Disallow */checkout*
Disallow */account*
Disallow */login*%60
Disallow */register*%60
Disallow */kibsonsregister*%60
Disallow /*/search%3D*
Disallow /_svc/

cazoodlebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot/1.0

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

onetrustbot

Rule Path
Allow *

gptbot

Rule Path
Disallow /

twitterbot

No rules defined. All paths allowed.

Back to top

Other Records

Field Value
sitemap https://kibsecommemorydump.blob.core.windows.net/kibsonsitemap/sitemap.xml
Back to top

Comments

  • _ __ _____ ____ _____ ____ _ _ _____
  • | |/ / |_ _| | _ \ / ____| / __ \ | \ | | / ____|
  • | ' / | | | |_) | | (___ | | | | | \| | | (___
  • | < | | | _ < \___ \ | | | | | . ` | \___ \
  • | . \ _| |_ | |_) | ____) | | |__| | | |\ | ____) |
  • |_|\_\ |_____| |____/ |_____/ \____/ |_| \_| |_____/
  • Our Vision: Farm to kitchen.
  • For all robots
  • Block access to specific groups of pages
  • Block CazoodleBot as it does not present correct accept content headers
  • Block MJ12bot as it is just noise
  • Block dotbot as it cannot parse base urls properly
  • Block Gigabot
Back to top