customon.com
robots.txt

Robots Exclusion Standard data for customon.com

Resource Scan

Scan Details

Site Domain customon.com
Base Domain customon.com
Scan Status Ok
Last Scan2025-09-29T08:16:04+00:00
Next Scan 2025-10-29T08:16:04+00:00

Last Scan

Scanned2025-09-29T08:16:04+00:00
URL https://customon.com/robots.txt
Redirect https://www.customon.com/robots.txt
Redirect Domain www.customon.com
Redirect Base customon.com
Domain IPs 104.26.8.12, 104.26.9.12, 172.67.69.93, 2606:4700:20::681a:80c, 2606:4700:20::681a:90c, 2606:4700:20::ac43:455d
Redirect IPs 104.26.8.12, 104.26.9.12, 172.67.69.93, 2606:4700:20::681a:80c, 2606:4700:20::681a:90c, 2606:4700:20::ac43:455d
Response IP 172.67.69.93
Found Yes
Hash 114e1ab154b5ad462b873385a20a7756efe4b9f458f42db34b6e7e296872b79f
SimHash 405dcbc27611

Groups

*
googlebot

Rule Path
Disallow

adsbot-google

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

baiduspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow

semrushbot

Rule Path
Disallow /

Comments

  • Baiduspider
  • Yandex
  • PetalBot
  • MJ12bot
  • DotBot
  • DotBot
  • AhrefsBot