c99withyou.com
robots.txt

Robots Exclusion Standard data for c99withyou.com

Resource Scan

Scan Details

Site Domain c99withyou.com
Base Domain c99withyou.com
Scan Status Ok
Last Scan2025-10-16T16:51:25+00:00
Next Scan 2025-11-15T16:51:25+00:00

Last Scan

Scanned2025-10-16T16:51:25+00:00
URL https://www.c99withyou.com/robots.txt
Domain IPs 13.35.238.105, 13.35.238.4, 13.35.238.42, 13.35.238.81, 2600:9000:2085:3600:15:3369:8bc0:93a1, 2600:9000:2085:4000:15:3369:8bc0:93a1, 2600:9000:2085:6600:15:3369:8bc0:93a1, 2600:9000:2085:9e00:15:3369:8bc0:93a1, 2600:9000:2085:b000:15:3369:8bc0:93a1, 2600:9000:2085:b600:15:3369:8bc0:93a1, 2600:9000:2085:bc00:15:3369:8bc0:93a1, 2600:9000:2085:f000:15:3369:8bc0:93a1
Response IP 13.35.238.105
Found Yes
Hash d5a1384f63b34bf177967cb2a18fcf6da79fd53332d5427b2c7f9b5a520f6c71
SimHash 3015df1164cb

Groups

*

Rule Path
Disallow /

googlebot
googlebot-image
googlebot-mobile
google-inspectiontool
bingbot
msnbot
slurp
baiduspider
duckduckbot
ahrefsbot
semrushbot
mj12bot
dotbot
screaming frog seo spider
moz
facebookexternalhit
facebot
twitterbot
telegrambot
gptbot
chatgpt-user
openai-user
anthropicbot
deepseekbot
google-extended
perplexitybot
youbot
applebot
ia_archiver
amazonbot
archive.org_bot

Rule Path
Allow /index.php
Allow /welcome*.php
Allow /images/club99_link_logo.png
Allow /welcome-images/
Allow /sitemap*.xml

Comments

  • Last updated: 20250618
  • Block all unknown crawlers
  • Allow major search engine crawlers
  • Allow trusted SEO tools
  • Allow social media bots for link previews
  • Allow AI and language model crawlers
  • Allow Apple, Amazon, Alexa bots
  • Allowed paths for all above bots
  • Sitemap declaration (not in use, remarked)
  • Sitemap: https://www.club99.com/sitemap.xml