Log In Sign Up

apexskin.com
robots.txt

Robots Exclusion Standard data for apexskin.com

Resource Scan

Scan Details

Site Domain apexskin.com
Base Domain apexskin.com
Scan Status Ok
Last Scan2025-11-01T21:58:59+00:00
Next Scan 2025-12-01T21:58:59+00:00

Last Scan

Scanned2025-11-01T21:58:59+00:00
URL https://apexskin.com/robots.txt
Redirect https://www.apexskin.com/robots.txt
Redirect Domain www.apexskin.com
Redirect Base apexskin.com
Domain IPs 35.184.218.156
Redirect IPs 35.184.218.156
Response IP 35.184.218.156
Found Yes
Hash 5b74752ea7d8d215d61ee739764da4349e2e911201e440bcf7a8f680a5d74708
SimHash 41b6d011f52f

Groups

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

gptbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

ccbot

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

anthropic-ai

Rule Path
Allow /

google-extended

Rule Path
Allow /

ia_archiver

Rule Path
Allow /

applebot

Rule Path
Allow /

bingbot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://www.apexskin.com/sitemap_index.xml
Back to top

Comments

  • START YOAST BLOCK
  • ---------------------------
  • Allow main search engines
  • Allow Google Image Bot
  • Sitemap for search engines
  • ---------------------------
  • END YOAST BLOCK
  • ==========================
  • AI Search Bots (explicitly allowed)
  • ==========================
  • OpenAI's GPTBot (used by ChatGPT to crawl content)
  • OpenAI's ChatGPT-User crawler (content fetching for AI)
  • Common Crawl Bot (CCBot) – feeds many AI models
  • Perplexity AI's crawler
  • Anthropic's Claude AI crawler
  • ==========================
  • Other Major AI / Search Bots
  • ==========================
  • Google AI crawler (controls Bard/Gemini training data usage)
  • Amazon's AI data crawler (used for Alexa/AI training)
  • Apple's crawler (used by Siri, Spotlight, and Apple AI search)
  • Microsoft Bing crawler (also used for Copilot & AI indexing)
Back to top

Warnings

  • 1 invalid line.
Back to top