Log In Sign Up

stealthagents.com
robots.txt

Robots Exclusion Standard data for stealthagents.com

Resource Scan

Scan Details

Site Domain stealthagents.com
Base Domain stealthagents.com
Scan Status Ok
Last Scan2026-02-16T00:41:59+00:00
Next Scan 2026-03-18T00:41:59+00:00

Last Scan

Scanned2026-02-16T00:41:59+00:00
URL https://stealthagents.com/robots.txt
Domain IPs 104.26.10.218, 104.26.11.218, 172.67.72.240, 2606:4700:20::681a:ada, 2606:4700:20::681a:bda, 2606:4700:20::ac43:48f0
Response IP 104.26.10.218
Found Yes
Hash a820753c9897269ba330afd62e4f0999363c1f58b88fc35d3158066407ffa1fe
SimHash 58da60d14576

Groups

*

Rule Path
Disallow /wp-content/uploads/wpforms/

oai-searchbot/1.0; +https://openai.com/searchbot

Rule Path
Allow /

mozilla/5.0 applewebkit/537.36 (khtml, like gecko); compatible; chatgpt-user/1.0; +https://openai.com/bot

Rule Path
Allow /

mozilla/5.0 applewebkit/537.36 (khtml, like gecko); compatible; gptbot/1.1; +https://openai.com/gptbot

Rule Path
Allow /

*

Rule Path
Disallow

*

Rule Path
Disallow /*?m=

perplexitybot

Rule Path
Allow /

googlebot

Rule Path
Allow /

google-extended

Rule Path
Allow /

ccbot

Rule Path
Allow /

youbot

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

bingbot

Rule Path
Allow /

adidxbot

Rule Path
Allow /

msnbot

Rule Path
Allow /

microsoft bingbot media

Rule Path
Allow /

archive.org_bot

Rule Path
Allow /

ia_archiver

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://stealthagents.com/sitemap_index.xml
sitemap https://stealthagents.com/sitemap_index.xml
Back to top

Comments

  • START WPFORMS BLOCK
  • ---------------------------
  • ---------------------------
  • END WPFORMS BLOCK
  • START OPENAI BLOCKS
  • ---------------------------
  • ---------------------------
  • END OPENAI BLOCKS
  • START YOAST BLOCK
  • ---------------------------
  • ---------------------------
  • END YOAST BLOCK
  • START BLOCK
  • ---------------------------
  • ---------------------------
  • END BLOCK
  • --- PERPLEXITY ---
  • --- GOOGLE ---
  • --- COMMONCRAWL ---
  • --- YOU.COM ---
  • --- DUCKDUCKGO (AI model feed) ---
  • --- BING / COPILOT ---
  • --- ANTHROPIC (No crawler – no entry needed) ---
  • --- GROK / xAI (No user-agent published yet) ---
  • --- ARCHIVE / RESEARCH (used for model training too) ---
Back to top