Log In Sign Up

nexoye.com
robots.txt

Robots Exclusion Standard data for nexoye.com

Resource Scan

Scan Details

Site Domain nexoye.com
Base Domain nexoye.com
Scan Status Ok
Last Scan2025-12-15T03:48:24+00:00
Next Scan 2025-12-22T03:48:24+00:00

Last Scan

Scanned2025-12-15T03:48:24+00:00
URL https://nexoye.com/robots.txt
Domain IPs 104.21.53.221, 172.67.219.44, 2606:4700:3033::6815:35dd, 2606:4700:3033::ac43:db2c
Response IP 172.67.219.44
Found Yes
Hash 86803795957cd2009da0eb6e7c818d4a339505a1b066fc13c3b6161094791d21
SimHash 6a14da00e4f0

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /xmlrpc.php
Disallow /?s=*
Disallow /trackback/
Disallow /comments/feed/
Disallow /wp-json/
Disallow /wp-content/cache/
Allow /wp-admin/admin-ajax.php
Allow /wp-content/uploads/

gptbot

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

claudebot

Rule Path
Allow /

applebot-extended

Rule Path
Allow /

youbot

Rule Path
Allow /

brave-ai

Rule Path
Allow /

googlebot

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

google-extended

Rule Path
Allow /

mediapartners-google

Rule Path
Disallow

bingbot

Rule Path
Allow /

bingpreview

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

yandex

Rule Path
Allow /

slurp

Rule Path
Allow /

baiduspider

Rule Path
Allow /

sogou spider

Rule Path
Allow /

exabot

Rule Path
Allow /

facebot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

twitterbot

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

google-adwords-adbot

Rule Path
Allow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

httrack

Rule Path
Disallow /

scrapy

Rule Path
Disallow /
Allow /llms.txt
Back to top

Other Records

Field Value
sitemap https://nexoye.com/sitemap_index.xml
Back to top

Comments

  • Sitemap
  • ------------------------------
  • ✅ AI Bots (for LLM indexing)
  • ------------------------------
  • ------------------------------
  • ✅ Traditional Search Bots
  • ------------------------------
  • ------------------------------
  • ✅ Social Media Bots
  • ------------------------------
  • ------------------------------
  • ✅ Google Ads Crawlers
  • ------------------------------
  • ------------------------------
  • 🚫 Block Spam & Bad Crawlers
  • ------------------------------
  • ------------------------------
  • ✅ LLM Data File Reference
  • ------------------------------
  • Helps LLM bots discover your AI data file
Back to top