aface1.com
robots.txt

Robots Exclusion Standard data for aface1.com

Resource Scan

Scan Details

Site Domain aface1.com
Base Domain aface1.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-12-18T02:12:12+00:00
Next Scan 2026-02-16T02:12:12+00:00

Last Successful Scan

Scanned2025-10-18T22:22:07+00:00
URL https://aface1.com/robots.txt
Domain IPs 104.21.18.3, 172.67.178.248, 2606:4700:3031::ac43:b2f8, 2606:4700:3035::6815:1203
Response IP 104.21.18.3
Found Yes
Hash 9a79a583e9f046b7378d4bdaa17441f956299f445f20336380712fcc8e35bc32
SimHash 6708ca5065f7

Groups

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

slurp

Rule Path
Allow /

google-extended

Rule Path
Allow /

gptbot

Rule Path
Allow /

claudebot

Rule Path
Allow /

amazonbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

*

Rule Path
Disallow /assets
Disallow /cache
Disallow /sources
Disallow /api
Disallow /script_backups
Disallow /updates
Disallow /install
Disallow /admincp
Disallow /admin-panel
Disallow /ajax_loading.php
Disallow /api.php
Disallow /xml
Disallow /system_status.php
Disallow /nodejs
Disallow /*/family_list
Disallow /*?cache=
Disallow /*%26cache%3D

Other Records

Field Value
sitemap https://www.aface1.com/sitemap-index.xml

Comments

  • ===========================
  • Robots.txt for aface1.com
  • ===========================
  • Sitemap location
  • ===========================
  • Allow Major Search Engines
  • ===========================
  • ===========================
  • Allow Useful AI Bots
  • ===========================
  • ===========================
  • Block Unnecessary/Heavy Bots
  • ===========================
  • ===========================
  • Global Rules
  • ===========================
  • Stop Google from crawling cache parameter URLs
  • ===========================
  • Notes:
  • - Sensitive/private folders blocked
  • - Public posts, profiles, pages are crawlable
  • - AI bots (Google-Extended, GPTBot, ClaudeBot) allowed
  • - Other scrapers blocked
  • ===========================