cryptoaware.site
robots.txt

Robots Exclusion Standard data for cryptoaware.site

Resource Scan

Scan Details

Site Domain cryptoaware.site
Base Domain cryptoaware.site
Scan Status Ok
Last Scan2025-08-20T15:21:40+00:00
Next Scan 2025-08-27T15:21:40+00:00

Last Scan

Scanned2025-08-20T15:21:40+00:00
URL https://cryptoaware.site/robots.txt
Redirect https://www.cryptoaware.site/robots.txt
Redirect Domain www.cryptoaware.site
Redirect Base cryptoaware.site
Domain IPs 216.198.79.65, 64.29.17.1
Redirect IPs 216.198.79.65, 64.29.17.1
Response IP 216.198.79.65
Found Yes
Hash 43d5a8b0385995a74f1590c4f03169265175ace956d8af6a63786ef6bfb781af
SimHash 6d14ca55b7a2

Groups

*

Rule Path
Allow /
Disallow /admin/
Disallow /private/
Disallow /api/

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

bingbot

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.cryptoaware.site/sitemap.xml

Comments

  • CryptoInsight robots.txt
  • Sitemaps
  • Disallow admin and private areas
  • Allow Google to index everything
  • Allow Google Images to index all images
  • Allow Bing to index everything