Log In Sign Up

hashlytics.io
robots.txt

Robots Exclusion Standard data for hashlytics.io

Resource Scan

Scan Details

Site Domain hashlytics.io
Base Domain hashlytics.io
Scan Status Ok
Last Scan2025-12-19T05:25:14+00:00
Next Scan 2025-12-26T05:25:14+00:00

Last Scan

Scanned2025-12-19T05:25:14+00:00
URL https://hashlytics.io/robots.txt
Domain IPs 104.21.73.138, 172.67.145.17, 2606:4700:3033::ac43:9111, 2606:4700:3036::6815:498a
Response IP 172.67.145.17
Found Yes
Hash 117ce4e011f3fe34873dc1834b6961106fdc7f6f50f4dafeccf68ed5b2fd22a8
SimHash 698adbc1e435

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-content/cache/
Disallow /wp-json/
Disallow /xmlrpc.php
Disallow /readme.html
Disallow /license.txt
Disallow /?s=
Disallow /search/
Disallow *?replytocom=
Disallow */trackback/
Disallow */embed/
Disallow /cgi-bin/
Allow /wp-admin/admin-ajax.php
Allow /wp-content/uploads/
Allow /wp-content/*.css
Allow /wp-content/*.js
Allow /llms.txt

a-ads

Rule Path
Allow /

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

googlebot-video

Rule Path
Allow /

google-inspectiontool

Rule Path
Allow /

googleother

Rule Path
Allow /

google-extended

Rule Path
Allow /

google-read-aloud

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

feedfetcher-google

Rule Path
Allow /

apis-google

Rule Path
Allow /

storebot-google

Rule Path
Allow /

bingbot

Rule Path
Allow /

bingpreview

Rule Path
Allow /

msnbot

Rule Path
Allow /

msnbot-media

Rule Path
Allow /

adidxbot

Rule Path
Allow /

yandex

Rule Path
Allow /

yandexbot

Rule Path
Allow /

yandeximages

Rule Path
Allow /

baiduspider

Rule Path
Allow /

baiduspider-image

Rule Path
Allow /

baiduspider-video

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

slurp

Rule Path
Allow /

ahrefsbot

Rule Path
Allow /

semrushbot

Rule Path
Allow /

mj12bot

Rule Path
Allow /

gptbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

claudebot

Rule Path
Allow /

anthropic-ai

Rule Path
Allow /

ccbot

Rule Path
Allow /

claude-web

Rule Path
Allow /

bytespider

Rule Path
Allow /

applebot-extended

Rule Path
Allow /

facebookbot

Rule Path
Allow /

diffbot

Rule Path
Allow /

cohere-ai

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

imagesiftbot

Rule Path
Allow /

omgilibot

Rule Path
Allow /

omgili

Rule Path
Allow /

mauibot

Rule Path
Disallow /

alphabot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

asterias

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

builtwith

Rule Path
Disallow /

bullseyebot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

domainstatsbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

turnitin

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

webstripper

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://hashlytics.io/sitemap_index.xml
Back to top

Comments

  • Hashlytics.io Robots.txt - Optimized for SEO, Tools & AI
  • Allow all crawlers by default
  • Allow important resources
  • === A-ADS Crawler Access ===
  • === GOOGLE BOTS - Full Access ===
  • === BING BOTS - Full Access ===
  • === YANDEX (Russian Search) ===
  • === BAIDU (Chinese Search) ===
  • === DUCKDUCKGO ===
  • === YAHOO (Uses Bing) ===
  • Allow SEO & Analytics Tools (You're Paying For These!)
  • Allow AI Crawlers (For llms.txt Integration)
  • Block genuinely bad/aggressive bots
  • Sitemaps
Back to top

Warnings

  • 2 invalid lines.
Back to top