Log In Sign Up

aaryavart.org
robots.txt

Robots Exclusion Standard data for aaryavart.org

Resource Scan

Scan Details

Site Domain aaryavart.org
Base Domain aaryavart.org
Scan Status Ok
Last Scan2025-08-11T16:44:11+00:00
Next Scan 2025-09-10T16:44:11+00:00

Last Scan

Scanned2025-08-11T16:44:11+00:00
URL https://aaryavart.org/robots.txt
Redirect https://www.aaryavart.org/robots.txt
Redirect Domain www.aaryavart.org
Redirect Base aaryavart.org
Domain IPs 76.76.21.21
Redirect IPs 216.198.79.65, 64.29.17.65
Response IP 64.29.17.65
Found Yes
Hash 99dfa994cbf064166281f629071e46720aa4eb33d2cdc5b03cdac24f57d05804
SimHash 2d065f72e4a2

Groups

*

Rule Path
Allow /

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

yandex

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

slurp

Rule Path
Allow /

twitterbot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

whatsapp

Rule Path
Allow /

instagram

Rule Path
Allow /

pinterest

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

gptbot

Rule Path
Allow /

ccbot

Rule Path
Allow /

chatgpt

Rule Path
Allow /

claude-web

Rule Path
Allow /

anthropic-ai

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

youbot

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

mozilla/5.0 (linux; android*

Rule Path
Allow /

mozilla/5.0 (iphone*

Rule Path
Allow /

mozilla/5.0 (ipad*

Rule Path
Allow /

ucweb

Rule Path
Allow /

opera mini

Rule Path
Allow /

samsungbrowser

Rule Path
Allow /

chrome mobile

Rule Path
Allow /

safari mobile

Rule Path
Allow /

jiobrowser

Rule Path
Allow /

mi browser

Rule Path
Allow /

justdial

Rule Path
Allow /

sulekha

Rule Path
Allow /

practo

Rule Path
Allow /

healthifyme

Rule Path
Allow /

pharmeasy

Rule Path
Allow /

netmeds

Rule Path
Allow /

healthbot

Rule Path
Allow /

medicalbot

Rule Path
Allow /

therapybot

Rule Path
Allow /

autismbot

Rule Path
Allow /
Disallow /admin/
Disallow /private/
Disallow /api/
Disallow /_next/
Disallow /node_modules/
Disallow /.env
Disallow /config/
Disallow /temp/
Disallow /cache/
Disallow /logs/
Allow /assets/
Allow /images/
Allow /css/
Allow /js/
Disallow /.git/
Disallow /.github/
Disallow /package.json
Disallow /package-lock.json
Disallow /yarn.lock
Disallow /README.md

Other Records

Field Value
crawl-delay 1
Back to top

Other Records

Field Value
sitemap https://aaryavartcentre.com/sitemap.xml
Back to top

Comments

  • Aaryavart Centre for Autism and Special Needs Foundation
  • Robots.txt file for search engine optimization
  • Last updated: December 2024
  • ========================================
  • ALLOW ALL SEARCH ENGINES
  • ========================================
  • ========================================
  • MAJOR SEARCH ENGINES
  • ========================================
  • ========================================
  • SOCIAL MEDIA CRAWLERS
  • ========================================
  • ========================================
  • AI AND CHATGPT SEARCH CRAWLERS
  • ========================================
  • ========================================
  • MOBILE-SPECIFIC CRAWLERS
  • ========================================
  • ========================================
  • ANDROID AND IOS MOBILE BROWSERS
  • ========================================
  • ========================================
  • INDIAN MOBILE BROWSERS AND APPS
  • ========================================
  • ========================================
  • INDIAN SEARCH ENGINES AND APPS
  • ========================================
  • ========================================
  • HEALTHCARE AND MEDICAL CRAWLERS
  • ========================================
  • ========================================
  • BLOCK SENSITIVE AREAS
  • ========================================
  • ========================================
  • SITEMAP AND CRAWL SETTINGS
  • ========================================
  • Crawl delay (helps with server load)
  • ========================================
  • ADDITIONAL SEO DIRECTIVES
  • ========================================
  • Allow crawling of all content for better SEO
  • Block access to development files
Back to top

Warnings

  • 2 invalid lines.
Back to top