Log In Sign Up

autozenapp.com
robots.txt

Robots Exclusion Standard data for autozenapp.com

Resource Scan

Scan Details

Site Domain autozenapp.com
Base Domain autozenapp.com
Scan Status Ok
Last Scan2026-01-10T19:23:54+00:00
Next Scan 2026-01-17T19:23:54+00:00

Last Scan

Scanned2026-01-10T19:23:54+00:00
URL https://autozenapp.com/robots.txt
Domain IPs 104.21.71.34, 172.67.142.201, 2606:4700:3031::ac43:8ec9, 2606:4700:3036::6815:4722
Response IP 172.67.142.201
Found Yes
Hash 235c1c4f9f3b5ed9143fc0d9e1fbcbfed8977d965841237b8bdbc4ee6cf74306
SimHash 644edf217cea

Groups

*

Rule Path Comment
Allow / -
Disallow /admin/ -
Disallow /wp-admin/ -
Disallow /wp-content/uploads/ -
Disallow /cgi-bin/ -
Disallow /tmp/ -
Disallow /logs/ -
Disallow /cache/ -
Disallow /backup/ -
Disallow /config/ -
Disallow /database/ -
Disallow /includes/ -
Disallow /dev/ -
Disallow /test/ -
Disallow /staging/ -
Disallow /beta/ -
Disallow /_dev/ -
Disallow /_test/ -
Disallow /phpinfo.php -
Disallow /readme.txt -
Disallow /license.txt -
Disallow /changelog.txt -
Disallow /.env -
Disallow /.git/ -
Disallow /.htaccess -
Disallow /composer.json -
Disallow /package.json -
Disallow /*?* -
Disallow /*%26* -
Disallow /search?* -
Disallow /*utm_* -
Disallow /*fbclid* -
Disallow /*gclid* -
Disallow /*sessionid* -
Disallow /*PHPSESSID* -
Disallow /*sid%3D* -
Disallow /forum/public/admin -
Disallow /forum/public/admin/ -
Disallow /forum/public/admin /
Disallow /forum/any -
Disallow /forum/any/ -
Disallow /forum/*/admin -
Disallow /forum/*/redirect -
Disallow /forum/redirect -
Disallow /forum/public/redirect -
Disallow /cdn-cgi/ -
Disallow /cdn-cgi/l/email-protection -
Disallow */cdn-cgi/l/email-protection -
Disallow /forum/admin/ -
Disallow /forum/moderator/ -
Disallow /forum/api/ -
Disallow /forum/ajax/ -
Disallow /forum/cache/ -
Disallow /forum/logs/ -
Disallow /forum/tmp/ -
Disallow /forum/backup/ -
Disallow /forum/install/ -
Disallow /forum/upgrade/ -
Disallow /forum/*?* -
Disallow /forum/*%26* -
Disallow /forum/*action%3D* -
Disallow /forum/*do%3D* -
Disallow /forum/*task%3D* -
Disallow /webmail/ -
Disallow /mail/ -
Disallow /email/ -
Disallow /roundcube/ -
Disallow /squirrelmail/ -
Disallow /horde/ -
Disallow /wp-login.php -
Disallow /wp-register.php -
Disallow /login -
Disallow /register -
Disallow /administrator/ -
Disallow /phpmyadmin/ -
Disallow /mysql/ -
Disallow /cpanel/ -
Disallow /*.php$ -
Disallow /*.inc$ -
Disallow /*.class$ -
Disallow /*.asp$ -
Disallow /*.aspx$ -
Disallow /*.cgi$ -
Disallow /*.pl$ -
Disallow /*.py$ -
Disallow /*.exe$ -
Disallow /*.dll$ -
Disallow /*.zip$ -
Disallow /*.rar$ -
Disallow /*.tar$ -
Disallow /*.gz$ -
Disallow /print/ -
Disallow /*print%3D* -
Disallow /mobile/ -
Disallow /m/ -
Disallow /wp-comments-post.php -
Disallow /comment/ -
Disallow /*/comment -
Disallow /trackback/ -
Disallow /*/trackback -
Disallow /index.php/ -
Disallow /*/index.php -
Disallow /home.php -
Disallow /default.php -
Allow /assets/ -
Allow /css/ -
Allow /js/ -
Allow /images/ -
Allow /fonts/ -
Allow /manifest.json -
Allow /sitemap.xml -
Allow /.well-known/ -
Allow /download -
Allow /app-ads.txt -
Allow /.well-known/assetlinks.json -
Allow /*.pdf$ -
Allow /facebook.html -
Allow /twitter.html -
Allow /instagram.html -
Allow /youtube.html -

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

slurp

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

yandexbot

Rule Path
Allow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

majesticseo

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot-sa

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

builtwith

Rule Path
Disallow /

wappalyzer

Rule Path
Disallow /

whatsmyip

Rule Path
Disallow /

zoombot

Rule Path
Disallow /

linkedinbot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

twitterbot

Rule Path
Allow /

whatsapp

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://autozenapp.com/sitemap.xml
Back to top

Comments

  • robots.txt for AutoZen - Car Navigation App
  • https://autozenapp.com/robots.txt
  • Allow all search engines to crawl the entire site
  • Block crawling of admin, private, and system directories
  • Block crawling of development and testing areas
  • Block sensitive files and directories
  • Block search and filter parameters to avoid duplicate content
  • Block session IDs and tracking parameters
  • Block forum admin and dynamic paths that cause soft 404s
  • Block Cloudflare email protection and CDN paths
  • Block common forum paths that generate soft 404s
  • Block dynamic forum URLs with parameters
  • Block subdomain mail server paths
  • Block common exploit attempts and security scanners
  • Block file extensions that shouldn't be indexed
  • Block printer-friendly and mobile redirect versions
  • Block comment and review spam areas
  • Block duplicate content from different URL structures
  • Allow important static resources for app functionality
  • Allow app-related files that help with app store optimization
  • Allow PDF files (useful for press kits, user manuals)
  • Allow social media verification files
  • Specific directives for major search engines
  • Crawl-delay for less important bots
  • Block known bad bots and scrapers
  • Allow important app ecosystem crawlers
  • Block AI training bots (optional - uncomment if desired)
  • User-agent: ChatGPT-User
  • Disallow: /
  • User-agent: Google-Extended
  • Disallow: /
  • User-agent: Amazonbot
  • Disallow: /
  • User-agent: Claude-Web
  • Disallow: /
  • Specify sitemap location for better crawling
  • Additional sitemaps for comprehensive coverage
  • Sitemap: https://autozenapp.com/sitemap-images.xml
  • Sitemap: https://autozenapp.com/sitemap-videos.xml
  • Sitemap: https://autozenapp.com/sitemap-news.xml
  • End of robots.txt
Back to top