Log In Sign Up

miriscope.com
robots.txt

Robots Exclusion Standard data for miriscope.com

Resource Scan

Scan Details

Site Domain miriscope.com
Base Domain miriscope.com
Scan Status Ok
Last Scan2025-11-13T05:45:24+00:00
Next Scan 2025-11-20T05:45:24+00:00

Last Scan

Scanned2025-11-13T05:45:24+00:00
URL https://miriscope.com/robots.txt
Redirect https://www.miriscope.com/robots.txt
Redirect Domain www.miriscope.com
Redirect Base miriscope.com
Domain IPs 216.198.79.1
Redirect IPs 216.198.79.65, 64.29.17.65
Response IP 216.198.79.1
Found Yes
Hash 42814b67216bb52b0b8a200472d1fb23060e1d305fd6758c4ad280e1383f680c
SimHash 3f805293c5a1

Groups

*

Rule Path
Allow /
Allow /articles/
Allow /category/
Allow /tags/
Allow /about/
Allow /contact/
Allow /trending/
Allow /breaking/
Allow /privacy-policy/
Allow /terms-of-service/
Allow /gdpr/
Allow /newsletter/
Allow /authors/
Allow /faq/
Allow /advertise/
Allow /careers/
Allow /press/
Allow /sitemap/
Allow /search/
Allow /api/rss/
Allow /api/news-sitemap/
Allow /api/sitemap/
Allow /feed.xml
Allow /images/
Allow /icons/
Allow /_next/static/
Allow /_next/image
Disallow /admin/
Disallow /dashboard/
Disallow /login/
Disallow /auth/
Disallow /api/auth/
Disallow /_next/
Disallow /api/upload/
Disallow /api/internal/
Disallow /*?utm_*
Disallow /*?fbclid=*
Disallow /*?gclid=*
Disallow /*?ref=*
Disallow /*?source=*
Disallow /*?fb_source=*
Disallow /*?fb_action=*
Disallow /*?fb_action_ids=*
Disallow /*?fb_action_types=*
Disallow /*?fb_ref=*
Disallow /*?fb_source=*
Disallow /*?utm_source=*
Disallow /*?utm_medium=*
Disallow /*?utm_campaign=*
Disallow /*?utm_term=*
Disallow /*?utm_content=*

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

googlebot-news

Rule Path
Allow /
Allow /articles/
Allow /category/
Allow /trending/
Allow /breaking/
Allow /tags/

Other Records

Field Value
crawl-delay 1

googlebot-image

Rule Path
Allow /images/
Allow /icons/
Allow /_next/image
Allow /articles/

Other Records

Field Value
crawl-delay 1

googlebot-mobile

Rule Path
Allow /
Allow /articles/
Allow /category/
Allow /tags/

Other Records

Field Value
crawl-delay 1

facebookexternalhit

Rule Path
Allow /
Allow /articles/
Allow /category/
Allow /tags/

Other Records

Field Value
crawl-delay 1

twitterbot

Rule Path
Allow /
Allow /articles/
Allow /category/
Allow /tags/

Other Records

Field Value
crawl-delay 1

linkedinbot

Rule Path
Allow /
Allow /articles/
Allow /category/
Allow /tags/

Other Records

Field Value
crawl-delay 2

whatsapp

Rule Path
Allow /
Allow /articles/
Allow /category/
Allow /tags/

Other Records

Field Value
crawl-delay 1

telegrambot

Rule Path
Allow /
Allow /articles/
Allow /category/
Allow /tags/
Allow /llms.txt
Allow /*.html
Allow /*.xml
Allow /*.json
Allow /api/articles/
Allow /api/categories/
Allow /api/tags/
Allow /mobile/
Allow /amp/

Other Records

Field Value
crawl-delay 1
crawl-delay 1
Back to top

Other Records

Field Value
sitemap https://miriscope.com/sitemap.xml
sitemap https://miriscope.com/sitemap-index.xml
sitemap https://miriscope.com/news-sitemap.xml
sitemap https://miriscope.com/top-stories-sitemap.xml
sitemap https://miriscope.com/api/sitemap
sitemap https://miriscope.com/api/enhanced-sitemap
sitemap https://miriscope.com/api/rss
sitemap https://miriscope.com/feed.xml
Back to top

Comments

  • Miriscope.com - Enhanced Robots.txt for SEO Optimization
  • Updated: 2025 - Enhanced for better Google crawling and indexing
  • ΕπιτρέποÏ
  • API endpoints για feeds και sitemaps
  • Images και assets
  • ΑποκλείοÏ
  • ΑποκλείοÏ
  • Special directives for Google
  • Special directives for Bing
  • Enhanced Google News support
  • Google Images with enhanced crawling
  • Google Mobile
  • Facebook crawler with enhanced access
  • Twitter crawler with enhanced access
  • LinkedIn crawler
  • WhatsApp crawler
  • Telegram crawler
  • Enhanced Sitemaps - Σημαντικό για SEO
  • RSS Feeds για discovery και Google News
  • AI Models Configuration
  • Host directive για consistency
  • Additional SEO directives
  • Ensure proper indexing of article content
  • Allow crawling of dynamic content
  • Enhanced mobile support
  • Performance optimization hints
  • Google recommends these settings for news sites
Back to top

Warnings

  • 4 invalid lines.
  • `host` is not a known field.
Back to top