Log In Sign Up

90min.com
robots.txt

Robots Exclusion Standard data for 90min.com

Resource Scan

Scan Details

Site Domain 90min.com
Base Domain 90min.com
Scan Status Ok
Last Scan2024-10-31T13:49:06+00:00
Next Scan 2024-11-07T13:49:06+00:00

Last Scan

Scanned2024-10-31T13:49:06+00:00
URL https://90min.com/robots.txt
Redirect https://www.90min.com/robots.txt
Redirect Domain www.90min.com
Redirect Base 90min.com
Domain IPs 35.83.190.225, 44.241.188.24
Redirect IPs 13.35.210.19, 13.35.210.40, 13.35.210.61, 13.35.210.94, 2600:9000:2078:2600:f:8f3b:1200:93a1, 2600:9000:2078:5400:f:8f3b:1200:93a1, 2600:9000:2078:6600:f:8f3b:1200:93a1, 2600:9000:2078:7400:f:8f3b:1200:93a1, 2600:9000:2078:800:f:8f3b:1200:93a1, 2600:9000:2078:b400:f:8f3b:1200:93a1, 2600:9000:2078:e000:f:8f3b:1200:93a1, 2600:9000:2078:f800:f:8f3b:1200:93a1
Response IP 13.35.210.94
Found Yes
Hash 46966395238f7dbe32a64893c343523c2224a64e30ef1e4eab4a89723b9f3189
SimHash 2a24410a9de2

Groups

*

Rule Path
Allow /

*

Rule Path
Disallow */?*utm_source=*
Disallow */?*utm_campaign=*
Disallow */?*utm_medium=*
Disallow /*?source=*
Disallow */?*utm_newsbreak=*

*

Rule Path
Allow /ads.txt

*

Rule Path
Disallow *?embed*

*

Rule Path
Disallow */?a_aid=*
Disallow /*%26a_aid*
Disallow /*?partner=*

*

Rule Path
Disallow */*?mm-experiments=*

*

Rule Path
Disallow /*?*s=*

*

Rule Path
Disallow *?app=*

*

Rule Path
Disallow *?fbclid=*

*

Rule Path
Disallow /*?setLocale=*
Disallow *?georedirect=*

*

Rule Path
Disallow /*?term=*
Disallow *?ref=*
Disallow /*?view_source=*
Disallow /*?view_medium=*
Disallow /*?initialLeagueId=*

*

Rule Path
Disallow *_ga_*
Disallow */?_gl=*

*

Rule Path
Disallow */api/*

*

Rule Path
Disallow */videos/undefinedc_fill%2Cw_360%2Car_16%3A9%2Cf_auto%2Cq_auto%2Cg_auto/undefined
Disallow */teams/mainNavigationChevron_icon.svg?*
Disallow */leagues/mainNavigationChevron_icon.svg?*
Disallow */undefinedc_fill%2Cw_360%2Car_16%3A9%2Cf_auto%2Cq_auto%2Cg_auto/undefined

*

Rule Path
Disallow */xposts/
Disallow */reader/
Disallow */embed/*
Disallow */ads/*
Disallow */editor/*
Disallow */posts/*/edit$
Disallow */posts/*/publish$
Disallow */singlepage/pipe/*
Disallow */singlepage/uncached_pipe/*
Disallow */singlepage/epipe/*
Disallow */reads/*/read$
Disallow */embed_code
Disallow */matches/*
Disallow */es/partidos/*
Disallow */it/partite/*
Disallow */de/spiele/*
Disallow */zh-CN/*
Disallow */ping
Disallow */sessions/*
Disallow */admin/*
Disallow */management/*
Disallow */castr
Disallow */unfeature$
Disallow */unfeature/*
Disallow */videos/*
Disallow */channels/videos/*

*

Rule Path
Disallow /_partial/*
Disallow */amazingfactgenerator/*
Disallow /_modules/*
Disallow /subscribe/v3/*
Disallow /search/*
Disallow */file/*
Disallow */node/*
Disallow /sites/*
Disallow /longform/*
Disallow /blogs/*
Disallow */store/*
Disallow */us/*
Disallow */store/*
Disallow */us/*
Disallow */HDYK/*
Disallow /dist/*
Disallow */shopping/*
Disallow /trivia/*
Disallow /worksheets/*
Disallow /music/*
Disallow /magazine/*
Disallow /puzzle/*

*

Rule Path
Disallow */files/*
Disallow */wp-admin/*
Disallow */?*utm_newsbreak=*
Disallow */wp-content/*
Disallow */wp-includes/*
Disallow */app/*
Disallow */embed_code*
Disallow */%7B%7Burl/*
Disallow */v2/*

twitterbot

Rule Path
Allow *

facebookbot

Rule Path
Allow *

*

No rules defined. All paths allowed.

Back to top

Other Records

Field Value
sitemap https://www.90min.com/news-sitemap.xml
sitemap https://www.90min.com/feed-sitemap.xml
sitemap https://www.90min.com/sitemap.xml
sitemap https://www.90min.com/es/news-sitemap.xml
sitemap https://www.90min.com/es/feed-sitemap.xml
sitemap https://www.90min.com/es/sitemap.xml
sitemap https://www.90min.com/it/news-sitemap.xml
sitemap https://www.90min.com/it/feed-sitemap.xml
sitemap https://www.90min.com/it/sitemap.xml
sitemap https://www.90min.com/fr/news-sitemap.xml
sitemap https://www.90min.com/fr/feed-sitemap.xml
sitemap https://www.90min.com/fr/sitemap.xml
sitemap https://www.90min.com/th/news-sitemap.xml
sitemap https://www.90min.com/th/feed-sitemap.xml
sitemap https://www.90min.com/th/sitemap.xml
sitemap https://www.90min.com/pt-BR/news-sitemap.xml
sitemap https://www.90min.com/pt-BR/feed-sitemap.xml
sitemap https://www.90min.com/pt-BR/sitemap.xml
sitemap https://www.90min.com/id/news-sitemap.xml
sitemap https://www.90min.com/id/feed-sitemap.xml
sitemap https://www.90min.com/id/sitemap.xml
Back to top

Comments

  • Allow all search engines to crawl
  • Disallow Parameters
  • GA traffic source parameters
  • Allow crawling ads
  • Embedded widget parameters
  • Influencers/Affiliate Links parameters
  • Experiments testing team
  • Search box param
  • Apps param
  • FB campaigns
  • GEO targeting:
  • Unknown Parameters
  • GA parameters - Generated from Google caching
  • Generated from Voltax API url
  • Voltax HTML Unknown crawled urls
  • Generated from 90min&TBL Monolith
  • Generated from Mentalfloss WP & Drupal
  • Generated from Fansided WP
  • Social Media Robots
  • Sitemap XML
Back to top

Warnings

  • 2 invalid lines.
Back to top