Log In Sign Up

playpilot.com
robots.txt

Robots Exclusion Standard data for playpilot.com

Resource Scan

Scan Details

Site Domain playpilot.com
Base Domain playpilot.com
Scan Status Ok
Last Scan2024-11-11T17:43:06+00:00
Next Scan 2024-11-18T17:43:06+00:00

Last Scan

Scanned2024-11-11T17:43:06+00:00
URL https://playpilot.com/robots.txt
Redirect https://www.playpilot.com/robots.txt
Redirect Domain www.playpilot.com
Redirect Base playpilot.com
Domain IPs 104.26.2.138, 104.26.3.138, 172.67.69.152, 2606:4700:20::681a:28a, 2606:4700:20::681a:38a, 2606:4700:20::ac43:4598
Redirect IPs 104.26.2.138, 104.26.3.138, 172.67.69.152, 2606:4700:20::681a:28a, 2606:4700:20::681a:38a, 2606:4700:20::ac43:4598
Response IP 104.26.2.138
Found Yes
Hash e93efbfe8b40d7a4f9bd1fcfbf5d7c53e481a050a6f25b2ba5fca4d2f67a0684
SimHash efea88234b35

Groups

*

Rule Path
Disallow /cdn-cgi/
Disallow /gravity/
Disallow /notifications/
Disallow /weekly-digest/
Disallow /search/
Disallow /*?imdb_score*
Disallow /*%26imdb_score*
Disallow /*?year*
Disallow /*%26year*
Disallow /*%26country
Disallow /*?country
Disallow /*%26length
Disallow /*?length
Disallow /podcast/*/episodes
Disallow /titles/*/
Disallow /titles/*/*sorting
Disallow /titles/service/*/*/
Disallow /titles/service/*/*/*sorting
Disallow /titles/*type
Disallow /titles/*type*sorting
Disallow /titles/*sorting*type
Disallow /titles/service/*/*type*sorting
Disallow /titles/service/*/*sorting*type
Disallow /titles/*imdb_score
Disallow /titles/*imdb_score*sorting
Disallow /titles/*sorting*imdb_score
Disallow /titles/service/*/*imdb_score*sorting
Disallow /titles/service/*/*sorting*imdb_score
Disallow /titles/*year
Disallow /titles/*year*sorting
Disallow /titles/*sorting*year
Disallow /titles/service/*/*year*sorting
Disallow /titles/service/*/*sorting*year
Disallow /titles/*length
Disallow /titles/*length*sorting
Disallow /titles/*sorting*length
Disallow /titles/service/*/*length*sorting
Disallow /titles/service/*/*sorting*length
Disallow /titles/*country
Disallow /titles/*country*sorting
Disallow /titles/*sorting*country
Disallow /titles/service/*/*country*sorting
Disallow /titles/service/*/*sorting*country
Allow /titles/?*sorting
Allow /titles/service/*/$
Allow /titles/service/*/?*sorting
Allow /user/serieradet
Allow /user/ronny547
Allow /user/ebbakleberg
Allow /user/blankensblankens
Allow /user/johanna547
Allow /user/miriampoulsen
Allow /user/frandforsjulia
Allow /user/sidselolivia
Allow /user/seriedrottningen
Allow /user/perusames
Allow /user/anne-marie-4941411031
Allow /user/felice-738877
Allow /user/kenneth-1580791031
Allow /user/edward3
Allow /user/tina841
Allow /user/kasper-2728891031
Allow /user/david03
Allow /user/daniellundgren
Allow /user/kimwickell-383957
Allow /user/mitchel1
Allow /user/jimmymerilainen
Allow /user/lorenzofontana
Allow /user/henrikbrandt
Allow /user/emil2241
Allow /user/jonashauger-794883
Allow /user/alexander23
Allow /user/isabella-531656
Allow /user/duddi-900544
Allow /user/connycarlsson-478833
Allow /user/hannadahlen
Allow /user/thomas444
Allow /user/pernierhaus
Allow /user/alexanderstahlberg
Allow /user/christianhedegaard-242028
Allow /user/jansankala
Allow /user/max52
Allow /user/mussefilmar
Allow /user/helenawitt
Allow /user/johannekohlmetz

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://sitemaps.playpilot.com/sitemap.xml
Back to top

Comments

  • This path is used internally by Cloudflare and Google encounters errors when crawling it
  • Disallowed Pages
  • Disallowed Explore Parameters
  • Podcast 'all episodes' pages are duplicate content compared to the main podcast page
  • Titles Page Settings
  • Disallow any genre page
  • Disallow any page that contains type
  • Disallow any page that contains imdb_score
  • Disallow any page that contains year
  • Disallow any page that contains length
  • Disallow any page that contains country
  • Allow sorting in main page
  • Allow service pages
  • Allow sorting in service pages
  • Allow creators pages (do not touch the line below)
Back to top