Log In Sign Up

spotern.com
robots.txt

Robots Exclusion Standard data for spotern.com

Resource Scan

Scan Details

Site Domain spotern.com
Base Domain spotern.com
Scan Status Ok
Last Scan2024-11-09T09:48:21+00:00
Next Scan 2024-11-16T09:48:21+00:00

Last Scan

Scanned2024-11-09T09:48:21+00:00
URL https://spotern.com/robots.txt
Redirect https://www.spotern.com/robots.txt
Redirect Domain www.spotern.com
Redirect Base spotern.com
Domain IPs 217.182.175.181
Redirect IPs 217.182.175.181
Response IP 217.182.175.181
Found Yes
Hash 082aaece8df4db631ce4ba9974dc5f79c689378c7259598ec7a6bd12ba736b24
SimHash 9d1942428567

Groups

*

Rule Path
Disallow /*/user/*
Allow /*/movie/tag/*
Disallow /*/tag/*
Disallow /*/login
Disallow /*/signup
Disallow /*/spot/new
Disallow /*/wanted/new
Allow /
Back to top

Comments

  • Disallow somes pages
  • Disallow filter parameters
  • Disallow: /*?*type=*
  • Disallow: /*?*items=*
  • Disallow: /*?*brandID=*
  • Disallow: /*?*query=*
  • Disallow: /*?*quality=*
  • Disallow: /*?*productType=*
  • Disallow: /*?*productID=*
  • Disallow: /*?*people=*
  • Disallow: /*?*orderDir=*
  • Disallow: /*?*orderBy=*
  • Disallow: /*?*mediaType=*
  • Disallow: /*?*media=*
  • Disallow: /*?*season=*
  • Disallow: /*?*episode=*
  • Disallow: /*?*category=*
  • Disallow: /*?*brand=*
Back to top