Log In Sign Up

coursereport.com
robots.txt

Robots Exclusion Standard data for coursereport.com

Resource Scan

Scan Details

Site Domain coursereport.com
Base Domain coursereport.com
Scan Status Ok
Last Scan2024-06-22T11:04:22+00:00
Next Scan 2024-07-22T11:04:22+00:00

Last Scan

Scanned2024-06-22T11:04:22+00:00
URL https://coursereport.com/robots.txt
Redirect https://www.coursereport.com/robots.txt
Redirect Domain www.coursereport.com
Redirect Base coursereport.com
Domain IPs 3.220.57.224, 3.232.242.170, 52.20.78.240, 54.91.59.199
Redirect IPs 3.220.57.224, 3.232.242.170, 52.20.78.240, 54.91.59.199
Response IP 52.20.78.240
Found Yes
Hash 76472b4aa9dee7e517d01023eee1cf845a91fa5d07d3ed7d5428fc2579edbb22
SimHash 26b80c14a240

Groups

*

Rule Path
Disallow /results
Disallow /login?*
Disallow /logout?*
Disallow /admin
Disallow /schools/*/admin
Disallow /schools/*?reviews_page=*&news_page=*
Disallow /schools/*?news_page=*&reviews_page=*
Disallow /reviews/*/votes
Disallow /*claim-page
Disallow /*redirect_path
Disallow /*contact_redirect
Disallow /do-not-sell
Disallow /posts/index*
Disallow /*cost%3D
Disallow /*focus%3D
Disallow /*location%3D
Disallow /*review_offer_id%3D
Disallow /*track%3D
Disallow /admin/
Disallow /users/auth/
Disallow /terms-of-service/

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://coursereport-s3-production.global.ssl.fastly.net/sitemaps/sitemap.xml.gz
Back to top

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /
  • Disallow: /schools/*?page=*
  • Disallow: /schools/*&page=*
  • Disallow: /*shared_review
  • Disallow: /*.modal$
Back to top