Log In Sign Up

lagsidene.com
robots.txt

Robots Exclusion Standard data for lagsidene.com

Resource Scan

Scan Details

Site Domain lagsidene.com
Base Domain lagsidene.com
Scan Status Ok
Last Scan2024-11-09T20:46:14+00:00
Next Scan 2024-11-16T20:46:14+00:00

Last Scan

Scanned2024-11-09T20:46:14+00:00
URL https://lagsidene.com/robots.txt
Domain IPs 54.154.91.134
Response IP 54.154.91.134
Found Yes
Hash eab892c7ea639a395aca294c88198b04ae0846182e2d2ea578280af0897ba12e
SimHash f2552bc5cc72

Groups

*

Rule Path
Disallow /main/team_name_url
Disallow */wp-admin/*
Disallow */wp-login.php
Disallow */wp-register.php
Disallow /password_forgot
Disallow /common/*
Disallow /advert_click
Disallow /advert/
Disallow /private_team_files
Disallow /map/static

mediapartners-google

Rule Path
Disallow
Back to top

Other Records

Field Value
sitemap /sitemap.xml
Back to top

Comments

  • robots.txt
  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • This url is for ajax calls only
  • Blog related
  • Disallow common, e.g. advert clicks
  • Never end up crawling private files
  • No indexing of static maps
  • Allow adsense crawler
Back to top