Log In Sign Up

mysw.info
robots.txt

Robots Exclusion Standard data for mysw.info

Resource Scan

Scan Details

Site Domain mysw.info
Base Domain mysw.info
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-08-01T01:13:10+00:00
Next Scan 2024-10-30T01:13:10+00:00

Last Successful Scan

Scanned2023-12-12T22:02:10+00:00
URL https://mysw.info/robots.txt
Redirect http://www.mysw.info/robots.txt
Redirect Domain www.mysw.info
Redirect Base mysw.info
Domain IPs 104.27.202.88, 104.27.203.88
Redirect IPs 104.27.202.88, 104.27.203.88
Response IP 104.27.202.88
Found Yes
Hash 7e4628d3adb8729240e025f023bb4c1b88e00bc84bc6aa464dd42346cae0d099
SimHash bc129d194574

Groups

*

Rule Path
Disallow /includes/
Disallow /misc/
Disallow /modules/
Disallow /profiles/
Disallow /scripts/
Disallow /themes/
Disallow /comment/reply
Disallow /comment
Disallow /contact
Disallow /search
Disallow /user/register
Disallow /user/password
Disallow /user/login
Disallow /search/
Disallow /search/google*
Disallow /search/node*
Disallow /search/user*
Disallow /filter
Disallow /node$
Disallow /archive/all$
Disallow /archive/all/2011$
Disallow /*?sort*
Disallow /*%26sort*
Disallow /*?destination*
Disallow /*%26destination*
Disallow /tracker?

Other Records

Field Value
crawl-delay 10
Back to top

Comments

  • $Id: robots.txt,v 1.9.2.1 2008/12/10 20:12:19 goba Exp $
  • robots.txt
  • This file is to prevent the crawling and indexing of certain parts
  • of your site by web crawlers and spiders run by sites like Yahoo!
  • and Google. By telling these "robots" where not to go on your site,
  • you save bandwidth and server resources.
  • This file will be ignored unless it is at the root of your host:
  • Used: http://example.com/robots.txt
  • Ignored: http://example.com/site/robots.txt
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/wc/robots.html
  • For syntax checking, see:
  • http://www.sxw.org.uk/computing/robots/check.html
Back to top

Warnings

  • `host` is not a known field.
Back to top