Log In Sign Up

weirder.earth
robots.txt

Robots Exclusion Standard data for weirder.earth

Resource Scan

Scan Details

Site Domain weirder.earth
Base Domain weirder.earth
Scan Status Ok
Last Scan2024-10-05T18:34:04+00:00
Next Scan 2024-10-06T18:34:04+00:00

Last Scan

Scanned2024-10-05T18:34:04+00:00
URL https://weirder.earth/robots.txt
Domain IPs 138.197.96.202
Response IP 138.197.96.202
Found Yes
Hash a4d3084a97d5f55151dd546253bff439d72a99937accede270e2e53c6118ff5d
SimHash b874194013d4

Groups

*

Rule Path
Disallow /

*

Rule Path
Disallow /media_proxy/
Disallow /interact/
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • lou, 2020-02-02:
  • Syntax checker, for adding changes:
  • http://www.tardis.ed.ac.uk/~sxw/robots/check/
  • disallow google entirely
  • see https://support.google.com/webmasters/answer/6062596?hl=de
  • User-agent: Googlebot
  • User-agent: AdsBot-Google
  • Disallow: /
  • alterantive: disallow indexing entirely
  • https://www.bjoernsworld.de/suchmaschinen/robots-txt.html
  • test here: https://de.ryte.com/free-tools/robots-txt/ (German)
  • or here: https://sitechecker.pro/robots-tester/ (English)
  • /lou
Back to top