Log In Sign Up

valme.io
robots.txt

Robots Exclusion Standard data for valme.io

Resource Scan

Scan Details

Site Domain valme.io
Base Domain valme.io
Scan Status Ok
Last Scan2024-10-17T05:15:49+00:00
Next Scan 2024-11-16T05:15:49+00:00

Last Scan

Scanned2024-10-17T05:15:49+00:00
URL https://valme.io/robots.txt
Domain IPs 38.45.64.207
Response IP 38.45.64.207
Found Yes
Hash c2f27b49ec349cc05e5035b404bd5a09a19b4a860579d30c8e54dee15811a543
SimHash 8e5dc5b82be1

Groups

*

Rule Path
Disallow /sandbox/

Other Records

Field Value
crawl-delay 10

ltx71 - (http://ltx71.com/)

Rule Path
Disallow /

velenpublicwebcrawler - (https://velen.io/)

Rule Path
Disallow /

idmarch

Rule Path
Disallow /

ias_crawler

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

npbot

Rule Path
Disallow /

slysearch

Rule Path
Disallow /
Back to top

Comments

  • Directories
  • Thank you https://www.videolan.org/robots.txt for the below
  • "This robot collects content from the Internet for the sole purpose of
  • helping educational institutions prevent plagiarism. [...] we compare
  • student papers against the content we find on the Internet to see if we
  • can find similarities." (http://www.turnitin.com/robot/crawlerinfo.html)
  • --> fuck off.
  • "NameProtect engages in crawling activity in search of a wide range of
  • brand and other intellectual property violations that may be of interest
  • to our clients." (http://www.nameprotect.com/botinfo.html)
  • --> fuck off.
  • "iThenticate® is a new service we have developed to combat the piracy
  • of intellectual property and ensure the originality of written work for
  • publishers, non-profit agencies, corporations, and newspapers."
  • (http://www.slysearch.com/)
  • --> fuck off.
Back to top