Log In Sign Up

speedycam.be
robots.txt

Robots Exclusion Standard data for speedycam.be

Resource Scan

Scan Details

Site Domain speedycam.be
Base Domain speedycam.be
Scan Status Ok
Last Scan2024-10-30T22:00:33+00:00
Next Scan 2024-11-06T22:00:33+00:00

Last Scan

Scanned2024-10-30T22:00:33+00:00
URL https://speedycam.be/robots.txt
Domain IPs 104.21.26.14, 172.67.135.34, 2606:4700:3031::6815:1a0e, 2606:4700:3033::ac43:8722
Response IP 104.21.26.14
Found Yes
Hash 35ff443558f3cdb73a0a9ca43623954149d02bfc2dd3db3aa31943d833aaf205
SimHash 951459844dd2

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /photos/

atspider

Rule Path
Disallow /

cherrypicker

Rule Path
Disallow /

dsurf

Rule Path
Disallow /

elitesys entry

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

mail sweeper

Rule Path
Disallow /

munky

Rule Path
Disallow /

roverbot

Rule Path
Disallow /

webemailextrac

Rule Path
Disallow /

xget

Rule Path
Disallow /

wget

Rule Path
Disallow /

webwalk

Rule Path
Disallow /

webvac

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

webmirror

Rule Path
Disallow /

webfetcher

Rule Path
Disallow /

webcopy

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webcatcher

Rule Path
Disallow /

webbandit

Rule Path
Disallow /

w3mir

Rule Path
Disallow /

vobsub

Rule Path
Disallow /

templeton

Rule Path
Disallow /

ssearcher100

Rule Path
Disallow /

spiderbot

Rule Path
Disallow /

shai'hulud

Rule Path
Disallow /

pbwf

Rule Path
Disallow /

lightningdownload

Rule Path
Disallow /

kdd exploror

Rule Path
Disallow /

internet explore

Rule Path
Disallow /

infospiders

Rule Path
Disallow /

httrack

Rule Path
Disallow /

havindex

Rule Path
Disallow /

geturl

Rule Path
Disallow /

getbot

Rule Path
Disallow /

esirover

Rule Path
Disallow /

download wonder

Rule Path
Disallow /

collage

Rule Path
Disallow /

mozilla/2.0 (compatible; ms frontpage 4.0)

Rule Path
Disallow /
Back to top

Comments

  • Robots.txt for Euregio.Net Hosted Web Site
  • Disallow indexing of specific file extensions
  • Disallow Collectors and Spam
  • Disallow Offline Browsers
Back to top

Warnings

  • `useragent` is not a known field.
Back to top