Log In Sign Up

bbbike.de
robots.txt

Robots Exclusion Standard data for bbbike.de

Resource Scan

Scan Details

Site Domain bbbike.de
Base Domain bbbike.de
Scan Status Ok
Last Scan2024-06-13T14:49:21+00:00
Next Scan 2024-07-13T14:49:21+00:00

Last Scan

Scanned2024-06-13T14:49:21+00:00
URL http://bbbike.de/robots.txt
Domain IPs 83.169.5.248
Response IP 83.169.5.248
Found Yes
Hash a85cd6e07f0f921230b3873dc6f7be557b70b34932471428ea4851b3124ffa39
SimHash 901c8119917e

Groups

*

Rule Path
Disallow /~slaven/
Disallow /cgi-bin/mapserver_comment.cgi
Disallow /cgi-bin/bbbike-snapshot.cgi
Disallow /cgi-bin/bbbike-data.cgi
Disallow /mapserver/brb/tmp
Disallow /cgi-bin/mapserv?map=
Disallow /cgi-bin/mapserv?layer=
Disallow /cgi-bin/mapserver_address.cgi?coords=
Disallow /cgi-bin/bbbike.cgi?pref_
Disallow /cgi-bin/bbbike.cgi?start=
Disallow /cgi-bin/bbbike.cgi?startc=
Disallow /cgi-bin/bbbike.cgi?startname=
Disallow /cgi-bin/bbbike.cgi?startplz=
Disallow /cgi-bin/bbbike.cgi?startmapimg
Disallow /cgi-bin/bbbike.cgi?via=
Disallow /cgi-bin/bbbike.cgi?viamapimg
Disallow /cgi-bin/bbbike.cgi?ziel=
Disallow /cgi-bin/bbbike.cgi?ziel2=
Disallow /cgi-bin/bbbike.cgi?zielc=
Disallow /cgi-bin/bbbike.cgi?zielc_wgs84=
Disallow /cgi-bin/bbbike.cgi?zielname=
Disallow /cgi-bin/bbbike.cgi?zielplz=
Disallow /cgi-bin/bbbike.cgi?zielmapimg
Disallow /cgi-bin/bbbike.cgi?appid=android-0%3B
Disallow /cgi-bin/bbbike.cgi?appid=android-0&
Disallow /cgi-bin/bbbike.cgi?scope=
Disallow /cgi-bin/bbbike.cgi?scvf=
Disallow /cgi-bin/bbbike.en.cgi?pref_
Disallow /cgi-bin/bbbike.en.cgi?start=
Disallow /cgi-bin/bbbike.en.cgi?startc=
Disallow /cgi-bin/bbbike.en.cgi?startname=
Disallow /cgi-bin/bbbike.en.cgi?startplz=
Disallow /cgi-bin/bbbike.en.cgi?startmapimg
Disallow /cgi-bin/bbbike.en.cgi?via=
Disallow /cgi-bin/bbbike.en.cgi?viamapimg
Disallow /cgi-bin/bbbike.en.cgi?ziel=
Disallow /cgi-bin/bbbike.en.cgi?ziel2=
Disallow /cgi-bin/bbbike.en.cgi?zielc=
Disallow /cgi-bin/bbbike.en.cgi?zielc_wgs84=
Disallow /cgi-bin/bbbike.en.cgi?zielname=
Disallow /cgi-bin/bbbike.en.cgi?zielplz=
Disallow /cgi-bin/bbbike.en.cgi?zielmapimg
Disallow /cgi-bin/bbbike.en.cgi?appid=android-0%3B
Disallow /cgi-bin/bbbike.en.cgi?appid=android-0&
Disallow /cgi-bin/bbbike.en.cgi?scope=
Disallow /cgi-bin/bbbike.en.cgi?scvf=
Disallow /cgi-bin/bbbike2.cgi?pref_
Disallow /cgi-bin/bbbike2.cgi?start=
Disallow /cgi-bin/bbbike2.cgi?startc=
Disallow /cgi-bin/bbbike2.cgi?startname=
Disallow /cgi-bin/bbbike2.cgi?startplz=
Disallow /cgi-bin/bbbike2.cgi?startmapimg
Disallow /cgi-bin/bbbike2.cgi?via=
Disallow /cgi-bin/bbbike2.cgi?viamapimg
Disallow /cgi-bin/bbbike2.cgi?ziel=
Disallow /cgi-bin/bbbike2.cgi?ziel2=
Disallow /cgi-bin/bbbike2.cgi?zielc=
Disallow /cgi-bin/bbbike2.cgi?zielc_wgs84=
Disallow /cgi-bin/bbbike2.cgi?zielname=
Disallow /cgi-bin/bbbike2.cgi?zielplz=
Disallow /cgi-bin/bbbike2.cgi?zielmapimg
Disallow /cgi-bin/bbbike2.cgi?appid=android-0%3B
Disallow /cgi-bin/bbbike2.cgi?appid=android-0&
Disallow /cgi-bin/bbbike2.cgi?scope=
Disallow /cgi-bin/bbbike2.cgi?scvf=
Disallow /cgi-bin/wapbbbike.cgi
Disallow /wap
Disallow /cgi-bin/bbbikeleaflet.cgi?coordssession=
Disallow /cgi-bin/bbbikeleaflet.en.cgi?coordssession=
Disallow /BBBike/html/fragezeichenform.html?supplied_coord=%3Bstrname=
Disallow /cgi-bin/bbbikegooglemap.cgi?wpt_or_trk=

feedfinder

Rule Path
Disallow /

msnbot

Rule Path
Disallow /cgi-bin/A327B109991F487BB01E1A38C1F3A941_00000

twengabot-discover

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

cms crawler

Rule Path
Disallow /

spinn3r

Rule Path
Disallow /feed
Disallow /atom
Disallow /rss
Disallow /index.rss
Disallow /index.rdf
Disallow /xml/rss

gnam gnam spider

Rule Path
Disallow /

pixray-seeker

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /
Back to top

Comments

  • Don't want automatic comments
  • These are quite large and unlikely to be usable for bots
  • Don't crawl temporary locations (googlebot-image does this, for instance)
  • Don't let bots to route searches
  • German
  • English
  • German, beta
  • don't advertize the wap version anymore
  • leaflet with coordsessions
  • fragezeichen pages with filled form
  • ancient googlemaps with filled form
  • We don't have online shops here,
  • and the robot generated too many 404
  • Crawls ftp URLs badly and does not have the advertised /robot page
  • No CMS here, randomly picks pages generating 404
  • Crawls some feed-like URLs
  • Comes too often (every three minutes), does not have
  • a proper http URL in UA, seems to be an advertising
  • company (oxamedia.com)
  • Broken bot
  • Broken bot, and we don't have any best buys
  • Broken bot, but at least does not sound useless, so give it another chance...
  • User-agent: 008
  • Disallow: /
  • May cause errors because of wrong encoding (and not interesting anyway)
  • Obviously broken behavior of the legal amazonbot
Back to top