噂のdistributed key-value storage Voldemortを動かしてみた。

cloud Voldemort key-value

今日は、動かしてみただけ。

って、分散で、Javaで、キーバリューで 格納できて何か凄いらしい。

Voldemort is a distributed key-value storage system

* Data is automatically replicated over multiple servers.
* Data is automatically partitioned so each server contains only a subset of the total data
* Server failure is handled transparently
* Pluggable serialization is supported to allow rich keys and values including lists and tuples with named fields, as well as to integrate with common serialization frameworks like Protocol Buffers, Thrift, and Java Serialization
* Data items are versioned to maximize data integrity in failure scenarios without compromising availability of the system
* Each node is independent of other nodes with no central point of failure or coordination
* Good single node performance: you can expect 10-20k operations per second depending on the machines, the network, the disk system, and the data replication factor
* Support for pluggable data placement strategies to support things like distribution across data centers that are geographical far apart.

It is used at LinkedIn for certain high-scalability storage problems where simple functional partitioning is not sufficient. It is still a new system which has rough edges, bad error messages, and probably plenty of uncaught bugs. Let us know if you find one of these, so we can fix it.

まぁ、何を言っているか良く分からないので、とりあえず動かす。
正直言って、

を動かしてみたかっただけなんだけどねぇ…


ダウンロードしたのは、

  • voldemort-0.51.zip

中身を開くとbashの起動スクリプトと、
テキトーに書かれたantスクリプトしかないので、
即動かすっていう訳にもいかない感じ。


僕の環境は、Windowsなので、NTスクリプトで書いて貰わないと非常にコマル。
いたしかたないので、本家のシェルスクリプトを参考にWindowsで動くスクリプトを書く等した。

  • setclasspath.cmd
set CLASSPATH=

for %%i in ( %~dp0..\dist\*.jar ) do ( call :setpath "%%i" ) 
for %%i in ( %~dp0..\lib\*.jar ) do ( call :setpath "%%i" )

goto endclasspath

:setpath
set CLASSPATH=%CLASSPATH%;%1
goto :EOF

:endclasspath
  • voldemort-server.cmd
@echo off
if "%1" == "" goto error
if not exist "%1" goto error

call setclasspath.cmd

java -Xmx512m -server -cp %CLASSPATH% -Dcom.sun.management.jmxremote voldemort.server.VoldemortServer %1

goto end

:error
echo USAGE: .\bin\voldemort-server.cmd [voldemort_home]
set ERROR_CODE=1

:end
  • voldemort-shell.cmd
@echo off
if "%2" == "" goto error

call setclasspath.cmd

java -Xmx512m -server -cp %CLASSPATH% voldemort.VoldemortClientShell %1 %2 %3

goto end

:error
echo USAGE: bin\voldemort-shell.cmd store_name bootstrap_url [command_file]
set ERROR_CODE=1

:end

細かい所で、ちゃんとした処理になってないのだけど、まぁ、とりあえず動いたヨって感じ。


サーバを起動するときには、

d:\voldemort-0.51\bin>voldemort-server.cmd ..\config\single_node_cluster

対話型シェルを動かす時には、

d:\voldemort-0.51\bin>voldemort-shell.cmd test tcp://localhost:6666
log4j:WARN No appenders could be found for logger (voldemort.store.socket.SocketPoolableObjectFactory).
log4j:WARN Please initialize the log4j system properly.
Established connection to test via tcp://localhost:6666
> help
Commands:
put key value -- Associate the given value with the key.
get key -- Retrieve the value associated with the key.
delete key -- Remove all values associated with the key.
locate key -- Determine which servers host the give key.
help -- Print this message.
exit -- Exit from this shell.

> put "hoge" "fuga"
> locate "hoge"
Node 0
host:  localhost
port: 6666
available: yes
last checked: 7250 ms ago

> exit
k k thx bye.

こんな感じ。