The Vose Way: Tag Mnesia

Complex commands with Rulang

vosechu@create-on.com (Chuck Vose) — Thu, 03 Jan 2008 12:17:00 -0500

Rulang was giving me a lot of trouble with regards to multi-line commands and complex commands in general. After hacking on it for a while I’ve developed a patch that will allow ruby code such as the following:

require 'rulang'

@mnesia = RulangBridge::Erlang.new("localhost", 9900)

def find
  @mnesia.eval(<<-EOF
    QH = qlc:q([X || X <- mnesia:table(shop)]),
    F = fun() -> qlc:eval(QH) end,
    {atomic, Val} = mnesia:transaction(F),
    Val.
  EOF
  )
end

puts find

PATCH

diff -u Desktop/rulangbridge/rulang.erl Current Schoolwork/Project/mnesia/rulang_test/rulang.erl
--- Desktop/rulangbridge/rulang.erl     2007-05-17 20:25:50.000000000 -0700
+++ Current Schoolwork/Project/mnesia/rulang_test/rulang.erl    2008-01-03 10:12:05.000000000 -0800
@@ -30,11 +30,15 @@
 handle_connection(Socket) ->
-       Reason = (catch communication(Socket)),
-       gen_tcp:send(Socket, io_lib:format("Error: ~w~n", [Reason])),
+       try communication(Socket)
+  catch
+    error:Reason ->
+         {gen_tcp:send(Socket, io_lib:format("Error: ~p~n", [Reason]))}
+  end,
        ok = gen_tcp:close(Socket).

+% Try to evaluate the code submitted throwing an exception if the evaluation
+% doesn't work. Return the code submitted.
 communication(Socket) ->
        {ok, Binary} = gen_tcp:recv(Socket, 0),
        {ok, Result} = eval(binary_to_list(Binary)),
@@ -43,9 +47,9 @@

 eval(Expression) ->
-       {ok, Scanned, _} = erl_scan:string(Expression),
-       {ok, [Parsed]} = erl_parse:parse_exprs(Scanned),
-       {value, Result, _} = erl_eval:expr(Parsed, []),
+       {done, {ok, Scanned, _}, _} = erl_scan:tokens([], Expression, 0),
+       {ok, Parsed} = erl_parse:parse_exprs(Scanned),
+       {value, Result, _} = erl_eval:exprs(Parsed, []),
        {ok, Result}.

diff -u Desktop/rulangbridge/rulang.rb Current Schoolwork/Project/mnesia/rulang_test/rulang.rb
--- Desktop/rulangbridge/rulang.rb      2007-05-24 10:42:22.000000000 -0700
+++ Current Schoolwork/Project/mnesia/rulang_test/rulang.rb     2008-01-03 10:02:19.000000000 -0800
@@ -79,7 +79,7 @@
                def eval(command)
                        socket = TCPSocket.new(@host, @port)
                        socket.write(command)
-                       socket.gets # ...?
+                       socket.read # ...?
                end

Mnesia + Ruby Announcement

vosechu@create-on.com (Chuck Vose) — Sat, 22 Dec 2007 01:17:00 -0500

My goal this year is to release a Ruby port of the Mnesia distributed database and hopefully start the process of moving to a true slice architecture. The port is an interesting project but I think the importance of the slice architecture is paramount.

For the last couple of years we’ve been working on the n-tier model with ruby. It’s well established and it has been working nicely for us. But the web server industry is starting to move more towards the idea of instances or clouds of ambiguous slices. Amazon is doing it, mongrel is a part of it certainly, Mnesia has always worked this way.

My hope is that my port will help us to create an EC2 instance that is both the master of it’s domain and a part of a cloud at the same time. I would like to see an EC2 instance that can autoconfigure itself and automatically find its neighbors, which contains a complete Mnesia instance, a couple mongrels, and a proxy/load balancer.

I’m not sure if we can do this quite yet but multi-processor theory suggests that it can be done. Whether it’s advantageous to remove all the bottle-necks and have to deal with the scheduling individually is where we’ll have to analyze but I’m confident that we are moving somewhere truly incredible.

In the future I hope to be able to drop in a new EC2 and just have it completely figure things out for me. No more MySQL master, no more apache proxying. Whether we use my new port or SimpleDB is of no concern to me at all.

Long Bets

vosechu@create-on.com (Chuck Vose) — Tue, 11 Sep 2007 23:42:00 -0400

Sam Ruby occasionally does an article about his long bets and I really respect his ability to predict the future. This time I feel that he was focusing too close and forgot the long part of the ‘long bet’. REST is already important, and there are further developments in edge rails that make it even more important (active resource). MySQL is already a pain for scaling and the plugins are starting to come out but haven’t really hit the fan yet.

Bet One – MySQL becomes the exception

First, I think Sam Ruby is dead on about databases. Right now there are only a few things that are a consistent pain for deployment and scaling and databases are the top of the heap. I know there are things like MaxDB for mysql and there are ways to set up ring replication but it’s hard; something that Ruby’s community is really good at solving.

My first bet is that MySQL dies out for the rails community and something new pops in. It wouldn’t surprise me if it was Mnesia from Erlang/OTP or something involving Hadoop and HBase. It would surprise me if CouchDB popped onto the scene in a big way but it’s the sort of thinking that could lead us somewhere interesting. The other option is that someone comes up with a really concrete stack of abstractions that makes it easy to balance mysql requests and writes.

Bet Two – Rails drops REST completely

Secondly, I think REST is the wrong way to move forward. REST maps very well onto the CRUD principles, but I feel like we very rarely actually use just CRUD. More often than not I want to run custom little things and create crazy associations. And I realize that this is all possible in the REST model, but it makes the controllers obscene sometimes.

What we really want is Query Language for the Internet and what better language to build that in than Ruby. I’ve seen DSL’s for direct database access and it seems like the routes would be just around the corner if this is where we decide to take it.

The question is whether DHH sees the writing on the wall or desperately wants to hang on to REST. Since he’s put so much effort into the REST idea it seems like he would be loath to drop it, but at the same time he’s an incredibly mature developer and would hopefully handle a change like this if it ever happened.

Conclusion

All respect to Sam Ruby, I really do respect his predictions over my own. But I think his predictions are too close to us right now. I would like to know what happens after REST and what happens in the database arena.