The Vose Way: Category Programming

Pagination, Sets, Checkboxes

vosechu@create-on.com (Chuck Vose) — Mon, 25 Aug 2008 15:29:00 -0500

When I was at RailsConf2008 I learned a lot about life and Rails but there were some nagging problems that lingered after me for a long time.

At one point in the talks there was a discussion of how to make listing pages have checkboxes and how to handle this data. The problem is complicated by pagination and filtering so it’s actually a really good problem to talk about.

The proposed solution was to send back a structure that looked something like this:

[{:id => 1, :activated => "+", :contact => "-"}, ...]

At this point I was probably playing pogs in the audience with Ryan Schenk so the details are lost to time but the concept remains.

It seems needlessly complex though. A very interesting idea, no doubt about it, and it certainly has its merit but it seems very complex to me when we’re talking about booleans. Also, I feel like it’s worth mentioning that this method is much faster than the below method but somehow that doesn’t turn me off that badly.

Instead of the above I took the already well accepted practice of using the check_box_tag in the view with the addition of a hidden field which would tell me which objects to care about:

<% @users.each do |user| %>
  <%= hidden_field_tag('seen[]', user.id) -%>
  <%= check_box_tag 'activated[]', user.id -%>
<% end %>

In the controller it’s as easy as loading up params[:seen] and loading each object:

def index
  if request.post?
    activated_ids = params[:activated].collect {|id| id.to_i} if params[:activated]
    seen_ids = params[:seen].collect {|id| id.to_i} if params[:seen]

    if activated_ids
      seen_ids.each do |id|
        r = User.find_by_id(id)
        r.activated = activated_ids.include?(id)
        r.save
      end
    end
  end
end

So what I don’t get is whether this is wrong on some moral stance or if I really didn’t understand what the presenter was getting at. Yes, he loads and saves fewer objects, which could be important. But he also introduced what seems to me to be a lot of complexity into a fairly simple process.

At any rate, this code is here in case someone needs to find it until I find what the presenter was proposing. I seem to recall being impressed at the time so there must be something I’ve forgotten.

Mnesia to ruby bridge evaluation

vosechu@create-on.com (Chuck Vose) — Wed, 02 Jan 2008 17:25:00 -0600

Owing to the complexities of building a Rails adapter for Mnesia I’ve been looking into using a Ruby to Erlang bridge and have looked at the various projects that seem to be available and want to share my comments on how each is stacking up so far.

Rebar

Rebar was announced on 2007-04-20 and has since never surfaced to the public as far as his blog and Google are concerned. The code looked to be among the easiest to understand but is thankfully very similar to the Japanese RulangBridge below. The existence of the Japanese project could explain why Tom Werner never finished the project but that’s merely a guess.

Erlectricity

Erlectricity by Scott Fleckenstein was the first project I really tried out and I am impressed by some aspects and disappointed by others. On one hand it was easy to obtain being hosted on RubyForge and Google Code and works well at using Ruby from Erlang. On the other hand it comes with no documentation or rdocs outside of two uncommented examples.

I could never figure out a way to get it to bridge Erlang commands from Ruby but there may be a way that I’m missing; I have to admit I’ve not asked Scott about whether this code can go both ways or not.

So if you need something that Ruby does well but Erlang doesn’t then this may be your project. The second example uses the bridge to generate a gruff graph from Erlang which seems like it’ll come in handy for a lot of people.

Outside of documentation I’m concerned about whether the bridge can handle asynchronous requests from Erlang. From first glance it doesn’t seem like there’s any blocking and each Erlang thread calls the script directly instead of central blocking thread so I’m guessing that there’s no concurrency built in; something that we would have to work with if we were to use this bridge for a many-threaded project.

RulangBridge

RulangBridge by Toshi Hirooka (?) is a Japanese project allowing Ruby to use Erlang functions. Google’s translation allows us to read the usage instructions and browse the code which has allowed me to start using it in earnest to build the Ruby to Mnesia bridge.

My first impressions are that the bridge is that it is a little immature despite the >1.0 release number. The example code is excellent (if in Japanese) but it isn’t packaged in a gem or hosted on RubyForge and doesn’t have a way to auto-start an Erlang server. Furthermore, using only the built in classes we have to make the choice of asynchronicity or complicated/multi-module code.

Of course the raw class (called Erlang) can be made asynchronous by wrapping it in a Thread.new which is essentially what the Rulang wrapper class does, but it would be nice to have this built-in. Putting the asynchronous switch in the wrapper class is fine but the wrapper class suffers from a desire to make the Erlang calls feel like ruby and therefore makes calling complicated code impossible through the wrapper and thus makes the asynchronous aspect moot also.

The last concern I have is that each class must connect to a node explicitly rather than being able to automatically find and balance between nodes. In order to be happy with the adapter we’ll have to figure out how to load-balance or make Erlang do it for us.

Conclusion

For the purposes of the the Ruby to Mnesia Adapter I believe that RulangBridge will be sufficient but Erlectricity definitely has its purposes in the Erlang community. Despite some concerns and drawbacks both are usable code and could easily move forward with a little love from the community or follow-up versions by the owners.

Any project involving Erlang and Ruby will have to deal extensively with concurrency. Ideally it would be nice to see a broker model between the two that supports communication in both directions and deals with concurrency issues and load-balancing transparently. Until that happens we’ll have to work with Erlectricity and Rulang and love the creators for their hard work.

Mnesia + Ruby Announcement

vosechu@create-on.com (Chuck Vose) — Sat, 22 Dec 2007 01:17:00 -0600

My goal this year is to release a Ruby port of the Mnesia distributed database and hopefully start the process of moving to a true slice architecture. The port is an interesting project but I think the importance of the slice architecture is paramount.

For the last couple of years we’ve been working on the n-tier model with ruby. It’s well established and it has been working nicely for us. But the web server industry is starting to move more towards the idea of instances or clouds of ambiguous slices. Amazon is doing it, mongrel is a part of it certainly, Mnesia has always worked this way.

My hope is that my port will help us to create an EC2 instance that is both the master of it’s domain and a part of a cloud at the same time. I would like to see an EC2 instance that can autoconfigure itself and automatically find its neighbors, which contains a complete Mnesia instance, a couple mongrels, and a proxy/load balancer.

I’m not sure if we can do this quite yet but multi-processor theory suggests that it can be done. Whether it’s advantageous to remove all the bottle-necks and have to deal with the scheduling individually is where we’ll have to analyze but I’m confident that we are moving somewhere truly incredible.

In the future I hope to be able to drop in a new EC2 and just have it completely figure things out for me. No more MySQL master, no more apache proxying. Whether we use my new port or SimpleDB is of no concern to me at all.

find_or_create by params (extension to dynamic attribute based finders)

vosechu@create-on.com (Chuck Vose) — Wed, 12 Sep 2007 22:40:00 -0500

Rails has a dynamic method where you can do find_or_create_by_attr_name(attr) but it the method names get incredibly long very quickly. So SJS wrote this article in which he tries to remedy the situation. His solution was pretty elegant but it still only worked for fields that were already defined in the database; if you often redefine setter methods it doesn’t work at all.

Here’s my attempt to remedy the situation.

module ActiveRecordExtensions
  def self.included(base)
    base.extend(ClassMethods)
  end

  module ClassMethods
    def find_or_create(params)
      begin
        return self.find(params[:id])
      rescue ActiveRecord::RecordNotFound => e
        attrs = {}

        # search for valid attributes in params
        self.column_names.map(&:to_sym).each do |attrib|
          # skip unknown columns, and the id field
          next if params[attrib].nil? || attrib == :id

          attrs[attrib] = params[attrib]
        end

        # call the appropriate ActiveRecord finder method
        found = self.send("find_by_#{attrs.keys.join('_and_')}", *attrs.values) if !attrs.empty?

        if found && !found.nil?
          return found
        else
          return self.create(params)
        end
      end
    end
    alias create_or_find find_or_create
  end
end

ActiveRecord::Base.send(:include, ActiveRecordExtensions)

Newb instructions

Create a file called active_record_extensions.rb in your the lib directory. Then add `require ‘active_record_extensions’` to your environment.rb at the bottom (without the ``). Restart your server and see what happens!

Long Bets

vosechu@create-on.com (Chuck Vose) — Tue, 11 Sep 2007 23:42:00 -0500

Sam Ruby occasionally does an article about his long bets and I really respect his ability to predict the future. This time I feel that he was focusing too close and forgot the long part of the ‘long bet’. REST is already important, and there are further developments in edge rails that make it even more important (active resource). MySQL is already a pain for scaling and the plugins are starting to come out but haven’t really hit the fan yet.

Bet One – MySQL becomes the exception

First, I think Sam Ruby is dead on about databases. Right now there are only a few things that are a consistent pain for deployment and scaling and databases are the top of the heap. I know there are things like MaxDB for mysql and there are ways to set up ring replication but it’s hard; something that Ruby’s community is really good at solving.

My first bet is that MySQL dies out for the rails community and something new pops in. It wouldn’t surprise me if it was Mnesia from Erlang/OTP or something involving Hadoop and HBase. It would surprise me if CouchDB popped onto the scene in a big way but it’s the sort of thinking that could lead us somewhere interesting. The other option is that someone comes up with a really concrete stack of abstractions that makes it easy to balance mysql requests and writes.

Bet Two – Rails drops REST completely

Secondly, I think REST is the wrong way to move forward. REST maps very well onto the CRUD principles, but I feel like we very rarely actually use just CRUD. More often than not I want to run custom little things and create crazy associations. And I realize that this is all possible in the REST model, but it makes the controllers obscene sometimes.

What we really want is Query Language for the Internet and what better language to build that in than Ruby. I’ve seen DSL’s for direct database access and it seems like the routes would be just around the corner if this is where we decide to take it.

The question is whether DHH sees the writing on the wall or desperately wants to hang on to REST. Since he’s put so much effort into the REST idea it seems like he would be loath to drop it, but at the same time he’s an incredibly mature developer and would hopefully handle a change like this if it ever happened.

Conclusion

All respect to Sam Ruby, I really do respect his predictions over my own. But I think his predictions are too close to us right now. I would like to know what happens after REST and what happens in the database arena.

Arrow keys in vi and iTerm

vosechu@create-on.com (Chuck Vose) — Thu, 26 Apr 2007 17:25:00 -0500

Arrow keys doing funny things when you use vi in iTerm 0.9.5? It’s because we’re too used to messing with things. Set your Default bookmark’s keyboard type back to Global.

“When the terminal type of your sessions is set to one of those “xterm” types, iTerm handles the arrow keys, the home key, and the end key in a special way.” says the FAQ while not actually addressing the solution.

But who can complain? Global works well now! Hooray!

Testing rails without fixtures

vosechu@create-on.com (Chuck Vose) — Wed, 25 Apr 2007 16:21:00 -0500

At On & On Creative we have a lot of sites on a common rails backend. It’s incredibly flexible so we can offer a lot of support to customers but because there are so many customers with different needs testing has become a huge pain. Not only does it let us run tests custom to each client, it provides a way to import the data to the test db really easily and it’s way faster than loading the fixtures each time you start a test. Yay!

After googling around I happened upon this caboo.se article about how to run without fixtures at all. It’s a very indepth article with lots of good things in it; far be it for me to try to compete.

Instead, here’s the low down on how we prepare each of our sites for testing.

Add the following to the end of your rails_root Rakefile:


class Rake::Task
  def detract(prerequisite)
    @prerequisites.delete(prerequisite)
  end
end

%w(units functionals integration recent uncommitted).each do |task|
  Rake::Task["test:#{task}"].detract('db:test:prepare')
  Rake::Task["test:#{task}"].enhance(['environment'])
end

Add the following to lib/tasks/fixtures.rake. Make sure to look at the two commented lines:


# This code courtesy of the "Rails Recipe's book":http://www.pragmaticprogrammer.com/titles/fr_rr/ I believe. 
namespace :db do
  namespace :fixtures do

    desc 'Create YAML test fixtures from data in an existing database. Defaults to development database. Set RAILS_ENV to override.'
    task :dump => :environment do
      # Remove the limit if you want to. It's a timesaver mostly
      sql  = "SELECT * FROM %s LIMIT 0,1000" 
      # Add tables you don't want dumped. 
      skip_tables = ["schema_info", "plugin_schema_info", "engine_schema_info", "sessions"]
      ActiveRecord::Base.establish_connection(:development)
      (ActiveRecord::Base.connection.tables - skip_tables).each do |table_name|
        i = "000" 
        File.open("#{RAILS_ROOT}/test/fixtures/#{table_name}.yml", 'w') do |file|
          data = ActiveRecord::Base.connection.select_all(sql % table_name)
          file.write data.inject({}) { |hash, record|
            hash["#{table_name}_#{i.succ!}"] = record
            hash
          }.to_yaml
        end
      end
    end
  end
end

Then the fun easy stuff happens:


rake db:fixtures:dump RAILS_ENV=production
rake db:fixtures:load RAILS_ENV=test
rake test:plugins PLUGIN=your_plugin

If you want to commit the fixtures to the repo you can but it seems a waste to me since you might be regenerating them often.

Also, check the two comments in the fixtures.rake, it could be doing things you don’t expect.

Sending messages to all users

vosechu@create-on.com (Chuck Vose) — Mon, 08 May 2006 03:19:00 -0500

One of the biggest fears of all system admins and developers is publishing work to a live server and having it break everything. Naturally we all go to great lengths to make sure this doesn’t happen but even the most well-tested code in the simplest environment still breaks sometimes.

In addition to these fatalities is the less intrusive but equally damning maintenance that sometimes just has to happen in the middle of the day whether there’s users in the system or not. I’m sure you can think of a few scenarios and horror stories.

At any rate I’ve come to realize that it’s extremely important to be able to tell the people in the system that things are going to break for a bit. This is fortunately extremely easy in Rails; a quick three or four lines in the application.rb will do it. In PHP it’s easy if there is even a little planning that went into the project.

Despite the ease I don’t think this is very common in the world of small-to-medium businesses and this seems like a shame.

Letting your users know what’s about to happen or what happened a moment ago will increase their sense of security in many cases and in the very least will keep them from calling you in a panic.

Blissful silence.

Subversion

vosechu@create-on.com (Chuck Vose) — Fri, 21 Apr 2006 10:32:28 -0500

I’m finally learning, oh yes.

The reason IDE’s make it so difficult to copy folders around is because it’s an incredibly difficult process. In order to copy in Subversion you have to copy the folders/files, delete any .svn directories in each level of a folder, then those folders/files have to be added to the repo.

Very complicated process which has been killing me lately. I should have known better; how not DRY of me to copy a folder…

Why Do Unit Tests Exist?

vosechu@create-on.com (Chuck Vose) — Thu, 20 Apr 2006 14:39:00 -0500

Holy crap I finally figured out why unit tests exist! To give me something to think about in the shower. Yep, simple as that.

No, wait, there are two really good uses off the top of my head: packaging and testing filesystem crap.

The first use is if you’re exporting a plugin for later use (or for general distribution). In this scenario there’s a fairly good chance that the user importing your plugin has messed up their install or is using a database that isn’t the same version or even the same type as the one you’re using.

In either case if you have unit tests and none of them fail then you know that the database probably isn’t at fault.

The second use is for filesystem crap. I can still count the number of times I’ve messed up the permissions on a folder only to find it after a half hour of debugging. This should never happen and there are wonderful tools to prevent it. `ri File` and `ri FileUtils` should get you started if you’re curious.

Specifically I’ve been working on a photo upload admin because I don’t understand file_column. In this I occasionally have to make a directory (okay, every time I upload something that isn’t a graphic) and it would be really nice to know that when I post it onto our production server that it’ll work when I release it into the wild.

Under this column is a wonderful addition to test_helper to check that things are correctly set in other places. Since rake is seperate from the dispatchers it could check their permissions, check the permissions of the public and log folders, and a myriad of other wonderful tests that could help newbies get on their way.

Rake is already pretty good at this but it serves as an excellent example of one use for unit testing.