The Power of Yes

Posted by Chuck Vose Fri, 30 Oct 2009 21:22:00 GMT

Cross-posted at metaltoad.com: The Power of Yes

Of all the powerful words in any language assertion is in my opinion the absolute most powerful. With affermative language we are able to create trust, enact people’s will, and begin processes. Silly of course, to take one word to mean so much but I believe everyone can agree that when a client asks for something ‘yes’ should be the thing that jumps to your lips immediately, hopefully without qualification but at times you have to add on things like ‘and it will cost x’ or ’, now lets look at the plan and see how we can make this a reality’.

But this post isn’t about clients, it’s about our language and framework choices.

When I was new to programming I said no a lot. This was in part an aspect of my first language PHP, which was perhaps as immature as I was at the time. ‘No’ was probably my biggest buzzword, maybe followed by ‘can’t’ which I justified by saying that PHP couldn’t do it. Turns out I was wrong, PHP really can do almost anything, but at the time I couldn’t. But ‘No’ was certainly an artifact of the culture. The things I do in PHP are often much harder than they need to be, at least they feel unnecessarily difficult to me now, and in this environment it is easy to say that it’s to hard. I’ve noticed since starting to use PHP again I’ve noticed ‘can’t’ creeping into my language more and more.

Ruby was the first language that was strongly oriented towards ‘yes’ both in the language and the culture. It could have been that I learned later in life or in a place that had a very strong Ruby culture (Seattle), but I suspect that the language had more to do with it than anything else. Ruby positively wants to make things faster for you so that the time between your thought and your expression of said thought grows shorter and shorter as you learn more Ruby. As such, when your language is saying ‘yes’ so often you naturally begin to start thinking this way yourself.

Drupal was a bit of a revelation to me in this regard. Drupal, as everyone knows, is a PHP app, yet despite the underlying language constructs the ui says ‘yes’ even more often than Ruby does. As my coworker pointed out, the amount of a website that gets finished before you hit the code is absolutely staggering. Maybe 90% of any given Drupal website could be accomplished by Drupal using only CCK/Views (et al) with the rest of the customizations being fairly quick using the copious APIs available to us. This is assuming that you find the correct project, in all things of course selecting the best tool for the job yields the best results.

So what are your languages saying to you every day? As products of our surroundings it is absolutely critical that we evaluate what our languages and tools say to us no just what they say about us. PHP may have a stigma, and Drupal through association, but I believe that the way it works with us is a much stronger reason to invest in exploring Drupal.

Posted in ,  | Tags , , , , , ,  | no comments

Multiple Dynamic Tabs in Brightcove 3

Posted by Chuck Vose Fri, 02 Oct 2009 17:57:00 GMT

Cross-posted at company blog: http://metaltoad.com/blog/multiple-dynamic-tabs-brightcove-3

This week I had the wonderful opportunity to work on an interesting problem that as far as I can tell hasn’t been documented. The call came out that we needed to generate a couple dynamic tabs on the top of our player for smart playlists. Now, we already have one dynamic playlist so I thought it was going to be a fairly simple logical step up to three but I was really, really wrong. If you read issue 3 of the pragprog magazine you’re probably thinking a lot about parallel and asynchronous processing. So it was extremely exciting to have come to a parallelization problem in my day to day activities. I’ll explain the problem:

In order to load a playlist into a brightcove player with the Player API you have to have already fetched the videos from the server using getMediaCollectionAsynch, getMediaInGroupAsynch, or getMediaAsynch. With one dynamic tab it’s pretty easy to use getMediaInGroupAsynch because you can assume that the only time the MEDIA_COLLECTION_LOAD event is going to fire is when your call has returned. But when you’re loading up two or more media items asynchronously you can no longer make that assumption.

If you’re just pulling down pre-defined playlists it’s super easy to drop something like the following in your onMediaCollectionLoad listener:

function onMediaCollectionLoad (e) {
  if (e.mediaCollection === null) {}
  else {
    tabBar.insertTabAt(e.mediaCollection, 0);
  }
}

However if you’re using getMediaInGroupAsynch or getMediaAsynch it can be difficult to figure out why the listener is getting called. For us we had three playlists of dynamic data but outside of comparing the resulting array to the original array (inefficient and error-prone) there is no way of knowing which asynch call asked for these videos.

But maybe we don’t care. All we care about is that the videos are loaded completely when we insert the tabs into the player right? So we stop relying on order and just make sure that all the videos are loaded into the player before adding some tabs. To do this I maintained a request counter (which could be called a counting semaphone were you so inclined. Before each request I increment the counter and when the request is filled I decrement the counter. When the counter hits 0 I know that all requests have been filled and I can safely add all the tabs at once.

Your code could probably look something like this:

var tab_count = 0;
var popular_tab;

function onTemplateReady (e) {
  tabBar = exp.getElementById("playlistTabs");

  if (popular.length > 0) {
    tab_count = tab_count + 1;
    popular_tab = true;
    content.getMediaInGroupAsynch(popular);
  }
}

function onMediaCollectionLoad (e) {
  if (e.mediaCollection === null) {
    // Do nothing
  }
  else {
    // Make sure that these playlists coming back are coming from 
    // getMediaInGroupAsynch. Otherwise their id would be positive.
    if (e.mediaCollection.id < 0) {
      // Add all media to the player's memory. 
      var mediaDTOs = new Array();
      jQuery.each(e.mediaCollection.mediaIds, function(i, val) {
        mediaDTOs.push(content.getMedia(val));
      });

      // Decrement the counter. When this hits zero we'll have filled all reqs.
      tab_count = tab_count - 1;

      if (tab_count == 0) {
        // Make sure this is one of the playlists we were going to add.   
        if (popular_tab) {
          popular_playlist = {
            displayName: "Most Viewed Videos",
            mediaIds: popular
          };
          tabBar.insertTabAt(content.createRuntimeMediaCollection(popular_playlist, 'playlist'), 0);
        }
      }
    }
  }
}

Here’s hoping this helps you out in your quest for multi-tabbed players and managing asynchronous loading.

Posted in  | Tags , , , , , , ,  | no comments

Fun with stakeout.rb

Posted by Chuck Vose Mon, 21 Sep 2009 22:55:00 GMT

Cross-posted to the company blog at: Fun with stakeout.rb

Having only left Rubyville a couple weeks ago there are still a lot of Ruby-based utilities that I still find incredibly handy. One such utility that I’ve recently fallen hopelessly in love with is stakeout.rb from: http://pragmaticautomation.com/cgi-bin/pragauto.cgi/Monitor/StakingOutFi…

Stakeout monitors your filestructure waiting for changes to happen and when they do it runs some command in the shell. Think about all the things you would like to happen when you change a file. Clear your cache? Refresh your browser via AppleScript? Run your PHPUnit tests? Program your Roomba vacuum to spin in a circle and sing for you? YES!

Take the code from the above webpage and drop it in a file called stakeout.rb. If you’re running Linux or OS X you probably already have Ruby. If you’re running windows I doubt the Ruby code would even work but do let me know if you do get it running and create a script for it.

After that simply run something like: ruby stakeout.rb ‘php drush.php clear cache’ sites/all/modules//

Now try editing and saving one of your module files. Success?

Rejoice!

Posted in  | Tags , , ,  | no comments

Converting from Brightcove 2 and loading arbitrary videos

Posted by Chuck Vose Fri, 18 Sep 2009 23:19:00 GMT

Cross-posted to: Company blog version

Over the last two weeks I’ve been working on a project to convert one of our client’s websites to Brightcove 3 players. As I am quite new to drupal but not to PHP you can imagine that many of the problems I had were unrelated to brightcove itself but there were a few aspects that I thought were worth documenting for those who are in the (surprisingly) enviable position of doing this migration.

First off I want to mention how huge of an upgrade it was going between 2 and 3. It may not look like it from the player aspect, many of the players look largely the same, but the backend work was tremendous. In addition to creating a spectacular new studio for uploading media and creating playlists they redesigned the players to allow greater customization without having to build your own swf player. What’s best about the backend changes is that if you really want you can actually keep your v2 players and still use the updated studio to manage your media as well as using the updated Media API to directly query their database for content to display. But what’s been crucial for us is the new Player API which allows us to use javascript to customize the actions of the player, load videos manually, and generally cause incredible ruckus without having to recompile the player itself.

To the point though, there is one mammoth gotcha in the upgrade between v2 and v3: if you’re specifying videos via the params (ie: almost every one of us) videos, the playlists they’re in, and the player both are in, must be from the same account. Additionally, videos must belong to the playlist that’s currently loaded as well or it just will ignore your selection.

Before I believe it was a little loosey goosey about where things came from but now there is this hard restriction. A little arbitrary feeling but there are ways of working around it. Here’s how we ended up working around the problem of having videos that didn’t have a playlist or that we just wanted to load in a different order:

In order to get a list of videos into javascript quickly and easily I used drupal_to_js and let it sort out the details. This was in my module file.

  <script language="JavaScript" type="text/javascript">
    videos = '. drupal_to_js($videos) .';
  </script>

In our js file we create the variables we need, (remember to put var in the front or ie will flip out) and add listeners. TEMPLATE_READY will be called when the template is fully loaded and MEDIA_COLLECTION_LOAD will be called anytime there’s a switch between playlists or if we use the getMediaInGroupAsynch() function.

var player;
var video, exp, social, content;
var tabBar, videoList;

function onTemplateLoaded(pPlayer) {

  player  = bcPlayer.getPlayer(pPlayer);
  video   = player.getModule(APIModules.VIDEO_PLAYER);
  exp     = player.getModule(APIModules.EXPERIENCE);
  content = player.getModule(APIModules.CONTENT);

  exp.addEventListener(BCExperienceEvent.TEMPLATE_READY, onTemplateReady);
  if (videos != null){
    content.addEventListener(BCContentEvent.MEDIA_COLLECTION_LOAD, onMediaCollectionLoad);
  }
}

When the player is loaded we want to load in a new set of videos. We don’t need to do anything with the return value because it will be caught by the listener we defined earlier in the js file.

function onTemplateReady(e) {
  tabBar = exp.getElementByID("playlistTabs");
  videoList = exp.getElementByID("videoList");

  if (videos != null) {
    content.getMediaInGroupAsynch(videos);
  }
}

Finally we get to actually use the videos returned to the player. It’s worth noting that the videos var here is exactly the same as above, a comma delimited list of video_ids. getMediaInGroupAsynch() doesn’t actually do anything with the videos it loads, it just loads them in the player so that we can use them later.

function onMediaCollectionLoad(e) {
  if (e.mediaCollection == null) {
  // Do nothing because no results came back. Must have been all disabled videos. 
  }
  else {
    //once the ids have been fetched from the service, create the runtime lineup.
    var playlist = {
      displayName: "Selected Videos",
      mediaIds: videos
    };
    var runtimeLineup = content.createRuntimeMediaCollection(playlist,"playlist");
    tabBar.insertTabAt(runtimeLineup, 0);

    // Select the new tab we created.
    tabBar.setSelectedIndex(0);

    // Select the correct video and play it.
    videoList.setSelectedIndex(0);
    video.loadVideo(list.getSelectedData());
  }
}

Now you may be saying to yourself, why can’t I just specify a list of videos in the @videoList param? To which I have to say that I have no idea. The documentation even suggests that this is perfectly possible but in fact it isn’t. In the meantime we’re just going to have to learn how to use the Player API, it’s probably good for us anyways. It certainly was good for me and highly entertaining!

Posted in  | Tags , , ,  | no comments

Pagination, Sets, Checkboxes

Posted by Chuck Vose Mon, 25 Aug 2008 20:29:00 GMT

When I was at RailsConf2008 I learned a lot about life and Rails but there were some nagging problems that lingered after me for a long time.

At one point in the talks there was a discussion of how to make listing pages have checkboxes and how to handle this data. The problem is complicated by pagination and filtering so it’s actually a really good problem to talk about.

The proposed solution was to send back a structure that looked something like this:

[{:id => 1, :activated => "+", :contact => "-"}, ...]

At this point I was probably playing pogs in the audience with Ryan Schenk so the details are lost to time but the concept remains.

It seems needlessly complex though. A very interesting idea, no doubt about it, and it certainly has its merit but it seems very complex to me when we’re talking about booleans. Also, I feel like it’s worth mentioning that this method is much faster than the below method but somehow that doesn’t turn me off that badly.

Instead of the above I took the already well accepted practice of using the check_box_tag in the view with the addition of a hidden field which would tell me which objects to care about:

<% @users.each do |user| %>
  <%= hidden_field_tag('seen[]', user.id) -%>
  <%= check_box_tag 'activated[]', user.id -%>
<% end %>

In the controller it’s as easy as loading up params[:seen] and loading each object:

def index
  if request.post?
    activated_ids = params[:activated].collect {|id| id.to_i} if params[:activated]
    seen_ids = params[:seen].collect {|id| id.to_i} if params[:seen]

    if activated_ids
      seen_ids.each do |id|
        r = User.find_by_id(id)
        r.activated = activated_ids.include?(id)
        r.save
      end
    end
  end
end

So what I don’t get is whether this is wrong on some moral stance or if I really didn’t understand what the presenter was getting at. Yes, he loads and saves fewer objects, which could be important. But he also introduced what seems to me to be a lot of complexity into a fairly simple process.

At any rate, this code is here in case someone needs to find it until I find what the presenter was proposing. I seem to recall being impressed at the time so there must be something I’ve forgotten.

Posted in , ,  | Tags , , , , , , , , , , ,  | no comments

Mnesia to ruby bridge evaluation

Posted by Chuck Vose Wed, 02 Jan 2008 23:25:00 GMT

Owing to the complexities of building a Rails adapter for Mnesia I’ve been looking into using a Ruby to Erlang bridge and have looked at the various projects that seem to be available and want to share my comments on how each is stacking up so far.

Rebar

Rebar was announced on 2007-04-20 and has since never surfaced to the public as far as his blog and Google are concerned. The code looked to be among the easiest to understand but is thankfully very similar to the Japanese RulangBridge below. The existence of the Japanese project could explain why Tom Werner never finished the project but that’s merely a guess.

Erlectricity

Erlectricity by Scott Fleckenstein was the first project I really tried out and I am impressed by some aspects and disappointed by others. On one hand it was easy to obtain being hosted on RubyForge and Google Code and works well at using Ruby from Erlang. On the other hand it comes with no documentation or rdocs outside of two uncommented examples.

I could never figure out a way to get it to bridge Erlang commands from Ruby but there may be a way that I’m missing; I have to admit I’ve not asked Scott about whether this code can go both ways or not.

So if you need something that Ruby does well but Erlang doesn’t then this may be your project. The second example uses the bridge to generate a gruff graph from Erlang which seems like it’ll come in handy for a lot of people.

Outside of documentation I’m concerned about whether the bridge can handle asynchronous requests from Erlang. From first glance it doesn’t seem like there’s any blocking and each Erlang thread calls the script directly instead of central blocking thread so I’m guessing that there’s no concurrency built in; something that we would have to work with if we were to use this bridge for a many-threaded project.

RulangBridge

RulangBridge by Toshi Hirooka (?) is a Japanese project allowing Ruby to use Erlang functions. Google’s translation allows us to read the usage instructions and browse the code which has allowed me to start using it in earnest to build the Ruby to Mnesia bridge.

My first impressions are that the bridge is that it is a little immature despite the >1.0 release number. The example code is excellent (if in Japanese) but it isn’t packaged in a gem or hosted on RubyForge and doesn’t have a way to auto-start an Erlang server. Furthermore, using only the built in classes we have to make the choice of asynchronicity or complicated/multi-module code.

Of course the raw class (called Erlang) can be made asynchronous by wrapping it in a Thread.new which is essentially what the Rulang wrapper class does, but it would be nice to have this built-in. Putting the asynchronous switch in the wrapper class is fine but the wrapper class suffers from a desire to make the Erlang calls feel like ruby and therefore makes calling complicated code impossible through the wrapper and thus makes the asynchronous aspect moot also.

The last concern I have is that each class must connect to a node explicitly rather than being able to automatically find and balance between nodes. In order to be happy with the adapter we’ll have to figure out how to load-balance or make Erlang do it for us.

Conclusion

For the purposes of the the Ruby to Mnesia Adapter I believe that RulangBridge will be sufficient but Erlectricity definitely has its purposes in the Erlang community. Despite some concerns and drawbacks both are usable code and could easily move forward with a little love from the community or follow-up versions by the owners.

Any project involving Erlang and Ruby will have to deal extensively with concurrency. Ideally it would be nice to see a broker model between the two that supports communication in both directions and deals with concurrency issues and load-balancing transparently. Until that happens we’ll have to work with Erlectricity and Rulang and love the creators for their hard work.

Posted in , , ,  | Tags , ,  | no comments | no trackbacks

Mnesia + Ruby Announcement

Posted by Chuck Vose Sat, 22 Dec 2007 07:17:00 GMT

My goal this year is to release a Ruby port of the Mnesia distributed database and hopefully start the process of moving to a true slice architecture. The port is an interesting project but I think the importance of the slice architecture is paramount.

For the last couple of years we’ve been working on the n-tier model with ruby. It’s well established and it has been working nicely for us. But the web server industry is starting to move more towards the idea of instances or clouds of ambiguous slices. Amazon is doing it, mongrel is a part of it certainly, Mnesia has always worked this way.

My hope is that my port will help us to create an EC2 instance that is both the master of it’s domain and a part of a cloud at the same time. I would like to see an EC2 instance that can autoconfigure itself and automatically find its neighbors, which contains a complete Mnesia instance, a couple mongrels, and a proxy/load balancer.

I’m not sure if we can do this quite yet but multi-processor theory suggests that it can be done. Whether it’s advantageous to remove all the bottle-necks and have to deal with the scheduling individually is where we’ll have to analyze but I’m confident that we are moving somewhere truly incredible.

In the future I hope to be able to drop in a new EC2 and just have it completely figure things out for me. No more MySQL master, no more apache proxying. Whether we use my new port or SimpleDB is of no concern to me at all.

Posted in , , ,  | Tags , , , , , , ,  | 2 comments

find_or_create by params (extension to dynamic attribute based finders)

Posted by Chuck Vose Thu, 13 Sep 2007 03:40:00 GMT

Rails has a dynamic method where you can do find_or_create_by_attr_name(attr) but it the method names get incredibly long very quickly. So SJS wrote this article in which he tries to remedy the situation. His solution was pretty elegant but it still only worked for fields that were already defined in the database; if you often redefine setter methods it doesn’t work at all.

Here’s my attempt to remedy the situation.

module ActiveRecordExtensions
  def self.included(base)
    base.extend(ClassMethods)
  end

  module ClassMethods
    def find_or_create(params)
      begin
        return self.find(params[:id])
      rescue ActiveRecord::RecordNotFound => e
        attrs = {}

        # search for valid attributes in params
        self.column_names.map(&:to_sym).each do |attrib|
          # skip unknown columns, and the id field
          next if params[attrib].nil? || attrib == :id

          attrs[attrib] = params[attrib]
        end

        # call the appropriate ActiveRecord finder method
        found = self.send("find_by_#{attrs.keys.join('_and_')}", *attrs.values) if !attrs.empty?

        if found && !found.nil?
          return found
        else
          return self.create(params)
        end
      end
    end
    alias create_or_find find_or_create
  end
end

ActiveRecord::Base.send(:include, ActiveRecordExtensions)

Newb instructions

Create a file called active_record_extensions.rb in your the lib directory. Then add `require ‘active_record_extensions’` to your environment.rb at the bottom (without the ``). Restart your server and see what happens!

Posted in ,  | Tags , , , , , , , ,

Long Bets

Posted by Chuck Vose Wed, 12 Sep 2007 04:42:00 GMT

Sam Ruby occasionally does an article about his long bets and I really respect his ability to predict the future. This time I feel that he was focusing too close and forgot the long part of the ‘long bet’. REST is already important, and there are further developments in edge rails that make it even more important (active resource). MySQL is already a pain for scaling and the plugins are starting to come out but haven’t really hit the fan yet.

Bet One – MySQL becomes the exception

First, I think Sam Ruby is dead on about databases. Right now there are only a few things that are a consistent pain for deployment and scaling and databases are the top of the heap. I know there are things like MaxDB for mysql and there are ways to set up ring replication but it’s hard; something that Ruby’s community is really good at solving.

My first bet is that MySQL dies out for the rails community and something new pops in. It wouldn’t surprise me if it was Mnesia from Erlang/OTP or something involving Hadoop and HBase. It would surprise me if CouchDB popped onto the scene in a big way but it’s the sort of thinking that could lead us somewhere interesting. The other option is that someone comes up with a really concrete stack of abstractions that makes it easy to balance mysql requests and writes.

Bet Two – Rails drops REST completely

Secondly, I think REST is the wrong way to move forward. REST maps very well onto the CRUD principles, but I feel like we very rarely actually use just CRUD. More often than not I want to run custom little things and create crazy associations. And I realize that this is all possible in the REST model, but it makes the controllers obscene sometimes.

What we really want is Query Language for the Internet and what better language to build that in than Ruby. I’ve seen DSL’s for direct database access and it seems like the routes would be just around the corner if this is where we decide to take it.

The question is whether DHH sees the writing on the wall or desperately wants to hang on to REST. Since he’s put so much effort into the REST idea it seems like he would be loath to drop it, but at the same time he’s an incredibly mature developer and would hopefully handle a change like this if it ever happened.

Conclusion

All respect to Sam Ruby, I really do respect his predictions over my own. But I think his predictions are too close to us right now. I would like to know what happens after REST and what happens in the database arena.

Posted in , ,  | Tags , , , , , , , , , , ,

Arrow keys in vi and iTerm

Posted by Chuck Vose Thu, 26 Apr 2007 22:25:00 GMT

Arrow keys doing funny things when you use vi in iTerm 0.9.5? It’s because we’re too used to messing with things. Set your Default bookmark’s keyboard type back to Global.

“When the terminal type of your sessions is set to one of those “xterm” types, iTerm handles the arrow keys, the home key, and the end key in a special way.” says the FAQ while not actually addressing the solution.

But who can complain? Global works well now! Hooray!

Posted in  | Tags , , , , ,

Older posts: 1 2