The Burlap Sack

find_or_create by params (extension to dynamic attribute based finders)

Posted by Chuck Vose Thu, 13 Sep 2007 03:40:00 GMT

Rails has a dynamic method where you can do find_or_create_by_attr_name(attr) but it the method names get incredibly long very quickly. So SJS wrote this article in which he tries to remedy the situation. His solution was pretty elegant but it still only worked for fields that were already defined in the database; if you often redefine setter methods it doesn’t work at all.

Here’s my attempt to remedy the situation.

module ActiveRecordExtensions
  def self.included(base)
    base.extend(ClassMethods)
  end

  module ClassMethods
    def find_or_create(params)
      begin
        return self.find(params[:id])
      rescue ActiveRecord::RecordNotFound => e
        attrs = {}

        # search for valid attributes in params
        self.column_names.map(&:to_sym).each do |attrib|
          # skip unknown columns, and the id field
          next if params[attrib].nil? || attrib == :id

          attrs[attrib] = params[attrib]
        end

        # call the appropriate ActiveRecord finder method
        found = self.send("find_by_#{attrs.keys.join('_and_')}", *attrs.values) if !attrs.empty?

        if found && !found.nil?
          return found
        else
          return self.create(params)
        end
      end
    end
    alias create_or_find find_or_create
  end
end

ActiveRecord::Base.send(:include, ActiveRecordExtensions)

Newb instructions

Create a file called active_record_extensions.rb in your the lib directory. Then add `require ‘active_record_extensions’` to your environment.rb at the bottom (without the ``). Restart your server and see what happens!

Posted in Rails, Programming | Tags ActiveRecord, attribute, based, create, dynamic, find, find_or_create, finders, Rails

Long Bets

Posted by Chuck Vose Wed, 12 Sep 2007 04:42:00 GMT

Sam Ruby occasionally does an article about his long bets and I really respect his ability to predict the future. This time I feel that he was focusing too close and forgot the long part of the ‘long bet’. REST is already important, and there are further developments in edge rails that make it even more important (active resource). MySQL is already a pain for scaling and the plugins are starting to come out but haven’t really hit the fan yet.

Bet One – MySQL becomes the exception

First, I think Sam Ruby is dead on about databases. Right now there are only a few things that are a consistent pain for deployment and scaling and databases are the top of the heap. I know there are things like MaxDB for mysql and there are ways to set up ring replication but it’s hard; something that Ruby’s community is really good at solving.

My first bet is that MySQL dies out for the rails community and something new pops in. It wouldn’t surprise me if it was Mnesia from Erlang/OTP or something involving Hadoop and HBase. It would surprise me if CouchDB popped onto the scene in a big way but it’s the sort of thinking that could lead us somewhere interesting. The other option is that someone comes up with a really concrete stack of abstractions that makes it easy to balance mysql requests and writes.

Bet Two – Rails drops REST completely

Secondly, I think REST is the wrong way to move forward. REST maps very well onto the CRUD principles, but I feel like we very rarely actually use just CRUD. More often than not I want to run custom little things and create crazy associations. And I realize that this is all possible in the REST model, but it makes the controllers obscene sometimes.

What we really want is Query Language for the Internet and what better language to build that in than Ruby. I’ve seen DSL’s for direct database access and it seems like the routes would be just around the corner if this is where we decide to take it.

The question is whether DHH sees the writing on the wall or desperately wants to hang on to REST. Since he’s put so much effort into the REST idea it seems like he would be loath to drop it, but at the same time he’s an incredibly mature developer and would hopefully handle a change like this if it ever happened.

Conclusion

All respect to Sam Ruby, I really do respect his predictions over my own. But I think his predictions are too close to us right now. I would like to know what happens after REST and what happens in the database arena.

Posted in Rails, MySQL, Programming | Tags Bets, CouchDB, Erlang, Future, Hadoop, HBase, Mnesia, MySQL, Rails, REST, Ruby, Scaling

Inheritance without STI: method_missing routing

Posted by Chuck Vose Mon, 13 Aug 2007 22:05:00 GMT

Every once in a while I find myself in the situation where I need to override a model or create an inheritance without using single table inheritance (STI).

The method_missing call can help us to create an intelligent router that will route calls to the parent class or the child while maintaining an extremely small code footprint.

Here’s an example of what I’m talking about. We needed a landing page that would be dynamic for each user that wanted one. But we needed a template landing page which just needed the values substituted in at the right spot. Here’s a quick Yaml of what the table might look like.

# Example Page
landing_page_template:
  title: Welcome to [COMPANY-NAME]
  meta_keywords: [COMPANY-NAME], [COMPANY-META-KEYWORDS]
  ...

# Example Landing Page
landing_page:
  company_name: CV, Inc. 
  company_logo: www.chuckvose.com/fake_logo.gif
  company_meta_keywords: happy, rails, method_missing, inheritance
  ...

So here’s the hope, when we load up the LandingPage object and call LandingPage#title it should display “Welcome to CV, Inc.”

Pseudo-code and implementation

def method_missing(method_name, args*)
  if method_name is in the extended database
    call method_name
  if method_name isn't in the extended database
    try calling parent.method_name
      return results directly or mangle them somehow

We chose to mangle the results but if you just wanted to add a couple columns you could. You could run into performance issues doing this instead of STI. However this is an absolutely ideal place to mangle the code too if you want. Probably as good as a controller after_filter.

Two steps needed to happen for this, we need to load up the template on initialization, then use method_missing to switch between calls to the LandingPage class (such as :company_name) and the Page class (such as title).

Step 1: Load the template

The first part is a little weird because of rails performance. If you haven’t used after_find/after_initialize rails-core had to put in a little kludge for performance reasons. The solution is just to define a blank function named after_find or after_initialize.

class LandingPage < ActiveRecord::Base
  after_initialize :find_template

  # Required to activate the after_initialize filter
  def after_initialize; end 

  private

  def find_template
    @template = Page.find_by_url("landing_page_template")
  end
end

Step 2: Create the method_missing switcher

The method_missing switcher had to incorporate two things and one trick. First, it needed to look in the LandingPage table for its own attributes. Then it needed to look in the Page table for anything else before passing it through a regex engine and returning. The last little trick was that the regex engine needed to only be run on Strings, trying to regex a boolean doesn’t work very well. This allows the plethora of query functions such as Page.valid?.

class LandingPage < ActiveRecord::Base

  private

  def method_missing(meth, *args)
    # See if this attribute is in this object already. 
    # If so just call out to super and let 
    # ActiveRecord::Base deal with trying to find the 
    # attribute in the database. 
    if @attributes.include?(meth.to_s) || @attributes.include?(meth.to_s.gsub(/(=|_before_type_cast)/, ""))
      super
    else
      # Call Page#meth and store the response. Usually something like @page.body or @page.title
      resp = @template.send(meth)

      # If the attribute returned from above is a String, toss it into the regex grinder
      # and return the results. 
      if resp.is_a?(String)
        return resp.html_sub(self)

      # If the response is a boolean, symbol, or something else, just return it since we
      # can't (and probably don't need to) regex it.
      else
        return resp
      end
    end
  end
end

Step 3: The regex grinder

Security Warning: Before I begin this I want people to know that it should never be used by people outside your direct trust. It calls ruby code directly off of text found in the database. It can be rewritten to be more secure but this wasn’t necessary for us. If you want slightly more security drop some constraints on the LandingPage.send() method. For even more security do one regex per tag to replace (such as html = html.gsub(/\[MERCHANT-NAME\]/, template.merchant_name || ””). Security Warning

The last step is totally up to you. We needed it to find things like [MERCHANT-NAME] and replace them with LandingPage#merchant_name. This can be done fairly simply monkey-patching String with a blocked gsub.

class String
  # Look for snippets like [MERCHANT-NAME] and replace 
  # them with LandingPage#merchant_name
  def html_sub(landing_page)
    html = self # self cannot and should not be modified directly.

    # Look for [WHATEVER] tags
    html = html.gsub(/\[[^\]]+?\]/) do |match|
      # For each match, call the LandingPage instance method 
      # with the same name and return those results or the empty
      # string.
      landing_page.send(match.gsub(/[\[\]]/, "").gsub(/-/, "_").downcase) || "" 
    end
  end
end

Conclusion

The method_missing call seems to be a rough spot for a lot of discussions. Why_ wrote an article (The Best of Method Missing) that shows this pretty well: he starts by explaining how he has hated every piece of code he’s written with it then proceeds to point out some method_missing code that he loves. Lots of people feel this way about it and about many aspects of Rails in general.

But sometimes it’s a wonderful thing. This example should have taken several hundred lines of code and lots of kludges and bugs, but method missing trimmed it to ~20 LOC and we haven’t been able to find a bug yet.

Hope this helps you with some project, I’ve enjoyed playing with it and writing it up.

Posted in Rails | Tags blocks, inheritance, method_missing, monkey, patching, Rails, regex, sti

Nginx http_perl_module

Posted by Chuck Vose Tue, 07 Aug 2007 17:24:00 GMT

Can't locate nginx.pm in @INC
Can't locate loadable object for module nginx in @INC

cd ~/src/nginx-0.6.5
./configure --with-http_perl_module --with-http_ssl_module --prefix=/opt/local/nginx/ --with-perl_modules_path=/opt/local/nginx/modules/perl
make
make install
sudo /opt/local/nginx/sbin/nginx

Tags: nginx, perl, rails

Posted in Rails | Tags errors, nginx, perl, Rails

Rendering partials with a 302 breaks ie6

Posted by Chuck Vose Wed, 02 May 2007 21:50:00 GMT

If you can spot what’s wrong with this code you get a cookie. Apparently rendering with a 302 breaks ie6 entirely. Just deleting the 302 makes it okay.

Whether it makes sense or not is not the question. I’m pretty confident that it doesn’t make sense after thinking about it for a while. But why would it make ie6 connections die while firefox et al are fine with it?

Posted in Rails | Tags 302, cookies, partials, Rails

Testing rails without fixtures

Posted by Chuck Vose Wed, 25 Apr 2007 21:21:00 GMT

At On & On Creative we have a lot of sites on a common rails backend. It’s incredibly flexible so we can offer a lot of support to customers but because there are so many customers with different needs testing has become a huge pain. Not only does it let us run tests custom to each client, it provides a way to import the data to the test db really easily and it’s way faster than loading the fixtures each time you start a test. Yay!

After googling around I happened upon this caboo.se article about how to run without fixtures at all. It’s a very indepth article with lots of good things in it; far be it for me to try to compete.

Instead, here’s the low down on how we prepare each of our sites for testing.

Add the following to the end of your rails_root Rakefile:

Add the following to lib/tasks/fixtures.rake. Make sure to look at the two commented lines:

Then the fun easy stuff happens:

If you want to commit the fixtures to the repo you can but it seems a waste to me since you might be regenerating them often.

Also, check the two comments in the fixtures.rake, it could be doing things you don’t expect.

Posted in Rails, Programming | Tags fixtures, plugins, Rails, rake, testing

RailsMachine / Slingshot image loader slowdown

Posted by Chuck Vose Mon, 23 Apr 2007 00:56:00 GMT

Background / Analysis

Recently a fellow asked me to help him diagnose a slowdown in his rails stack at the Ruby Brigade meeting in Seattle.

After going through the normal procedures of checking mysql indexes, talking about caching, and checking the code for bizarreness I still couldn’t find anything really out of the ordinary.

Finally we turned to Firebug and found that the images were taking an eternity to download. This had happened to On & On Creative earlier on this year so the solution was quick at hand.

Solution

RailsMachine and the mongrel docs themselves recommend the following code for apache:

But there’s a hidden gotcha that I still haven’t fully figured out. Sometimes, usually when images aren’t in the top level of /images, apache will not match the RewriteCond and pass the image to mongrel instead of loading it itself.

¹ http://mongrel.rubyforge.org/docs/apache.html

So if you see a slowdown on images, you run mongrel through apache’s mod_proxy_balancer, and you used the config from the mongrel website or from RailsMachine/Slingshot’s capistrano script, try this code and see if it helps:

Did It Work?

Checking the rewrite logs

My computer is exceedingly slow so I can’t rely on my browser to tell me how fast an image is really downloading. If you’re like me it might be worth watching the rewrite logs to see if the images are being passed through rather than being rendered.

Drop this into your virtual host configuration right after the RewriteEngine On:

Then check the apache logs folder (often /var/log/httpd or /usr/local/apache2/logs) for the myapp_rewrite_log. After loading a page it should look something like this:

Rather than this contrived example:

Checking the rails logs

The rewrite logs are cool and handy so I put them first but you should also be seeing entries in your development.log or production.log. Try the following snippet, you may be surprised to see the results:

Posted in Rails | Tags Apache, firebug, image, mod_proxy_balancer, mod_rewrite, mongrel, proxy, Rails, Rails, slow

Rmagick OS X Verdana

Posted by Chuck Vose Sat, 17 Mar 2007 22:18:00 GMT

If you’re getting errors about Verdana not being found, specifically:

“unable to read font `/Library/Fonts/Verdana’ (Magick::ImageMagickError)”

try:

sudo apt-get install applesystemfonts
cp /sw/lib/X11/fonts/applettf/Verdana.ttf /Library/Fonts/
sudo gem install rmagick

Posted in Rails

Recursive children

Posted by Chuck Vose Wed, 24 May 2006 21:13:00 GMT

One of the things sorely missing from Rails (and with good reason) is a function to find all the children in a tree. Below is my attempt at it, you can drop it in any model that acts_as_tree and call all_children to get a flattened list of all children, grandchildren, etc.

  def all_children
    all_children = []
    all_children << list_children(self)
    return all_children.flatten
  end

  private

  def list_children(branch)
    all_children = []
    for child in branch.children
      all_children << list_children(child)
    end

    if branch.children.empty?
      return branch
    else
      all_children << branch
      return all_children
    end
  end

There’s probably a lot of reasons for this not existing but sometimes you need the limited functionality that this provides and can work within the constraints of making sure that no children have two parents.

The other approach is to use acts_as_nested_set but you sometimes lose valuable data such as the self_and_siblings function from acts_as_tree.

Posted in Rails

Cascading page attributes

Posted by Chuck Vose Mon, 08 May 2006 06:15:00 GMT

Intro and Concept

One of the first things I tried to do in Rails was to figure out how to make a nice, generic website that anyone could update. After about sixteen iterations I’ve finally stumbled on one that I think will finally work.

As of now it’s only partially complete. I’m really only using this method to generate the titles and meta information but each time I have to make a change to the website I find more and more information going into what I call the welcome message.

• Making an action for each page is almost always rediculous and hard to update. So that’s pretty much out.
• Making a loader page that loads a Page object and displays the content works for the most part if you don’t have special content (and as long as you remember to change the :parameter if you ever try to paginate a series of pages.)
• Making the content an optional part of an initialize function in the application.rb will ensure that you can always access page related information on any page (ever dynamic pages).

In order to do this I have a function called `initialize` in my application.rb that parses the @params and looks through the database for a entry in the welcomes table. As I said, I’m only using parts of this but it would be easy to extend; initially this was just to display a welcome message on any page they wanted.

If there’s an entry in the welcomes table with controller params[:controller], action params[:action], and id == params[:id], then it returns a raft of information such as the page title, a background image, meta values, etc. These override the defaults in the page (and there must be defaults) and make the page dynamic.

The beauty of this approach is that it’s very easy to take an action that actually has an action (such as /admin/delete_user) and pop a new title on it, or meta values, or custom css, or whatever you can think of. If you have this table contain content you could have a blank template by default that gets overridden by the content in the row.

Steps

• Create the table and figure out what you want to be able to apply to any action. Titles and meta values are a great start. Make sure to make a unique index on the controller, action, and id rows (This is very important).
• Create an initializer function that searches this new table for the controller, action, and id in the params.
• Create before_filters in each controller you want this to be available in. This may be possible in the application.rb but I haven’t tried it (and it can be extremely picky).
• Create forms that make editing these items easy. Definitely note that it’s entirely possible to have /page/page_loader override /page/page_loader/3. This may be intended and a good idea but it’s pretty important to note that it could completely wreck things until you figure out why things aren’t working.
• Lastly, create sane defaults in the initializer function for pages that don’t have entries. This is incredibly important.

Conclusion

I realize that I breezed over all the details, this was intentional. Because I’ve not actually done this entirely in a site (only as far as titles and whatnot) I don’t want to give fake code samples. I’ll get there eventually but it may be a while until I start a new project.

At any rate, the idea is sound and is working in part very well in my sites so far. There are a lot of gotchas, mostly in the defaults, but there’s also a lot of room to innovate. The idea of the defaults cascading is a wonderful extension especially when so many sites are in tree/nested set formations.

Keeping the attributes away from the actual code is a great way to make it easy for customers to interact with the website. Customers often want to change little things and it’s important that they be able to change as much as possible even when the code is mostly hidden in a template or you’ll end up making little changes for the rest of your existence.

Posted in Rails, MySQL | no trackbacks