Posted by Chuck Vose
Tue, 11 Sep 2007 19:42:00 GMT
Sam Ruby occasionally does an article about his long bets and I really respect his ability to predict the future. This time I feel that he was focusing too close and forgot the long part of the ‘long bet’. REST is already important, and there are further developments in edge rails that make it even more important (active resource). MySQL is already a pain for scaling and the plugins are starting to come out but haven’t really hit the fan yet.
Bet One – MySQL becomes the exception
First, I think Sam Ruby is dead on about databases. Right now there are only a few things that are a consistent pain for deployment and scaling and databases are the top of the heap. I know there are things like MaxDB for mysql and there are ways to set up ring replication but it’s hard; something that Ruby’s community is really good at solving.
My first bet is that MySQL dies out for the rails community and something new pops in. It wouldn’t surprise me if it was Mnesia from Erlang/OTP or something involving Hadoop and HBase. It would surprise me if CouchDB popped onto the scene in a big way but it’s the sort of thinking that could lead us somewhere interesting. The other option is that someone comes up with a really concrete stack of abstractions that makes it easy to balance mysql requests and writes.
Bet Two – Rails drops REST completely
Secondly, I think REST is the wrong way to move forward. REST maps very well onto the CRUD principles, but I feel like we very rarely actually use just CRUD. More often than not I want to run custom little things and create crazy associations. And I realize that this is all possible in the REST model, but it makes the controllers obscene sometimes.
What we really want is Query Language for the Internet and what better language to build that in than Ruby. I’ve seen DSL’s for direct database access and it seems like the routes would be just around the corner if this is where we decide to take it.
The question is whether DHH sees the writing on the wall or desperately wants to hang on to REST. Since he’s put so much effort into the REST idea it seems like he would be loath to drop it, but at the same time he’s an incredibly mature developer and would hopefully handle a change like this if it ever happened.
Conclusion
All respect to Sam Ruby, I really do respect his predictions over my own. But I think his predictions are too close to us right now. I would like to know what happens after REST and what happens in the database arena.
Posted in Rails, MySQL, Programming | Tags Bets, CouchDB, Erlang, Future, Hadoop, HBase, Mnesia, MySQL, Rails, REST, Ruby, Scaling
Posted by Chuck Vose
Thu, 26 Apr 2007 13:25:00 GMT
Arrow keys doing funny things when you use vi in iTerm 0.9.5? It’s because we’re too used to messing with things. Set your Default bookmark’s keyboard type back to Global.
“When the terminal type of your sessions is set to one of those “xterm” types, iTerm handles the arrow keys, the home key, and the end key in a special way.” says the FAQ while not actually addressing the solution.
But who can complain? Global works well now! Hooray!
Posted in Programming | Tags arrow, iTerm, keyboards, keys, vi, vim
Posted by Chuck Vose
Wed, 25 Apr 2007 12:21:00 GMT
At On & On Creative we have a lot of sites on a common rails backend. It’s incredibly flexible so we can offer a lot of support to customers but because there are so many customers with different needs testing has become a huge pain. Not only does it let us run tests custom to each client, it provides a way to import the data to the test db really easily and it’s way faster than loading the fixtures each time you start a test. Yay!
After googling around I happened upon this caboo.se article about how to run without fixtures at all. It’s a very indepth article with lots of good things in it; far be it for me to try to compete.
Instead, here’s the low down on how we prepare each of our sites for testing.
Add the following to the end of your rails_root Rakefile:
class Rake::Task
def detract(prerequisite)
@prerequisites.delete(prerequisite)
end
end
%w(units functionals integration recent uncommitted).each do |task|
Rake::Task["test:#{task}"].detract('db:test:prepare')
Rake::Task["test:#{task}"].enhance(['environment'])
end
Add the following to lib/tasks/fixtures.rake. Make sure to look at the two commented lines:
# This code courtesy of the "Rails Recipe's book":http://www.pragmaticprogrammer.com/titles/fr_rr/ I believe.
namespace :db do
namespace :fixtures do
desc 'Create YAML test fixtures from data in an existing database. Defaults to development database. Set RAILS_ENV to override.'
task :dump => :environment do
# Remove the limit if you want to. It's a timesaver mostly
sql = "SELECT * FROM %s LIMIT 0,1000"
# Add tables you don't want dumped.
skip_tables = ["schema_info", "plugin_schema_info", "engine_schema_info", "sessions"]
ActiveRecord::Base.establish_connection(:development)
(ActiveRecord::Base.connection.tables - skip_tables).each do |table_name|
i = "000"
File.open("#{RAILS_ROOT}/test/fixtures/#{table_name}.yml", 'w') do |file|
data = ActiveRecord::Base.connection.select_all(sql % table_name)
file.write data.inject({}) { |hash, record|
hash["#{table_name}_#{i.succ!}"] = record
hash
}.to_yaml
end
end
end
end
end
Then the fun easy stuff happens:
rake db:fixtures:dump RAILS_ENV=production
rake db:fixtures:load RAILS_ENV=test
rake test:plugins PLUGIN=your_plugin
If you want to commit the fixtures to the repo you can but it seems a waste to me since you might be regenerating them often.
Also, check the two comments in the fixtures.rake, it could be doing things you don’t expect.
Posted in Rails, Programming | Tags fixtures, plugins, Rails, rake, testing
Posted by Chuck Vose
Sun, 07 May 2006 23:19:00 GMT
One of the biggest fears of all system admins and developers is publishing work to a live server and having it break everything. Naturally we all go to great lengths to make sure this doesn’t happen but even the most well-tested code in the simplest environment still breaks sometimes.
In addition to these fatalities is the less intrusive but equally damning maintenance that sometimes just has to happen in the middle of the day whether there’s users in the system or not. I’m sure you can think of a few scenarios and horror stories.
At any rate I’ve come to realize that it’s extremely important to be able to tell the people in the system that things are going to break for a bit. This is fortunately extremely easy in Rails; a quick three or four lines in the application.rb will do it. In PHP it’s easy if there is even a little planning that went into the project.
Despite the ease I don’t think this is very common in the world of small-to-medium businesses and this seems like a shame.
Letting your users know what’s about to happen or what happened a moment ago will increase their sense of security in many cases and in the very least will keep them from calling you in a panic.
Blissful silence.
Posted in Programming | no trackbacks
Posted by Chuck Vose
Fri, 21 Apr 2006 06:32:28 GMT
I’m finally learning, oh yes.
The reason IDE’s make it so difficult to copy folders around is because it’s an incredibly difficult process. In order to copy in Subversion you have to copy the folders/files, delete any .svn directories in each level of a folder, then those folders/files have to be added to the repo.
Very complicated process which has been killing me lately. I should have known better; how not DRY of me to copy a folder…
Posted in Programming | no trackbacks
Posted by Chuck Vose
Thu, 20 Apr 2006 10:39:00 GMT
Holy crap I finally figured out why unit tests exist! To give me something to think about in the shower. Yep, simple as that.
No, wait, there are two really good uses off the top of my head: packaging and testing filesystem crap.
The first use is if you’re exporting a plugin for later use (or for general distribution). In this scenario there’s a fairly good chance that the user importing your plugin has messed up their install or is using a database that isn’t the same version or even the same type as the one you’re using.
In either case if you have unit tests and none of them fail then you know that the database probably isn’t at fault.
The second use is for filesystem crap. I can still count the number of times I’ve messed up the permissions on a folder only to find it after a half hour of debugging. This should never happen and there are wonderful tools to prevent it. `ri File` and `ri FileUtils` should get you started if you’re curious.
Specifically I’ve been working on a photo upload admin because I don’t understand file_column. In this I occasionally have to make a directory (okay, every time I upload something that isn’t a graphic) and it would be really nice to know that when I post it onto our production server that it’ll work when I release it into the wild.
Under this column is a wonderful addition to test_helper to check that things are correctly set in other places. Since rake is seperate from the dispatchers it could check their permissions, check the permissions of the public and log folders, and a myriad of other wonderful tests that could help newbies get on their way.
Rake is already pretty good at this but it serves as an excellent example of one use for unit testing.
Posted in Rails, Programming | no trackbacks
Posted by Chuck Vose
Thu, 20 Apr 2006 10:33:00 GMT
Background
After a lot of thought and an enlightening article by Joel Spolsky , I’ve come to the following conclusion: Hiring new developers is almost never the answer to the problem of having too many programming projects.
We have that problem right now. Demand is very high and Tynan no longer has much time to code due to business requirements. So we have at least four pending projects, two of them very large, and three on-going projects but only 1.5 developers.
No, we really have much closer to 3/4 of a developer. If that.
I spend roughly 1/3 to 1/2 of each day doing server related projects. Unbreaking the repo is a pretty consistent project (probably because Tynan and I are bad citizens but we don’t know any better), as well as maintaining the email, doing backups, messing with mysql, doing security checks and making sure apache is still running.
I usually consider it a good day if I get in two hours of programming before I’‘m too tired to continue. It happens fairly often that after all the server management I’m just too tired to get in the zone so I pour out crap that I have to re-program later on.
Tynan is in an even more painful place. I’ve only seen him work once but he spent the whole day on the phone with customers. He’s figured out what to do with his problem to some degree but I estimate him at somewhere around 1/4 to 1/3 of a developer.
Solutions
What the hell do you do? Adding more developers is the natural idea; it seems like it would work brilliantly. But anyone that’s worked with another developer knows that this is dead wrong. As Brooks points out in The Mythical Man-Month adding a developer increases the complexity of communication exponentially.
Gamble with time
So that’s out, the only thing left is to reclaim lost developer time and gamble a little bit. By hiring a secretary, a server admin and a tester will the two developers produce enough to offset the three extra salaries?
It’s a business choice, the sort of thing I really suck at. If the developer is twice as productive will it be worth twice the money or will something happen where the developer isn’t twice as productive? How long will it take to train and hire a server admin? How much of this time will be from the developer? Can we afford to lose the main developer for the time it takes to train the server admin?
What about a secretary? How long will it take to train the secretary to find all the information in the office? How long will it take to organize the information so they can find it period?
Time management
Maybe that’s not the only solution. The conflict is that the developer doesn’‘t have time to do all the projects. So you have two approaches, make the developer more effective or give them more time.
The other option is one that requires months of foresite. We have a project starting in May, two weeks from now. It’s not going to happen. I knew it wasn’t going to happen when it was proposed a month and a half ago. I had hoped but I think that in the back of my mind I knew it wasn’t going to happen.
But telling a client you’ll get to their project seven months from now is going to be a hard sell.
Conclusion
Point is, hiring developers is pretty much out. There are two options in our case which are to make the developer better or make the projects farther apart.
There are things blocking both, all we need to do is figure out what the blocks are, which are biggest, and how to deal with them.
Posted in Programming | no trackbacks
