Posted by Chuck Vose
Fri, 09 Jul 2010 23:50:00 GMT
Cross-posted at the Metaltoad Blog
While I’m certain that I could never work up to the standards of Peter Cooper his “Interesting Ruby Tidbits That Don’t Need Separate Posts” series was a great help to me when I was a ruby developer. You can see the original here: IRTTDNSP #1. I’ve not found similar for the Drupal community but I believe a lot of the things I do during the week are interesting but not at all worthy of a complete post of their own. So this will be my attempt to categorize and commit those very thoughts to memory. I hope that even one of these things sparks something for you.
How to get Views’ exposed filters to display in a block
If you create a block display and have some exposed filters you may have trouble getting them to actually show up. The problem is that blocks don’t seem to like to display their exposed filters unless two prerequisites are met: there must be a full page display somewhere and it must have a path, the block must be set to use ajax. I would have never guessed this either but if you look at the requests that are going across the wire they’re going to that page display and the results are coming back over ajax.
How to reorder taxonomy terms in Views’ exposed filters
On that same page we realized that we needed to order the dropdown in a reasonable way in part just to keep people from using it on the awards day to stave off some of the traffic. Analysis of traffic suggests we actually got the ordering wrong but whatever, you just want to know how to reorder taxonomy terms in an exposed filter.
Despite the ordering that Views seems to be suggesting below, the actual display order of terms is determined by the taxonomy weights page at /admin/content/taxonomy/. So you can feel confident that even though views doesn’t have a means to order your taxonomy selections, they will be ordered for you. Now the real question is though, is it possible to reorder those terms on just one page and not on another? I don’t know, maybe I’ll get to post about that later.
How to get an F5 Load Balancer to stop putting the Vary headers on your responses
When you’re using a CDN like Akamai having a Varies header in your response absolutely kills your caching despite all the hard work drupal puts into getting things right. We were seeing about 50-75% cachability through Akamai because of it. Mostly this was because Akamai couldn’t cache the JS/CSS which was clearly incorrect. After we figured out that it was the Varies header (props to Grendzy as usual :P), it was a hop skip and a jump to figure out which settings could be causing the actual problem.
Turns out the Varies header setting isn’t the culprit though it does in fact fix the problem we were having it also introduces some others. The culprit is in fact having Browser Workarounds on which fixes IE6 and gzip and not having HTTP/1.0 Requests on. Of course the manual doesn’t really describe what’s going on so massive props out to spark on the F5 forums for answering this question before I had the chance to ask it.
Conclusion
I’ve run out of time but I’d be grateful for any feedback. If people don’t find this useful I’ll not post it on planet but if you do please keep reading and encouraging me with your great comments.
Posted in Drupal, Programming | Tags drupal, f5, header, ordering, snippet, taxonomy, varies, views | no comments
Posted by Chuck Vose
Fri, 04 Jun 2010 04:48:00 GMT
Cross posted at the Metal Toad Blog
I was trolling around the Internet today looking for benchmarks and I actually had a little trouble finding something current. Dries has one comparing D6 on PHP4 vs D6 on PHP5 but that was clearly ages ago. I also realize that this is going to be out of style in about 12 minutes, will probably be fraught with contention, and generally mocked by everyone; but in the interest of those few souls out there that really actually just want to know what is reasonable to expect from production hardware under some load I want to post these stats anyways.
Our setup
We’re using a gaggle of 1U SuperServers by Super Micro. The basic stats are here. We have them loaded with a pair of 4-core opterons and 32GB of RAM. Not entirely unaffordable nowadays for production kit.
These sit behind some firewalls and an F5 load balancer which helps make SSL a little quicker and makes sure that if things go down we can fail over to a different webhead. The truth is that all this redundancy stuff up front actually slows our pages down pretty heavily for small loads but when doing a lot of traffic on those happy spikey days it helps out a lot.
We’ve done a lot of the normal things to speed up the servers:
- Turn on APC, this is huge!
- Get rid of your .htaccess files and configure apache to do this at restart
- Use Boost if you can’t run Varnish, but run Varnish wherever you can
- Turn off all the devel modules
- Turn on as many of your drupal caching options as you can without breaking the site. Then figure out why the site broke and try to get those breaking caches online
- Turn on your MySQL Query Cache, it’s off by default and makes an amazing difference
- Put MySQL on its own disk, put logs and even the MySQL binlogs on a different disk if you can
- MAKE SURE YOU HAVE ENOUGH BANDWIDTH! Getting capped by your ISP will kill you during spikes since your servers will be resending lost data constantly
Commands Used
Since this article is about ab I’m not going to post our configs from SIEGE or JMeter.
Running from a box with an extremely fast network connection we were doing the following:
ab -n 10000 -c 100 -k -H 'Accept-Encoding: gzip,deflate' www.site-name.tld/
I like to run from a box within the network directly to the webhead as well to find out how much things are getting slowed down in the load balancers and firewalls. Since our webheads often do more than one site we have to use a Host header and the actual IP address as shown in the following snippet.
ab -n 10000 -c 100 -k -H 'Accept-Encoding: gzip,deflate' -H 'Host: www.site-name.tld' 192.168.0.27/
I also like to use relatively high concurrency and number of requests so that momentary spikes even out a little bit. There is a rule out on the net that you should take the average of three attempts for any benchmark and I think that’s totally necessary.
We’ve noticed that our servers start to really crack around 500 concurrent anonymous users. Peak performance seems to be around 100-200 so I stick with that so that the differences in config changes are the most obvious. If I go from 100 #/s to 250 #/s I know it’s a big change (or vice versa in a lot of cases).
Results
Without further ado, these are my results. I realize that there is certainly more that we should be doing to squeeze out performance and I also realize that my methods are pretty unscientific but I hope they give you an idea of what you might be looking for.
Internal – Test 1
Document Path: /
Document Length: 10716 bytes
Concurrency Level: 100
Time taken for tests: 40.694054 seconds
Complete requests: 10000
Failed requests: 9993
(Connect: 0, Length: 9993, Exceptions: 0)
Write errors: 0
Keep-Alive requests: 0
Total transferred: 112128752 bytes
HTML transferred: 106842680 bytes
Requests per second: 245.74 [#/sec] (mean)
Time per request: 406.941 [ms] (mean)
Time per request: 4.069 [ms] (mean, across all concurrent requests)
Transfer rate: 2690.81 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 2 84.8 0 2999
Processing: 61 379 809.9 147 12255
Waiting: 53 344 799.4 134 12255
Total: 61 381 814.4 148 12259
Percentage of the requests served within a certain time (ms)
50% 148
66% 170
75% 265
80% 345
90% 599
95% 1371
98% 3159
99% 4725
100% 12259 (longest request)
Internal – Test 2
Document Path: /
Document Length: 10683 bytes
Concurrency Level: 100
Time taken for tests: 29.175060 seconds
Complete requests: 10000
Failed requests: 5389
(Connect: 0, Length: 5389, Exceptions: 0)
Write errors: 0
Keep-Alive requests: 0
Total transferred: 111006011 bytes
HTML transferred: 105724371 bytes
Requests per second: 342.76 [#/sec] (mean)
Time per request: 291.751 [ms] (mean)
Time per request: 2.918 [ms] (mean, across all concurrent requests)
Transfer rate: 3715.64 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 5 123.8 0 3033
Processing: 41 273 445.8 135 11841
Waiting: 36 245 430.6 124 11840
Total: 41 278 463.5 136 11851
Percentage of the requests served within a certain time (ms)
50% 136
66% 150
75% 176
80% 287
90% 491
95% 1305
98% 1534
99% 3118
100% 11851 (longest request)
External – Test 1
Document Path: /
Document Length: 10472 bytes
Concurrency Level: 100
Time taken for tests: 31.161653 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Keep-Alive requests: 0
Total transferred: 110620000 bytes
HTML transferred: 104720000 bytes
Requests per second: 320.91 [#/sec] (mean)
Time per request: 311.617 [ms] (mean)
Time per request: 3.116 [ms] (mean, across all concurrent requests)
Transfer rate: 3466.66 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 13 199.5 0 3051
Processing: 46 273 414.5 138 4558
Waiting: 41 246 404.3 126 4545
Total: 46 287 457.2 139 4558
Percentage of the requests served within a certain time (ms)
50% 139
66% 156
75% 299
80% 336
90% 482
95% 1119
98% 1580
99% 3143
100% 4558 (longest request)
External – Test 2
Document Path: /
Document Length: 10472 bytes
Concurrency Level: 100
Time taken for tests: 29.336590 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Keep-Alive requests: 0
Total transferred: 110000000 bytes
HTML transferred: 104720000 bytes
Requests per second: 340.87 [#/sec] (mean)
Time per request: 293.366 [ms] (mean)
Time per request: 2.934 [ms] (mean, across all concurrent requests)
Transfer rate: 3661.67 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 3 95.1 0 3034
Processing: 50 264 447.7 129 4532
Waiting: 45 240 435.7 119 4528
Total: 50 268 459.6 129 6128
Percentage of the requests served within a certain time (ms)
50% 129
66% 143
75% 162
80% 295
90% 423
95% 1307
98% 1532
99% 3125
100% 6128 (longest request)
Posted in Drupal, Programming | Tags ab, benchmark, drupal | no comments
Posted by Chuck Vose
Thu, 06 May 2010 04:44:00 GMT
Cross posted at the Metal Toad Blog
When Webchick announced that Drupal was moving to Git at Drupalcon 2010, our office erupted in pleasure at the news. Lots of great Drupalists are already using Git and there’s even an unofficial Github branch of Drupal for your branching and stashing pleasure Github mirror. However, Metal Toad Media has been an SVN shop for a long time we still have a lot of processes that use SVN, so we elected unanimously to do a gradual rollout: new sites get a private repo on Github, old sites just use git-svn. Looking back I wondering why we ever delayed.
Standard Git-svn setup
If you’ve got a standard SVN setup (with trunk under trunk, branches under branches, and tags under a tags directory), then you’re lucky, because the standard Git-SVN setup is pretty much all you’ll ever need. In that case you merely need to use: `git svn clone -s https://repo.tld/repo`. After this step you’ll magically be able to switch branches super fast, commit to stage, etc.
If you have an old svn repo with just stackloads of changes use the -r flag to specify a range of revisions you’re actually interested in. We have one repo that’s well over 10k commits with tons of branches, this literally takes 20 hours to checkout with Git-svn.
Now that you’re here what do you do? There are only a couple commands you need:
git svn rebase | svn up
git svn dcommit | svn ci
git update-index --assume-unchanged path | ignore this file, great for settings.php
git checkout -- path | delete my current changes
git reset --hard | really, really delete my current changes
git merge branch_name | merge in the changes from the branch called branch_name
git stash | temporarily store my changes and check out master
git stash pop | restore my stashed changes and take those changes off of the stash list
When in doubt check out the git-svn cheatsheet and the git cheatsheet or install the cheat gem and get this direct to your command line:
sudo gem install cheat
cheat gitsvn
If you’re going to work with Github or a some other straight Git repo it’s well worth setting your Name and email address so that people know who you are and GitX can pull down your gravatar. You can do so with the following commands:
git config --global user.name 'John Doe'
git config --global user.email johndoe@example.com
I know that’s a lot of commands, it’s okay to bookmark this or the cheatsheets if you don’t want to forget. The beauty of Git is that you get to find the way that works best for you; these and a few others are all I ever need from git-svn.
Non-standard Git-svn setup
Okay, so the standard setup is all well and good but since SVN gives you the flexibility to drop branches and tags anywhere this can cause some problems for Git. Normally this is as simple and manually specifying the directories that it actually uses via the -T (for trunk), -t (for tags), or -b (for branches). However if you’re in the unlucky situation of having branches sitting alongside your trunk directory there is some additional love that you need to give.
First set up the repo as if it were a standard repo, get as much done right as you can by using -s or -T/-t/-b.
Once you have the repo checked out edit the .git/config file
In the svn-remote section take a peek at the braches/tags/fetch lines. If something doesn’t fall completely into one category add it as another fetch line. For instance, if you had a client branch that wasn’t in the branches folder you could do the following:
[svn-remote "svn"]
url = https:// svndomain.tld/svn/project
fetch = trunk:refs/remotes/trunk
fetch = client:refs/remotes/client
branches = branches/*:refs/remotes/*
tags = tags/*:refs/remotes/tags/*
After you’ve edited this you can do a `git svn fetch` to pull down all remote branches. Now when you’ve done that you should see other remote branches when you do `git branch -r` and you should be able to do awesome things like `git merge client && git commit -m “happiness is a fresh merge”`
Merging to client branch
After a while you’re going to get out of date with the client branch unless they’re doing a similar thing with git-svn. But merging into the client branch isn’t quite as easy as a normal git merge, or at least it’s not the same. Do merge from trunk -> client do the following:
git reset --hard remotes/client
git merge trunk
git svn dcommit
I’ve had varying results doing this. On some projects it seems to work great, on others it seems to switch me back to trunk instead of dcommit’ing. Worth a try at least.
Conclusion
Git has been amazing for us so far. Grendzy has been using it for about a year and I’m at about 9 months. Getting everyone else on board has been pretty easy and already I’ve seen more smiles around the office. Hope your transition is as easy and fruitful as ours is so far!
Posted in Drupal, Programming | Tags branches, drupal, git, merge, svn
Posted by Chuck Vose
Tue, 02 Mar 2010 22:50:00 GMT
Cross posted on the Metal Toad Blog
There’s a lot of good to be said about the Brightcove iPhone SDK, it’s easy, it seems to work really well, but there’s also a ton of gotchas and the docs are a little conflicting and/or out of date. The OnePlanet app is also out of date. This post is my attempt to rectify some of these things.
There are three things needed to get your app going:
You must be using your Read with URL Acccess token to fetch videos. Old accounts don’t have this so you’ll have to contact brightcove support to get them. Access this page from Home -> Account Settings -> API Management:
Next thing you need is to make sure that your UDS is enabled. You can find this page at Home -> Account Settings -> Publishing Settings:
Despite the documentation saying you need your videos to say Progressive Download it doesn’t actually matter. All you need to do is adjust your call to be like this from this excellent doc:
bc = [[BCMediaAPI alloc] initWithReadToken:@"your read with URL Access api"];
// This makes UDS happen. Without it there is no happiness in the world.
[bc setUdsSupportOn:YES];
videoList = [ [ NSMutableArray alloc ] init ];
NSError *err;
BCPlaylist *bcp = [ bc findPlaylistById:1234512345L error:&err ];
if (bcp != nil) {
for (int i = 0; i < [ bcp.videos count ]; i++) {
[ videoList addObject: [ bcp.videos objectAtIndex: i ] ];
}
}
Other notes:
- When using the OnePlanet app beware that the Headers/Libs have been set on the target as well as on the project so you need to amend their instructions to remove the Header Search Path and Library Search Path from the target.
- Also with the OnePlanet app the BCMoviePlayer.bundle needs to be reimported from your SDK or it’s not going to be happy.
- The README states that you can call the BCMediaAPI calls without the additional frameworks MediaPlayer, OpenGLES, and QuartzCore but you actually need all three or no project you build is ever going to compile.
- It’s already been stated in the docs but Geo restricted videos just don’t work yet. I’m not sure about the status of Localization yet.
- Use the error catching logic from the best practices page, it helps a lot and it’s actually pretty mandatory if you don’t want blank players and app crashes.
Posted in Objective-C, iPhone Development, Brightcove, Programming | Tags BCPlayer, brightcove, iPhone | no comments
Posted by Chuck Vose
Tue, 16 Feb 2010 22:53:00 GMT
Cross posted on the Metal Toad Blog
Imagination time: Imagine for a moment that you have an app, it has one button and one little window. When you click the button it changes the text within the window to something else. Now, when you get into flash development it seems like the easiest and clearest way to do this is to wipe out the contents in the window, but you would be wrong for thinking so. While it is the most obvious I intend to prove to you that to do so is both slower and prevents you from turning on the juice later with caching.
On to story time: We’re building an app for the OpenPeak tabletop device, it’s totally rad and you’ll love it I promise but that’s really not the point of the post. When we started the app we had what I considered a pretty awesome solution, we’d clear the state, build it onto a global, then write the global out to a window on the app. It actually worked great for a long time until we decided to start doing some caching where it became wildly apparent that I had actually written all three parts of that app dead wrong. It’s pretty exciting to do something wrong enough that you can write about it later. :)
Building on Globals (Horrible)
Okay, lets start with the most awful part of the lesson for me, that using globals is horrible and should never be done. Oh sure, they make certain listeners easier but it really does blow up fast when you start trying to cache things or thread. Here’s two scenarios that will inevitably bite you if you start using globals:
- When you start threading you’ll end up painting two scenes on to the global stage object at once.
- Even if you try to be synchronous you have to descend into an event listener hell since AS3 often decides to not dispatch Event.COMPLETE which means you’ll end up with a deadlocked app while a URLLoader tries in vane to do nothing.
- At some point you’ll accidentally use the same URLLoader twice or some other critical variable and the results will be completely random seeming as one gets overwritten on occasion.
So, how can you deal with this? As annoying as it is, define your variables at the beginning of your functions. I know it seems like a waste, but unlike that one global TextFormat object your URLLoaders really are totally different beasts and need to be treated as such. Same goes for almost all objects and display containers. I know it seems nice and elegant to use the same object all the time but what you may not be seeing is that all those globals need to have IPC methods defined on them or they’re completely useless in the inherently async world that is AS3.
Clearing the State (Wrong)
This one is simple really, I should have thought of it to begin with it but I wasn’t in OO space yet. Rather than thinking about pages or DisplayObjects as something physical that needs to be removed from the stage it’s much, much more efficient to just build a new Sprite() and replace the old stage that you were using. Best part is that when you just replace the display container you’re using you can easily stash this away in a cache object somewhere. Simple caching is only moments away but if you wipe the slate clean not only did you spend time killing all those children objects, you lost the ability to cache it away somewhere.
Metaphorical you of course, my apologies.
So what we did was to use an Object (we should have used a Dictionary but I didn’t know any better yet) with attributes that described the path we were writing to. We never wrote to a global stage object (we actually deleted those so we couldn’t). This way async calls could continue to write to the correct Sprite even in the background, the only place we needed to put a lock around was the drawing part as we drew out the content but rather than drawing a global DisplayObject we took the data from the cache and drew that which meant it was much, much faster, handled multiple threads just fine, and was extremely cacheable.
Conclusion
I hope this gives you some ideas; my hope was merely to get people thinking about two things: global objects require IPC, and that wiping the slate clean is wasteful. Let the garbage collector take care of clearing memory and make sure that you have only a few global objects.
Next time I’m going to talk about how I got a requirements engine running which would allow me to ensure that certain steps were always completed before anything else without creating messy spaghetti code. I think it’s pretty fantastic and I hope that you do too.
Posted in Actionscript, Programming | Tags 3, actionscript, cs4, evil, flash, globals, state | no comments
Posted by Chuck Vose
Wed, 02 Dec 2009 22:59:00 GMT
Cross posted on the Metal Toad Blog
If you’ve ever worked with a rich data model in drupal you know it can be a pain
to load up all the children and parents of a node within the templating engine.
One method that could save you a lot of time is to load the data recursively
in node_load and save your poor front-end guy some wrist pain (or yourself if you’re
that guy!)
Edit: Please look at the get_metadata() definition towards the bottom or none of this is going to make sense.
Here is the initial bit which loads on details about a node. Here of course you
could load all manner of things like read/write attributes, cck fields, etc.
/**
* Implementation of hook_load().
*/
function wrapper_load($node) {
$metadata = get_metadata($node);
// Load our attributes without children. We'll load children later.
$node = wrapper_load_without_friends($node);
One thing we found was that search would break when trying to index since it was
trying to load all the data from associated nodes as well. One way of dealing with
this is to use hook_nodeapi(‘update index’) to only load a subset of data instead
of the whole shebang. But I didn’t do it like that and I’m not going to put untested
code on the blog (well, aside from slightly edited code).
This is what I did instead. It’s interesting for its hackishness. There must be a
better way though.
$backtrace = debug_backtrace();
if ($backtrace[4]['function'] == 'node_update_index') {
return $node;
}
Here we get to the fun part though. This bit loads up the parents, those that this
node belongs to. Due to performance constraints when doing this you need to pick
a direction to load infinitely. If you choose to load parents recursively you have
easier code and it’s actually a lot faster as far as SQL is concerned. If you choose
to load children recursively, as I will do in a moment, the SQL is a little slower
but on-the-fly SQL is easier to write for a belongs_to relationship. When all the
data necessary is already in the db row you load to build yourself it’s easy to
include your parents too.
So you’ll note that here we load up a collection of parents being careful to make
sure that the recursive function knows who called it by the parent attribute on $obj.
// Load up any parents
if ($metadata['belongs_to']) {
foreach ($metadata['belongs_to'] as $drupal_attr => $legacy_attr) {
// When we load up a child it shouldn't load its parents
if ($node->parent != $drupal_attr) {
$node->{$drupal_attr."_collection"} = array();
// Get all the node attributes for our new object
$obj = db_fetch_object(db_query(
"SELECT * FROM %s as extra WHERE node.nid = %s",
$node->$drupal_attr
));
// This ensures that a child doesn't reverse and load its parent in the
// next call
$obj->parent = $node->type;
// Push our recursively loaded object into the empty collection. Here we
// choose to only load one level but you could use hook_load again to load
// deeper structures. The parent attribute should prevent us from getting
// into loops.
array_push(
$node->{$drupal_attr."_collection"},
wrapper_load_without_friends($obj)
);
}
}
}
Very similarly, here we load up the children. This time we load recursively with
no end condition. This is prone to cycles so you may have to go with a non-recursive
loader here if you have a cyclic loading cycle or some other way of terminating
the recursion.
// Load any children
if ($metadata['has_many'] && $node->nid) {
foreach ($metadata['has_many'] as $drupal_attr => $legacy_attr) {
// Bail out if this node has a parent at all. We just want to keep it
// simple for now.
if (!$node->parent) {
// Get the list of things this object owns
$res = db_query(
"SELECT * FROM $drupal_attr as extra WHERE extra.%s = %s",
$node->type,
$node->nid
);
// node_load all children and drop them in an array
$node->{$drupal_attr."_collection"} = array();
while ($obj = db_fetch_object($res)) {
// Once again, prevent our children from loading us and creating loops.
$obj->parent = $node->type;
// Push our recursively loaded children and their children onto the empty
// collection. This time we go all the way and create a much deeper data
// model. These two could be reversed, loading belongs_to indefinitely
// but I find this way easier.
array_push($node->{$drupal_attr."_collection"}, wrapper_load($obj));
}
}
}
}
return $node;
}
Here’s where we load extra attributes from the node addon table that you see associated
with every custom content-type. These attributes just get added onto the node directly
so there is some concern about columns named with php reserved words. Conflicting
column names like title just need to be carefully considered, they may not actually
be bad a bad idea.
// Hook load that doesn't recursively load children/parents, just the attributes
// of another table
function wrapper_load_without_friends($node) {
$metadata = get_metadata($node);
// Load up extra info from the node addon table
if ($node->nid) {
$extra_attributes = db_fetch_object(db_query(
"SELECT * FROM %s WHERE nid = %s",
$node->type, $node->nid)
);
// Foreach of the linking attributes which will allow us to find children
// load them onto the node directly.
foreach ($extra_attributes as $key => $value) {
$node->$key = $value;
}
}
return $node;
}
This is just a sample of our metadata loader. Naturally you could do this some other
way but it works pretty well for us. There is room for improvement though, using
a different format like YAML could buy some extra win for instance.
// Sample metadata describing has_many and belongs_to relationships as well as
// the read/write attributes each table has.
function get_metadata($node=NULL) {
$metadata = array(
'belongs_to' => array(
'staff' => array(
'office' => 'office_id',
)
),
'has_many' => array(
'office' => array(
'staff' => 'office_id',
)
),
);
$return['belongs_to'] = $metadata['belongs_to'][$node->type];
$return['has_many'] = $metadata['has_many'][$node->type];
return $return;
}
Posted in Drupal, Programming | Tags data, drupal, model, node_load, recursive, rich | no comments
Posted by Chuck Vose
Fri, 30 Oct 2009 20:22:00 GMT
Cross-posted at metaltoad.com: The Power of Yes
Of all the powerful words in any language assertion is in my opinion the absolute most powerful. With affermative language we are able to create trust, enact people’s will, and begin processes. Silly of course, to take one word to mean so much but I believe everyone can agree that when a client asks for something ‘yes’ should be the thing that jumps to your lips immediately, hopefully without qualification but at times you have to add on things like ‘and it will cost x’ or ’, now lets look at the plan and see how we can make this a reality’.
But this post isn’t about clients, it’s about our language and framework choices.
When I was new to programming I said no a lot. This was in part an aspect of my first language PHP, which was perhaps as immature as I was at the time. ‘No’ was probably my biggest buzzword, maybe followed by ‘can’t’ which I justified by saying that PHP couldn’t do it. Turns out I was wrong, PHP really can do almost anything, but at the time I couldn’t. But ‘No’ was certainly an artifact of the culture. The things I do in PHP are often much harder than they need to be, at least they feel unnecessarily difficult to me now, and in this environment it is easy to say that it’s to hard. I’ve noticed since starting to use PHP again I’ve noticed ‘can’t’ creeping into my language more and more.
Ruby was the first language that was strongly oriented towards ‘yes’ both in the language and the culture. It could have been that I learned later in life or in a place that had a very strong Ruby culture (Seattle), but I suspect that the language had more to do with it than anything else. Ruby positively wants to make things faster for you so that the time between your thought and your expression of said thought grows shorter and shorter as you learn more Ruby. As such, when your language is saying ‘yes’ so often you naturally begin to start thinking this way yourself.
Drupal was a bit of a revelation to me in this regard. Drupal, as everyone knows, is a PHP app, yet despite the underlying language constructs the ui says ‘yes’ even more often than Ruby does. As my coworker pointed out, the amount of a website that gets finished before you hit the code is absolutely staggering. Maybe 90% of any given Drupal website could be accomplished by Drupal using only CCK/Views (et al) with the rest of the customizations being fairly quick using the copious APIs available to us. This is assuming that you find the correct project, in all things of course selecting the best tool for the job yields the best results.
So what are your languages saying to you every day? As products of our surroundings it is absolutely critical that we evaluate what our languages and tools say to us no just what they say about us. PHP may have a stigma, and Drupal through association, but I believe that the way it works with us is a much stronger reason to invest in exploring Drupal.
Posted in Ruby, Programming | Tags can, client, drupal, languages, Ruby, t, yes | no comments
Posted by Chuck Vose
Fri, 02 Oct 2009 16:57:00 GMT
Cross-posted at company blog: http://metaltoad.com/blog/multiple-dynamic-tabs-brightcove-3
This week I had the wonderful opportunity to work on an interesting problem that as far as I can tell hasn’t been documented. The call came out that we needed to generate a couple dynamic tabs on the top of our player for smart playlists. Now, we already have one dynamic playlist so I thought it was going to be a fairly simple logical step up to three but I was really, really wrong.
If you read issue 3 of the pragprog magazine you’re probably thinking a lot about parallel and asynchronous processing. So it was extremely exciting to have come to a parallelization problem in my day to day activities. I’ll explain the problem:
In order to load a playlist into a brightcove player with the Player API you have to have already fetched the videos from the server using getMediaCollectionAsynch, getMediaInGroupAsynch, or getMediaAsynch. With one dynamic tab it’s pretty easy to use getMediaInGroupAsynch because you can assume that the only time the MEDIA_COLLECTION_LOAD event is going to fire is when your call has returned. But when you’re loading up two or more media items asynchronously you can no longer make that assumption.
If you’re just pulling down pre-defined playlists it’s super easy to drop something like the following in your onMediaCollectionLoad listener:
function onMediaCollectionLoad (e) {
if (e.mediaCollection === null) {}
else {
tabBar.insertTabAt(e.mediaCollection, 0);
}
}
However if you’re using getMediaInGroupAsynch or getMediaAsynch it can be difficult to figure out why the listener is getting called. For us we had three playlists of dynamic data but outside of comparing the resulting array to the original array (inefficient and error-prone) there is no way of knowing which asynch call asked for these videos.
But maybe we don’t care. All we care about is that the videos are loaded completely when we insert the tabs into the player right? So we stop relying on order and just make sure that all the videos are loaded into the player before adding some tabs. To do this I maintained a request counter (which could be called a counting semaphone were you so inclined. Before each request I increment the counter and when the request is filled I decrement the counter. When the counter hits 0 I know that all requests have been filled and I can safely add all the tabs at once.
Your code could probably look something like this:
var tab_count = 0;
var popular_tab;
function onTemplateReady (e) {
tabBar = exp.getElementById("playlistTabs");
if (popular.length > 0) {
tab_count = tab_count + 1;
popular_tab = true;
content.getMediaInGroupAsynch(popular);
}
}
function onMediaCollectionLoad (e) {
if (e.mediaCollection === null) {
// Do nothing
}
else {
// Make sure that these playlists coming back are coming from
// getMediaInGroupAsynch. Otherwise their id would be positive.
if (e.mediaCollection.id < 0) {
// Add all media to the player's memory.
var mediaDTOs = new Array();
jQuery.each(e.mediaCollection.mediaIds, function(i, val) {
mediaDTOs.push(content.getMedia(val));
});
// Decrement the counter. When this hits zero we'll have filled all reqs.
tab_count = tab_count - 1;
if (tab_count == 0) {
// Make sure this is one of the playlists we were going to add.
if (popular_tab) {
popular_playlist = {
displayName: "Most Viewed Videos",
mediaIds: popular
};
tabBar.insertTabAt(content.createRuntimeMediaCollection(popular_playlist, 'playlist'), 0);
}
}
}
}
}
Here’s hoping this helps you out in your quest for multi-tabbed players and managing asynchronous loading.
Posted in Brightcove, Programming | Tags asynchronous, brightcove, dynamic, jquery, parallel, processing, requests, tabs | no comments
Posted by Chuck Vose
Fri, 25 Sep 2009 20:49:00 GMT
Cross-posted from the Company Blog
In my experiences in the past working with external contractors is often a pain, especially the final merge where you try to incorporate all their code. I’ve always been a huge proponent of Subversion for SCM but today I saw an example where Git knocked the socks off of Subversion.
Time came and code needed to be merged into trunk. While discussing our options someone noticed that there was a .git file in the project directory and put the pieces together. While Subversion relies on a central repo Git has no such requirement and the contractor had included his git repo with the file which meant that we had his complete revision history available to us as well as an extremely easy method of applying diffs.
Normally merging was a pain but today we issued these commands, watched the output, then shared high-fives and a beer.
cd imported_files
git log
*write down the first couple chars of the SHA1 of the first commit
git diff --no-prefix {sha of first commit}..HEAD > patchfile
cd trunk
patch -p0 < ../imported_files/patchfile
Who knew it was going to be so easy!?
So the moral of the story: If you’re working with people that may not be in your office or you expect to get a tarball/zip of their work you could do no better than Git for version control. You get to see their complete history and work with it.
I’ll bet that someone out there smarter than myself actually knows how to replay the git commits so you can keep the revision history as well. Git is cool!
Note: Patch instructions based on these directions
Posted in Git, Programming | Tags contractors, git | no comments
Posted by Chuck Vose
Mon, 21 Sep 2009 21:55:00 GMT
Cross-posted to the company blog at: Fun with stakeout.rb
Having only left Rubyville a couple weeks ago there are still a lot of Ruby-based utilities that I still find incredibly handy. One such utility that I’ve recently fallen hopelessly in love with is stakeout.rb from: http://pragmaticautomation.com/cgi-bin/pragauto.cgi/Monitor/StakingOutFi…
Stakeout monitors your filestructure waiting for changes to happen and when they do it runs some command in the shell. Think about all the things you would like to happen when you change a file. Clear your cache? Refresh your browser via AppleScript? Run your PHPUnit tests? Program your Roomba vacuum to spin in a circle and sing for you? YES!
Take the code from the above webpage and drop it in a file called stakeout.rb. If you’re running Linux or OS X you probably already have Ruby. If you’re running windows I doubt the Ruby code would even work but do let me know if you do get it running and create a script for it.
After that simply run something like: ruby stakeout.rb ‘php drush.php clear cache’ sites/all/modules//
Now try editing and saving one of your module files. Success?
Rejoice!
Posted in Ruby, Drupal, Programming | Tags drupal, drush, Ruby, stakeout | 1 comment
