Eastern US Ruby Jobs and Consultancies

As I was perusing jobs on Rubynow I thought I was seeing a concentration of jobs in Florida, New England, and the Washington DC area. To test the idea a little, I decided to throw a sampling of the recent jobs onto a map (see Ruby on Jobs and Consultancies »). To that map I also added some of my favorite Ruby consultancies:

• ENTP (Portland, OR)
• Hashrocket (Jacksonville Beach, FL)
• Intridea (Washington, DC)
• Planet Argon (Portland, OR)
• thoughtbot (Boston, MA)
• Thought Propulsion (Portland, OR) « hey that’s me »
• Viget (Falls Church, VA)

If you look at the whole (world) map I think you will agree that there is a definite concentration of Ruby jobs and kickass consultancies in the Eastern US. There are many biases at work here of course, especially when it comes to the consultancies I’ve listed. Nevertheless, it does seem like something’s going on here.

I’m intrigued by a few points:

What’s going on down there in Florida? Seems like I see lots of Ruby jobs down there.

What’s up with that DC area. I’d say the presence of Intridea and Viget within what, 10 miles of each other qualifies DC as a hotbed.

Look at that New England cluster. I think there is general awareness a big presence in New York, but the number of jobs in New England surprised me.

What is it about the Eastern US that makes it such fertile ground for Ruby? Is there some bias in the job listings on Rubynow? Or is this simply a reflection of Ruby moving into the enterprise. Is there a particular kind of enterprise that’s predominant in the East and that’s adopting Ruby technology?

Now what if you don’t use cookie-based session storage at all? Well, just because you aren’t using cookie-based session storage doesn’t mean you aren’t using cookies. If your application has sessions at all, be they memcached ones or ActiveRecord ones, it is probably using cookies. It’s using cookies to store the session id so that when a request arrives, that id can be mapped to the corresponding session storage.

“So what” you say. “Well” I say… isn’t it cool that Rails generates that big random secret for you when you use cookie-based session storage? When we are not using cookie-based session storage, and that secret is not generated, don’t you wonder what secret is being used to secure your session id’s? You see, a session id must be hard to guess lest bad people gain access to your site. Usually when you want to make something hard to guess, you start with a secret and mix that with something that changes a lot and hash the whole shebang. So I went in search of this other secret.

What I found was that Rails calls CGI::Session#create_new_id to generate new session id’s. That routine uses no secrete per se. It hashes (MD5) a combination of:

1. the current date and time (expressed as a human-readable string)
2. the microseconds elapsed since the last second (expressed as a human-readable string)
3. a pseudo-random number greater than zero and less than one (from Kernel#rand)
4. the current process id number
5. the string ‘foobar’

Notice there is no secret keying material there. “But what about the Kernel#rand call Bill!” I hear you saying. If you go have a look at Kernel#rand and Kernel#srand you’ll see that if rand is called before srand is called with a number parameter then the random number will be generated from a combination of:

1. the current time
2. the process id number

So the security of these session ids hinges on the secrecy of current time (on the server running Ruby) and the process id. Given that the system time is returned in HTTP headers and process id’s are often in the hundreds or thousands, it’s only really the microseconds that are hard to guess here, from a statistical standpoint. Others have expressed similar concerns.

If you’re worried about this two suggestions come to mind:

• time out sessions on the server so that an attacker has to guess faster
• monkey-patch CGI::Session#create_new_id to hash its result with a great big old 128 character secret

Updated: October 15, 2008 expanded analysis of Kernel#rand and Kernel#srand and updated suggestions.

The Thought Propulsion™ corporate site is up and running. For the technically inclined, here are some interesting facts:

• hosted on Amazon EC2
• using the extremely slick EC2 On Rails Ubuntu/Ruby on Rails virtual appliance
• there’s a tasty microformatted hcard on the contact page (check it out with Operator)
• OpenID login (foreshadowing)

And of course a nice gray and orange theme just in time for Halloween.

This thing uses Comatose 0.8.1. Unfortunately, Comatose 0.8.1 isn’t compatible with Rails 2.1. Fine, I’ll just upgrade to Comatose 2.0 (uber-alpha) and that’ll work. Oops, Comatose 2.0 uber-alpha breaks Rails migrations. Fixing that breakage requires a patch to Rails itself.

Oh and did I mention that this project of mine also requires a patch to Comatose proper (adds before_filters to the Comatose configuration object).

So I need a patched version of Comatose and a patched version of Rails. “OK” you say, just use Piston to manage those vendor branches. Not so fast. Piston only works with Subversion and the Rails project is no longer hosted on a Subversion repository. The old repository was deprecated after Rails 2.0.2. Rails is now hosted on Github. Oh and so is Comatose.

Never fear, Github support for Piston is coming Real Soon Now™. Until then I’m stuck in limbo. I suppose I’ll do the manual vendor branch thing—essentially manage my own private Subversion repositories for Rails and Comatose.

The fact that François Beausoleil is implementing Piston support for all-Git projects leads me to believe that there is no convenient alternative (to Piston) for vendor branches in Git. Hum, that’s hard to believe. Anyhow, I can’t migrate this project to Git yet so all-Git alternatives are sort of moot.

In this pursuit I learned a few things. Firstly, plain old JQuery does not “do” UJS in the following sense: if you bind a behavior to an element (via a CSS selector), that binding will not be reapplied as the DOM is manipulated. For dynamic binding, or “live” binding that gets re-applied after every DOM update, you need Brandon Aaron’s livequery plugin for JQuery. Livequery is to JQuery as Dan Webb’s Low Pro is to Prototype — kind of. Difference being that livequery hooks each of JQuery’s DOM manipulation routines, whereas Low Pro hooks each of Prototype’s Ajax.Responders “onComplete” event — the event that happens when an XHR is complete.

What all this means is that you can use JQuery’s UJS (livequery) if you use JQuery for all your DOM manipulations, or you can use Prototypes UJS (Low Pro) if you use Prototype for all your Ajax calls. The unfortunate bit is that you cannot (yet) use livequery with Prototype Ajax calls, or Low Pro with JQuery DOM manipulations. Can’t we all just get along?

I wonder if a UJS package could remain stack agnostic. I think livequery is on to something by hooking DOM manipulation routines. That seems more robust (if potentially less performant) than hooking the Ajax returns. By hooking DOM manipulation it seems that you’ve plugged more holes and have a more general-purpose solution. Could the livequery approach (of watching for DOM manipulation) be done in a stack-agnostic fashion rather than by hooking JQuery routines? If we could do that then we’d have a UJS package we could use no matter which JavaScript base library we choose — or perhaps whichever library chooses us.

• it separates behavior (JavaScript) from markup (HTML) and this is good for the same reasons separating style (CSS) from markup is good
• it can reduce page weight

I had been using ujs4rails but that thing is being deprecated so it isn’t an option for me. If I want to do UJS I can either use Prototype or JQuery. I don’t know of any other good options.

I’m using JQuery a bit now. Pretty light use: a highlight effect here, a show/hide toggle there; some AJAX form submission. In thinking about how to proceed I’m kind of torn. I don’t want to use both JQuery and Prototype longer term. For starters, that’s too much mental baggage to carry around — one must gain fluency in one or ‘tother I think. Also it’s a lot of page weight to load both libraries.

Another dimension is RJS vs. POJS. RJS is Rails’ templating and API that lets you generate JavaScript from Ruby. POJS is “Plain Old JavaScript”. If we do RJS then we almost have to stay on the Prototype stack. But with POJS a guy could go with either stack and in fact I think the scale tips to the JQuery stack for POJS as I’ll show in a sec.

So here’s how I see the top two options:

RJS: Prototype + Scriptaculous + RJS + Dan Webb’s Lowpro:
+ Lowpro supports UJS
+ most of us have used prototype + scriptaculous and are somewhat comfortable with it
+ native Rails (RJS) support
- Lowpro is not as well documented (nor as widely used) as JQuery for UJS
- I see no centralized library of plugins (other than Scriptaculous itself) for this stack

POJS: JQuery + MinusMOR:
+ it supports UJS out of the box in a pretty clean way
+ people I respect seem to be moving to JQuery
+ there is a large library of plugins including autocompleters and flash upload progress indicators
- very little native Rails (RJS) support — you can do some things but it’s hard to know exactly what will work
- if you already know scriptaculous effects, you have to learn new effects

A big part of this decision hinges on the value of RJS. After using Rails RJS templates for a year and a half or so my conclusion is that they are more trouble than they are worth. Many believe RJS is easier to write/maintain that POJS — especially for a Ruby programmer. I believe RJS is actually a false economy. In real applications you really have to code in real JavaScript.

An RJS template generates JavaScript from Ruby code — but that Ruby API is insufficiently documented (in particular with regard to the scriptaculous effects) and insufficiently capable (e.g. there is no easy way to update DOM elements matching a CSS selector through that API). Essentially a guy can waste a lot of time trying to get RJS to work for anything but the simplest demos. So I lean toward plain-old JavaScript (POJS) instead.

I wonder what you think. Do you place higher value on RJS for real applications? Have you found an alternate stack, or perhaps a different combination of stack elements? Must I pick one stack or the other, or is it possible and profitable to use both at once?

ActiveRecord::Base.logger.debug "value of x is #{x}"

read on for the older solution…

I just spent five minutes figuring out how to log from a model class in Rails (or any class that isn’t a controller or view). Y’see, there is no “logger” method on an ActiveRecord for instance.  Once again Robby to the rescue with an oblique mention of the RAILS_DEFAULT_LOGGER global. I love that guy.

Related: I wonder what Jamis circa ’05 was talking about when he said:

Note: logging is a bad example here, because Rails already has very good support for logging. Furthermore, recent Rails releases support a service keyword, for declaring and using system-global services in the same manner described here.

Huh? Service keyword?

puts %w{A B}.collect{|e| e.downcase}.inspect

This is my cooler code:

puts %w{A B}.collect( &:downcase).inspect

This is how the Ruby Extension Project makes it work:

class Symbol
  def to_proc
    proc { |obj, *args| obj.send(self, *args) }
  end
end

This is the explanation.

This is the love (see day 38).

Don’t ever forget this.

For the Rails folks out there I thought you’d be interested to know that this time around I chose to use Locomotive for my Rails stack and it has mostly worked very well. Last time I configured a brand new development machine (late ’05) I went through the heck of installing a slew of ports for all the pieces (ruby, gem, gems, app server, web server, etc.) This time around I eschewed the 14 pages of instructions in favor of the single download and got myself some needed RMagick support in the bargain. I had successfully put off installing RMagick for about 18 months (thanks Captchator!) but recently started a project that just had to have RMagick. If you wonder why I’ve been avoiding RMagick then read this.

In moving my development environment from the old machine to the new, mysql didn’t automatically come over the firewire so I had to download and install it on the new machine. I needed to dump the database and move it to the new machine and load it. Turns out that moving that file over was about the most annoying part of this whole process.

So I created the dump on the old machine:

mysqldump -u root -pyour-password-here --single-transaction --all-databases > mysql_backup.sql

And then went into System Preferences/Sharing and turned on Personal File Sharing. On the new machine I saw the shared directory, but the newly created file wasn’t there. All the other files were there. I’ve seen this before and presume it must be some sort of caching or timing issue. Old files show up, new files don’t. And of course there is no way to “refresh” a finder view.

So I switched gears and tried “FTP Access” in Sharing. Neither Transmit nor Interarchy were able to connect via FTP to the old machine. Finally, a friend recommended SFTP (SSH SFTP). And um, after clicking on each option in Sharing I saw that “Remote Login” mentions SSH so I thought I’d run that and sure enough I was able to use SFTP via Transmit to get the file. Whew.

Once the database dump was moved over I simply loaded it on the new database server. Remember a newly created mysql database has no passwords.

mysql < mysql_backup.sql

You’ll notice that logins you brought over don’t work until you restart mysqld. Once you do that, you’re database has been replicated and your various logins and security restrictions will be in effect.

One other odd thing (besides the file sharing annoyance) was that while Locomotive purports to be a Universal Binary application it doesn’t actually work properly after the firewire transfer. It seemed like it was working — the application itself ran and it would actually start certain Rails apps. Problems had to do with RMagick not working (there’s a recurring theme here :) My solution was to reinstall Locomotive and the RMagick bundle afresh. After that everything worked great.

To summarize, here’s the main things I did to move my Rails development from PowerBook to MacBook Pro:

1. run the automatic firewire system transfer to get most of your environment moved over
2. install Apple Software Updates and reboot (you should do this early on with any new machine)
3. install mySQL 5.0
4. dump mySQL database on old machine, transfer file using “remote login” and SFTP, load database on new machine and reboot mySQL
5. install X Code 2.4.1
6. re-install Locomotive (even though it transferred via Firewire):
1. move Applications/Locomotive2 to trash
2. download/install Locomotive
7. install Locomotive RMagick bundle if you need it
8. finally, under the Locomotive Terminal I had to install a few gems for my projects: ruby-debug, chronic

That’s pretty much it. All my Rails projects are up and running on the new machine. A little faster and with 60px less vertical space to work with. Not nearly as bad as it might have otherwise been I suppose.

Bear in mind these are not complete instructions for getting from a clean MacBook Pro to a Rails development environment. This is just about migrating from the PowerPC to Intel with a minimum of pain. It’s been pretty painless so far. Time will tell how well the Locomotive choice holds up. I’ll keep you updated.

In Joyomi I create an Omi when I lend you a book. I can optionally share that Omi with you. If I do, then the Omi shows up in the “Omis Shared With Me” tab in your dashboard. Sharing. Mkay. Social networking 101.

Now the query for a Joyomi user to find his own Omi’s (one’s he’s created) is trivial. The user model simply has_many omis. No brainer. On the other hand, going in the opposite direction and finding all the Omi’s that my friends have shared with me is decidedly non-trivial. In fact it’s downright tricky. It involves a big long series of joins. This is further complicated by the fact that in order to address potential SPAMming we don’t show you Omi’s that were created by folks who are not in your contact list. Got that? Good.

So I set out to implement the “Omis Shared With Me” tab hoping to reuse a lot of stuff from the “Outstanding Omis” and “Archived Omis” tabs. In particular I’ve got a nice clean little pagination routine that takes an association as a parameter and then proceeds to paginate the Omis in that association. Here’s the routine:

    1   def paginate_omis( association, order)
    2     @omis_pages = Paginator.new self, association.count, 10, params[:page]
    3     @omis = association.find(
    4       :all,
    5       :order => order,
    6       :limit => @omis_pages.items_per_page,
    7       :offset => @omis_pages.current.offset)
    8   end

(the order parameter lets you send in nice little SQL ordering clauses like ‘-expected_at DESC’ ) You can see this routine relies on the association parameter responding to count and find. Since “outstanding” and “archived” Omis are associations:

    1   has_many :outstanding_omis, :class_name => 'Omi', :conditions => "`archived_at` is null"
    2   has_many :archived_omis, :class_name => 'Omi', :conditions => "`archived_at` is not null"

This works fine. So when it came time to implement “Omis Shared With Me” I thought great, I’ll just add another association for that. Granted this is a more complex association, but from reading about the :joins option to ActiveRecord::Base#find it seemed doable. I’d just use that :joins option on my new association and everything would Just Workâ„¢. Boy was I wrong.

You may wonder why I would think that just because :joins was documented under find, that it would also work for has_many. Well it’s because those two methods mostly implement the same options.

Options Implemented by Both find and has_many

• :conditions
• :order
• :group
• :limit
• :offset
• :include
• :select

But as it turns out there are a handful of discrepancies. Most of the discrepancies make sense. For instance, it wouldn’t make sense for find to support a :foreign_key option. But there are a few discrepancies that do not really make sense.

Options Implemented by find but not by has_many

• :joins
• :readonly
• :lock

Hum, it seems to me that all three of these options would be useful on has_many. I groveled around the Rails source and I don’t see any particular reason why these aren’t supported by has_many. In fact it turns out that the find and has_many sources have many similarities. They just happen to have very little reuse going on.

It occured to me while reading the Rails source and seeing all the duplication between find and associations that there was a missing class, perhaps it should be called a “relation”. I’d like to instantiate a “relation” as a first class object. If such an object existed then both find and has_many could certainly use one. But that is the subject of a different post. I digress.

So finding myself in need of an association with :joins I did what any self-respecting Ruby programmer would do — I made one myself. Now before you freak out, I didn’t implement all 23-odd methods of has_many associations. I implemented only the two I needed in my pagination method (find and count). Remember, in Ruby, if it quacks like a duck that’s good enough. And I knew exactly which duck sounds this “association” would be called upon to emit.

So here is what the has_many would look like if has_many actually supported :joins:

    1 has_many :omis_shared_with_me, :class_name => 'Omi', :joins => 'inner join identities as i2 on i2.id = omis.identity_id inner join email_addresses as e1 on e1.address = i2.name inner join contacts as c1 on c1.id = e1.contact_id inner join contacts as c2 on omis.counterparty_id = c2.id inner join email_addresses as e2 on c2.id = e2.contact_id', :conditions => 'identities.id = c1.owner_id and e2.address = identities.name'

My workaround was to define a method on the user class (model). That method would return an instance of my new “association” class. Here is the method:

    1   def omis_shared_with_me( *args)
    2     proxy = Object.new
    3     instance_eval <<END
    4     class <<proxy
    5       def find( *args)
    6         with_scope do
    7           Omi.find( :all, *args)
    8         end
    9       end
   10       def count( *args)
   11         with_scope do
   12           Omi.count( *args)
   13         end
   14       end
   15       private
   16       def with_scope
   17         # without :select omis.* you get superfluous columns an the Omi objects aren't right (description is often NULL)
   18         Omi.with_scope(:find => { :select => 'omis.*', :joins => 'inner join identities as i2 on i2.id = omis.identity_id inner join email_addresses as e1 on e1.address = i2.name inner join contacts as c1 on c1.id = e1.contact_id inner join identities as i1 on i1.id = c1.owner_id inner join contacts as c2 on omis.counterparty_id = c2.id inner join email_addresses as e2 on c2.id = e2.contact_id', :conditions => ['e2.address = i1.name and :identity_id = i1.id', {:identity_id => #{id}}]}) do
   19           yield
   20         end
   21       end # def
   22     end # <<proxy
   23 END
   24     proxy
   25   end # def

This code bears some explanation. (darn tootin!) The most obvious way to define the association class would be to simply define a new Ruby class at the top level or maybe within the scope of the user class. If we did that however, we wouldn’t have access, in instances of that class, to the id attribute of the user. And we need that id when the association is called upon to do its thing.

Now we could add an instance variable to our association class to hold the id and we could initialize that thing on construction but that wouldn’t be very educational now would it? Nope, no instance variables for us — we’re going to use a little meta-programming instead.

So instead of defining the new class at the top level or in the context of the user class, we define an object-specific class (specific to the proxy object) and we evaluate that definition in the context of the user object. In this way, the reference to id in the private with_scope method will reference the id attribute of the appropriate user at runtime.

So that’s it. We are leveraging find to create an association class that supports the :joins option in 25 lines of code. This approach could easily be extended to add some of the other methods of has_many depending on your project needs. Also, I’m sure some Ruby Rock Star could DRY it up more.

On the other hand, doing a whole lot more to turn this hematoma into a second head is not a very wise use of resources. It’d be interesting to go in and steal (factor out) the best bits of find and has_many and build that general purpose ‘relation’ class I spoke of earlier, DRY-ing up find and associations, normalizing the options between them and giving us first-class ‘relations’ to boot. Anyone of you ActiveRecord warriors out there interested in some deep hacking?