RadRails 0.2 Released, and RadRails' ancestry
I'd noticed the launch and subsequent talk of the new Ruby on Rails IDE, RadRails. The idea intrigued me, especially since I'm one of the lead developers on RDT and we'd heard a number of folks who had wanted to spin off their own Rails IDE separately or on top of RDT. The interest has been piqued yet again by today's anouncement of RadRails 0.2.
RadRails is fairly impressive. It's a nice fully self-contained copy of Eclipse with the relevant plugins installed so all you do is unzip and double-click the executable. It has custom loading graphics, Rails project generation and all sorts of neat little helpers. So where did it come from and how'd they get a full Rails IDE up so fast?
So, I took a peek inside the zip and I was slightly irked by one thing. RadRails is built on top of RDT as well as the Subclipse plugin. You wouldn't know that by their webpage, but they did mention us in the about dialog under the help menu. Not for nothing guys, but if you're building on top of RDT and Subclipse (not to mention Eclipse), it'd be nice to share the attribution and love. RDT could always use the support to drum up usage or more developers which in turn would help your own efforts...
OK, personal rant aside, I do see a lot of improvements that they should consider. First, you have these wizards for generating new controllers or models. But they're hidden. You have to right click in the Rails Navigator, select Other... and then open the Rails folder to find these wizards. They should have nice big clickable icons up top on a menubar. Next, If I enter the name AccountsController in the controller wizard, you should be able to recognize that I typed Controller as a suffix and strip it out of the name before running the generator so I don't get accounts_controller_controller.rb generated. Third, try using the latest RDT 0.6.0 - it looks like you're running an old nightly build under the hood there because Rakefile and the Rails scripts are all not recognized as Ruby files and syntax highlighted, which they ought to be with 0.6.0. Fourth, the RI view we have in RDT? It doesn't look or work too well where you guys have moved it to be default. It should be fairly big like the console view in the bottom left, otherwise you can't read anything. Fifth, there should be shortcuts to show the other views RDT has such as the Test::Unit view or the Regexp view. They're still useful when developing a Rails project.
Rails Tip: What goes where (directory structure) - A cheat sheet
I work with Rails on a pretty much daily basis in my day job. While I own Agile Development with Rails, am subscribed to the Rails mailing list, and read a number of blogs where the authors touch on Rails I've been slow to pick up some of the subtleties of using the framework.
A few coworkers also have all those resources available, but can't seem to wrap their heads around how the project is stuctured.
What Goes Where (or the Rails framework structure)
Rails defines a whole hierarchy of folders when you first generate a project. While some are very noticable in their intent, others can be confusing. Here's a simple breakdown of what goes where:
While a lot of this should be simple and straightforward to pick up, I've found that I generally tend to get a little sloppy about my division of code between app/models, lib and vendor. Remember, if you didn't write it, it goes in vendor. If you wrote it and it doesn't derive from one of the big base classes of Rails (ActiveRecord, ActionMailer or Actioncontroller) it probably goes in lib - even if it is considered a "model" class. At least I like to try and follow that rule. Otherwise in complex apps you'll either end up with two undesirable results - you'll place tons of code in the controllers which could be encapsulated in lib models; or you'll end up putting all these custom models which have nothing to do with the Database in app/models and 'pollute' the folder - making it difficult to discern which models are DB abstractions and which are non-persistent.
Rails, Page Caching, Lighttpd and Stupidity
I thought I would pass on the trouble (and solution) I ran into this past week trying to understand why Rails page caching was just not working for me. I run this blog and pervwatch.org on TextDrive. The recommended and typical setup for Rails is to run with lighttpd, proxied by Apache. I managed to get the site operating fine with this setup but for whatever reason page caching simply didn't work.
I forgot about it, until some of the pages became slow to load due to large amounts of DB queries (particularly my rails generated sitemap for pervwatch.org). It became a big nuisance for the sites to laod so slowly when they were both easy and amenable to full page caching. I played around with my lighttpd config and posted pleas for help to the textdrive forum, support, and rails mailing list to no avail. The problem? Two swapped characters.
The wrong way
url.rewrite = (
"^/$" => "index.html",
"^([^.]+)$" => "1$.html"
) The right way
url.rewrite = (
"^/$" => "index.html",
"^([^.]+)$" => "$1.html"
) Work continues on PervWatch.org
I'm continuing to expand the sex offender registries that I screenscrape for PervWatch.org, adding Texas this weekend. So I will have the sex offender data for New York, California and Texas mapped. I've also added RSS and Atom feeds to the individual address search results, and I've been tweaking the site to behave better with this additional data - for instance, the zipcode list was getting way too long, so now I'm forcing the user to enter some filter text before I show any.
I thought I'd also share some cautionary tips for people out there who may be working on similar projects.
Internet Explorer still sucks and everyone uses it
I fail to remember this every time I deal with web pages. I use Firefox all the time, so I don't run into these problems until the odd day where I'm at a friend's trying to show them the site. Pervwatch was broken under IE. The odd thing was that the pages loaded fine, but the map pages would load well and then as soon as they were done would display a modal dialog saying that it couldn't open the page. After much googling, I realized that IE can't handle Google Maps unless you place the map script code at the end of the HTML body. Always remember, if you're altering the DOM (like Google Maps does), you must place the javascript code at the end of the page! IE just can't handle it otherwise.
Be careful with live searches
I was very excited to use AJAX to do live searches or filtering of the list of zipcodes - since the list was often large and I don't imagine people would want to scroll to the zipcode every time. So, I had set up a text field which would be polled every quarter-second and the text of it would filter the list of zipcodes. This worked well on my development box, but on the real server it didn't turn out so well. I took my cue from the engine that runs this blog - Typo. The problem was that the query and rendering could take some time, and the nature of the filtering meant that the first query would take much longer than later ones. Often the results would be out of sync with what the user had typed. For instance a user would type the first digit, say a '1', and Rails would start getting all zipcodes beginning with a 1. In the meantime the user enters a couple more digits, so Rails does more queries. The later queries would often finish first and display results. Then a split second later the first, slow query would return and the results containing all zipcodes beginning with the digit 1 would overwrite the more refined smaller result list. Rather than deal with temporarily disabling the text field until results appeared, I just decided to force the user to hit Return/Enter or click a submit button to force the query themselves. I'm still not sure if this is the best way...
Screenscraping is a delicate art
Since the sex offender registries don't offer up their data in programmatic ways, I have to manually screenscrape the contents. I originally was trying to do this by using open-uri and lots of GET requests, but this limited me to only a few registries which didn't create session IDs and cookies to track their users. If you need to do things like this, please check out Michael Neumann's WWW:Mechanize.
It's available as a gem so you should be able to grab it by doing:
# gem install mechanizeIt's a port of Perl's mechanize library which allows you to pretend to be a browser. One drawback is that it isn't very tolerant of bad HTML code, so in some cases you can't use the built-in method to traverse the page contents if the HTML you grabbed is bad. You will likely need to fall back on regex's to do a lot of the dirty work.
A word of caution about Mechanize as well. In my scripts I was using open-uri and mechanize, but open-uri was being require'd first. Mechanize uses custom Ruby 1.9 versions of net/http and net/https, but if those libraries have already been loaded via require, the custom versions will not get loaded and you'll get complaints about the missing method get_fields. I edited mechanize locally to use the [] and []= methods whenever it called for get_fields and add_header, respectively. I'm not sure if those changes would break anything if we had multiple cookies or values for a given header key, but it worked for what I needed.