Sunday provided a wonderful conclusion to RailsConf 2008. Here are some brief notes for the sessions I attended to whet your appetite with links to their RailsConf pages (where you can grab slides).

Waxing Ballroom Floors on the Titanic (and Other Less Seaworthy Vessels) - Rick Bradley, Yossef Mendelssohn, Kevin Barnes (OG Consulting) (RailsConf page)
Plugins:
object_daddy
timely
shmemeter
autochronic
freshtrack
nihilist_bot
flame

Genomes on Rails - Matt Wood (Wellcome Trust Sanger Institute) (RailsConf page)
Frequently and rapidly changing data

• Don’t have fields – store data as Descriptor/DescriptorValue objects (key/value pairs)
• With careful indexing it’s not that slow
• If one field is consistent across versions, that can be added to the db
• Can map Descriptors between different versions

They specify task the same way and allow users to create pipelines for various projects
Throughput: Data → 320tb/450 cpu cluster → Archive (2 Petabytes of data)

• Multiple instances of multiple applications (loosely coupled)
• Deployment is difficult because you need to push to all your instances
• reverse proxying with fairnginx in front of mongrels, deploying with cap

The Profitable Programmer: Creating Successful Side Projects - Geoffrey Grosenbach (Topfunky Corporation), Tom Preston-Werner (Powerset, Inc.), Chris Wanstrath (Err Free), PJ Hyett (Err Free), Ben Curtis (Catch the Best) (RailsConf page)
Project Ideas:

• Build something simple
• Build something you want to use
• Take something that’s already popular in one area and move it to one where it doesn’t exist

Implementation:

• Build it yourself
• Sub-contract

Promotion:

• Allow free accounts with limited, but useful functionality
• Piggyback on already established people and sites
• Form symbiotic relationships with existing sites
• Drive traffic to their sites
• Promote their product through yours

Success:

• Don’t listen to critics as you go through issues while expanding
• Try to build apps that are a source of constant income and require little effort to maintain
• Avoid VC → basically just getting a boss

Notes:

• Donationware? Ads?
• Ads are tricky because you have to become a marketer
• Market Research
• Track how people interact with the app and not how they say they interact or are going to interact with it
• Try to do something where its easy to get to people you want to sell to
• Think about how often people will use it (including you)
• Keep in mind who’s actually going to be paying for it

That's it. What an amazing conference. Can't wait until next year!

More nuggets:

Advanced RESTful Rails by Ben Scofield (Viget Labs)
Pick out the nouns from your app to determine the resources
Decide what methods to expose to those resources
Ajax actions generally use middle man which is a join model

Fast, Sexy, and Svelte: Our Kind of Rails Testing Dan Manges (ThoughtWorks), Zak Tamsen (ThoughtWorks)

Unit Test

• one method or object being tested at a time
• Avoiding Unit Test Britleness:
• gem “unit_record”
• ActiveRecord::Base.disconnect! keeps tests from hitting db
• To avoid db you could use fixtures (Difficult because you need to go back and forth to the fixture file)
• Or you could construct the object

Functional Testing Speed

• Gonna be slower – involve more objects and hit the db
• DeepTest speeds up functional tests
• Read DeepTest gem ReadMe to set up distributed tests – SUPER FAST!

Integration Tests

• Sometimes useful for special purpose tests
  • performance tests
  • smoke tests
  • tests simulating multiple users

Acceptance tests

• Better for low levels of granularity
• Selenium RC – way to control the browser
• Writing Acceptance Tests:
  • Encapsulate Pages
  • Data Access:
    • Can use Factory again to create entries
  • Parallelization – multiple Selenium RCs

Integration Testing with RSpec's Story Runner by David Chelimsky (Articulated Man, Inc)

• Process
  1. Feature Injection
  2. Focus on outputs
  3. Acceptance Criteria – if tests pass, customer accepts it
• Popping the “Why?” stack – ask “Why?” until value comes out

Integration Testing:

• Make sure the component parts play nice together
• Automated User Stories/Scenarios
  • Document the expected behavior of the system
  • Verify that behavior by executing a thin vertical slice of the system
  • Narrative Format: identify goal, user, action
  • Should test edge cases!
    • ex. goal of 200, 1 should return 1% and 199 should return 99%
  • Store DOM in memory
  • manipulates DOM
  • builds POST from DOM

Webrat Ruby gem (http://github.com/brynary/webrat)

Selenium RC (lets you test JavaScript/AJAX & you can watch it!)

• Because it’s slow you should only use for AJAX and use Webrat otherwise
• Multiple Sessions

The Great Test Framework Dance-off by Josh Susser (Pivotal Labs)

• RSpec for the structural level
• Shoula for test cases
• Test/unit for assertions
• Refactor with shoulda

Let's talk about the nuggets. Quick note though - I heard the complaint that the talks weren't technical enough. I don't particularly agree with that criticism. There's only so much you can cover in 45 minutes or whatever it is so I think the real purpose of most of these talks was to show us what's out there and it's up to us to dig further. My two cents. Anyways, the nuggets:

"Multi-core Hysteria": FUD about CRUD? by Andrea O.K. Wright (Chariot Solutions)-
Learned Mongrel mechanism for handling concurrent threads
def initialize(dir, meme_map = {}{
@guard = Mutex .new
}
Mutex Doc
Got this little forking snippet:
walking = fork do
100.times do
puts “walking”
sleep 1
end
end

chewing = fork do
100.times do
puts”chewing”
sleep 1
end
end

Surviving the Big Rewrite: MOVING YELLOWPAGES.COM to Rails by John Straw (YELLOWPAGES.COM) -
(This talk was awesome for me personally as it dealt with moving a large, established Web site to Rails)
Why the move? Absolute control of URLs (of utmost importance for their site), stateless HTTP, being agile & writing less code
His team and their development process which was pretty cool: ~20 people (5 core developers), sat together, 3 week (approx.) cycles - 1 week wireframe development, 1 week UI design, 1 week+ development
Web Service performance issues they tackled:

• Slow page performance caused more by asset download times than speed of web framework
• Worked through the Yahoo! performance guidelines
• Minified and combined CSS and JavaScript with asset_packager
• Moved image serving to Akamai edge cache
• Apache slow serving analytics tags – moved to nginx for web tier

Also, they debated about other technologies (Java/EJB3, Django) but eventually chose Rails because, well, it's the bomb.

CRUD Doesn't Have an 'S' in It: Managing Complex Searching in Rails by Stephen Midgley (Hutz.com)
4 architectural steps (in order):

1. POST/GET Params representing Search Criteria
   • e.g. ?s=search_parameters
2. Merging Search Criteria with persistent criteria
   • Session/backend, hidden input tags or on the URL line
3. Converting Search criteria to Search Rules to SQL
4. Paginating the Search & Render

I loved this: For clean URLs use POSTs. Make sure your URLs are distinct for core search options. If your URLs aren’t clean, you’re doing it wrong!!!

1. Clean: http://mysite.com/find-vacations/p/1/United-States/Califronia
2. Not Clean: http://mysite.com/controller/action?wtf=is&all=this

You want to store your SQL in an object so that you can pass it around

1. This makes writing your Search Rules more modular
2. Doesn’t have to be session (maybe you want searches sshared across users?)
3. Caboose::EZ

SQL Tools

1. EZ-Where
2. Squirrel
3. Sequel

Alright so that's what I got for Friday. Saturday and Sunday coming soon...

**Please see my follow-ups on individual days: Friday, Saturday, Sunday**

RailsConf 2008, two words: passionate and fascinating.

I met so many people over the past 3 days who are absolutely in love with the work they're doing. I met pod/screencasters Gregg Pollack (Rails Envy Podcast), Ryan Bates (Railscasts), Chistopher Haupt and Michael Slater (Learning Rails), and Geoffrey Grosenbach (Peepcode & Ruby on Rails Podcast). Everyone's familiar with their work I'm sure, but needless to say their depth of knowledge and eagerness to help out and get others involved was amazing. I've got a whole list of tips & tools for podcasting and screencasting now and several friendly contacts to help me along the way. I met Evan Phoenix, Brian Ford, and Wilson Bilkovich from Engine Yard who are the lead developers of Rubinius. Their presentation was great and I feel like I have a much better understanding of why Rubinius is important and useful. And I'm actually itching now to get commit rights (it doesn't take much!) I also met David Chelimsky, fellow sports fanatic and RSpec guru. We had a really interesting 5 minute chat about the new story runner feature in 1.1 and I'm really excited to explore ways to integrate it into my development process. And that's just the tip of the iceberg! There are about 20 other people I had conversations with, learning about their projects, what tools they use, what they like (love) about developing with Rails, what they don't, etc etc.

And all of the talks were so interesting! There was a nugget or two (or six) of programming goodness in literally every talk I attended. I'll expand on the talks I attended in more depth coming up, but needless to say there wasn't a single striking disappointment among the bunch. They were all fascinating, which of course is a direct by-product of having so many people doing something they're so invested in and that's so much fun to do.

In Conclusion,

Can't wait for RailsConf 2009!

What a fantastic day at the Web 2.0 Conference (and Expo) in San Francisco. It was really fun being surrounded by so many people, most of whom are doing such interesting things using (mostly) such cutting edge techniques and technologies. Today was a very fun and engaging day that covered a wide swath of the Web 2.0 landscape.

My first session was with Jia Shen of RockYou. He talked about how his company approaches Facebook application development. A lot of his points were quite useful. He highlighted the need for rapid development so your product and brand don't get stale. He likes telling the story about how SuperWall was developed in a weekend - started Saturday morning, launched Sunday at 3 am! He highlighted the need for brainstorming and coming up with fun, creative apps that will engage their existing user base (which is surprisingly 60% women) while attracting new Facebook users to sign up. One of his major points was that one's approach to app development should be from an advertising and marketing viewpoint. He promoted A/B testing as a good way to figure out the optimal flow for the install process since that's when the majority of potential users get lost. Overall it was a really fun talk about a very interesting topic.

Next I attended a session that was more philosophy than tutorial but was pretty cool just the same. Joseph Smarr of Plaxo touched on the very important topic of Data Portability. In a nutshell, he's basically concerned with the fact that a user isn't able to define him/herself in one place and have that information show up everywhere. For example, I love The Magnetic Fields. I love them everywhere, all the time. But despite this, I'm sure every site on which I have an account doesn't list them in my favorite artists section. Wouldn't it be neat to define my favorite artists in one spot and have that show up everywhere? Or another example. I'm friends with Jorge. I'm friends with Jorge everywhere. But I'm sure there's a site we're both on but don't realize we're both on and therefore aren't friends (probably eons.com). Why can't I store friends information in one centralized spot? If you stop and think about it, it's actually a really cool idea. I friend you on Facebook. It updates my account. That information gets pushed everywhere else, and if you're anywhere else we're friends there too, automagically! I think this is (or should be) the direction that Google is headed with its OpenSocial platform. But the clear question is, what's in it for the sites? If I can get all my Orkut contacts on MySpace - whose site I absolutely love - why would I ever go to Orkut? This was a question I posed and he had a very interesting response. Imagine I can take my LinkedIn contacts everywhere. Wouldn't I spend that much more time on LinkedIn solidifying my contacts since I knew if I did it there I wouldn't have to do it again anywhere else? Same goes for finding friends taggings on ma.gnolia or tweets on twitter. The logic isn't foolproof - sites that don't have any particular niche become sorta useless - but it's the first real argument I've heard for sharing data. Cool talk.

At 1:30 I sat in on a tea party with 3 panelists who talked about nothing. That was frustrating so I left after 20 minutes and caught the end of what seemed to be a pretty neat talk about JavaScript hijacking. All I really caught was the conclusion which detailed how only a few of the JS Frameworks actually implement protection for it and how we as developers can protect against it server-side. But from what I saw it looked like a really good topic to go more in depth with from both a curiosity and actual security perspective.

The final session of my day was entitles "Design your API" and was presented by two Twitter guys (who assure me they're getting paid). Actually one officially works for Twitter and one was brought in to help build the Twitter API. Twitter delivers an astounding 80% of its traffic through its API. Which means this thing is getting hammered. Which means it needs to be done right. There were several key features of effective APIs that they highlighted including:

• Have a dedicated subdomain for your API and implement caching immediately
• Version the API from the get-go
• Make life easy for Flash developers, i.e. pass back skinny XML and not heavy RSS
• Help your developer community as much as possible and listen to their feedback
• Make it secure
• Make it RESTful
• Make it awesome

Then there were the keynotes. O'Reilly got people all O'riled up and read a 50-line Rilke poem (which was not part of the O'riling). Max Levchin (dude who created PayPal and Slide) basically told us to, in the words of Winston Churchill, "never surrender." I think Churchill was actually talking about Internet startups too when he said that. Can't remember. Amit Mital from Microsoft showcased their newest product "Microsoft Mesh". This new product basically syncs files across any number of devices and greatly eases the ability to share files with others. I feel like there's going to be a fair amount of overhead to learn how to actually use the system for the average user - Microsoft has a pretty spotty history in terms of designing for the computer-savvy. I know, for example, that no member of my family would be able to figure out what he was showing in his video. (Paraphrasing) "You open up the mesh, attach your devices or link to them via your network, open up your target folder, and drag and drop files you want to add to your mesh. Files you want to share can be dragged to your shared mesh, etc etc." My Mom and Dad would be mouth agape after "You open up the mesh"... It's just too complicated. Not to mention it only works with Microsoft products, though they claim Mac support is coming. So we'll see how useful it actually turns out to be. It's an okay idea though. Steve Gillmor likes it: Microsoft Says Yes With Mesh

Clay Shirky's keynote "Here comes everybody" was the most interesting thing I'd heard all conference. He set the stage by taking us back to the early-to-mid 19th century. Streets of London. Gin carts (carts!) are being pushed up and down the streets, the salesmen pedaling their wares to keep an entire generation inebriated and useless. Similar alcoholic binges were occurring en masse, all over the world. Then the human race sobered up and actually started doing stuff. And we get the period in world history known as the Industrial Revolution. Now it's his premise that we are coming out of a similar period of cognitive suppression. And what, pray tell, does he believe to be our drink of choice? Wait for it. Sitcoms. TV! How many hours of TV do we watch as a nation, or worldwide even?? He threw numbers out (which I can't completely remember) but they were in the tens of billions of hours. One overall number was in the trillions. Needless to say, and anyone could've told you this, we watch a lot of TV. But what's happened over the past couple years? People have been turning away from the tube and turning towards the web. Now instead of watching reruns, people are editing Wikipedia articles. Or even if they're not doing something that cognitively engaging, if they're just editing their 1,000th LOLCat, at least they're engaging their brains in some fashion. He told a funny anecdote about his friend's kid watching a movie. At some point she got up and walked behind the television set and started rooting around in the wires. "What are you looking for?" his friend asked. "The mouse," she replied. We now expect our media to be engaging. We are coming out of the decades-long stupor produced largely by mindless TV and are starting to participate in a worldwide movement of creativity and collaboration. Typing it all out, it is pretty exciting to be a part of.

In Conclusion,

The revolution will not be televised...

The 2008 Web 2.0 Conference (and Expo) got off to a raucous start today. This morning I participated in a workshop (emphasis on the "shop") covering Facebook and opensocial application development. While extremely disappointing in its basicity (a word which I am coining to mean "When a group of people watch some girl click around her MySpace profile to show where you can put really neat widgets"), the latter half of the workshop did semi-touch on important concepts concerning how one should develop in this rapidly changing social network application development environment. I wish the entire 3 hours had been spent developing a working application from scratch, though, even if it had just been a dumb "Hello World" app that says "Hello World" to your friends. Then they could've discussed specific issues encountered with working with Facebook and OpenSocial. How you deal with the differing implementations in the OpenSocial containers. The best way to update and maintain your code to work on the various sites. That's stuff I would've liked to learn about. But if anything it provided a few scraps for thought about OpenSocial coding.

In the afternoon I sat in on a pretty interesting talk about "Innovating on Time" (which may or may not be the official title, but close enough). This topic is something I'd like to go more in depth with at a later point when I've had the opportunity to read a little more and collect my thoughts, but the premise that someone or some group should allot time in their schedules for the simple act of idea creation - or "ideating" - is pretty interesting to me. And while this was the primary focus of the first half of the workshop (there was a 15 minute break for snacks which were nowhere to be found...sob), the second half of the workshop focused on the issue of how you act on these ideas in a timely manner. The presenter, Scott Berkun, brought up several key issues and roadblocks we all face when trying to complete a project - be it innovative or not. At one point he started presenting a series of charts and was throwing around "quantities of work-time" and calculus-based derivative explanations of how projects finish in a hyperbolic fashion. It got a little goofy. But the last 10 minutes of wrap-up definitely left some good messages in terms of how one (or, again, one group) should focus on developing innovative products in an efficient and timely manner.

Early-to-late evening after the conference is mostly a vague wash of various scary homeless people harassing me.

In conclusion,

Web 2.0 FTW

I recently sat in on part of ABC's two-day Digital Media Developer Summit.

Lex Sisney, CEO of ELC, which we teamed up with with for development and training, spoke about Agile and Iterative development, open source and how they could fit into ABC's developer stack.

Lex touched on three trends:

1) Success of agile/iterative project management
2) Emergence of developer friendly programming languages (specifically Ruby)
3) Commoditization of hosting (Amazon Web Services)

I spoke about how the ESPN Community Group utilizes these three trends for our development process. Oh, ESPN and ABC are part of Disney, which is why I got the invite in the first place.

We got a great reception from the various developers and managers there, which was great. These folks use the same proprietary technology and project management methods we used to use and I got the impression there is a lot of excitement for trying new technologies, methods and hosting solutions.

One of the best questions was what are the fringe benefits we've enjoyed from using all open source methods and technologies. As a manager and developer, I have noticed several:

1) Greater number of applicants for positions
2) Developers acquire skills more quickly
3) Developers acquire a much broader range of skills
4) Developers are more eager to improve their knowledge and skills (because they are acquiring marketable assets)
5) We have become part of a much larger community. Both in terms of taking and giving, we are active in the community through code sharing, Google Groups and much more.

Another great question regarded the combination of Test Driven Development, Agile Development and Ruby on Rails and if they can be separated.

I think Lex answered this best by saying TDD and Agile are methods and RoR is a great tool to use with these methods.

But it is interesting because Agile and TDD almost go hand-in-hand with Rails because the stack is so conducive to those methods that developers would almost have to fight against the methods NOT to use them with Rails.

Any language or stack (especially MVC stacks) can utilize TDD and Agile as processes, but both were baked in with Rails.

I can't think of any reason why that's bad, but I'd be interested to hear arguments on negatives to that.

The last big theme that I tried to stress is that we are not forcing or trying to force any one to adopt our methods or technologies, but merely present that, even in a large company with established methods and technologies, change still can happen, and languages, methods and hosting can be chosen to fit a particular application or product.

For example, I said that moving from Java to Ruby to do the heavy SAX-based parsing of statistical data feeds probably wouldn't be the best idea, but Ruby may be a much better solution for building a new social network.

So I am very glad Lex and got to share knowledge with developers from another part of the community. I hope the company as a whole embraces different solutions, so we can move beyond boundaries created by a particular technology and join a very smart and open community.

One of our group goals as the Community Development team here at ESPN.com is to be a productive member of the global community of developers.

Each member of the team is a habitual social networking user with a deep love for the latest and greatest technology used to build and innovate the best web sites on the internet.

We're doing a number of things in our effort to contribute back to the community, including starting this blog, and one of the other main aspects of our effort is attending technology conferences. There are a slew of conferences happening all the time and though we can't make it to as many as we'd like, we did manage to make it out to our first a short couple of weeks ago. We were fortunate to have a fun and interesting one take place in nearby Hollywood at the Community Next event Next Generation: Media and the Web.

The one day event took place a couple of weeks ago now but we thought it would be a good idea to still give a little round up of our experience there.

The event featured some of the most well known sites on the social landscape and some that even we've never heard of so there was a good cross section of speakers and sponsors.

Here's a list of some of the sites that were represented: MySpace, Gaia, Songza, Geni, Warner Bros Records, Pandora (blog), The Hype Machine, YouTube, Scoble, Mashable, Reddit, 8020 Publishing, Veronica Belmont, Meebo, Playboy U.

None of us here have been regular conference goers so we didn't know really what to expect from the event. We were stoked to find out it was a very casual affair with a lot of cool people in attendance. The food was good and the schwag the sponsors were giving away was pretty good to boot!

We, all three of us, took in the event in shifts though there was a good couple of hours or so when we were all there together. I took the early shift since I love breakfast the most and got to listen to some interesting talks and presentations before I had to take off for a previously planned engagement. I'm a huge music lover so I got the most out of Aza Raskin's presentation about his songerize site. I never knew about the site prior but have since checked it out and now count it in my repertoire of music consumption. Two other peeps that were part of the Music/Radio discussion represented their sites which I had heard of before included Pandora and The Hype Machine. Though I hadn't checked either out in a while, it was cool to hear what they were up to and get their thoughts on what's happening in the ever changing world of music and how the dynamics of the internet are changing the way we as music lovers get and listen to music.

There's some great video that was taken at the event up on Stickam and there are a slew of photos uploaded on Flickr that are tagged with communitynext.

We're looking forward to attending many more conferences as we grow as a group and get the word out about what we're working on here at ESPN.