Sometimes you need to make a mess

Edit: This is a follow up on another blog post by my colleague Andy Maleh that you can read here. I wanted to expand on his interpretation and add my own thoughts.

What do we mean when we say some code is a mess? We're usually referring to its structure or lake thereof. Interactions between components are awkward and poorly named. There's often logic leaking between the layers causing odd couplings and generally making our lives a pain. Software like that is certainly a mess. That sort of mess is usually traceable to poor discipline on the part of the programmers or poor management of the team. We know these messes well.

However, that's not the kind of mess I want to talk about. I want to talk about the kind of mess that happens while you're learning about a problem space by programming in it. These are flaws that you intend to clean up once you've wrapped your head fully around the problem, but you're quite incapable of doing so until you *do* understand what's going on.

Would you say code in that state is messy? I would, but I think these messes are actually natural and good. Trying to avoid all kinds of messes early can be a mistake.

It is premature optimization of the design to force structure into the system when your understanding of the problem is too weak to support it.

Here's an example. I wanted to build a pure-ruby png spiriting library to use in our web applications. It would take multiple png images, smash them together into one "sprite" png, and generate the css you need to access each of the images contained within. Now, I had never done any of this before. I didn't know how to parse a png or splice them together. I had a lot to learn to solve this problem.

What did I do? First, I went and found the official png documentation, then I found a png library in ruby I could refer to. Finally, I made a huge gigantic mess trying to get it to work. This mess was a little bit like laying all the puzzle pieces out on the floor before you start to assemble it. I had some functional tests, I had some code that could parse a png - but it was far from an ideal form. Once I had something basic working, even though the code was a disaster, I could start to refactor. More importantly, I had allowed myself to explore the problem space by not getting hung up on the optimal design. I could worry about that later. Once I had enough understanding of the solution that I felt good about introducing new concepts I would do so and refactor the code into its new home.

The end result was a nice little spiriting library you can check out here. How does this technique work? How can you make a big mess without tossing all the code in the end? I've certainly done that before. Well, most importantly, you need to write the right kinds of tests. Whenever I'm working in a problem space that I don't feel comfortable with I always start my testing at the highest possible level, most of the time this means integration tests. I do this because integration tests give me more freedom to refactor.

A system that is easy to refactor doesn't punish me as much for making mistakes and I certainly do make a lot of them. A good suite of integration tests shouldn't break unless you've truly broken the application, that is very useful. In the case of the spiriting library, I started by writing a test that expressed how I wanted to open a png and inspect its data. Once I had that working I would refactor what I thought I understood well enough and move on to the next test. When I got stuck, I tried to think up an easier test that doesn't require me to take on so much of the problem. So foremost, when I have a really awesome set of integration tests I'm never afraid to attack a mess. I know the test suite has got my back.

Second, I need to be really good at refactoring. As I understand more of the problem I need to be able to take all these mistakes and turn them into solid code. Another important aspect of refactoring is knowing when the mess is getting too big. I need to understand how to refactor as much as I need to understand when a refactoring is necessary. I try to do this by always keeping the current state of the solution in my brain in sync with the solution in the code. As I learn and as code starts making sense I change it to match that new knowledge. That way when I come back to a problem I can pretty clearly tell where I need to focus my learning - wherever the code is the messiest.

Therefore I endeavor to only allow a mess to live as long as my ignorance of the problem space.

Groupon Launches Personalized Deals

Last week Groupon's founder Andrew Mason announced, as he put it, "the biggest thing we've done since launching Groupon". That announcement was personalized deals and I'm very proud to say that Obtiva was intimately involved in the development this new initiative.

Collaborating as a team comprised of Obtivians, Groupons, and other consultants we were able to take this project from inception to prototype to launch in just a few months. I believe Groupon and Obtiva have shown once again that taking the time to do things right, to be craftsmen, is the most efficient way to build software.

I'm confident that the growing Obtiva team at Groupon will continue contributing to the great software that has helped make Groupon one of the fastest growing internet companies of all time.

craftsman Swap – Day 5 at Relevance

It was the last day of an amazing week and I wanted to end it with a bang. I poked around after standup to see what people were doing and it was apparent that it was going to be a light day in the Relevance office. I think it was a coincidence, but this Friday found most of the Relevance staff traveling.

Fridays are special at Relevance as it gives everyone the opportunity to work on his or her own R & D projects with the stipulation that these projects are open source. Not only is this an awesome perk for their staff it is also great for their customers too. A lot of the tools they build on Fridays help them deliver better software faster and more reliably for their clients. If you’ve used any of the projects on this page, then you owe a nod to Open Source Friday at Relevance. Personally, I find their commitment to the community inspiring. I wish more companies found a way to give back like this (nudge, nudge, Obtiva).

This Friday began as every other day this week had, with the company stand-up. Nothing new to report there. After standup everyone broke to do their own thing, either as a pair or solo. I think it was split evenly between those working alone and those working as pairs.

I was attempting to sell Larry on my Clojure QuickCheck port when Brian Marick found his way into the conversation. For those who are unfamiliar, QuickCheck is a Haskell testing library where test data is automatically generated and assertions made against “properties” of a function under test. What I wanted to attempt was a BDD / QuickCheck Hybrid that supported both styles of testing.

Eventually it was just Brian and I working to bootstrap the framework. We’re both new-ish to Clojure and we ended up bumping our head on a rough edge or two. For instance, using a binding to override the behavior of the "=" function in Clojure is a really bad idea. And it is bad in a way that isn’t immediately obvious. I think we killed a good hour figuring that out. We didn’t make a ton of progress, but it was really great to finally work with Brian. He’s an exceptional guy.

My Open Source Friday ended too soon, as did my time at Relevance. The people are top notch, the culture is wonderful and the software they write is exceptional. Without a doubt, a most wonderful place to spend a week or a career. So, a huge Thank You to all the Relevance folks for having me out. Let’s do another one soon!

craftsman Swap – Day 4 at Relevance

The fair weather abandoned Durham sometime last night. I awoke to a brisk, rainy day. I had treated myself to an early bedtime and I was feeling especially good even though the weather wasn't spectacular. I made it in early thanks to Chad and spent about 40 minutes working on the blog post from the day before. I’m still behind from Tuesday.

Standup went with a now expected cadence. I filled everyone in with the delightfully busy day I had yesterday. Today I would continue working with Larry on the Compojure web application. Excellent.

Larry Karnowski and I began our pairing session by reviewing the card. Our review turned out a missing behavior. We had yet to allow the caller to provide the recipient's identifier in the request. Our next step was to look to see what was currently supported in the model. Someone had already wired up the model so that a message could have multiple recipients, but it wasn't entirely finished. Multiple recipients is also clearly beyond the scope of our card. I thought we might as well go through with it but Larry was adamant that we rip it out. It was not necessary for the delivery of this card - or any of the cards in this iteration.

This intrigued me. We spoke about it a bit and it became clear to me that this is part of what makes Relevance guys different. If the code is dead or if the implementation isn't ideal it gets ripped out right away.

I realized that this could be why I found it so easy to integrate myself with their projects. I didn't waste any cycles trying to sift out what was relevant from a year's worth of dust and debris. Great code doesn't say that way long if you're not disciplined about taking out the garbage. Relevance employs some tidy fellows. Larry had to run to a meeting and left me to extract the unnecessary bits. I managed to muddle through and finished the job just before lunch.

After lunch Larry and I ran into another little issue. This story we were working on wasn't exactly user facing. We were building a programmatic API for creating messages. In order for the customer to sign off that the story was complete we needed a way for them to create messages through the API. It didn't need to be perfect, just functional. Given this particular API is for a web application we were able to construct a simple form that the customers could use to see the system work on their own.

Our afternoon ended early when we broke to do an estimation session. This estimation was to be done in preparation for the next iteration of the project Larry and I were working on. This project was also different. The outcome was to be a proof of concept that would be used to demonstrate some of the application's core features, not an application they'd be putting into production.

Before we talk about what happened let me roughly explain what I learned about how Relevance estimates cards. There are two estimates taken, one at a high level and another at a low level. The high level estimate is used to size stories into an iteration. An iteration is “full” when the sum of the high level estimates meets the expected velocity of the team.

These estimates are taken using a relative scale derived from completed stories in the project. Before each estimation session begins they baseline themselves for this project by looking at completed stories from the previous iteration.

The low level estimate is done by the pair picking up the card for development. This low level estimate is the one that the development pair is expected to perform to. It makes a lot of sense then that they get to chose what this number is.

Now, back to this estimation session. Muness, Chad, Larry and I began by reviewing the available cards in Mingle. We were debating priority from the customer's perspective as well as what sequence made the most sense for development. This gave us a subset of the available cards to plan for the upcoming iteration as well as a rough sense of priority.

At this point we began estimating cards. I abstained as I was unfamiliar with the project and I didn’t want my poor estimates to cause someone trouble down the road. Each estimate for a card was collected by counting down 3-2-1, then everyone showing a point value simultaneously. If the variance was small the higher estimate is taken, if the variance is large then the card is debated again and another estimation is taken.

We continued this process until the iteration was full. And that marked the end of my 4^th day at Relevance.

Craftsman Swap – Day 3 at Relevance

Morning standup went as expected, Rob and I were back to work again on the story we'd began the week with. The module we had extracted for allocating addresses was more or less done. We spent a few minutes discussing our strategy for integrating back into the model that would use it. All of the 'old' behavior was still in place through various lifecycle hooks. There were two before_save hooks and one validation hook we'd need to change. Given there was a lot of code already that depended on this existing behavior Rob and I were hesitant to just start ripping it out. It made more sense to write the new code in parallel with the old keeping everything working until the last possible moment. This way we could have some certainty that our new code in the model worked before we were forced to consolidate our changes with the rest of the system.

Over another great Relevance provided lunch it was decided that I would be switching projects in the afternoon. I felt bad about leaving Rob midstream, but consoled myself with the fact that I probably wasn't very much help anyway. He picked up Jason as a pair for the afternoon and I went with Chad.

Chad and I spent a couple hours spiking out client side certificate validation in JRuby and Jetty. We generated a little Rails project then pulled down warbler and jetty-rails. Getting Rails up and running on Jetty with jetty-rails was mindlessly easy. We hit our first bump trying to get an SSL listener up and running in Jetty. We spent a few minutes reviewing the jetty-rails documentation and found no mention of how we could achieve this. In the end we just copied the jetty-rails launcher script and modified it to suit our purposes. This was just a spike after all. We launched our new script and surprisingly the thing worked! Chad and I reported our success back to Stuart and Larry, then moved on. We knew we could make JRuby work, if we needed to.

Chad and I had a few free moments and I decided to take him up on his offer of a 10 minute tour of RSpec2. He's quite enthusiastic about the new metadata model and spent most of our time telling me about that. My very basic understanding is that everything you declare about a specification is captured in metadata which is accessible from pretty much anywhere you'd care to get to it. This makes RSpec2 extensions a breeze to write. Oh, and its really fast too, he'd want you to know that.

With my RSpec2 introduction complete I moved on to pair with Larry on a Compojure web application. When I sat down Larry was finishing up a card for a part of a messaging API. Specifically, message creation. You'd post to the controller and eventually your message would end up in the database. Compojure looks quite a bit like Sinatra. There was a simple block of functions that mapped routes and http methods to controller functions. These controller functions served the same purpose as actions do in Rails controllers.

Larry mentioned that the create method was getting a little out of hand. There were two conditionals that were essentially doing validation. If we got past the two conditionals a transform operation would execute and turn the form parameters into a Clojure data structure. We'd validate the record and if everything was fine it'd be persisted to the database. If validation failed or if either of the conditional guards failed the method would return a 422 response and a message.

I noted that the conditional guards were serving as as second type of validation. It was easy to move them into the clj-record validations and so we did. I also suggested to test that a system failure would return an appropriate response for an API - not some HTML 500 error page. We wanted to test this by throwing an exception out of the create action. We monkeyed around trying to get dynamic re-binding working through Clojure's binding form as a cheap form of mocking, but we couldn't figure it out. The day was over, so we tabled the test until tomorrow.

A note on my first day of Clojure experience. Personally, I find the language itself immensely pleasing to work with. As I've grown as a developer I keep finding myself writing smaller and smaller functions. A lot of languages have syntactic overhead that punish you for writing really small functions. Clojure does not, Ruby does not. I really like that.

However, Clojure is a young language and there are definitely rough edges. Naturally, the tools available in a young language are also necessarily young. There's a lot of missing pieces in Compojure that Rails has had baked in for years. Does that mean you shouldn't use Clojure or Compojure for your project? Absolutely not. But you should know what you're getting in to.

Now's the time to act if you want to make a name for yourself in the Open Source Clojure community. There's a ton of work to be done and not nearly enough people doing it.

Craftsman Swap – Day 2 at Relevance

Day 2 begin as I expected. We gathered at 8:45 in Relevance's large conference room to discuss yesterday's happenings across the company. I said that I had been working on Rob's project and expected more of the same today. We weren't giving story-level progress in this meeting so it wasn't drawn out with the minutia of everyone's day. Ten minutes later I was back at my desk with Rob continuing work on our story.

Yesterday we had worked through a few model changes that were necessary to support the behavioral work we'd be tackling today. The old model had a one-to-one association that we changed to a one-to-many. We needed to alter the algorithm that populated this relationship.

The existing algorithm was embedded as part of the model's lifecycle hooks. These hooks fired on save and would populate the association. This algorithm was already fairly complex and it was going to get worse. Rob suggested we extract it into its own module. Rob wanted to TDD this new module from scratch where I thought it would be best to refactor into the module we wanted with the support of the existing tests. After some more discussion and thought on my part Rob convinced me the greenfield TDD approach was best.

Our next debate was: How would the module communicate with the model we were extracting it from? Rob wanted something we would mix in. You'd execute it and it'd do whatever it needed to the model to populate the association. I was worried this approach would create coupling between the model and the module that wasn't immediately obvious. It seemed a bit magical. I suggested that we could reduce the coupling between two and make their interaction obvious by calling out to the module explicitly from the model, then letting the model handle hooking up the associations. Rob agreed to my plan.

After breaking for another excellent lunch provided by Relevance, Rob and I began TDDing our new address allocation module. Rob ran out for a phone call, so in the interim I put together a set of pending specs for all the scenarios that I thought we'd want to test. I knew I'd missed some things, but it was a good place to start the conversation when Rob came back.

The rest of the afternoon was spent building our module, test-first. Many times we would write a failing spec, then realize the behavior we wanted should live in another model. We'd leave that test failing, hop over to the other model's spec and write a second failing test there for the new method we wanted. If everything went well we'd go from two failing tests to a green bar in one fell swoop. Nice.

This is different than my usual method of inlining the new behavior to get the outer test passing, then refactoring into a new method after I've seen the green bar. Rob's approach has the advantage of making you write that unit test for the new function first. It is easy to rationalize not writing it after you've seen the green bar. I know I've done it. We used the 'fit' (f means focused) feature in RSpec2 a lot for this style of work. It really came together well.

The evening had four of us geeking out with a Magic The Gathering draft + tournament, which is the reason I am so late in delivering this post. Although I didn't fare well (1-6) it was a lot of fun.

Two days down, three to go. I'm having a lot of fun. There's rumblings that I may move on to a Clojure project tomorrow. That'd be fantastic.

Craftsman Swap – Day 1 at Relevance

My first day at Relevance actually began Sunday night when Chad Humpries and Kris Singleton were kind enough to entertain me for the evening. I had dinner with Chad at Piedmont, an excellent local Italian place. The food was great and the conversation interesting. Chad talked to me about his inspiration for Micronaut, which is now Rspec2, and went over some of its features. He’s on the hook for giving me a 10 minute overview of the framework and some of its innovative features. Pretty excited about that. He also offered to meet me for breakfast the following morning while we walked to the Relevance office for a late night tour.

Kris met Chad and I at the office as he’d kindly offered to shuttle me to and from his place for a night of gaming. We played two rounds of Dominion (the seaside expansion) and a train game I cannot remember the name of, but that was after a glass of bourbon. We played to about ten. It was a great way to finish out the evening.

I met Chad the next morning at Dos Perros a little Mexican place next door to Relevance. About half way through the meal I look up to see a familiar face outside the window. Holy crap! Its Brian Marick!

He’s in town doing his own craftsman swap – so to speak. I don’t know if he’d call it that, but his plan seems similar to mine – do some work with the top-notch Relevance crew. It’s always a good day when Brian walks through your door. How exciting.

We all made it up stairs and hung around a bit before the 8:45 all-hands standup. As stand ups go it was pretty much what I expected from an Agile shop. What did you do yesterday? what’s the plan for today? what’s blocking you? Brain and I were set up with our pairing partners as well, which looked to have been discussed before hand. No problem.

Rob Sanheim and I (my pair for the day) started off with an architectural overview of the project I was joining. The project itself is really interesting, but I can’t say much about it. NDAs and all. We had the usual sort of arrowed diagrams describing the major components of the system and their interactions. I was interested in how their testing strategy played out across the tiers of the application, so we drew some lines to show the boundaries between the test suites and where they overlapped. It was a useful experiment. It turned out that in a past life I had worked quite a lot in the domain, so most of it made sense to me at a high level. Good enough to get started.

The work environment at Relevance is really nice. Everyone sits together in one large room. The desks are huge with ample room for whatever you think you’ll need for the day. Each pairing station has two monitors, two keyboards, and two mice. Most of the shops I’ve visited do it that way and it works. The office itself had a really warm, relaxing atmosphere. Exposed wood beams, pine plank flooring and lots of windows.

We settled in and began our pairing session by reviewing the story in Mingle. It was larger than I usually like a story to be and Rob agreed that it was a bigger card than they usually played. We discussed breaking the story down. A significant refactoring was necessary, so I pitched doing that first as an “internal” dev-facing card. Rob seemed to like the idea initially, but thought it would be a hard sell for the customer – no value. The second pitch was to break the story by its two major functional components – configuration and execution. That didn’t make sense from a value standpoint either – the configuration part had no value to the customer without being able to execute it. All that discussion took about 5 minutes and in the end we decided to take on the whole card.

The card itself broke down like this. It had the usual title and a paragraph or so describing the story. That was followed up by four cucumber features that loosely described the acceptance criteria. These were written by a developer beforehand. The bottom of the card contained a Q/A section that listed responses to developer questions and anything outstanding. The whole card was sent to the customer for approval before we began work. I liked that a lot.

The part of the system we’d be changing is a Rails webapp. First things first, we made sure the test suite passed. I noticed Rob was using something like Autospec, but it wasn’t. It is called Watchr and it looks pretty awesome. My first big knowledge win for the week. It works like Autospec in that it automatically re-runs the test suite whenever something changes, but it gives you a lot more control over what's getting executed. Going to be a big win for me on my client project back home.

We had spent about an hour on the card when we were pulled into the (I think daily) project standup meeting with the customer. It was efficient and well run. We discussed all of the outstanding issues from last week and how they were being addressed. Ran through the story work for the day. It was pretty much your standard stad-up meeting.

15 minutes later we’re back to work. The communal stereo offered an eclectic mix of tunes to set the mood – pretty much any genre you can imagine. By lunch we’d managed to translate most of the first acceptance test from the story into something we could execute.

Relevance brings in lunch every day for its employees. Well that’s nice, but what’s the value? Is catering lunch every day just a perk or does it add something to Relevance’s culture? I think it does. First, everyone eats together every day. A lot of interesting discussions went on over lunch and everyone was there to participate. Second, we all got back to work at the same time. That is a fantastic feature if you’re pairing a lot – and Relevance certainly is.

The rest of the afternoon was standard Rails fare – a few migrations, some Refactoring, fixing specs. We did get a little hung up in our acceptance test where we wanted to assert the presence of a few fields in the view. We got something working with rspec’s include matcher, but the failure output was terrible. It dumped the entire response body. We blew a good 20 minutes trying to clean it up to no avail. We're using Micronaut (RSpec2) so have_tag wasn't available and webrat's selector doesn't work with assertions. We benched the cleanup when it became apparent it was going to be harder than we thought. It wasn’t that important.

Our day ended with an 1.5 hour company retrospective. I was told these happen every two weeks here at Relevance. We don’t do these at Obtiva and I really liked the idea. We do a lot more on-site work than Relevance does, at least it seems that way. An all-hands company retrospective is a great idea for us and I’m definitely going to take back to Obtiva with me.

Overall, it was a pretty awesome first day. The way they put together their projects really resonates with my own personal style. All the code we worked with is exactly what I expected to see. Very high quality, easy to comprehend and simple to test. I had enough confidence to take the driver’s seat after a few hours and I’m happy to say I was able to contribute a couple useful ideas on day 1. That speaks not to my own ability but to the quality of my pairing partner and the expressiveness of the system he was helping me to learn. Well done.

Looking forward to day 2.

Managing dotfiles with git

I've never managed dotfiles before, but now that I'm a serious vim user I thought it was time to get on with it. Naturally, I googled around to see what other fellows had done. I found a few different schemes. They all seemed more complicated than they should be. Some had scripts to run and symlinks everywhere. Others appeared to require a lot of manual bit pushing to maintain.

all of this because dotfiles are in your home directory along with a lot of other stuff you want to ignore. There is a very simple way to take care of this, let's get started.

First, you'll need a git repository. So cd into your home directory and type

git init

Now run the following command

git status

you should see a gigantic list of files. Some we want to version with git, most we do not.

Now here's the first trick. We want git to ignore everything by default, so we type:

echo "*" > .gitignore

Now run 'git status' again and git will tell you there's nothing to add to the repository. Excellent!

At this point you have a choice. You can manually add the dotfiles you want to version by prefixing their name with a '!' like this:

echo "!.vim" >> .gitignore

This makes for a short .gitignore file, but git won't tell you when new dotfiles are added - they'll be globally ignored along with everything else. That may be fine for some, but I like to know when new dot files are created.

To do this, I told git not to ignore anything that was prefixed with a '.'. My .gitignore file now looks like this:

/*
!.*

Now when you run 'git status' you get a big list of all the dotfiles that have not been versioned, most of these we want to ignore. So we capture the output in a file:

git status > unversioned

Open 'unversioned' and '.gitignore' in your favorite editor, copy the file list from 'unversioned' to the bottom of '.gitignore'. Now remove everything that you *DO* want to version. Be careful not to ignore your '.gitignore' file! That'd be silly.

Finally, add and commit your changes normally:

git commit -a -m "hey dotfiles! woohoo!"

That's it! If you want to store your dotfiles on github create a repo and follow their instructions for setting up github with an existing git repository.

You can see my dotfiles repo here

Cheers!

A response to "Failures in Isolation"

Following Corey Haines on twitter is such a good idea. Today he dropped this gem:

Please read, I'm not going to paraphrase it. Nevermind, I will. But it will be really short. Poorly designed code that is tested with a high level of isolation causes a problem. You can't refactor it with confidence because the tests, or the mocks, are blocking your progress. So let's talk about how they impede refactoring first.

Tests prevent behavior from flowing 'up'. For instance, if you mistakenly coupled a model class to a view tier class you could not 'refactor' this problem without breaking the tests for your model. Most likely, the test will fail because the API of the class has changed, you no longer pass the request.

For Mocks it is the opposite. You cannot move behavior 'down'. Let's say I have leaked some model behavior into my controller and I wish to move it to the appropriate model. If my test uses the real model, not a mock, this is not a problem. I can move the behavior and my controller test will still pass. Now I can safely write model tests for the new behavior and delete the controller test that now duplicates it.

If I have mocked the model, however, my controller test will fail when I attempt to move the behavior 'down' into the model. One offense is not an issue. I was involved in one system where an entire test suite was thrown away because of improperly used mocks. Death by a thousand paper cuts.

I have no universal solution for this problem - as it is a problem of my skill as a designer and the skill of my peers. However, I have come up with two helpful guidelines.

Only directly test code you like

I dislike code when I'm unsure of its design. I also dislike code that appears volatile. As my confidence has grown I find myself more comfortable with my early design decisions and more willing to glorify my creations with their own test sooner. When I am not confident I keep my tests at a high enough level where I can refactor freely. As my confidence grows in a system I move towards higher levels of isolation.

Only mock code you like

For all the same reasons as above. Mocks crystalize an API for a class and can cause a lot of problems if the class is used often in a system and the class it is poorly designed. As the ugliness and volatility of code increase, mock it less.

In Conclusion

As with everything, the value of isolation in testing is not an absolute. For younger developers I suggest beginning with a state-based testing approach over using mocks pervasively. A state-based style will afford you maximum flexibility to refactor your design. As your experience grows begin to use mocks to solve some of the problems that state-based testing will invariably bring about.

That is the path I travelled and it worked out alright for me.

A log4j adapter for Rails

This current client application I'm working on is a mixed bag of Ruby and Java parts. All our existing logging for the older Java parts uses log4j and we wanted to unify our Rails logging with that. I spent a couple of hours putting together this adapter we now use. If you need something similar it should give you a good place to start.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53

class Log4jAdapter include ActiveSupport::BufferedLogger::Severity L4JLevel = org.apache.log4j.Level SEVERETIES = { DEBUG => L4JLevel::DEBUG, INFO => L4JLevel::INFO, WARN => L4JLevel::WARN, ERROR => L4JLevel::ERROR, FATAL => L4JLevel::FATAL, } INVERSE = SEVERETIES.invert def initialize @logger = org.apache.log4j.Logger.getLogger('com.shc.mmh.Rails') @root = org.apache.log4j.Logger.getRootLogger end def add(severity, message = nil, progname = nil, &block) message = (message || (block && block.call) || progname).to_s @logger.log(SEVERETIES[severity], message) end def level INVERSE[@logger.getEffectiveLevel] end def level=(level) raise "Invalid log level" unless SEVERETIES[level.to_i] @root.setLevel(SEVERETIES[level.to_i]) end def enabled_for?(severity) @logger.isEnabledFor(SEVERETIES[severity]) end #Lifted from BufferedLogger for severity in ActiveSupport::BufferedLogger::Severity.constants class_eval <<-EOT, __FILE__, __LINE__ def #{severity.downcase}(message = nil, progname = nil, &block) # def debug(message = nil, progname = nil, &block) add(#{severity}, message, progname, &block) # add(DEBUG, message, progname, &block) end # end # def #{severity.downcase}? # def debug? enabled_for?(#{severity}) # DEBUG >= @level end # end EOT end def method_missing(meth, *args) puts "UNSUPPORTED METHOD CALLED: #{meth}" end end

A Scala version of Unle Bob's lazy PI sequence in Clojure

I follow Uncle Bob on twitter and I'm also currently learning Scala. When I saw his tweet about a lazily evaluated infinite PI sequence in Clojure I couldn't resist giving it a shot in Scala. Here's my result (it works!). Feedback is absolutely welcome.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56

//Gives us the :: syntax for streams. From http://www.codecommit.com/blog/scala/infinite-lists-for-the-finitely-patient. class RichStream[A](str: =>Stream[A]) { def ::(hd: A) = Stream.cons(hd, str) } implicit def streamToRichStream[A](str: =>Stream[A]) = new RichStream(str) object M { //Gives us a shorthand for declaring BigDecimals that Scala lacks def apply(i:Int) = BigDecimal(i) } //Hack to work around poor bigdecimal support val mc = new java.math.MathContext(300,java.math.RoundingMode.HALF_UP) def divide(a:BigDecimal, b:BigDecimal) = { new BigDecimal(a.bigDecimal.divide(b.bigDecimal, mc)) } def piSummands(n: BigDecimal): Stream[BigDecimal] = { divide(M(1), n) :: piSummands(n + M(2)).map(_ * -1) } def scaleStream(stream:Stream[BigDecimal], factor:Int) = { stream map {_ * factor} } def addStreams(s1:Stream[BigDecimal], s2:Stream[BigDecimal]) = { s1 zip(s2) map {z => z._1 + z._2} } def partialSums(s:Stream[BigDecimal]):Stream[BigDecimal] = { s.head :: addStreams(s.tail, partialSums(s)) } def piStream = scaleStream(partialSums(piSummands(1)), 4) def square(s:BigDecimal) = s * s def eulerTransform(s:Stream[BigDecimal]):Stream[BigDecimal] = { val s0 = s(0) val s1 = s(1) val s2 = s(2) val head = s2 - divide((square(s2 - s1)), ((s0 + (M(-2) * s1) + s2))) head :: eulerTransform(s.tail) } type Transform = (Stream[BigDecimal]) => Stream[BigDecimal] def makeTableau(transform: Transform, s:Stream[BigDecimal]):Stream[Stream[BigDecimal]] = { s :: makeTableau(transform, transform(s)) } def acceleratedSequence(transform:Transform, s:Stream[BigDecimal]) = { makeTableau(transform, s) map {_.head} } val set = acceleratedSequence(eulerTransform, piStream) println(set.take(150).last)

Rails threadsafe! and Engines

If you're planning to use the combination of engines and threadsafe! you'll need to do some extra work. Due to a bug in Rails the engine load paths are not added to the eager_load_paths at startup time. This makes engines unavailable in threadsafe! mode.

Fortunately, working around this problem is simple. Just subclass the plugin loader like this:

1 2 3 4 5 6 7 8

class EagerLoader < Rails::Plugin::Loader def add_plugin_load_paths super engines.each do |engine| configuration.eager_load_paths += engine.load_paths end end end

And wire up your new loader in your environment.rb:

1 2 3 4 5 6 7 8 9

#require the plugin loader require File.join(File.dirname(__FILE__), '..', 'lib', 'eager_loader') Rails::Initializer.run do |config| #override the default loader config.plugin_loader = EagerLoader # ... end

Big O for all you fellows who slept through the algorithms course

http://rob-bell.net/2009/06/a-beginners-guide-to-big-o-notation/

Mathematicians should never code alone.

Case in point:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

if (xGrad * yGrad <= (float) 0 /*(1)*/ ? Math.abs(xGrad) >= Math.abs(yGrad) /*(2)*/ ? (tmp = Math.abs(xGrad * gradMag)) >= Math.abs(yGrad * neMag - (xGrad + yGrad) * eMag) /*(3)*/ && tmp > Math.abs(yGrad * swMag - (xGrad + yGrad) * wMag) /*(4)*/ : (tmp = Math.abs(yGrad * gradMag)) >= Math.abs(xGrad * neMag - (yGrad + xGrad) * nMag) /*(3)*/ && tmp > Math.abs(xGrad * swMag - (yGrad + xGrad) * sMag) /*(4)*/ : Math.abs(xGrad) >= Math.abs(yGrad) /*(2)*/ ? (tmp = Math.abs(xGrad * gradMag)) >= Math.abs(yGrad * seMag + (xGrad - yGrad) * eMag) /*(3)*/ && tmp > Math.abs(yGrad * nwMag + (xGrad - yGrad) * wMag) /*(4)*/ : (tmp = Math.abs(yGrad * gradMag)) >= Math.abs(xGrad * seMag + (yGrad - xGrad) * sMag) /*(3)*/ && tmp > Math.abs(xGrad * nwMag + (yGrad - xGrad) * nMag) /*(4)*/ ) { magnitude[index] = gradMag >= MAGNITUDE_LIMIT ? MAGNITUDE_MAX : (int) (MAGNITUDE_SCALE * gradMag); //NOTE: The orientation of the edge is not employed by this //implementation. It is a simple matter to compute it at //this point as: Math.atan2(yGrad, xGrad); } else { magnitude[index] = 0; }

Ruby's object.methods reflection in Scala

Update: Just found out the Scala REPL in trunk has tab completion, which does pretty much exactly this with a lot less fuss. Cool.

If you're coming from the Ruby world to learn Scala you'll learn quickly how much you miss Ruby's elegant reflection. On day 5 of my Scala adventure I took a crack at using Java's reflection APIs to do the job. I think it worked out pretty well. I'd love to hear what you think about it.

get the code here

1 2 3 4 5

//names of all methods, including super classes namesOf(methods("Foo")) //signatures of all methods, excluding super classes signaturesOf(declaredMethods("Foo"))

Hacking tail call optimization into ruby

Found this via Twitter. Really interesting hack.

http://judofyr.net/posts/tailin-ruby.html

.:format feature breaks URL compliance (rfc1738) in Rails

I put in a lighthouse ticket in Rails which you can read here. +1 me if you agree.

Software Craftsmanship > Certification

There's been a lot of discussion about Agile Certification, a Rails Maturity Model, and some such. Are we always doomed to repeat the mistakes of the past? Remember the MS/Java/Sun/Linux certification craze of a few years ago? Yeah, same thing.

I did some hiring back then. I'd float right over any resume that prominently listed a big set of certifications because I thought people that misguided couldn't have a clue as to how real software systems are built. Mostly I think I was right. The first resume I see sporting a Rails or Agile certification is going to get framed and put on The Wall of Shame. I don't have a Wall of Shame, but I'll create one just for it. I promise.

For those of you coming into this industry, coming into Agile or Rails. Or for those somewhat experienced fellows looking for what to do next. Certification is not the answer. Let me now bestow upon you the few truths I know about this:

First. I can tell you with much certainty that there is only one way to learn how to build great software: from a master. I read this Chinese proverb once and it stuck with me:

A single conversation with a wise man is better than ten years of study.

A bit of an exaggeration, I know, but truth none the less. Nothing compares to working day in and day out with a master of the craft. Literature, conferences, training. They're all good places to start, but they are not enough. You need a mentor, the best one you can find.

Second. It will take you years of hard work to become generally competent. I've been at it for about seven years now and I feel as if I've climbed to stand upon one great peak of knowledge only to look down on the vastness of my ignorance. Yes that was as bit dramatic. I apologize.

Third. The only way to gauge the true skill of a craftsman (that's you) or organization is by their work and the opinions of their colleges. Not where you got your graduate degree, especially not the number of certifications pinned to your wall.

Now that I have you convinced. What can you do? How can you apprentice yourself to a master?

Most are not terribly hard to find. They write lots of books, they have blogs, they speak at conferences. Find one who's style resonates with you, then figure out how to get your foot in the door wherever their opinions hold sway. If it does not work out, find someone else and try again. And again.

If you wanted to get your foot in the door with us, for instance, you can undertake an apprenticeship. Otherwise, There are a lot of great places to learn this craft beyond our doors. I know this list is short, but here's the few companies I can recommend (mostly in the Chicago area): Obtiva, 8th light, Atomic Object, Object Mentor, ThoughtWorks.

Join the Software Craftsmanship community!

Interested in Software Craftsmanship but haven't found a good peer group to share your thoughts with? Then this google group is for you.

http://groups.google.com/group/software_craftsmanship/

Watch Paul Graham write an essay

Neat technology. It is fascinating to see how often phrases are re-worded and paragraphs completely restructured. The essay isn't half bad either.

http://etherpad.com/ep/pad/slider/13sentences