Fun development

Several things came together today, and though not evident at first, they are all somewhat related.

FIRST

My pair kept asking why I was so adamant about not writing any code without specifying it first. I explained how specifying kept us focused on what we needed to do, helped us drive out the overall design, and eventually provided a safety net for testing and refactoring. In particular, we were practicing outside-in development in Rails, using Cucumber and RSpec. The goals we were achieving, in order, were:

1. understanding our problem
2. designing and developing our solution
3. building up tests for the future.

The cucumber scenarios help us focus on understanding the requirements, from the end-user's point of view. They also drive out the high-level interface of the app, particularly the URL's. The URL's then lead us to create routes, and the routes lead us to create controllers. The cucumber scenarios also drive out any expectations we may have on any models, which the controller will usually set as instance variables.

Once we know we need a particular controller, we jump in to specify the controller, and the detailed behavior we expect there. Fleshing out the controller may drive out any additional functionality needed on the model. Finally, we spec out the model. At this point, we work through any additional functionality in the cucumber scenario. Once all scenarios and specs are passing, and we can't think of any more to write, we're done.

Obviously, my description is quite simplified. There is alot of subtlety here. That's why I suggested that I send some links to some good videos online, which talk about outside in development, specification, integration tests, and design in general. My friend Tom had done something similar earlier this year.

Here is my list, in suggested viewing order. The first is fairly concrete, and then they get more abstract and theoretical.

1. Joseph Wilk's Working Outside-in with Cucumber and RSpec
2. J.B. Rainsbergers Integration Tests are a Scam
3. Michael Feathers' The Deep Synergy Between Testability and Good Design

SECOND

On the way home, I saw that my coworker Mike tweeted about a blog post he wrote today. This struck a chord with me, especially when I got to the end. The entire article rings true in the Enterprise Java space as it does in .NET. Then he clinched it with this:

If I do ever go back to working on a .NET project, I'll bring a lot of new knowledge and ideas that will likely make that a more enjoyable experience.

Over the last few months, I've been having tremendous amounts of fun developing with Cucumber, RSpec and Rails. There seems to be a huge sweet spot, where it's very easy to get into the flow. You start writing the scenarios and specifications, and the application starts to write itself.

I had practiced acceptance testing and unit testing in the java world, and it was alot of hard work! With these tools, it's great fun, and it practically does the work for you. There is still alot of thinking involved, but it is much more focused on the problem domain.

THIRD

Andy talked about a powerful technique to write and grow your specs. In addition, he talks about how the developer specifications, when printed up pretty, made great documentation for a non-technical client. Consider how much stronger his collaboration was because he could share his specifications with his client. Shared language and shared understanding lead to better communication and increased collaboration.

FINALLY

Syntactically, the Ruby language is not very large. You can start reading ruby code very quickly. And, you can read most any ruby code that you want. As an interpreted language, all libraries are shipped in very readable files. In addition, the community seems very open to keeping things open and readable. I have yet to see any library that is obfuscated.

Which brings us to Colin, and his tweet at the end of the day:

pro-tip: every week read up on a new class in the #ruby stdlib. i don't think enough people know what's available to them.

You could wield alot of power when you are given the chance to understand every piece of every library that you are using. This power comes in the form of understanding what you're using.

TAKEAWAY

Learn powerful tools and techniques. Read as much code as you can. Understand the guts of what you're working with. Go out and have fun developing!!!

Your days will be alot more enjoyable, and your work will be alot more professional.

A Perfect Programming Day

I had what I would consider a nearly perfect programming day today. It really started yesterday before I left, when Jim (tech lead) reviewed an upcoming task with me. I was in the middle of another task, so I suggested we pick up in the morning.

The api team was processing a product file, and we needed to adapt that process so that the web team could also consume it. The next day, Jim and I reviewed the current implementation of the process, as well as the goals of the updates. The process took a file, filtered and translated it, and created a new file. We wanted to add to that process, to create a second file to be used by the web team.

Our BA came by to discuss the details of the fields and the formats we would be getting, as well as how we wanted to filter the records. We all agreed to an approach. Our BA would do the write-up for the sake of sharing the understanding with QA and other team members. We also agreed not to wait for a detailed write-up before starting development.

I was going to work with Matt, who had written the initial process. When Matt arrived, he asked what we needed to do. I gave him an overview of what I understood. He immediately knew where to jump into the code and start. We were working at a station that had not been used for this project before, so first we synched with source control. Then we ran the test suite. After installing a couple of gem dependencies, the tests passed.

Matt showed me the classes involved in the process, and gave me a detailed walk through of how the input was filtered and transformed. This took about 10 minutes. We then talked about what parts were common and could be re-used, and what would change to support the creation of a second file. At that point, instead of digging in, we decided to refactor the current implementation. This would allow us to more easily add the code to create the second file.

This process consisted of two classes: a Processor and a Line. The Processor looped through source file, passed each line to a Line instance to be filtered and transformed, and wrote the output file. The Line class was responsible for the actual filtering and transforming.

We spent a few minutes creating a superclass, and moving common processing up into the superclass. We ran all tests, and they passed. We didn't change any tests along the way. We only changed the implementation by introducing a superclass. This was pure refactoring. We were ready for an initial local commit.

We spent a few minutes trying different names for the superclass and the subclass with the current implementation. Initially, the Line class was called SocLine, named after the file which was being filtered and transformed. We tried Line for the superclass, but that was too generic. We finally went with ProductFileLine as the superclass, representing the source of the data, and ApiLine for the subclass.

In afterthought, these classes' responsibilities were filtering and transforming. Maybe better names would have been ProductFileLineTransformer and ApiLineTransformer.

Next, we looked at the code that was looping through the input file, filtering and transforming, and creating the output file. We added acceptance tests to check that a second file was being created. We modified the loop to create a second file, and our tests passed.

Now we wanted to implement the new filtering and transformation for the web file. This we decided would be better served through unit tests. First we wrote the specs for the new filtering rules, and then the code to get those to pass. Then we wrote the specs to transform each line to the correct format.

At this point, we were done with the happy path for this story. What remained were a couple of specific transformations, and any final cleanup. We also wanted to look at an example input file for any special data considerations. We continued after lunch.

We transformed several columns to change their data types from text flags to booleans. Our specs failed as expected. We updated them to pass, and continued.

We reviewed the order of the data we were exporting, and arranged the fields in a more logical order. We also reviewed the BA's write-up (which was done at this point) for anything that we may have missed. We identified a small item, fixed it, and continued.

At this point, we squashed our local commits down to one, pushed our changes up to source control, and called it a day.

Why perfect?

The obvious point is that the day flew by, and it was incredibly fun. But there is more to it than just fun.

Working with Jim and our BA, we were able to start development quickly with a minimal amount of ceremony. I was able to start developing while the BA was detailing the story. At the end of the day, we were able to do a final check before moving the story to QA. We went with a light a process as we could, and we worked in parallel once we had a good common understanding.

We never strayed too far from all tests passing. We kept our changes small. We worked in small increments, and committed (locally) whenever we were at a stable point. Though there was no need today, had we needed to go back and take a different approach, it would have been as easy as reverting a commit or two.

The code that we started with was already well-factored. This made it easy to make our changes.

There were tests in place when we started, so that refactoring was safe. When we finished refactoring, we were quite confident that we hadn't broken anything.

Finally, we were pairing. Both Matt and I were at the keyboard, and taking turns driving and thinking throughout the episode.

PS: Towards a stronger design

The extension to the design that we did today was sufficient. We could have taken it further, having our Processor class follow the Open-Close Principle. That way, it's core functionality would not need to change when we needed to add another output file. We would simply need to add a new Line class with it's own filters and transformations.