laststation.net

Sustaining a 14-day Release Cycle

We talk about it all the time - agility. At GoodData it’s in our DNA. We understand being agile is not necessarily easy. You need the right people, attitude and tools. And exactly as we’re trying to deliver the GoodData platform to help our customers bring agility into a BI arena, we need the very same tooling internally to sustain the pace of a 14-day release cycle. We bring our customers new features every two weeks - unheard of in the stodgy world of BI and analytics.

Let’s get some perspective since 14 days might not sound so difficult for agile developers. How do you stay confident that all 2,713 data marts, 1,344 dashboards and 19,086 reports (our internal count as of June 2010) are valid from release to release? For a long time, we performed very expensive and time consuming validation directly with our customers. But how many reports you can actually test? Where does the attention to every single detail end and become a boring chore?

It turns out it’s at the second report you have to validate. Then you have to add security and privacy concerns. We motivate our customer to be as self-service as possible, so why we should destroy this experience by a need to manual verification? GoodData is about the Cloud. And the cloud need automation to scale.

A few weeks back, we quietly and successfully deployed our Regression Testing Toolkit. Its purpose is simple: take two different code branches and select data mart(s) you want to compare and bang - results. All anonymized, with no data being exposed, and still providing enough information for our engineers to fix the possible problems. For me and you, the results might look a bit sci-fi but engineers love them, which is critical if they need to resolve the problems as fast as possible.

Here’s what it looks like:

For the rest of us, the final OK message is pretty much what we are looking for.

Then there are the unexpected benefits.

As the devops supporter, I’m glad we found a way to get our engineers more engaged in our production environment. We got also a new performance tracking tool as a by-product. For every single report, and as aggregate for whole data mart, we now have a runtime value that is easy to compare across the releases or different environment. That’s what I call getting the good business value from our engineering team!

No matter how passionate we’re about this tool, there’s more work needed. And hopefully soon you will be able to see validation of your data and model directly. No delays. Enable you to iterate through the changes in your data model faster.

Building confidence and trust for on-demand solution is a long run. Tools like our Regression Testing Toolkit helps us to go that extra mile and me, an ops guy, help to sleep better…

5 Hours A Week

Priorities do change, but old habits die hard. The luxury of having more than 10 or 15 hours a week for my personal software projects is now history. To keep up means to adapt. In last 6 months I’ve embarked on a personal quest to protect the last development activities and to be able to stay up-to-date with the latest technologies. This is a practical review of what I’ve found out and what works for me (so far).

Plan ahead the software project you’re working on. With the right milestones it’s easy to track the progress and avoid unnecessary distractions. There are hundreds of ideas you can always implement later. But somebody has to use the software first. I started to make a list and every time a feature comes to my mind as something useful I do increase it’s planning priority. Review it once a week, during some dull activity (like daily commute to work), to asses if the features have real benefit for the productivity or project effectiveness - reprioritize accordingly. In most cases 99% of features end up on ‘one day’ list.

Pick the right tools, because it’s nothing more annoying than to wait 10 minutes for Maven to download all the packages, and 3 minutes for your container to start/refresh application. Actually, in last year or so, I’ve completely abandoned Java as a platform and after some surprising turnaround started using Ruby. While working on a automated processing backend for Le Bebe (czech only; website is run by somebody else) I’ve finally understood the real power of Rails. Now I’m starting to be Ruby addict.

Don’t do something you can buy somewhere else for a good price. Going back to the Le Bebe, I’ve only written the engine to automate the sales handling from a different sales channels. E-Commerce solution is nowadays so cheap, it’s better to pay $30 per month or so and get it as a service. Even when using Rails, I can’t really say I would be sane to value hour of my life for about $1.50 - given the service fee $30 per month / (4 weeks * 5 hours I have a week).

Document, Backup, Version everything. Yes, Planning for 5 hours a week software development activities can be challenging and in some cases I don’t get back to the project for as long as 2 - 3 weeks. With all what’s going on with my day job, I’m effectively starting to loose the track of the details after couple of days. Only now I’ve learned to document all my commits (Mercurial and BitBucket) properly, and document every single decision (Google Documents).