Skip to content

The Locality of Code Changes

Wednesday, June 24, 2009

The probability that you will change a piece of code in the near future increases when you make changes to that code or to code in its vicinity.

Recently, we were about to make changes to parts of a system that hadn’t been changed in about ten years. When we discussed how to approach the change, the argument that we probably wouldn’t change that code again in the foreseeable future came up. So why should we bother to refactor it and make sure it was testable? The cost for refactoring that code was deemed extremely high, and was an investment that would not pay off any time soon.

We opted for the quick fix, and made as minor changes to the code as possible. The reasons for not touching the code more than necessary seemed overwhelming.

Well, the following weeks we found ourselves making changes to that code over and over again. While testing that part of the code, we found that we needed a few more changes after all. One change led to another, and it was like we’d brought that code back to life, and now had to take care of it. Since we did not invest much in code quality at any stage as “this is the last change in ten years – promise”, the cost for each change was very high.

I don’t think this is an uncommon phenomenon. My theory is that once you make a change to a file, the probability that you’ll revisit that file increases. Whenever you make changes to a piece of code you start testing it. When you start testing it you might find things that needs to be fixed. When your customers see your new functionality they might come up with new suggestions. All this leads to new changes to the code and the process starts all over again.

Finding Some Data

To test this theory out, I created a small ruby application that reads a subversion log. For each path in the log, it records the dates that path has been changed. It then records the time between each consecutive change to each file and counts the number of times the different intervals are found. This is put into a histogram with each bar representing a week. If the time between two consecutive changes to a file is two and a half week, the bin in the histogram that represents two week intervals is increased by one. Files that only occurs once in the log are discarded as they have only been added to the repository, but never been worked on (this might be an oversight on my part, but I don’t think so).

Armed with the histogram, we can plot the probability density of the time interval between changes to a file.

When I ran this against our subversion repository as well as two open source repositories (gcc and python), the resulting histograms showed similar characteristics (the histogram below shows the results for gcc).  The bar on the far left shows the amount of changes that occurred to files within a week. The bar on the far right represents the sum of all changes with over a year between them. This bar represents an interval well over a week, so it should probably be represented lower and wider bar.

Histogram for gcc svn log

Histogram for gcc svn log

The data from these repositories shows a 50% chance that a file will be revisited within a 4-5 week period, and around a 30 % chance that the file will be revisited within the same week.

Implications

So, if we’re correct in our assumption that each change to a file increases the chance of us revisiting it, might that affect how we go about and make those changes?

I think that one thing this shows us is that the probability that an investment in code quality made now will pay off in the near future is quite high. Invest a bit each time you change something and chances are that you’ll reap the benefits of that investment within a few weeks.

I think it also shows that you might get away with smaller refactorings at each time you change code than you might think. As you’ll probably work on the same code in the near future, you can opt to spread that investment in code quality over a couple of times rather than doing it all at once. Hence you can strike a balance between making code better and moving forward.

In our particular case with the ten year old corpse brought back to life, the rationale for the quick fixes was that since we won’t touch this code often, it does not matter that much if each change costs a bit more than usual. Besides, it would mean a lot of work to make code better and we saw little benefit from that work in the near future.

What in fact happened was that we ended up with quite a few changes to the code that were each quite hard because no investment was made to make subsequent changes easier. The accumulated cost was quite high. Had we from the beginning worked under the assumption that we would revisit this code regularly, things might have gone differently.

We hopefully learned our lesson.

Filed in Uncategorized | | Comments (12)

Begin with the End in Mind

Sunday, June 7, 2009

To have a software development process that focuses on continually delivering value, we need to take a look at the process from the perspective of a feature/user story. What happens to an idea as it first pops up in someones head and then gradually transforms into something that can be used to solve real, concrete problems? What steps does it go through? How long does it lay around and wait without anyone working on it? How much time does it on average take for a single idea to flow through the system?

With this perspective it comes more natural to look at how the process of delivering value works as a whole. That way we might avoid the trap of just looking at part of the process and ignoring the rest.

We did an exercise last week where we attempted to draw a map of what kind of work needs to be done in order to deliver an idea to our customers. We started by drawing a large light bulb to the far left of a whiteboard. “Ok, here is an idea in someones head. What shall he or she do with it?”

This sparked a lot of great discussions about responsibilities in the organization and how we decide which features makes it into a release and which don’t. Good discussions, but we had a hard time focusing them. When the time was up, our idea hadn’t made it far on its travel on the whiteboard.

When we picked up the session the next day, we did it a bit differently. Instead of starting the discussions at the beginning of the process, we started at the end. We started out by drawing a happy customer that had just received a new cool feature to the far right on a whiteboard. Then we asked ourselves -

“What had to happen just before the customer received this piece of functionality?”

“Well, we need to put the functionality up on our server for automatically updating the software.”

“What do we need to do to accomplish that?”

“Well, we…”

And so on.

The discussions seemed to become more focused as we began with the end in mind and walking backwards one step at the time. It was kind of a pull system for our discussions, pulling new topics to be discussed as the meeting progressed. We made enough progress in that second session to begin creating our first version of our kanban board.

Yay!

Filed in Uncategorized | | Comments (4)

New New Beginnings

Wednesday, April 22, 2009

Today – Marcus, Andrés, Chris, and myself decided to close down Blueplane. We’ve had a great run. We’ve had the opportunity to work with a lot of smart people, and we’ve learned a lot in the process.

Thanks to all of you who invited us in to your organizations to create better ways of working, and to all of you who joined our seminars and made them a challenging and fun experience.

Right now, I don’t know what I will do next but when I do I’ll let you know.

Now, all of Blueplane is off to drown our sorrows with a couple of beers.

Cheers!

Filed in Uncategorized | | Comments (2)

Goal-Oriented Daily Stand-Ups

Thursday, February 19, 2009

The most widely used format for the daily scrum, or daily stand-up, is having everyone answer three questions:

  1. What did you do yesterday?
  2. What will you do today?
  3. What is blocking progress?

For a description of the rationale behind using these questions, see this article by Jeff Sutherland.

In addition to these questions, I like to add the following:

  1. What is the goal we as a team have for today?
  2. How did we as a team do on our goal for yesterday?
  3. What, if anything, hindered us as a team from reaching that goal?

First, these questions look at what we as a team wants to accomplish. This helps us focus the individual tasks towards a common goal for that day.

Second, these questions sets up goals as opposed to just focusing on what tasks to work on. This gives the tasks we work on a more meaningful context. When we strive for goals, and reach those goals together, we share a sense of accomplishment. That helps in building a team.

Third, as we look at impediments that hinders us collectively, every team member gets a better understanding of what our major impediments are, and how it affects our progress.

Filed in Agile | | Comments (0)

There Can Be Only One?

Tuesday, February 17, 2009

It’s not easy being a product owner. For a lot of you out there it’s a lonely job. It’s also a job that brings with it a lot of responsibility. You are responsible for the vision and the profitability of the product.

That’s not something to be taken lightly.

A recent discussion on the Scrum Development List on whether it was advisable to have more than one product owner on a project or not, got me thinking. The prevalent view seems to be that there should only be one product owner. What is required of that person?

Vision

As a PO, you are responsible for the product vision and for making sure that it’s communicated to all project stakeholders.

Balance

A big part of it is about balance. You have to balance the demands from the other stakeholders:

  • Different customers have different ideas about what absolutely has to be in the product – now!
  • The team requests to work on esoteric parts of the system that won’t result in any visible new functionality, otherwise all development will come to a complete stop any day now
  • Your own organization holds you accountable for releasing something that generates revenue – ASAP!

Answers

You’re also supposed to be able to answer questions ranging from minute details about how the users want their data displayed, and whether the credit card payment functionality has higher priority than switching to version 5.2 of the OR-mapper (whatever that is), to answering the more general question from management “So, how’s that project of yours coming along?”.

So, either you put on your superhero costume, cape and all, and do it all yourself, or you better start finding ways to collaborate with the people around you.

There Can Be Only One?

When an organization is at the beginning of introducing agile software development, it might be a good idea to have one product owner. Chances are that the constraint of the organization is the team’s ability to use the agile engineering practices.

But as the team gets more experience, it will eventually cease to be the constraint. It is highly likely that the new constraint is the single product owner.

That’s when you need to review the responsibilities of your PO. Which are the core ones? Can some of the non-core responsibilities be delegated to someone else?

Perhaps you’ll find that your PO has more responsibilities than can be handled by one person.

For another perspective on the product owner role, you might wanna check out Rich Mironov’s article, Revenue Products need Product Managers, not Product Owners

Filed in Agile | | Comments (0)

Agile Planning at Øredev 2008

Friday, January 9, 2009

My talk about agile planning at Øredev 2008 is now available here.

Filed in Agile | | Comments (0)

Why Estimate When You Can Sort?

Saturday, November 8, 2008

Chris and I have been helping a team introduce agile software development for a while now. When we are running a planning meeting, we have usually used planning poker for estimation. But we seem to be using planning poker less and less.

I like planning poker. It has a lot of advantages. In particular it helps with giving everyone on the team a voice. If you make sure to have everyone show their estimate before you begin discussing the details, you’ll stand a better chance of getting estimates that are not biased by the opinions of the stronger and louder team members. Brilliant!

There are some shortcomings to planning poker if you don’t have a good facilitator though.

It is easy to lose track of the relative nature of estimates. As you are addressing each story one by one it is quite easy for teams to forget to compare it to the other stories. This leads to uneccessary discussions about details that really aren’t relevant at this point.

As an alternative to using planning poker, you might want to try a sorting exercise. Instead of trying to attach an estimate to a story directly, we first have the team sort index cards. The smallest/easiest/least complex on the far left, and the largest/hardest/most complext on the far right. Don’t think in terms of points, ideal days, gummy bears, or whatever estimation unit you’re using. Just sort!

When that part of the exercise is complete, you have all the story cards lined up. Next, have the team think of the possible story points as buckets. We usually use 1, 2, 3, 5, and 8. The mission is to decide what cards belong in what bucket.

Begin with the leftmost card and ask “Is this a 1?” Have all team members do a thumb up or thumb down. If all agree, continue with the next card and repeat until you get a thumbs down. Have the team quickly negotiate which cards make in to the first bucket and which don’t. Continue the same procedures with the next estimation bucket, and repeat until you got an estimate for all cards.

Oh, and in order to keep bias from the more vocal team members to a minimum – have the team perform the sorting in silence.

For more information about this, check out this post by Kane Mar.

Filed in Agile | | Comments (1)

Daily Stand-Ups

Thursday, June 26, 2008

Gears
If you’re doing any kind of agile software development, you’re probably having some form of daily stand-up. In my experience most of the daily stand-ups out there are, quite frankly, pretty uninspiring things. A lot of meetings are spent mechanically reporting status to a project manager or discussing details of interest to only a few.

Like all practices, you need to constantly keep in mind the underlying principles and values behind the daily stand-up to keep it from deteriorating. Here are a few things to consider so that you don’t end up with just another meeting that people attend just because they have to.

Keep the Goal in Mind

The purpose of the daily stand-up is to give the team a good sense of how it’s doing on the commitment it made for the current iteration. Is someone stuck somewhere? Are things taking longer than we expected? Do we need to drop things to be able to deliver a coherent product increment?

This implies an obvious prerequisite – you need to have a goal for the iteration and a commitment to reach it. That goal is set at the iteration planning meeting. A commitment to reach it is created by giving the team responsibility to decide how many user stories they can implement, no one else.

It’s for the Team

It’s the team’s job to keep track of how they’re doing on their commitment. A daily stand-up that is run by a few persons where a major part of the team members shows up to passively attend is of limited use. Meetings that consists of one way communication from the project manager to the team, or from the team to the project manager, do not help create an environment where the team actively owns and cares for it’s project.

In a working stand-up, the team members address each other, reminding themselves of what’s currently on the agenda, and everyone is participating to the best of their ability.

If you are in a lead position in the team, either formally or informally, consider taking a step back and see what happens.

Discuss Elsewhere

The daily stand-up is not for solving problems. It’s about sharing information. If someone thinks he or she has input on a problem reported by another team member, the stand-up is the time to make the others aware of that and schedule a discussion afterwards. Anyone who has input may join.

If you allow too much discussion during the stand-up, there’s a risk that these take up most of the meeting. Chances are that some has nothing to add and resort to passively await the end. We don’t want to waste the time of the team members, so keep discussions to a minimum.

One tool that might help you get a more organized stand-up is to use a token. This a small physical item that is passed between the team members during the meeting. The person currently holding the token is the one that’s currently allowed to speak.

Come Prepared

Daily stand-ups are intended to be short and effective. If people don’t come prepared, time will be wasted when members try to remember what happened since last time. Make sure that you think through what you want to bring up on the next meeting. What you have accomplished and how much work is left. Having done that you can focus on what is being said by the other team members and get something out of the stand-up. If you focus on what to say when the token comes your way, you will miss what the others have to say.

Accomplishments, Not Activities

Focus on sharing what you accomplished since the last stand-up rather than reporting what you have been working on. For one thing, talking about accomplishments made provides a more positive attitude than just providing information on what parts of the system was worked on.

When the rest of the team hear about these micro-increments it has a tendency to trigger new activities. “Now that we can do X, I might actually begin doing those performance tests that I have been putting off for far too long.”

Make Commitments

Have team members bring up what they plan to have accomplished at the next stand-up. This way everyone makes a commitment to the rest of the team. The team can then compare the commitments they’ve made since last time to what was actually accomplished. That way they’re at a better position to find if there’s any problems or opportunities that needs to be addressed.

If members mechanically bring up what they worked on since last time instead, the team risks missing if there is anything that could potentially hinder their progress.

Bring up Impediments

In my experience, developers can take a lot without complaining. I’ve seen developers sigh and pull their hair while making changes to 60 files just to add a seemingly trivial thing to the code base. Still, at the next stand-up, they don’t report any problems.

If we don’t bring up problems, we will never be in a position to fix them. Hidden problems will affect quality, velocity, and our chances to reach our commitments in general. So we need to breed a culture where it’s expected of everyone to constructively bring up impediments before they get out of hand.

Conclusion

Daily stand-ups is a great tool for keeping a team in sync. When the stand-up works well everybody leaves it with a good sense of what they’re focus for the day is, what the others are working on, and how that connects to the team’s goal. Everybody should leave that meeting invigorated and ready to get going.

In the complex environment that is software development, you will always need to make adjustments to your plan. The daily stand-up is one of the best tools for this.

So, remember why you are there!

Filed in Agile | | Comments (0)

New Beginnings

Tuesday, April 8, 2008

It’s official. I am now part of Blueplane. Blueplane is a consulting firm that specializes in helping organizations become more efficient by applying what we’ve learnt about agile software development over the years. We help teams by coaching them on how to use sound engineering practices, find better ways to run projects, and start communicating more efficiently.

I’m in this together with Andrés Taylor, Chris Hedgate, and Marcus Widerberg. I know I will learn a lot from those guys. They’re very sharp, they have loads of experience, and they’re just plain fun to be around.

If you’re situated in southern Sweden, don’t hesitate to contact us if you think your organization could use a vitamin injection.

Blueplane Business Card

Filed in Misc | | Comments (0)

Anchoring

Thursday, December 13, 2007

Anchors

Anchoring and adjustment is a psychological heuristic that influences the way people intuitively assess probabilities. According to this heuristic, people start with an implicitly suggested reference point (the “anchor”) and make adjustments to it to reach their estimate.

– Wikipedia

The team managing the project portfolio in a large enterprise need to come up with a rough estimate for when projects can start and what their duration might be. The master plan for all projects in the portfolio may span more than a years worth of work. This is ok. Everybody usually understand that these are extremely rough estimates that are needed to get some understanding of what type of resources are needed when and so on. No harm no foul.

When a team is assembled to work on one of those projects they are usually asked to come up with a new release plan. A new estimate of the duration of the project. However, when doing that the team tend to, unintentionally, steer towards the initial rough estimate in the project portfolio. Even if that estimate, if viewed objectively, is way off. This is anchoring. A seed has been planted in the minds of the team members that can be quite hard to get rid of.

A team is having a sprint/iteration planning meeting where the team looks at the product backlog to determine how many of the most prioritized items they can commit to. The product owner or project manager says in passing: “I think we should be able to get at least these four items into the iteration, but it is you who decide”. This is anchoring. Once the number four is introduced into the meeting that will act as an anchor. It will make it hard for the team to commit to just one item or even twenty items now that the number four is readily in their minds.

It is really important to be aware of when anchoring comes into play. Both when you are anchored by someone else and when you are about to provide the anchor your self.

An excellent example of a practice that addresses anchoring is planning poker. When not using it there will usually be one or two members that are always quick to come up with an estimate for the task at hand. It will be hard for the rest of the team members to come up with something radically different once the initial anchor is set. With planning poker everybody selects a card with an estimate on it and then show it to each other at the same time. This way you can enter the discussion without being anchored. This will hopefully lead to better decision making.

Filed in Agile | | Comments (0)