Adam Bowen

The Path to Agility Part 11: Yet Another Retro

2017-05-09T00:00:00+01:00

More retrospective! This time I wrote up my retrospective plan before the retrospective. Read on to find out how it went…

Setting the stage

This time I didn’t go through the sprint before we began, in the last two retrospectives I’ve written up I said I felt that going through the content of the sprint in a rather rote fashion was dry, unengaging and ultimately unhelpful. No-one noticed, and I don’t think the retrospective was any worse for it.

I also set a focus - we did a more in depth planning than normal for this sprint because we had some tricky commitments, and I wanted to make sure we had a plan to meet them. Since we did something a bit different, I thought it would make a nice focus. This part I don’t think I got right, there wasn’t a noticeable change in the type of information we gathered, the insights, or the actions. I didn’t steer the retrospective back toward the focus because there was nothing wrong with the discussions the team were having and - crucially - I the team should own the retrospective - if they don’t agree with my focus that’s on me, not on them.

Finally, I used the “one word” activity to get everyone involved. Hyphenation became popular this time, so in the future I think allowing up to three words would be helpful. I wrote all the words on a board, as this time I decided to follow each activity with Derby and Larson’s ¹ debriefing questions. Upon starting the debriefing, I immediately hit a problem - it wasn’t clear whether we were trying to analyse the activity, or it’s content. I’m pretty sure the intent is to debrief the content, but the team (and - initially

myself) found this confusing. I forgot to take a picture of the words.

Gather data

To gather data I used the timeline activity again, only this time I just asked for facts and not feelings. I find the timeline activity regularly useful, I often see something on there that I wasn’t aware of because my main interaction with the team is during the daily stand-ups. This time was no exception as several people mentioned changing requirements.

Generate insights

For this step we used “learning matrix”. The learning matrix consists of four quadrants, labelled :), :(, idea, and appreciate. The idea is to get the team to examine the data on the timeline and add insights to the quadrants.

Looking back, the end result was decent - we managed to get a good set of ideas, good, and bad points - and they were a good distillation of the timeline. However, there were a couple of problems. Firstly, it took a while to get going, as the team gets used to the activity I expect it will get better - but this first time there was a long period of silence. Secondly, there were a lot of appreciations - I like giving the team an opportunity to recognise the things they do for one another, but in order for the activity to be effective we do need a balance between the four quadrants.

Having written some thoughts in each quadrants we then used “dot voting” to pick out items we thought worthy of action. This immediately leads to a third problem - two quadrants aren’t generating useful insights to take forward. It’s fairly obvious that the :) and appreciate quadrants have other benefits, but toward the end of this activity we need to make sure we bring the focus back to things we can act on.

Decide what to do

As I’ve mentioned in previous retrospectives, I’d noticed a trend to not follow through on retrospective actions - so this time I used an activity designed to generate SMART goals. SMART goals are intended to be easier to understand and measure than less well defined objectives, in turn this is supposed to make them easier to follow through on. The acronym stands for:

Specific
Measurable
Attainable
Relevant
Timely

For this activity I split the team into groups, and asked each group to create a SMART objective for one of the most highly voted items on our learning matrix.

This activity was partially successful, with the success dictated mostly by how well the teams spent their time creating their objectives - one of the teams finished fairly quickly, and gave a reasonable objective, while one of the teams spent a lot of time discussing the problem without creating a firm objective.

I think if I were going to do this activity again I would try to structure the creation of the goals a bit more to help those unfamiliar with goal setting get started.

I don’t have a picture, but here are a couple of the goals we set

At the start of each sprint someone on the team will review each of the requirements of the stories in the sprint and clarify any that are not clear or up to date. We will count the number of requirements that change during the sprint and revisit this activity in 2 sprints time.

Prior to each sprint, someone from the test team will review the requirements for the stories planned for the next sprint. They will use their knowledge to identify any missing requirements and add them to the stories. We will count the number of requirements that are missing during the sprint and revisit this activity in 2 sprints time.

Both of these activities centre around requirements changing during the sprint, of course, one of the Agile principles is that we “welcome changing requirements, even late in development” but we are concerned with identifying easily avoidable changes in the requirements. The first activity centres around when our requirements have not been updated - often there is discussion about stories during the planning meeting, and some of this discussion doesn’t get written up. This first objective compels the team to spend some time making sure the stories are clear and up to date.

The second objective comes from problems identified during testing, we test during the sprint, with the test team reviewing requirements early in the sprint and testing development work as soon as it is in a suitable state - however, we have found that the test team are often adding requirements once they begin testing, indicating that we could be doing a better job of identifying corner cases and other problems earlier in the development of a story. This objective compels the team to spend some time seeking missing requirements before the sprint has been planned.

Close the retrospective

To close the retrospective I used a different activity this time, I used “return on time invested” to gauge how well I had used the team’s time.

The basic idea is to ask the team how well they think their time was spent, and then ask why they chose the answers they did. I was well over time at this point, so I kept it fairly short. For the most part the team seemed to feel like the time was moderately well spent, with a few individuals deriving little benefit (mainly because of how their roles interact with the development team).

Agile Retrospectives: Making Good Teams Great by Esther Derby and Diana Larson ↩

The Path to Agility Part 10: Another Retro

2017-05-01T00:00:00+01:00

Continuing the theme of real retrospective examples, this post is another example of a real retrospective. These posts lag somewhat behind my team’s actual work, so please forgive me if I appear to have failed to learn any lessons highlighted in the last post!

Setting the stage

I opened this retrospective by listing the stories we completed (and those we didn’t), showed the burndown, and our velocity over the last few sprints. I also summarised the actions we’d decided on in the last few retrospectives.

As with the previous retrospective I was trying to refresh the team’s memory about the last sprint, and trying to set an implicit focus on the team’s velocity (the recent trend had been in the wrong direction, and I wanted to see if there was anything underlying that).

As before, I found that the opening was not very engaging - and therefore I suspect that it didn’t serve to refresh the team’s memory as well as it could have done.

I again used plans-for-retrospectives.com to generate some activities, then tweaked them so that I could get a clean flow throughout the retrospective. This time I used an activity called “Last Retro’s Actions Table” to set the stage. I’d noticed a tendency to creation actions without following through, and I wanted to help focus the retrospective back on what we’d previously decided.

I wrote down our previous commitments, and added columns labelled ‘More of’, ‘Keep doing’, ‘Less of’ and ‘Stop doing’. I got everyone to vote on which column they felt each action belonged in. For almost all the actions the consensus was wither “keep doing” or “more of”, so I went through the list and tried to get the team to suggest ways we could follow through on the actions we weren’t doing. When we felt that an action had become “part of our day-to-day” we removed it from the board.

Overall this was a pretty useful activity, and I can easily imagine building a retrospective where this was part of the “gather data” phase.

Gather data

For gathering data I used “Analyze Stories”. Writing this up several weeks after we did the retrospective I remember nothing about this activity…which probably means it wasn’t very useful. In essence, I went through each story and asked the team how they felt it went and how it could have gone better. This is a bit like the “three questions” I have discussed in the past, but helps to focus the team on the actual work.

Generate insights

I used an activity called “the worst we could do” to generate insights. The idea of this activity is to ask the team for ways we could make the next sprint a certain disaster. I picked it because it sounded fun, which may not be the best reason - but helps to liven up the retrospective at a time when it might be starting to lose energy.

Having gathered the team’s answers, I then flipped all the ideas around and looked to see what we were doing to prevent them from happening.

So…this activity…firstly, you need to think very carefully how you set this up - I just threw it out there and waited, but the team was (understandably) confused about what I was looking for. After all, “we just don’t come in to work” pretty much guarantees the sprint will fail, but doesn’t really help us find ways to improve. If I use this activity again I will have to think more carefully about how I focus the activity and the boundaries I set on answers.

On the plus side, once I started to get answers they were fairly easy to flip around and look at what we’re doing to prevent them (for example, “delete every other line of Javascript” was a suggestion, which is easily prevented with our code review and automated regression testing).

Decide what to do

Err…yeah…looking back at my notes I somehow didn’t really do this. I was trying to keep the retrospective on time, and some of the earlier activities did generate actions (in particular the very early focus on our outstanding actions).

Close the retrospective

I closed the retrospective with the same activity again - asking everyone for

One thing that pleased or surprised them this retrospective
One person they would like to acknowledge for something they did this iteration.

This was less helpful than the previous retrospective, but I still got the very interesting feedback that looking over our previous actions was a useful activity.

The Path to Agility Part 9: Our Retro

2017-04-25T00:00:00+01:00

In my last post I rambled on about how important retrospectives are. This blog is meant to be a real story of our adoption of agile, so this time I want to write about one of our actual retrospectives - and what I learned from it.

Setting the stage

I opened the retrospective with a summary of the stories we got done, and those we didn’t. The main aim was to help refresh everyone’s memory about what they’ve worked on, although so far I’m not sure this little opening has quite achieved the right result - in the future I might stick with something more brief to avoid spending the time unproductively.

Next, I gave everyone one minute to come up with one word to describe their sprint. This forces everyone to think back over what happened in the sprint and reduce their overall impression to a single word. Going round the room and getting everyone to read out their word encourages the whole team to participate in the retrospective. To save time I could have done this activity without the one minute thinking time, but my worry with that is that instead of listening, everyone who hadn’t spoken would instead be thinking of their word.

I’m pretty happy with how this turned out, and I was pleased only one person used “frustrating”. Even though the activity got a few funny looks when I described it, everyone managed to contribute. Having collected these answers I didn’t do anything with them, instead my hope was that the team would carry forward their thoughts to the gathering data phase.

Incidentally, our team structure lends itself to my participating in the retrospectives. While this goes very much against my role as a facilitator it’s also difficult to convince the team to take part in these weird activities if I don’t do so myself. My word was “opaque”, because I felt I wasn’t sufficiently engaged with the sprint to know what was going on throughout it.

Gather data

We didn’t have any particular focus this retrospective, so “gathering data” is a pretty important phase - it’s where we’ll figure out what areas need addressing. To do this I used an activity called “three wishes”, I gave everyone 5 minutes to come up with (and write down, on individual cards), three wishes:

one for themselves,
one for the team, and,
one for the company.

Wishing for more wishes is cheating (these are engineers, and working around limitations is basically in the job description so you’ve gotta watch them!)

By-and-large this worked ok, I got the obvious questions about wishing for faster cars and bigger houses - but for the most part we managed to keep the wishes “on topic” as it were. In hindsight, this is where failing to set a focus for the retrospective caused problems. Without the focus, it is difficult to limit the responses in activities like this to ones that are immediately useful.

After the time was up everyone read out their wishes, which in turn gave us a whole load of useful data on what was frustrating the team. Having got that data we needed to focus it down to a more manageable size - so we used dot voting to figure out which “wishes” were most widely appreciated. To dot vote you basically give every participant a fixed number of dots (I used three), and ask them to place them wherever they wish.

Generate insights

To generate insights I split the team up into groups of three, I asked each group to pick three highly voted cards and discuss why they think it came up and what they think we can do about it. I deliberately tried to create groups of people who don’t normally work together to allow ideas to spread to the whole team and build relationships. I gave everyone 15 minutes for this.

I don’t know what the name is for this activity, but it was probably my favourite. There was a lot of productive discussion between the various team members, and it was really nice to see even the quieter and newer team members participating. Having seen this I’m a lot more likely to pick activities that break up the team a little and help to avoid the one-speaking-eight-listening (if you’re lucky…) mode that feels like it could be more productive (let’s face it, anyone who’s done a computer science degree is familiar with how merge-sort is a better algorithm than bubble sort!)

Decide what to do

The tricky part with my insights activity was turning the productive discussion into real actions. For this I asked each group to come up with an action for each of their cards; I gave them 10 minutes.

I then got the groups to read out their actions, and we dot voted on the ones we were going to take. I took the top three actions, and asked for an ambassador who would try to ensure that we followed through on it. Our actions were:

Replace one of our ageing servers (or rather, come up with a specification for the replacement so that we could get one purchased).
Document a particularly tricky part of our system architecture.
Write more (or make it easier to) end-to-end tests.

Looking back I can see problems with these actions, firstly we didn’t allocate time for them in the next sprint - this made them difficult to follow through on. Secondly, we didn’t find a way to measure the effectiveness or set any limits on the activities to help the ambassadors focus the effort. At the time it felt like the group was starting to get tired and the quality of discussion was starting to dip - which is something I will have to look out for and address in the future.

Close the retrospective

I closed the retrospective with a simple activity - I asked everyone for

One thing that pleased or surprised them this retrospective
One person they would like to acknowledge for something they did this iteration.

A lot of the team struggled to name something that pleased or surprised them, but one memorable comment was that someone was “surprised that they enjoyed a retrospective”. That’s a really nice comment to hear, and a strong indication that changing our retrospective format was long overdue.

I really enjoyed the round of thanks, most people could name someone who had done something they appreciated this sprint - and I think creating opportunities for this kind of praise is really good for overall morale.

As much as I liked this activity, it’s not great at delivering useful feedback on the retrospective itself. I’d use it again, but at some point will need to change it to make sure our retrospectives are also continuously improving.

This retrospective was basically extracted from plans-for-retrospectives.com, but before you go there, hit random, and run your retrospective ten minutes later I suggest making sure the activities gel. In this retrospective I tried quite hard to make sure each activity acted as an input to the next. It was by no means perfect, but each activity needs to build on the previous one otherwise you’re basically just messing around for no reason. Productive retrospectives need to be tightly focussed on delivering value (actually, this is true of any kind of meeting - something more people would do well to remember) so every activity needs to contribute something. Please also learn from my mistakes, set a focus to keep people on task, find ways to measure the outcomes of your actions and make sure to allocate time for them!

The Path to Agility Part 8: Retro

2017-04-17T00:00:00+01:00

What’s the most important Scrum meeting? Is it the stand-up? That’s the most frequent meeting, the main synchronisation point for the team. Without the stand-up we might all end up working on the same thing, we might fail to collaborate on a story and end up with two mismatched halves, someone might be stuck and we would never know!

How about the demo? Without the demo how will our stakeholders know what we’ve done? How will we gather feedback on the product? How will we demonstrate working software?

Maybe it’s the planning? How will we know what’s valuable and what isn’t? When will we have the opportunity to clarify requirements? How can we make commitments to our stakeholders without it?

Does any of the above make sense to you? If so, then I’m sad to say, you’re mistaken. The most important Scrum meeting is the retrospective, for one simple reason, if anything is wrong with any of the above meetings, or any other aspects of your development process, the retrospective is your opportunity to fix it. Without this period of introspection, problems and impediments may go unaddressed for long periods of time or even indefinitely. The retrospective is also the only meeting that can fix itself if it’s broken. If you’re going to put all your effort into getting one meeting right, then put it all into getting the retrospective right.

It probably won’t surprise you to learn, then, that the retrospective is the only Scrum meeting directly codified in an Agile principle.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behaviour accordingly.

In fact, retrospectives are not just applicable to the Agile software world, they are useful wherever you’re looking to do continuous improvement.

How to retrospect

It’s the end of the sprint again, and there’s a meeting in the shared calendar for the 2 hour “retrospective” from 10am to 12pm. At five past ten the Scrum master has to go and remind everyone that they’re supposed to be in the meeting room now. Over the next ten minutes the team trickles in, perhaps with some protracted effort to dial in team members who are working remotely.

Eventually everyone is gathered around a large meeting table. The room is slightly too hot, because, as usual, the meeting room is really slightly too small for the team it accommodates. The Scrum master pipes up, “OK everyone, lets go round the table and everyone can answer the three questions -

What went well?
What could have gone better?
How can we improve?

let’s start with Fred.” Fred looks briefly startled, he was working on a bug before the meeting, and his brain is still trying to work through what might cause it. He wasn’t really paying attention, but he eventually gathers himself and says “Um, I can’t really remember what happened last sprint…I guess…the thing that went well is that we got the Jingly Jangly feature done. It could have gone better if we’d managed to get it tested during the sprint. I don’t know how we can improve on that - it’s just not possible to test code until we’ve finished developing it!”. The meeting carries on in much the same vane, with most people offering vague or similar statements to Fred - apart from the one guy who takes copious notes, and reels off a list of complaints long enough that no-one can remember the first thing he said by the time he gets to the last.

Does any of this sound familiar to you? Does this sound like a meeting you want to be in? It seems pretty clear that in this fictitious scenario the team isn’t excited to join the meeting, the environment isn’t comfortable and the content is uninspiring.

I suspect that something like this is common among small and inexperienced Agile teams. When I first heard those three questions I thought they were a great idea. We can focus on the good along with the bad - and then come up with some real actions we can take to improve! This will be great. Some six or seven years after I started trying to work in an Agile way I see my mistake, in fact, I would go so far as to say that that format is poison to the Agile retrospective. It seduces you into the idea that that is all that you need - when in fact a retrospective needs a lot more to be effective.

Esther Derby and Diana Larsen literally wrote the book on Agile retrospectives ¹. In it they set out the 5 basic phases of a good retrospective:

Setting the stage
Gather data
Generate insights
Decide what to do
Close the retrospective

Setting the stage

Did the example above strike you as focussed? Did it seem like the team had a goal in mind? In Derby and Larson’s model the first part of a retrospective is to set the stage.

Setting the stage creates the context and atmosphere for the retrospective, so there are a few of things we might want to do here:

Outline the rules of the retrospective, from the time box to the prime directive:

Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand.

Set the focus, perhaps the team above might want to focus on how they test if their stories are not really done because they are not tested.
Set the mood, for example, a very common activity is to ask everyone for a one or two word description of how they felt about the last sprint. This has the added bonus of helping to trigger memory associations that will be used in the next stage.

Gather data

What’s the first thing Fred said? “Um, I can’t really remember what happened last sprint”. If you’re hearing that in your retrospectives, then you need to beef up this part. Gathering data is all about helping people to collectively remember the last sprint, and to construct a common narrative about what happened. Gathering data is based on facts, not opinions. One example of a way to do this is by creating a timeline.

Generate insights

In our example meeting do you see any analysis of the things that could have gone better? We jump straight from “what could have gone better” to “how can we improve” without asking why things weren’t as good as we’d like them to be. To decide on the most effective actions we really need to do some analysis of the problems first.

This phase is about taking the facts and identifying trends, themes and root causes. If we look at Fred’s answer again (poor Fred), he said “we got the Jingly Jangly feature done” - this is a fact (well, assuming their “definition of done” does not include testing). But why did Fred consider that a good thing? Was there something specific about that feature that meant completing it was significant? What did we do that meant that the feature got done when it otherwise might not have done?

One common activity for this area is to ask “five whys”, it is the most basic technique one can apply to get to the root cause of a problem. The idea is to keep asking “why” until the root cause is uncovered, the trick is to ask the right why - sometimes a response will yield multiple avenues, and it requires real knowledge of what the team was doing at the time to ask the “why” that will yield an actionable cause.

Decide what to do

Again, going back our our example meeting, it looks like there are actions - after all everyone gets to come up with some! I bet there is even some discussion as the team goes around the table of what those actions should be, and at the end of the meeting the Scrum master probably has a big list written down. Are those actions really team commitments though? Does it seem like the process is likely to identify the two or three best actions the team can take to improve the process?

In the most self explanatory phase of the retrospective, the goal is for the team to decide on some concrete actions to take during the next iteration. In fact, Derby and Larson use the term “experiments” for some actions, implying that the team should not only come up with an action, but design a way of measuring the consequences of that action.

If you have the same problems every retrospective this is the part to beef up, because the actions you’re taking are not having the desired effect. If you’re having difficulty following through on actions this is the time to build a plan and get team members to commit to following through on it.

The best actions are ones that will happen automatically as part of the team’s day to day work. For example, committing to do more pair programming is easy because pair programming is often part of the normal day to day. On the other hand committing to write documentation for a poorly understood part of the system may not yield results if writing documentation for legacy code is not part of anyone’s day-to-day. When actions generate work that is not the norm it is important to make sure time is allocated in the iteration to carry out those actions.

Close the retrospective

Closing is used for a few different purposes, obviously when closing you need to summarise the actions, and the commitments made. It’s also a great time to do a round of “appreciations”, where team members can appreciate the little things other people have done to help them out. Finally, it’s a good time to do a “retrospective retrospective”, to figure out how to make retrospectives more useful and productive in the future.

Agile Retrospectives: Making Good Teams Great by Esther Derby and Diana Larson ↩

Slicing: A Practical Example

2017-02-11T00:00:00+00:00

I guarantee that the first time you try vertical splitting with a team someone will say “some stories can’t be split”. There is a kernel of truth in this statement (it is obviously the case that there will be some level below which a story cannot be split any further) that makes it easy to say - but the truth is a team splitting stories almost never reaches this limit; you have to push through your instinctive reluctance to split vertically until it starts to become natural.

Merging organisations

Our example is taken from my current team’s work (slightly simplified and paraphrased):

As a corporate administrator I would like to be able to merge duplicate organisations through the online portal so that I can report on the organisation correctly.

Test that a list of all organisations can be viewed

Test that the list of organisations loads in under a second with 1,000 organisations

Test that when an organisation is deleted through the portal it is possible to select an organisation to migrate all existing records to

Test that only users with the manage organisations permission can delete organisations

Test that sites cannot see deleted organisations

Test that site records are updated to reflect the merged organisation

Test that sites on old versions continue to work correctly even when connected to an updated online portal

Test that when a site tries to save a record with a deleted organisation, it is automatically migrated to the new organisation

The basic idea is pretty simple, we have a big list of organisations in our database, and we would like to merge some together to get rid of duplicates. We have a bunch of “test cases” that cover some behaviour we would like it to have

in particular there are requirements around making sure the updated information synchronises correctly.

The first estimate for this story was pretty high, either 40 or 100 points and we know that those don’t really fit into a single sprint - so we had to split it, but where? If you just look at the title of the story splitting it can be a bit difficult, how can you deliver “part” of the merge functionality and have it be useful?

Let’s apply the patterns!

Is it a workflow? Nope - one button, one operation.
Are there multiple operations? No…wait…not in the title, but looking at the test cases I can see “Test that a list of all organisations can be viewed” and “Test that when an organisation is deleted through the portal it is possible to select an organisation to migrate all existing records to”. That’s two operations! Hint: writing out your requirements like this makes splitting a lot easier!
- We can split into adding a view to see the organisation list and the merge operation.
- We can split the merge and delete operations.
Are there business rules we can tweak?
- We could split out the permission requirement.
- We could split out the migration requirement.
Are there any data variations to consider?
- We could defer making it work with old versions.
Are there any interface variations?
- We could perhaps come up with a simpler select/delete/merge UI that doesn’t need the entire list of organisations and therefore avoid the performance requirement.
Is there an obvious split? No.
Is there a simple core? No.
Can we defer performance? Yes!
- We can defer the requirement to have the organisation list load in under a second.
Do we have any unknown quantities that need a research spike?
- The technical details of how to perform the merge and maintain backward compatibility could do with some examination.

So, we’ve got some splits - we need to try to find ones that will let us devalue one story. It turns out that the only split we can apply that truly devalues work is the permission requirement (because we suspect our sales and support teams will end up doing most of the migration, we can just restrict the functionality to administrators.

However, we can still use the splits to get useful incremental work. We ended up splitting the story like this:

Implement listing organisations on the portal.
Implement deleting organisations through the portal.
Implement migrating organisations after deletion.
Implement automatic correction of data synchronised from older versions.
Add configurable permission controls to organisation management.

The keen eyed among you might have spotted that there appear to be dependencies between these stories - but there’s a way of thinking about this that helps explain why this ends up being acceptable. Steven Thomas writes about how dependencies can be more apparent than real ¹. The core idea is that if one of these stories had value without the others we could just go ahead and implement it. Take two obviously related stories - “Implement migrating organisations after deletion” and “Implement automatic correction of data synchronised from older versions”. We can’t implement the correction without the migration, right? Wrong! Of course we could, if there were some way that old versions could be syncing bad data (perhaps if we were manually performing merges at the database level), then this story would have value without the interface for generating the migrations. Thomas calls this kind of dependency a “natural order” for the stories - but the reality is that we can implement them in whatever order we want, it just so happens that, once implemented, some stories can increase the value of other stories.

Having done this split it turned out some of these stories were still too big. We could perhaps go further, for example with the migration we could have an interface for creating a list of migrations and a separate story to perform the migrations. Again, the act of performing migrations only depends on a source of migrations - that source does not have to be a user interface! When you have gotten as far as you can splitting a story and it’s still too big the last question to ask is “why?” - chances are there’s some technical debt in your system that’s holding your team back from delivering more value to your customers. As slow accumulation of debt is extremely likely to become a normalisation of deviance, when your stories are too big to implement and impossible to split then there’s a good chance your system is the problem. Generating and estimating subtasks might be one way to try and pick apart what it is about a story that is complex or debt ridden - it may even reveal vertical splits you hadn’t considered. If debt is the problem (perhaps one component is very complex, and integrating the new feature with it is a lot of work) your sustainable pace has been compromised, and you need to take action to change that!

User Story Dependencies are more Apparent than Real by Steven Thomas ↩

The Path to Agility Part 7: Slicing

2017-02-01T00:00:00+00:00

Lately I’ve been spending a lot of time thinking about slicing work. This article is about my struggle with it.

Vertical slicing

As a software developer one of the concepts I find most difficult to apply is vertical slicing. My brain, equipped with many years of development experience, naturally sees the horizontal splits in the system, the traditional “layers” we find in a multi-tier system where we try to split features into a data layer, a business logic layer, and a presentation layer.

The problem is, as someone trying very hard to be agile this is the wrong way to slice a system. If we go all the way back to the first of the Agile principles we have the following:

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software ¹

The implementation of the data layer is not valuable software. The implementation of the business logic is not valuable software. Even the implementation of the presentation layer is not valuable software. It is only when we have all three that we have valuable software.

Vertical slicing is about taking a story and simplifying it in one or more of the horizontal layers instead of splitting it along those layers. The extension from the simplified work to the original intention becomes a new story that can be prioritised separately.

How to create vertical slices

Richard Lawrence describes a number of patterns for splitting stories ², and even offers up a helpful little cheat-sheet for applying those patterns ³.

The core idea is to find simplifications of a story that allow the the complex elements to be de-prioritised or deleted. It’s best to read Lawrence’s original material, but for the lazy here’s a quick summary of his patterns:

Split workflows by doing the beginning and end first, then enhancing with the middle later; or by applying other patterns to the individual steps of the workflow.
Split out operations, for example a typical CRUD (Create, Read, Update, Delete) interface could be split into four stories - one for each operation.
Simplify business rules - look for flexible or vague language in the requirements and simplify the story by tying down the language to specifics. Look for business rules are are adding complexity and try to move them to separate stories.
Split out different input data. Simplify the story by implementing one type of input data first, then enhancing with other types of data later.
Simplify the interface. Sometimes an element will have a complex UI - split the story into a simple UI and enhance it with other stories later. If there are multiple ways to do the same thing, build one interface first and add the others on later.
Look for the “obvious split”. The “Independent” part of the INVEST rules sometimes makes the obvious split difficult, for example we often have stories to implement a report with several variations. The obvious split is into the various reports - but the first report will be harder than the others because it will have to build most of the scaffolding. We can split this story into a “implement one of X,Y,Z variations” story and a “implement X,Y,Z given one is already done” story. The stories are not truly independent, but we are at least making the dependencies minimal and explicit.
Remove complexity. Some stories are a simple core with most of the value, and a series of complex requirements built on top. Build the core first.
Defer performance. Sometimes stories have performance requirements that make them hard, in this case build the story first without the performance requirement; then enhance it with the performance requirement later.
Create a spike to reduce uncertainty. Sometimes the reason something is hard is because there is a lot of uncertainty. Often the easiest way to reduce uncertainty is to build something valuable first then use the knowledge gained to define the future stories. Failing that, we will need a spike to reduce the uncertainty. With the knowledged gained from the spike we can better estimate or split the original story. This is a last resort, because a spike is a research task which won’t deliver valuable software. One thing I cannot emphasise enough is to define the outputs of the spike and hold the team to them. It’s easy to get lost in research and overrun a timebox, spikes need to be small, well defined and focused otherwise they will fail to reduce the uncertainty.

How to know if you created a vertical slice?

This is actually pretty simple; can you give a build to the test team and have them test the new feature? Saying “test nothing changed” does not count; that is literally the definition of refactoring and a waste of your test team’s time. It doesn’t matter whether or not you have an actual test team, at the end of the day someone probably needs to test something - whoever it is needs to be able to test that the changes are correct. If the answer is no, then you probably created a horizontal slice.

The other question to ask is “could you accidentally ship it?”. Again, if the answer is “no” you probably didn’t create a full slice through the system. Sometimes compromises need to be made in order to have a vertical slice that is accidentally shippable (for example, dropping a performance requirement might introduce the need for a simple “busy” or “progress” display).

That’s it for now, next time I want to introduce some practical, real world, examples of slicing.

The Path to Agility Part 6: Burndown

2017-01-25T00:00:00+00:00

In the last couple of parts of this series I’ve been talking about how we’ve been coping with sprints that went wrong. This time I want to look at a sprint that appeared to go right.

The burndown

Anyone familiar with Scrum will know about burndown charts. The basic idea of a burndown chart is to plot effort remaining vs. time up to the “commitment”. The are various different commitments we can track (by project, by release date, by epic, or by sprint) - but for the purposes of this post we’re interested in the sprint burndown.

There are also various ways we can track effort remaining. The obvious one is to track the number of story points that have not been completed, but there are others - such as counting the number of subtasks remaining, or constantly re-estimating the number of hours work left in a task.

To decide how to plot the burndown, we need to know what it’s for:

To empower the team to take action when the sprint is off target (finding ways to unblock themselves when behind, and new work to take on when ahead).
To enable the Scrummaster to detect and address impediments early in the sprint.
To enable the Product Owner to pro-actively manage stakeholder expectations and alter sprint scope when necessary.
To help analyse the team’s performance during the retrospective when looking for ways to improve.
To communicate progress to stakeholders.

If we look at these we can see one common trend: it’s about knowing whether or not the sprint is on track. If we plot by story points, and have a relatively small number of stories with a large number of points in them then our burndown will be stepped, like this:

In fact, it could be worse, because if those stories are tricky to split between multiple team members then we may end up with a chart like this:

While just looking at completed stories is what matters to the product owner and the stakeholders, it’s not a realistic view into the progress of the sprint - the only time it will be useful for taking actions during the sprint is when we have a large number of stories with few points attached which is not always practical.

There is, however, one case where we do want to only look at completed stories - and that’s when we’re trying to minimise our “work in progress”. In particular, since we consider “working software” to be the primary measure of progress we need to make sure the team is finishing stories as soon as possible.

So, we need to look at both story points completed and effort remaining, but we want to focus on the latter for most purposes because the granularity of the former can make it difficult to interpret.

The real world

Remember in the last couple of posts in this series we looked at a sprint that wasn’t burning down, half way through the sprint the burndown looked like this:

Fast-forward to our most recently completed sprint, and half way through the sprint the burndown looked very similar:

This time though, I wasn’t worried - because I knew that around 80% of the work in the sprint was made up of two stories, and by looking at the subtasks I could see progress was being made. At the end of the sprint our (story) burndown looked like this:

In the retrospective we looked at this and could clearly see that the burndown wasn’t a true reflection of the progress we were making on these difficult-to-slice stories. We decided that we’d divide story points equally between subtasks, and look at that instead. In fact, the story points aren’t even that important - simply counting the subtasks remaining produced a very similar burndown. The “effort” burndown looked like this:

This isn’t a brilliant burndown, but compared to the previous sprint it’s a lot closer to what we want. There are, however, two features we should be concerned about - the first is that cliff near the start of the second week - clearly something stalled progress around that time and we should look at what that is. The second is the significant increase in velocity (the gradient) in the second week. Like a stall there are many reasons why this might be - but the one that should really concern us is whether or not corners are being cut to meet the deadline. If they are then we have probably incurred some technical debt that we will have to pay off later.

The positives

We got all our work done, through a combination of not allowing bugs raised against the previous sprint to affect the scope of the current sprint and by generating a decent set of subtasks to help divide the work.

The negatives

It turns out, we did incur debt toward the end of the sprint - something that we’re paying for in the current sprint. We knew this when we planned the sprint, so we set a sprint goal around ensuring the product was in a releasable state. This means the team can modify the scope of the sprint to meet the goal, rather than being torn apart by the twin forces of the need to add new features or to fix old ones.

Our actions

Carry on generating decent subtasks
Track our effort burndown more closely to better detect problems

Refactoring

2017-01-18T00:00:00+00:00

Refactoring is something that you hear about almost constantly in software engineering these days. Agile practices centred around minimal documentation and avoiding big up-front design drive a need to constantly reduce any accumulated technical debt in order to maintain the “sustainable pace”. In this post I wanted to talk a bit about my views on the practice.

What is refactoring?

Refactoring is the process of changing a software system in such a way that it does not alter the external behaviour of the code yet improves its internal structure ¹

The key idea here is that we’re making some changes to the internal structure of the code (to improve it), but we are confident that we are not making changes to the behaviour. How can we be confident that we have not changed the behaviour? Because we have behaviour driven tests that prove the behaviour has not changed.

What if we don’t have tests? Then we are not refactoring! We can’t be sure that our changes won’t take a working system and break it, so we shouldn’t be making the change. The only refactoring we can do in this case is to add tests since adding tests cannot change the behaviour (assuming your test and production code are separated

if not then you have problems my rambling is not going to help you with!).

Why refactor?

Since our changes do not affect the product at all (by definition) you might ask “why refactor?”, since there is no tangible benefit for the customer - why invest resources in refactoring at all? The answer lies in the idea of technical debt. There are multiple definitions of technical debt, but the one I like is the one by Steve McConnell:

[Technical debt is] a design or construction approach that’s expedient in the short term but that creates a technical context in which the same work will cost more to do later than it would cost to do now (including increased cost over time) ² ³

In other words, technical debt describes a characteristic of a software system that makes future work more expensive. When we refactor, we are attempting to eliminate these characteristics so that future work will be cheaper.

There are many sources of technical debt, many of which can never be eliminated. Two in particular are a lack of up-front knowledge and the fact that technology tends to evolve over time. Technical debt caused by lack of knowledge occurs because we almost always learn more about the requirements of a system and the best way to build it as time goes on. Technical debt from the evolution of technology occurs because the most expedient design or construction today may not be the most expedient design or construction in a years time. This last is surprising, because the debt can suddenly “appear”, one day you’re working with version 4 of a library and doing things in the most efficient way - the next day version 5 is released and suddenly you’re sitting on a pile of debt because the approach of using version 4 means you can’t use the more time or cost efficient methods from version 5.

Other definitions of debt focus on debt incurred as a result of a business decision to take an expedient approach in the short term in order to meet some deadline or other requirement. I don’t like these definitions because they imply that the only source of debt is these decisions, when in fact there are other sources that all have the same effect of making future work more expensive.

When to refactor?

Not all debt is worth refactoring to remove; all debt can incur a cost of development at a later date - but it only does so when we’re working on code that is affected by the debt.

This makes the answer obvious - tackle debt in the components you are working on, when you are working on them. There’s no benefit in addressing debt in a rarely changed component, the “interest rate” is so low as to be negligible. Conversely, a component that’s touched frequently needs debt tackling aggressively, as it has a comparatively high interest rate.

One last thing to watch out for when deciding when to refactor - look out for your own vanity. It’s surprisingly easy as a developer to decide to refactor someone else’s code because it’s “not how you would have done it”, but the reality is, when you do this you have to be sure you are making the code better. Sometimes (and I’m as guilty of this as anyone), we’re not making the code better - we’re just making it different.

Tracking debt

The technical backlog is an established best practice to define purely technical work packages ⁴. This is effectively how we are managing the debt in our system, whenever we encounter an aspect of the system that is slowing down development, we log it as a technical debt task. We let the product owner prioritise and schedule these tasks the same way all our other backlog tasks are scheduled - subject to the usual rule that the planning meeting is a negotiation between the product owner and the scrum team.

In this way we keep track of what debt we have in the system, and align tackling the debt with the development work we are doing in any given sprint.

Writing a Better Options Library Part 2

2017-01-11T00:00:00+00:00

In the last post on this topic we talked about some general characteristics we would like to see in an options library, and what some of the existing libraries do.

In this post we’ll do some high level feature design.

Features

We wanted the library to be minimal and complete - so it should provide everything one would expect from an option parsing library and no more. For inspiration, let’s look at what options some existing tools accept - if we can’t cope with these options then we would obviously not be providing a complete interface.

$ cp --help
Usage: cp [OPTION]... [-T] SOURCE DEST
  or:  cp [OPTION]... SOURCE... DIRECTORY
  or:  cp [OPTION]... -t DIRECTORY SOURCE...
Copy SOURCE to DEST, or multiple SOURCE(s) to DIRECTORY.

Mandatory arguments to long options are mandatory for short options too.
  -a, --archive                same as -dR --preserve=all
      --attributes-only        don't copy the file data, just the attributes
      --backup[=CONTROL]       make a backup of each existing destination file
  -b                           like --backup but does not accept an argument
      --copy-contents          copy contents of special files when recursive
  -d                           same as --no-dereference --preserve=links
  -f, --force                  if an existing destination file cannot be
                                 opened, remove it and try again (redundant if
                                 the -n option is used)
  -i, --interactive            prompt before overwrite (overrides a previous -n
                                  option)
  -H                           follow command-line symbolic links in SOURCE
  -l, --link                   hard link files instead of copying
  -L, --dereference            always follow symbolic links in SOURCE
  -n, --no-clobber             do not overwrite an existing file (overrides
                                 a previous -i option)
  -P, --no-dereference         never follow symbolic links in SOURCE
  -p                           same as --preserve=mode,ownership,timestamps
      --preserve[=ATTR_LIST]   preserve the specified attributes (default:
                                 mode,ownership,timestamps), if possible
                                 additional attributes: context, links, xattr,
                                 all
      --no-preserve=ATTR_LIST  don't preserve the specified attributes
      --parents                use full source file name under DIRECTORY
  -R, -r, --recursive          copy directories recursively
      --reflink[=WHEN]         control clone/CoW copies. See below
      --remove-destination     remove each existing destination file before
                                 attempting to open it (contrast with --force)
      --sparse=WHEN            control creation of sparse files. See below
      --strip-trailing-slashes  remove any trailing slashes from each SOURCE
                                 argument
  -s, --symbolic-link          make symbolic links instead of copying
  -S, --suffix=SUFFIX          override the usual backup suffix
  -t, --target-directory=DIRECTORY  copy all SOURCE arguments into DIRECTORY
  -T, --no-target-directory    treat DEST as a normal file
  -u, --update                 copy only when the SOURCE file is newer
                                 than the destination file or when the
                                 destination file is missing
  -v, --verbose                explain what is being done
  -x, --one-file-system        stay on this file system
      --help     display this help and exit
      --version  output version information and exit

We can see several types of command line option here:

“Arguments” like SOURCE and DEST, which can occur in various combinations (for example, one SOURCE and one DEST, or multiple SOURCE... with one target DIRECTORY).
“Flags” like --archive or --verbose which can turn specific behaviour on or off.
“Parameters” like --target-directory=DIRECTORY which can set some configuration setting.

We can also see a couple of interesting features:

Flags and parameters can have a short form (-v) and a long form (--verbose).
Short form flags can be specified all together (-dR)
Some parameters or flags could affect the interpretation of later command line options.
Some parameters (--reflink[=WHEN]) have an optional argument.
Parameters can be specified in several ways (e.g. --suffix=.orig, --suffix .orig -S.orig and -S .orig).

cp is one of the simpler options, when it comes to command line options gcc has more than most. If we look through the GCC man page we can see a few variations on the types of option accepted by cp:

GCC does not accept combined single letter flags.
Some options (e.g. -Idir) can be specified multiple times with different parameters.
Some options (-f and -W) could be considered an option with a parameter (-Wall could be thought of as -W with parameter all), or as a self contained option (just -Wall). Some usages suggest the latter as there is an additional parameter (-fabi-version=n).
Some options (-DMACRO=VALUE) use the equals sign as part of their parameter.

We also said we wanted our library to be cross platform, so we should look at some Windows programs too.

> cl /help

Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24213.1 for x86
Copyright (C) Microsoft Corporation.  All rights reserved.

                         C/C++ COMPILER OPTIONS


                              -OPTIMIZATION-

/O1 minimize space                      /O2 maximize speed
/Ob<n> inline expansion (default n=0)   /Od disable optimizations (default)
/Og enable global optimization          /Oi[-] enable intrinsic functions
/Os favor code space                    /Ot favor code speed
/Ox maximum optimizations               /Oy[-] enable frame pointer omission
/favor:<blend|ATOM> select processor to optimize for, one of:
    blend - a combination of optimizations for several different x86 processors
    ATOM - Intel(R) Atom(TM) processors

                             -CODE GENERATION-

/Gw[-] separate global variables for linker
/GF enable read-only string pooling     /Gm[-] enable minimal rebuild
/Gy[-] separate functions for linker    /GS[-] enable security checks
/GR[-] enable C++ RTTI                  /GX[-] enable C++ EH (same as /EHsc)
/guard:cf[-] enable CFG (control flow guard)
/EHs enable C++ EH (no SEH exceptions)  /EHa enable C++ EH (w/ SEH exceptions)
/EHc extern "C" defaults to nothrow
/EHr always generate noexcept runtime termination checks
/fp:<except[-]|fast|precise|strict> choose floating-point model:
    except[-] - consider floating-point exceptions when generating code
    fast - "fast" floating-point model; results are less predictable
    precise - "precise" floating-point model; results are predictable
    strict - "strict" floating-point model (implies /fp:except)
/Qfast_transcendentals generate inline FP intrinsics even with /fp:except
/Qpar[-] enable parallel code generation
/Qpar-report:1 auto-parallelizer diagnostic; indicate parallelized loops
/Qpar-report:2 auto-parallelizer diagnostic; indicate loops not parallelized
/Qvec-report:1 auto-vectorizer diagnostic; indicate vectorized loops
/Qvec-report:2 auto-vectorizer diagnostic; indicate loops not vectorized
/GL[-] enable link-time code generation
/volatile:<iso|ms> choose volatile model:
    iso - Acquire/release semantics not guaranteed on volatile accesses
    ms  - Acquire/release semantics guaranteed on volatile accesses
/GA optimize for Windows Application    /Ge force stack checking for all funcs
/Gs[num] control stack checking calls   /Gh enable _penter function call
/GH enable _pexit function call         /GT generate fiber-safe TLS accesses
/RTC1 Enable fast checks (/RTCsu)       /RTCc Convert to smaller type checks
/RTCs Stack Frame runtime checking      /RTCu Uninitialized local usage checks
/clr[:option] compile for common language runtime, where option is:
    pure - produce IL-only output file (no native executable code)
    safe - produce IL-only verifiable output file
    initialAppDomain - enable initial AppDomain behavior of Visual C++ 2002
    noAssembly - do not produce an assembly
    nostdlib - ignore the default \clr directory
/Gd __cdecl calling convention          /Gr __fastcall calling convention
/Gz __stdcall calling convention        /GZ Enable stack checks (/RTCs)
/Gv __vectorcall calling convention     /QIfist[-] use FIST instead of ftol()
/hotpatch ensure function padding for hotpatchable images
/arch:<IA32|SSE|SSE2|AVX|AVX2> minimum CPU architecture requirements, one of:
   IA32 - use no enhanced instructions and use x87 for floating point
   SSE - enable use of instructions available with SSE-enabled CPUs
   SSE2 - (default) enable use of instructions available with SSE2-enabled CPUs
   AVX - enable use of instructions available with AVX-enabled CPUs
/Fm[file] name map file                 /Fo<file> name object file
/Fp<file> name precompiled header file  /Fr[file] name source browser file
/FR[file] name extended .SBR file       /Fi[file] name preprocessed file
/Fd: <file> name .PDB file              /Fe: <file> name executable file
/Fm: <file> name map file               /Fo: <file> name object file
/Fp: <file> name .PCH file              /FR: <file> name extended .SBR file
/Fi: <file> name preprocessed file
/doc[file] process XML documentation comments and optionally name the .xdc file

                              -PREPROCESSOR-

/AI<dir> add to assembly search path    /FU<file> forced using assembly/module
/C don't strip comments                 /D<name>{=|#}<text> define macro
/E preprocess to stdout                 /EP preprocess to stdout, no #line
/P preprocess to file                   /Fx merge injected code to file
/FI<file> name forced include file      /U<name> remove predefined macro
/u remove all predefined macros         /I<dir> add to include search path
/X ignore "standard places"

                                -LANGUAGE-

/Zi enable debugging information        /Z7 enable old-style debug info
/Zp[n] pack structs on n-byte boundary  /Za disable extensions
/Ze enable extensions (default)         /Zl omit default library name in .OBJ
/Zs syntax check only                   /vd{0|1|2} disable/enable vtordisp
/vm<x> type of pointers to members
/Zc:arg1[,arg2] C++ language conformance, where arguments can be:
  forScope[-]           enforce Standard C++ for scoping rules
  wchar_t[-]            wchar_t is the native type, not a typedef
  auto[-]               enforce the new Standard C++ meaning for auto
  trigraphs[-]          enable trigraphs (off by default)
  rvalueCast[-]         enforce Standard C++ explicit type conversion rules
  strictStrings[-]      disable string-literal to [char|wchar_t]*
                        conversion (off by default)
  implicitNoexcept[-]   enable implicit noexcept on required functions
  threadSafeInit[-]     enable thread-safe local static initialization
  inline[-]             remove unreferenced function or data if it is
                        COMDAT or has internal linkage only (off by default)
  sizedDealloc[-]       enable C++14 global sized deallocation
                        functions (on by default)
  throwingNew[-]        assume operator new throws on failure (off by default)
  referenceBinding[-]   a temporary will not bind to an non-const
                        lvalue reference (off by default)
/ZH:SHA_256             use SHA256 for file checksum in debug info (experimental)

/Zo[-] generate richer debugging information for optimized code (on by default)
/ZW enable WinRT language extensions
/constexpr:depth<N>     use <N> as the recursion depth limit
                        for constexpr (default: 512)
/constexpr:backtrace<N> show <N> constexpr evaluations
                        in diagnostics (default: 10)
/constexpr:steps<N>     terminate constexpr evaluation after
                        <N> steps (default: 100000)
/ZI enable Edit and Continue debug info
/openmp enable OpenMP 2.0 language extensions

                              -MISCELLANEOUS-

@<file> options response file           /?, /help print this help message
/bigobj generate extended object format /c compile only, no link
/errorReport:option Report internal compiler errors to Microsoft
    none - do not send report
    prompt - prompt to immediately send report
    queue - at next admin logon, prompt to send report (default)
    send - send report automatically
/FC use full pathnames in diagnostics   /H<num> max external name length
/J default char type is unsigned
/MP[n] use up to 'n' processes for compilation
/nologo suppress copyright message
/sdl enable additional security features and warnings
/showIncludes show include file names   /Tc<source file> compile file as .c
/Tp<source file> compile file as .cpp   /TC compile all files as .c
/TP compile all files as .cpp           /V<string> set version string
/w disable all warnings                 /wd<n> disable warning n
/we<n> treat warning n as an error      /wo<n> issue warning n once
/w<l><n> set warning level 1-4 for n    /W<n> set warning level (default n=1)
/Wall enable all warnings               /WL enable one line diagnostics
/WX treat warnings as errors            /Yc[file] create .PCH file
/Yd put debug info in every .OBJ        /Yl[sym] inject .PCH ref for debug lib
/Yu[file] use .PCH file                 /Y- disable all PCH options
/Zm<n> max memory alloc (% of default)  /FS force to use MSPDBSRV.EXE
/await enable resumable functions extension
/Wv:xx[.yy[.zzzzz]] disable warnings introduced after version xx.yy.zzzzz
/source-charset:<iana-name>|.nnnn set source character set
/execution-charset:<iana-name>|.nnnn set execution character set
/utf-8 set source and execution character set to UTF-8
/validate-charset[-] validate UTF-8 files for only legal characters

                                -LINKING-

/LD Create .DLL                         /LDd Create .DLL debug library
/LN Create a .netmodule                 /F<num> set stack size
/link [linker options and libraries]    /MD link with MSVCRT.LIB
/MT link with LIBCMT.LIB                /MDd link with MSVCRTD.LIB debug lib
/MTd link with LIBCMTD.LIB debug lib

                              -CODE ANALYSIS-

/analyze[-] Enable native analysis      /analyze:quiet[-] No warning to console
/analyze:log<name> Warnings to file     /analyze:autolog Log to *.pftlog
/analyze:autolog:ext<ext> Log to *.<ext>/analyze:autolog- No log file
/analyze:WX- Warnings not fatal         /analyze:stacksize<num> Max stack frame
/analyze:max_paths<num> Max paths       /analyze:only Analyze, no code gen

There are a couple of new points here:

The character used to start a flag or parameter is /, but this would obviously be inappropriate on Unix filesystems.
A colon (:) is frequently used to separate an option from some kind of parameter.
An @ can be used to read options from a file.
A - can be used to turn options off.
Some options take a variable number of parameters (/Zc:arg1[,arg2]).

As a final use case we need to consider programs like GDB and Valgrind which wrap another program and forward arguments. For example, we can start GDB with gdb a.out --args arg1 arg2 arg3 to pass arg1 arg2 arg3 through to a.out, similarly with Valgrind we would have valgrind a.out arg1 arg2 arg3. With CMake we can use cmake --build <dir> -- <native build tool options> to forward options to the build tool.

If we want to support programs like this we will need to make sure our argument parser is able to recognise constructs like this, and treat them appropriately.

Yet another options library

So, parameter wise we need to support

Positional arguments, that may be optional or occur multiple times.
Options, which are named switches to the program and must be specified separately.
Flags, which are named switches that may be collapsed into a single argument.
Parameters to options and flags.
Options and flags can be specified any number of times.
Configurable “switch” prefix (- or /).
Configurable parameter separation.
Arguments, options, and flags need to be able to modify the interpretation of later arguments, options, and flags.

In the next post of this series we’ll start to flesh out an interface design.

The Path to Agility Part 5: When it still goes wrong

2016-12-20T00:00:00+00:00

Last time we talked about a sprint whose burndown was going wrong, and the actions we took. One week later and our burndown still looked like this:

What now?

At this point we knew we were in trouble, we had external commitments which required us to have something to show at the end of the year - but even fixed time/variable scope projects don’t function when there is no burndown at all.

As it turns out, we still have options to help us recover - and in this situation I called a stop, wrote everything we were doing up on a white board, and tried to establish what work was outstanding and what was blocking progress. For us, it turned out that almost every story in the sprint had a requirement linked back to one other story, in other words we had a dependency that if we could just finish it would unblock almost all of the work and get us into a much better state.

We were desperate, so I decided to fly in the face of everything we’re taught about the mythical man month ¹ and add developers to a late project. Because both myself and our product owner have a strong technical background we pitched in to help unblock the blocked stories.

Adding knowledgeable developers is not as bad as you would think

The core problem with adding developers to a late project is the resulting increase in communication required because the number of communication channels grows as the square of the number of team members. Developers not familiar with a project will also have significant knowledge hurdles to overcome before they can become productive team members, and thanks to Agile’s “documentation-light” philosophy there’s likely to be a significant amount of communication required at those early stages.

Fortunately, our product owner and I already have all the knowledge necessary to contribute to the development, we are (after all) heavily involved with the team and their work on a day-to-day basis. Even so, the increase in communication required when going from 4 active developers (on this project) to 6 was very noticeable. Scrum may suggest team sizes of 3-9 ², but our experiences on this project lead me to strongly favour sizes at the lower end of the spectrum.

The Analysis

Last time I said one of the most important things you can do is to analyse the failure. This is exactly what we did at the retrospective, we started by going round and having everyone say how they thought the sprint went - pleasingly not everyone said “terribly”, some of the failings were obvious but there were also some positive points around the knowledge gained and the way the team worked together as a whole.

We then broke out into a more unusual activity, a sprint time line. A time line consists of two parts

Have everyone write down the events they remember on different coloured post-it notes (or your favoured medium). The colours correspond to:
- Good events
- Problematic events
- Significant events
Have everyone mark how they were feeling at various points of the time line (I went with happy, neutral, and sad faces).

The idea is to build a shared record of what happened, when - and how it affected the team.

There were a few things that became obvious from this activity:

Our failure was right at the beginning of the sprint, between the sprint planning on Friday and kicking off the sprint on Monday there was a significant shift in how the team planned to implement the work. This was exacerbated by a number of people being on holiday on one day or the other, and a lack of suitable subtasks that would have clued us in to the fact that we had changed the implementation to one that wouldn’t fit in the time available.
Our interventions (restarting the sprint, adding developers) were generally viewed positively, but ultimately could not overcome the hidden scope change.
Unsurprisingly, the team felt pressured and stressed for the majority of the sprint

Overall, the activity was very time consuming - and I’m not convinced I would do it every retrospective, but the shared view of what went wrong was very useful.

The Actions

Having identified problems, the next step is to identify concrete actions we can take to prevent those problems occurring again.

We have regular “planning and estimation” sessions with our product owner to help estimate and forecast upcoming stories. We decided that after these sessions we would take some time, either in individuals or pairs, to add subtasks to stories planned for the next sprint. We want these subtasks to be sufficiently detailed that any team member can take on the work.

We also said that if there’s not a subtask for it, we should be questioning whether or not we should be doing it.

Finally, we decided to stop using “As an X I would like Y so that I can Z” for the titles of our stories. While this kind of title captures the whole story, it makes it almost impossible to grasp the state and content of a sprint at a glance, instead our PO will try to write brief descriptions that are easier to take in quickly.

These two changes should help to tackle the problem from two directions, we should be more easily able to see a high level overview of our progress - and use that information to spot discrepancies in the high level, feature work. We should also be able to spot discrepancies at low level, technical work, thanks to having pre-planned subtasks.