> gvim -d file1 file2
        > vimdiff file1 file2

Today I (re)learned about diffopt, which makes gvimdiff more wonderful.

    :set diffopt=filler

By default, diffopt is set to “filler”. Filler doesn’t do much, it just adds vertical spaces to keep the text of the left and right pane aligned. It’s a good setting, and I don’t much like working without it. But there are other settings which are overlooked, even by experts.

    : set diffopt+=iwhite

My second favorite is ‘iwhite’, which stands for “ignore whitespace”. It makes gvimdiff ignore leading, trailing, and embedded spaces so that simple acts like retabbing or deleting vertical spaces won’t obscure actual changes. It really helps you to merge changes in situations where you’ve adopted code that was built to entirely different standards.

    :set diffopt+=icase

iCase ignores capitalization, which is not as useful as the other options, since programmers are normally interested in case, but it may be handy when someone has corrected and reformatted SQL. I don’t expect to use it much.

    :set diffopt+=horizontal

The last interesting bit is the choice between vertical splits (the normal case) and horizontal splits (for very long lines on cinema-aspect screens). I hope to never need this option again, but I have needed to deal with absurdly long lines.

If you can use VIM at all, you will find vim’s diffmode to be a very powerful and useful tool. If you can’t use vim already, then maybe vimdiff will be the reason you will learn vim.

I collected their reasons for importance. Some were about keeping promises, some were about providing increased quality (uptime, correct function), and there were other kinds of importance. We wrote them on a 3×5 card. Ultimately, we realized that the company’s reputation is their source of importance. Users needed to know that the company will do the right things, and provide software that worked and kept them working. I thought it was a good criteria.

Next I explained that urgency is different from importance. There may be a hot demand for some cosmetic changes that don’t really impact the company’s reputation one way or another. There may be some changes that in the direction we want to go. These may be hot topics, but may not be truly important.

We could see a difference between the forces of urgency and the force of importance, but there is gray area as a sufficient pent-up demand for otherwise unimportant changes could in fact effect our reputation for service. Still, we could usually see a crisp-enough boundary for any given user story.

I then (re-)introduced the team to Stephen Covey’s four-quadrant system. It has two axes. The vertical is importance (important at the top, unimportant below). The horizontal is urgency with urgent things to the left and non-urgent on the right:

Q1 Important and Urgent	Q2 Important, not Urgent
Q3 Unimportant and Urgent	Q4 Neither important nor urgent

We took our set of cards and the team determined groupings. Was this a quadrant one activity? Quadrant 3? Quadrant 2? There were some real surprises, especially when we realized our “most important” bit of work was not urgent and should wait until quadrant one activities were finished.

Covey’s recommendation is that we tackle first those items that are both important and urgent. When we knock that stack down, we move on to those that are important but not urgent. If we have time left, those non-important-but-urgent items can “fly standby” in our sprint. We reordered the priorities in accordance with Covey’s recommendation.

We had no cards that were both unimportant and non-urgent.

At this point in time, we had what looked very much like a magic funnel. Ideas came in from everywhere, and the items that moved from the planning team to the developers came out in the “magically correct” order. We agreed that the selection and the ordering were the best we could do at the time.

Our next sprint planning meeting was remarkably efficient.

My first strategy was to get the right people in a room and have them fight it out. I think that the prioritization process should be a lot like local government (maybe a school board?) in that people should argue and complain and push and eventually settle on compromises and deals. Ultimately, I believe that people who have a strong interest in the company can make the right decisions. At least they can be right enough for the next 5 days. When you have one-week iterations, the next chance to change the agenda is never far away.

This first strategy didn’t work out, and so we went to a backup plan. A C level manager said he knew what we should do, and so we scheduled an hour or so with him, myself, our priority manager (“funnel guy”) and the senior technical developer.

Our guys used the sticky post-it 3×5 cards and papered the CIO’s windows and whiteboards. They listed the various categories of work from the various stakeholders and pasted them up in priority order.

I asked my first pedantic question: “What is the single most important thing we can work on? If you had only one story that you knew for sure would be finished this week, which would it be?” That led to a nice discussion.

When they placed the card on the table, signifying that it was definitely in the build, I asked again if there was any one card anywhere else in the room more important. I asked if that was really the single most important one.

When the answer was “yes, absolutely” I was ready for my second pedantic question: “Now that this card is off the board, what is the single most important card left for us to do, if only two stories were guaranteed to be done.” Now the pedantry was fully exposed, but the idea had carried. The team collected all the most important stories and placed them in order on the table.

Now I was ready for my next round of pedantry.

Recently, I’ve been working to establish another magic funnel. One of the first things we did was to find the person who formerly handed out work to the developers and made him a single point of contact. In our case this is not the scrum master, but is another trusted line-manager type. He has been given a number of people to work with on the Customer side of the house and also works with technical people.

We have tried to establish story feeds from all the various stakeholders. Developers, operations people, technical writers, customers, sales people, marketing people, inside security consultants, and others have been feeding their ideas in to our point-of-contact man. This part of the funnel is working fairly well.

The next thing we have tried to do is to match the feed of work to the rate at which work can be done. This has created a fair amount of back-pressure on the stakeholders (which I believe to be healthy).

We have also worked on making the bottom of the funnel narrow, meaning that our guy doesn’t scatter new work to the team, but feeds it through the scrum master who protects the team’s productivity and keeps the work flow and tasks visible on the status board. He makes sure that the team is not expected to “absorb” changes, but that adding two points of work to the iteration results in two points of unstarted work coming off. This also creates a healthy back-pressure.

As a pun on “in-flight”, I named another area of the board “flying standby”. This is for stories that aren’t important enough to swap a story off the board (or for stories that are displaced by more important ones). If the developers finish more work than they expected, there are stories that can be picked up even though they’re not a scheduled part of the iteration. Stakeholders are told that there is no guarantee of these stories being picked up at all in this iteration, but there is some small chance that it could happen if the team discovers that it has overestimated some other stories.

The bottom of the funnel is working pretty well.

What’s missing is the “magic” bit.

All kinds of stories come pouring into the customer group. Some are from actual users in the field who want improvements to existing features. Some suggest entirely new areas of functionality into which the program could expand to good effect. Some are from the operations group, generally describing ways to make the product more reliable, configurable, or manageable. Some are from the developers who realized that some small changes in code could provide large changed in capability or usability. Some are from external regulatory bodies, some are from quality control, but for the most part they’re all good ideas and many have urgency.

When the magic funnel is working, we pour all these stories into the top, and the thing that falls out the bottom is the single most important, urgent task we could possibly do. When it comes time for a new iteration, we fill in our allotted space with whatever comes out the end of the funnel. Whatever we start doing on Monday, it is the best thing we could possibly start doing. We know that it is true because it came out of the funnel that way.

Sadly, magic funnels are hard to come by. We approximate them with human beings in multi-disciplinary small groups. If we leave the funnel purely in the hands of operations, then operational stories are the first to come out. If we leave the funnel in the hands of marketing people, we tend to do things that are more speculative even though the product users don’t really need them right now. If it’s sales, then whatever will sell more product this month will be the first thing out of the funnel. Give control to customer support and we find ourselves constantly appeasing customers and not moving forward on larger issues. If we leave it to developers, it will tend to turn out those stories that improve the experience of developing the application. Of course the problem is about conflicting perspectives (all of them right!) with competing desires being processed in fixed time.

I’m not an expert in creating magic funnels, though it would be a very interesting direction to take at this point in my agile coaching career. I believe that the answer is ultimately to be found in people working together, well-informed people arguing and debating and even trading favors.

I think that part of the answer is Covey-style prioritizing to select the urgent and important (Q1), then the important items that aren’t urgent (Q2), and holding all unimportant work until there is nothing better to do.

Importance in this case relates to how much the change will affect the future of the product and the company. This is not just a matter of closing a contract or soothing an angry customer. It may be that the future is larger and brighter than our immediate concerns. Immediacy is “urgency”, but the leverage to create a more greater future is “importance”. I had to read Covey’s First Things First several times to really understand the difference, as may the kind reader. But I did manage to absorb the fact that until you know what you want your future to be, you don’t know what is important.

But imagine how great it will be when you have established your magic funnel, and the work being assigned to each iteration really is the most important and valuable thing that you could do next. Imagine what it would be like if you really were building the future you would like to live in.

The group has been working to stabilize and improve a product that was developed by a tiger team of outsider contractors and handed over to the in-house team post-facto. The developers had to overcome many obstacles to come up to speed on the code, to learn the new programming languages and tools, and to try to keep the feature set moving forward. None of them had been involved in the original design, but it was now their product, and its problems were their problems.

At the time of this meeting, I was one part of a coaching team which had introduced a great many changes. We were trying to help the organization to build a “whole team” mentality that encompassed documentation, security, systems administration, QA, customer representatives, and developers alike. We’d attacked the problem of matrix management. We reorganized the seating floorplan. I think at some point we’d been an inconvenience to just about everybody. The developers were in the middle of their second two-week iteration.

On this occasion, the team was in the midst of a recurring production difficulty, and had been gathered into the CIO’s office to work through a top-20 list of problems to solve.

“Your attitude has been affecting other departments”, he said. “And I want to thank you for it.”

I guess I get hung up on words. Essentially, “velocity” is fine. It is the rate at which we’re “burning through” our stories. We use “yesterday’s weather” to set our level, and we are always seeking any sustainable way to increase our velocity. Why would we produce only 12 points when we could produce 20? As motivated developers, we aren’t yearning to accomplish less. But somehow this doesn’t tell the right story.

When our velocity is decreased, it always seems like a tragedy. We have all felt it when our 23-point iteration is followed by a 21-point iteration. We can justify it a lot of different ways, but it always feels tragic. Worse, hitting any kind of plateau seems frustrating.

You increase velocity by pushing harder. If you push harder, and the numbers come up, then you’ve successfully increased the velocity and that seems to be intuitively all there is to it.

I’ve reflected about this for a few years before I realized that I really like the term capacity much more. It is also accurately describing the rate at which we can truly complete stories. Since we use yesterday’s weather, we can even use the term “proven capacity” to describe that we’ve demonstrated the speed. But it also says some other things.

Intuitively it seems that we know that you don’t push your people beyond their capacity to get work done. Instead, you want to increase their capacity. When we talk about longer hours, it is still obvious that working people harder diminishes their capacity. Instead, we want to be working at capacity on the one hand, and increasing our capacity on the other.

Increasing the capacity sounds like the right problem. To improve capacity, you want fresh workers working hard, pairing and communicating, building and/or using better tools (or using tools more capably), using more powerful hardware, learning, and performing the kind of housekeeping that keeps the team productive. Life balance enters into it. Sufficient team size enters into it so you’re not so small you’re short-handed, yet not so plentiful you trip over each other.

I don’t suppose that anyone is going to rename standardized eXtreme Programming terms like “velocity” just because some guy from Indiana says to, but I wonder if maybe we can’t rephrase the problem in such a way that we are intuitively driven toward more reasonable solutions. Maybe we should explain that velocity is just capacity, and then work from that point onward.

I’m a sucker for a good one-liner. That one had me thinking, and as I’ve had other conversations about innovation, I keep coming back to that line.

In our Agile practices, we work really hard for a week or two, and then hold a retrospective. The purpose of the retrospective is to find ways to work more effectively for the next two weeks. As we develop better software, we also evolve a better team. We use “tricks” such as tracking our velocity and recording blockages on our ‘waste snake’ to provide data for our decisions, and we use gut feel to evaluate those things that feel like collateral effort to us.

If the practice works, we will see incremental improvement in the team. We will develop ways of avoiding special variations, and we will learn to accept our normal variations. It will make us better at the way we do things now.

XP didn’t come from a series of incremental improvements to the waterfall technology. I wasn’t there when it happened, but it seems that they took on an change in axioms. They didn’t strengthen the contracts between groups, but pulled all the decision-makers onto the same team. They didn’t find more careful ways to examine the code they were changing, but rather decided to lean radically on volumes of tests. They didn’t build practices to improve their anticipatory design, they decided instead not to anticipate at all and simplify their design to allow future change. At the time, this was radical stuff.

I’m sure there have been many other less-successful process mutations, but there is no evolution without mutation.

The man behind the iPod, iPhone, and MacBookPro has had some less-successful product ideas, too. Some exciting high-concept products didn’t make it in the wild. But then some new ideas become category killers.

How do we learn to make the axiomatic changes that lead us to radically better ways to get what we want?

Yes, I know that volumes have been written about TDD and BDD, and that it’s a large topic with many many branches of application, but I didn’t have time for that. I had time for three index cards. I figure that an index card is a token, and it represents a conversation, and that one can always dig deeper later.

They looked more or less like this:

---

Card 1: Uncle Bob’s Three Laws (Object Mentor)

1. Write no production code except to pass a failing test.
2. Write only enough of a test to demonstrate a failure.
3. Write only enough production code to pass the test.

---

Card 2: FIRST Principles (Brett and Tim at Object Mentor)

• Fast: Mind-numbingly fast, as in hundreds or thousands per second.
• Isolated: The test isolates a fault clearly.
• Repeatable: I can run it repeatedly and it will pass or fail the same way each time.
• Self-verifying: The test is unambiguously pass-fail.
• Timely: Produced in lockstep with tiny code changes.

---

Card 3: Flow (using the famous three-node circle diagram) – origin unknown.

• Red: test fails
• Green: test passes
• Refactor: clean code and tests

Sure there is plenty more, but I didn’t know how I could provide significantly less. As is, I’m pretty happy with the exercise. Now I am wondering if I couldn’t produce most of the important information I wish to convey as a series of index cards. Would that be cool or what?

Where to start?

Documents

Why not look at the documentation? <laughs> Just kidding. You know there’s no documentation, and it’s probably not useful.

People and Interactions

You need a partner who is familiar with the code. Having a guide is better than having a map. Not only will you know where to go, but you’ll know ’’how’’ to go and where not to step. Other people are a wonderful resource. Just don’t settle for someone doing the work in front of you… the goal is to learn as much as to do.

No, really, documents!

OTOH, if you have somethng in the way of a summary or architectural overview, that might help. I’d read the abstract and look at the pictures. We’re after big-picture, so read it only, don’t trust it. Documents are seldom accurate, and seldom for long. It might be good preparation for your partner’s visit.

Use The Tests, Luke

Do you have tests? If you are in a test-driven shop, looking at unit tests is a good idea. If they’re written well, they are specifications (by example) of the system. If they’re written poorly, you know where to start your work at least. Your partner could do a great service by helping clean up the code instead of explaining it.

No tests? Uh, oh. No tests and no doc? Now you’re in trouble.

Scan the File Space

Are the classes well-named? Do you know where to begin working? If you can find code by looking at file names then you’ve got the handle you need. Your partner can help improve file naming if you find the code you’ve been handed is in ill-named files.

So you’re in a file, you have a mission. Either you have tests, or you’re ready to write some.

Sense Of Smell

What if the code is still not obvious? You have one more resource before you have to print out the listing and break out the markers to reverse-engineer your way out of code hell. You have your nose and your refactoring editor. You can use refactorings such as ‘rename’, ‘extract method’, and ‘introduce variable’ to clarify an existing method. You can spot duplication and eliminate it. Maybe you can bring it to a point of clarity, and then you will know

Deep Code Spelunking

It’s time to pour a really big cup of tea, get some food, take a couple of preemptive aspirin, and make sure your printer has paper.

Sometimes, you will have to reverse-engineer some of the code you were handed. Your only hope may be to rebuild it from the inside-out.

You might want to either copy the code to a scratch space to do this. Maybe you want to branch the project in version control. You want to be free to dig in an learn this the hard way, and make it better as you emerge from the depths.

You still can/must use refactoring tricks to capture the knowledge you gain, but you may need to take the code off line. And look for new partners.