Heedful Programming: The Collective Mind of Agile Teams

As iron sharpens iron, so one man sharpens another. Proverbs 27:17

Though one may be overpowered, two can defend themselves. A cord of three strands is not quickly broken. Ecclesiastes 4:12

Software development teams consist of humans with varying degrees of self-control and conflicting self-interests. While this diversity adds to the complexity of team dynamics, there is an aspect of teams that can align human nature with disciplined behavior. When humans relate closely with each other they experience peer pressure.

As a child, my parents warned me about the dangers of peer pressure. Searching for acceptance, young people are particularly vulnereable to following the crowd despite their better judgment. While adults are not as suseptible to abandoning their convictions when their peers head in a different direction, there is no denying that peer pressure's powerful influence is involved in group interaction at any age level.

There is an upside to peer pressure, though. Peer pressure can act as a support system, as a saftey net to fall back on when our own discipline falters. This phenomenon provides the foundation for support groups built around those who are recovering from substance abuse, mental illness, and addictive behavior. In the field of software development, proponents of pair programming (Williams & Kessler, 2000) use peer pressure as a means to improve developer discipline. Humans are less likely to fall prey to our own selfish instincts when there is a witness present.

While sitting two developers next to each other often enhances disclipline, it does not in itself end the struggle for discipline in software development projects. There is an attribute of successful teams that produces the discipline required for success.

The concept of Heedful Programming helps capture this struggle for discipline. Heedful programming is based on a study by Weick and Roberts (1993) on the 'collective mind' of aircraft carrier crews. During their study, Weick and Roberts observed a phenomenon they referred to as 'heedful interrelating'. They found that as heedful interrelating increased, organizational errors decreased. With an everpresent potential for catastrophic failure, heedful interrelating is critical on aircraft carriers. Although the majority of software development projects do not share in this severity, they share a fundamental similarity: complexity.

Roy Miller's work on growing software (2003) attests to the complexity of software development: "Software development is a messy problem. The only way to solve it is to interact with each other, and to let our understanding and path forward emerge." This flies in the face of traditional approaches of software development in which complexity is controlled through rigorous scheduling, analysis, and design prior to beginning development. Miller argues that our attempts at control are based on faulty assumptions, passed down to us from the manufacturing mindset of the industrial revolution. A growing number of software professionals are recognizing that our grand attempts at prediction and control are a facade.

A Heedful Programmer...

• Sees her actions from her team's perspective.
• Communicates openly, deliberately, and continually with her team.
• Constantly refines her activities based on continual feedback.

Weick, K. & Roberts, K. Collective mind in organizations: Heedful interrelating on flight docks. Administrative Science Quarterly, 1993, 38, 357-81

Williams, L. & Kessler, R., The Effects of "Pair Pressure" and "Pair Learning" on Software Engineering Education, The Thirteenth Confefence on Software Engineering Education & Training, 2000, 59-65

...a high-discipline methodology is fragile.

Weick and Roberts developed the notion of 'collective mind', which they take to be, not a given property of a collectivity, but the pattern whereby individuals interrelate their actions. The more heedfully individuals interrelate their interactions, the more likely it is that unexpected events will be handled adequately. The significance of this cannot be overestimated because in high-reliability organizations it is extremely important that interactions between small, unexpected events do not escalate to yield catastrophes.

How might heedful interrelating be increased? Weick and Roberts (1993) suggest three ways, the first of which is directly relevant to our discussion of distensio: by making connections across time, activities, and experience. Weick and Roberts (1993: 366) explain: '[by connecting longer stretches of time] more know-how is brought forward from the past and is elaborated into new contributions and representations that extrapolate farther into the future'. By making connections between the past, the present, and the future, collective mind becomes more complex and, thus, is strengthened, because, 'the scope of heedful action reaches more places' (Weick & Roberts, 1993: 366). In this regard, Weick and Roberts (1993: 368) extol the significance of organizational members developing their 'narrative skills', because it is through them that collective mind becomes richer and more complex. 'Stories', argue the authors, 'organize know-how, tacit knowledge, nuance, sequence, multiple causation, means-ends relations, and consequences into a memorable plot' (Weick & Roberts, 1993: 368).

...Complex patterns can be encoded by patterns of activation and inhibition among simple units, if those units are richly connected. This means that relatively simple actions may be able to apprehend complex inputs if they are organized in ways that resemble neural networks. p. 359

Need to learn more about neural networks. They sound like complex adaptive systems.

...Connections between behaviors, rather than people, may be the crucial "locus" for mind and that intelligence is to be found in patterns of behavior rather than in individual knowledge. p. 359-360

Need to compare this behavioral notion with Berger and Luckmann's social constructionism.

Contributing, representing, and subordinating actions that form a distinct pattern external to any given individual, become the medium through which collective mind is manifest. p. 364

Need to study these three actions a bit more.

We suspect that organic systems, because of their capacity to reconfigure themselves temporarily into more mechanistic structures, have more fully developed minds than do mechanistic systems. p. 377

Need to define what they mean by "organic systems".

If heedful interrelating is visible, rewarded, modeled, discussed, and preserved in vivid stories, there is a good chance that newcomers will learn this style of responding, will incorporate it into their definition of who they are in the system and will reaffirm and perhaps even augment this style as they act. p. 367

Agile teams must continually and explicitly encourage and reward heedful interrelating. This is a topic that requires more attention. Something novel may emerge here...

No matter how visionary or smart or forward-looking or aggressive that one brain may be, it is no match for conditions of interactive complexity. Cooperation is imperitive for the development of mind. Reliable performance may require a well-developed collective mind in the form of a complex, attentive system tied together by trust. p. 378

Need to look more closely at "interactive complexity". The point that is being driven home here is that in the face of complexity, a team with a developed mind will prevail where individual minds, even brilliant ones, will falter.

Related to trust, Peter Merel writes: "Compassion isn't mere kindness, but a willingness to enter into long-term relationships, to accept and nurture the viewpoints of developers, managers, and customers, with any of whom you may disagree, to help them find ways to accept each other and build their community." Extreme Programming Examined: "The Tao of Extreme Programming"

Agents working alone have less grasp of the entire system than they do when working together... p. 357-358

A case for pair programming and switching pairs frequently. Open, constant collaboration.

Overlapping knowledge allows for redundant representation that enables people to take responsibility for all parts of the process to which they can make a contribution... p. 360

Find agile quotes to reflect this idea. Pair programming, collective code ownership.

From Beck: "By accepting responsiblity for your own development process, you are accepting responsibility for being aware and fixing it when there is a problem." Extreme Programming Explained

As seasoned people become more peripheral to socialization, there should be a higher incidence of serious accidents. p. 368

Those people who may be viewed as architects or lead programmers must not be physically seperate from the team. Removing talented and/or experienced people from the workspace removes them from the collective mind. Their social and technical contributions will be dramatically diminished. See Cockburn's "Expert in Earshot"...

Comprehension can be increased if more levels of experience are connected, as when newcomers who take nothing for granted interrelate more often with old-timers who think they have seen it all. p. 366

Pair programming between master craftsman and apprentice. Find quotes from Cockburn, others.

Pete McBreen: "As soon as a person stops practicing, her mastery fades. Hence, beginners are apprenticed to a craft, because people cannot afford to take time away from their craft to teach others." Software Craftsmanship

The quality of collective mind is heavily dependant on the way insiders interact with newcomers. p. 368

Agile teams will gradually lose their agility unless they can effectively incorporate new members. There seems to be a newcomer/oldtimer theme emerging.

The collective mind that emerges during the interrelating of an activity system is more developed and more capable of intelligent action the more heedfully that interrelating is done. p. 365

Communication should be deliberate, continual, and highly valued. From the Agile Manifesto: "We have come to value individuals and interactions over processes and tools." Manifesto for Agile Software Development

...Organizations concerned with reliability enact aggregate mental processes that are more fully developed than those organizations concerned with efficiency. p. 357

I would venture to guess that the vast majority of software development projects are more concerned with reliability than efficiency. I may be taking reliability and efficiency out of context, though. They may be speaking in terms of a team's execution rather than the product a team produces.

Narrative skills are important for collective mind because stories organize know-how, tacit knowledge, nuance, sequence, multiple causation, means-end relations, and consequences into a memorable plot. The ease with which a single story integrates diverse themes of heed and action foreshadows the capability of individuals to do the same. p. 368

The quality of a team's narratives will directly impact the ability of the team to incorporate new members. Agile teams must deliberately monitor and tend to the narratives they construct.

People know the locations rather than the details of common events and rely on one another to contribute missing details that cue their own retrieval. p. 358

Knowledge is shared and distributed throughout a team that continually collaborates.

If participants know how to coordinate their activities with the technologies and people with which they interact, the global structure of the task performance will emerge from the local interactions of the members. p. 360

Favors emergent direction from the team over following a predefined plan. Confirm this. Roy Miller's "Growing Software" probably has some material to back this up.

In habitual action, each performance is a replica of its predecessor, whereas in heedful performance, each action is modified by its predecessor. p. 362

We must be heedful programmers, constantly tracking our actions and decisions. Constantly refining our processes based on continual feedback. Habituation prevents creativity. Kerth's work on retrospectives can add to this.

Cockburn on discipline: "Discipline involves a person choosing to work in a way that requires consistency. Process involves a person following instructions." Agile Software Development

The more heed reflected in a pattern of interrelations, the more developed the collective mind and the greater the capability to comprehend unexecpected events that evolve rapidly in unexpected ways. p. 366

Agility is the outcome of a well-developed collective mind. Per Highsmith: "Agile development is not defined by a small set of practices and techniques. Agile development defines a strategic capability, a capability to create and respond to change, a capability to balance flexibility and structure, a capability to draw creativity and innovation out of a development team, and a capability to lead organizations through turbulence and uncertainty." What is Agile Software Development?

People can't be careful unless they take account of others and unless other do the same. Being careful is a social rather than a solitary act. p. 373

Heedful programming requires us to view our individual actions from the team's perspective. Pair programming would seem to encourage this.

Heedful interrelating of activities constructs a substrate that is more complex and, therefore, better able to comprehend complex events than is true for smart but isolated individuals. p. 373

This agrees with Robert Martin's idea that a team of average developers who communicate effectively will outperform a team of isolated superior developers: "A team of average developers who communicate well are more likely to succeed than a group of superstars who fail to interact as a team." Agile Software Development Principles, Patterns, and Practices, p. 4

...Carrier operations are a struggle for alertness and that the concept of heedful interrelating helps capture this struggle. p. 373

Software development projects are a struggle for discipline and the concept of heedful programming helps capture this struggle...

If dispositions toward individual and collective heed were increased in most organizations in conjuction with increases in task-related interdependencies and flexibility in the sequencing of tasks, then we would expect these organizations to act more like high-reliability systems. p. 376

"Increases in task-related interdependencies" sounds like the depedencies that exist between the XP practices. Or I could be wrong.

A culture that encourages individualism, survival of the fittest, macho heroics, and can-do reactions will often neglect heedful practice of representation and subordination. Without representation and subordination, comprehension reverts to one brain at a time. p. 378

Their use of "subordination" is not used in terms of subordinates blindly following the orders of superiors. "Subordination" means to submit to the will of the team and to heedfully contribute to it.

Heroics are an interesting phenomena in software development. Most managers seem to look more favorably on a team that develops a faulty system and works late to get it working, than on a team who works forty hour weeks while developing a well-designed system with few problems. That first sentence is probably difficult for most US natives to accept, since the list describes what our culture esteems in people.