Chapter 12 - Managing the environment

Why should a Software Architect care about management?

You can invent all of the magnificence your intellect, your experience and your insight might conjure together. Invention, and this is what most inventors fail to grasp, is not delivery. All of the good ideas in the world are worth less than a cloud on a planet of water if there is no-one to deliver them.

The world is full of soap box inventors screaming ‘why will no-one listen?’ We do not need any more. We need inventive deliverers. Invention is only the beginning of software architecture. The rest is delivery, and delivery takes management and salesmanship.

You may work with a project manager, or a team of project managers if delivering large systems. If you have created something a project manager cannot grasp, and cannot be sold, then it is unlikely to be delivered. Project managers are under significant pressure to deliver, and if you, as software architect, can help them to do so, they will help you deliver your vision. If you have a good relationship with a project manager, they may even go out of their way to deliver your integrated, multi-magical system if they have faith in you as a person.

It is your job to manage the technology. It is the project manager's job to deliver. If you can help each other you will do far better than if you antagonise each other.

Why should a Software Architect care about selling?

You need to sell your vision to yourself, your business, your managers, your co-workers and your juniors. You cannot simply deliver an architecture. It is an ongoing struggle against the fundamental nature of matter immortalised in the 2nd law of thermodynamics.

The 2nd law states that the entropy of isolated systems always increases. Entropy is energy lost to the internal workings of the system and no longer available to do useful work. In other words, it is a measure of the internal disorder of systems. 

What is almost unique about it as a physical law is that it is not always true, just in the vast majority of cases. Only you, through selling your vision, process and self, can reduce entropy in your software systems. To do so, you must stand against nature. If you ever wanted a challenge, you will be hard pressed to find a greater one.

What makes projects succeed or fail? 

Far more projects succeed or fail because of people factors rather than technical factors. Alistair Cockburn[2] has suggested that people are by far the most important influences on project success. I would agree with this to a point, but believe that a defined and followed software process is key to helping us address the people problem. 

As for technologies, I would be surprised if a poorly led team with the latest tools and technologies, produced better software than a well motivated team of amateurs using technology from a decade ago .

Why projects fail

Projects fail because of:

If you can’t agree what you’re doing at the beginning of a project, you will never finish it.

How many times does this have to be said before people get it?

My favourite word in the English language is Chaaaarge! Especially when it’s into the valley of the death with the six hundred. All of our greatest moments and greatest follies have been crowned with that glorious hurl into the teeth of fate. Many believe it is suitable for software projects too. It isn’t.

You and I both know this means no documentation.

If anyone ever says to you ‘but I didn’t understand the design.’ It is your fault, not theirs. Don’t use the old ‘Why didn’t you ask?’ chestnut.

Why projects succeed

A project will succeed if:

The CSSA published the following list of success and failure factors[3]:

Success factors Element Failure factors
Short and realistic
Split into phases 
Flexible
Timescale Long
Unrealistically short
Tight process
Mutual commitment 
Speedy decisions
Approvals and acceptance No formal process 
Procrastination 
Lack of delegated authority
Well specified 
Clear acceptance criteria 
Under management control
Requirements High level 
Ambiguous and fluid 
Open to interpretation
Flexible 
Contingency plans in place 
Payment by deliverables
Budget Piece-meal 
Tight 
No contingency plans
Skilled staff available 
Trained and experienced 
Effective structure
Project management Skills not available 
Overloaded or inexperienced 
Structure too layered
Fully committed, accountable 
Attention to detail 
Success oriented
Business attitude Lacks interest, delegated 
Hostile to bad news 
‘Must win’ at all costs
Complete 
Well defined 
Relevant metrics
Objectives Incomplete 
Vague 
No metrics
Applied from start 
Regular assessment 
Contingency plans
Risk management

Ignored
Spasmodic
No contingency plans

Firmly applied 
Documented
Change control Loose 
Not documented
Close Partnering 
Good communication
Supplier relationship Adversarial 
Poor communications
Balanced 
Success oriented 
Stable 
Qualified and experienced PM
Project team Unbalanced 
Procedure oriented 
Volatile 
Diffuse project management
Thorough 
Fully documented criteria 
Pre and post integration 
User involvement
Testing Sketchy 
No benchmarks 
No user involvement
Integrated in team 
Own the requirement 
Involved throughout project lifecycle
User involvement Not in project team 
Kept at arm’s length 
Only involved after delivery
Planned at start 
Involves users 
Tracked
Training End of development 
Done by supplier only 
Incomplete
Established solutions 
Realistic expectations 
Not over complex
Technology Drives the project 
Lure of the leading edge
Modular 
Based on standard software 
Core first, functionality added
Design Monolithic 
Bespoke
‘Big bang’ implementation

There are further elements of success and failure which depend upon the arena the software is to serve.

Public projects have:

Finance projects have:

Retail, information and manufacturing have:

Applications for the control of hospital equipment are far more rigorously tested than business applications, and applications for real time control are developed with poorer tools, and higher quality requirements than business applications.

Should software teams be driven by quality or productivity?

Faster, Cheaper, Better rings the latest sales pitch at NASA. It used to be faster, cheaper, better: choose any two of the above. It was intended to explain that if you chose faster and cheaper, it would not be better.

NASA appears to have succeeded on all three by making spacecraft that meet all three goals simultaneously. 

As pointed out in Peopleware[4], when we think of highly productive environments, we also think of highly efficient processes and high quality output. This implies that more of the right kind of thinking is done up front in defining these environment. We must bring that thinking into the general chaos that is software development, and make it faster, cheaper and better. We can do so as software architects by defining the vision, the environment and process, and managing delivery within that environment.

Getting someone to do something

Peopleware claims:

There is nothing more discouraging to a worker
than the sense that his own motivation is adequate
and has to be supplanted by that of the boss.

You may be doing entirely the wrong thing by trying to get someone to do something. They may be fully committed to doing it and your foolish impositions on them can remove their will to do so. This is why so many project managers fail to deliver. They view progress as their will imposed on others by means of a Gannt chart and list of project milestones which must be met.

It's something akin to trying to make a hole through a piece of soil with a worm. You have the following options:

I am not suggesting software development has anything to do with worms, just that forcing something is the wrong thing to do. Do not overlook this fact. It may save you an awful lot of work and an equal amount of pain.

When you must ask for something, it is important to ask in the correct way. Your request must be in tune with the listener, or nothing will happen. Children are the absolute experts at selective listening. You can shout exact orders from the distance of one atom directly into the ear of a child, without the slightest recognition of your efforts. However, whisper that you have chocolate in your pocket, and you will find the deafness magically lifts. Then you will see how to get something. Just ask for it over and over again. No-one can withstand the whining ...but I want... repeated until the said item arrives.

If you ask someone to do something, there can be responses numerous enough to gladden the heart of any statistician. They may do it, they may not.

If they are highly motivated with the ability to do the task, and they are not yet doing it, ask and get out of the way.

Figure 12.1 Motivation and ability

Equally, if someone is not motivated, and has no ability, ask them exactly the same way. After some time this will filter through into at least some motivation. It is far better to raise motivation than to raise ability. Motivation in itself will seek out ability. It is a facet of our nature.

Ability is the result of understanding, training and experience. Motivation is the result of a desire to achieve, confidence, security and possibly incentive.

Your insoluble problems will not be getting an individual to do something, but entire organisations to do something.

Motivation

Abraham Maslow[5] suggested that there are five motivational levels to we humans.

They are:

Man's most basic needs are physiological. They are the ones which help us stay alive: eat, drink, sleep, keep warm etc. When we are deprived of them, we die. When they are satisfied, we become motivated by safety. Being safe is our desire for protection against danger or deprivation. We seek shelter and security.

When our basic needs are satisfied, and we feel safe, then we yearn for love, or belonging. We become members of groups: family, class 4B tearaways, Kingston chess club. In these groups, we give and receive friendship and understanding.

Then we desire, and are motivated to do things which give us self-esteem and self-respect. We build our reputations and have a need for recognition, appreciation and the approval of those we relate to.

Finally, at level 5, we are on our own. We want self-fulfilment. We are motivated for self-development, creativity and job satisfaction.

How does this relate to software development, you may ask, and the answer is that all of those people you are working with are striving equally for self-fulfilment. It is peddled as the goal of life in all major religions, and by all psychologists. The motivation for self-fulfilment is a huge source of energy, and if you can get out of the way and let someone reach for self-fulfilment while helping you to deliver an architectural system, your life will become and endless sea of ease. And you will be helping people reach what it is they strive for.

Motivation is the key to everything. If we are motivated and do not have the ability, then we will find the ability. That is why it is a mistake to try and take someone from the low ability, low motivation state to the high ability, low motivation state. All the ability in the world will not mean someone will do something.

To change an organisation, you must have a vision to sell, and be able to sell it. The vision will come from your technical and artistic excellence. Selling the vision will depend upon your sales skills, and most importantly you.

You!

This is the trickiest part of the equation. You may have technical, even artistic skills. If you have grown up in the technocentric software world, you are less likely to have selling and personal skills. It is not a criticism. It is simply a fact. I shall deal with neither here, but point you in a good direction. For selling skills, see [6]. To improve yourself, I recommend The Seven Habits of Highly Effective People[7].

In the 7 habits, Covey leads us through seven layers of self improvement, beginning with personal victories, and ending up with maintaining yourself in your improved state, or as he calls it, sharpening the saw. You will have heard all of the habits before in management geek speak, and men of many clichés. Yet Covey’s book is profound. It will change the you in you.

To be a Software Architect you must become a leader. You must take a lead in the development process, the development community and the communication between customer and supplier. To become a leader, you must study other leaders. You must be prepared to take advice from your peers, your juniors, your superiors and your customers.

The world of should

The land of should is a painful environment to inhabit. Some live, forever trapped there by their own inability to escape. They announce with great herald and oyezing that a certain approach is the one that should be taken. The should is seen as an unattainable goal. It is the yearned for perfection that will not be delivered.

You must change your should for the world of will, and state your will with the conviction of one who knows, and will do, better.

We should do more testing, must become:
We will do more testing, and here is what we will do, how we will do it, and when we will do it.

Before further consideration, digest the following definitions:

Want-got gap

This is the gap between an ideal and reality. If I have a Jaguar and want a Rolls-Royce, I have a want-got gap. If a business wants its development groups (architects, analysts, developers, testers, project managers) to work together to create superlative software, but instead have intergroup strife, there is a want-got gap between what they want and what they've got.

Causal chain

These want-got gaps create causal chains. If I want the respect of my coworker, but do not have it, we have between us a want-got gap. I can fill that gap by earning my coworkers respect, by helping them with an intractable problem, or by putting in extra effort to help them deliver something. In the meantime, that lack of respect that I want will affect our relationship. It will not be as easy for me to interact with that person, setting off a causal chain.

Contributing problem Source problem Business Problem
personal clash
creates
lack of interaction
-> business does not respect awkward techies -> projects go less well than they could

The business sees that its projects are not going well and identifies the source problem as techies being awkward. Many smaller individual problems contribute to the source problem. It is a vicious circle as the lack of respect from the business goes to reinforce the personal clash as each believes they are doing their best for the business and that the lack of respect is caused solely by the other person.

Organisation

Figure 12.2 Organisational model

An organisation is made of individuals. An organisation is also hierarchical in that the architecture team is an organisation, as is the IT department, and the business.

The architecture team has its own climate, culture, tools and structure, and this must support the climate, culture and other facets of the next highest hierarchy in the structure, i.e. the IT department. If it does not, then it creates want-got gaps between the layers (and possibly the siblings) of the organisation.

Goal convergence

Different parts of the organisation may have different goals. If these goals lead to a want-got gap being created anywhere between layers or siblings in an organisational hierarchy, it is known as goal divergence. Filling, or aligning those want-got gaps is the process of goal convergence. Converged goals make for a smoothly run and efficient organisation. Unfilled gaps create entropy.

The world of should is at least a desire to fill those want-got gaps. However, the should is unlikely to be the will without an understanding of the gaps to be filled to make it so. To escape the world of should, start understanding and mitigating the gaps. This is further discussed in [8].

The office environment

Peopleware[4] discusses the development environment, and factors for software success.

Through a development challenge - named code wars - between software teams in professional life and at university, they identified the following.

This means there is slightly greater than a 3:1 difference between organisations. Research into the causes of difference revealed that the most significant factor was simply the area that each person had around their desks.

A larger area means a less dense population. A less dense population means less noise (background and localised), less interruption, and all the inherent problems that go with being disturbed. Writing code is a task requiring the writer to be immersed in their work. This is why programmers may seem aloof at times. It is not that they non-communicative, it is that they are struggling with mental problems.

I once showed someone around our workplace. I knew her well, and had worked with her previously. She worked as a buyer; that is, she bought in what the company sold. Her day was filled with making sure things got ordered when they needed to, and that once they had been ordered, were delivered. Her office was loud, fast moving, and what I would generally refer to as frantic at certain times of the day.

We entered the development area. It was fairly open plan, containing a dozen people, and was absolutely silent. 'Frantic, isn't it.' she said. I was at a loss to point out that indeed it was, but only in the developers heads. Frantic to her meant movement and noise. She saw nothing happening.

Immersion is important when writing code. If you cannot get immersed, you will be in an organisation at the failing end of the 3:1 difference in productivity, quality, and output.

The sound of silence

Another astounding piece of information in Peopleware is a piece of research carried out where two groups of developers were asked to develop a piece of software to solve a standard problem.

One group had piped music, the other group had silence. No difference was observed in the solution, either the speed at which it was written or the quality of the output. What was surprising was the fact that the solution took a series of numbers as input, and produced another series of numbers as the output. The two series of numbers were related. This relationship was noted by half of the people who had silence, yet hardly any of those who had piped music.

The recognition of patterns is done by a separate part of the brain to that which performs the logic required to program. The pattern recognition bit also recognises the patterns in music, so is active when listening. The upshot of the listening meant that the pattern between input and output numbers was not seen, because that part of the brain was occupied elsewhere.

We are beginning to see programmers herded together in tighter and tighter offices. To block out the mass of noise, and interruption and allow themselves to get to a point of immersion, they listen to music. I leave you to draw your own conclusion.

Hold on there a moment pardner...

If Peopleware claims that no methodology or programming language makes a difference, why is half of this book dedicated to process?

I have a few answers for that:

In making these comments, I am not suggesting that Peopleware stopped short of the mark. I consider it one of the most excellent and important books in the software library. What I am saying is that we, as software architects, exist beyond the project level and require a defined process in which we can work.

If there is no design, it will not be discovered in the code. If there is no approach, then it will not be seen in the execution. If there were no coding standards, then it will appear as if in a foreign language.

Even simple coding standards issues like different tab settings or data prefixes, and management issues such as version control and saving the source code once it is compiled can be hugely important to getting to version 2. And what if version 1 is such a mess, it must be scrapped?

Getting to version 2 is far (I would dare to say 2 to 10 times) more difficult if the version 1 programmer had no coding standards, no methodology, no requirements, no analysis, no design, and no iterated development. If they have no documentation about the final implementation, you will have to discover all of that yourself, and it is difficult, frustrating work. Not only will it be more difficult, but the resulting output will be worse.

I would like to see a study on getting to version 2 from a version 1 missing the above. I have no idea how one might go about it.

References

  1. A Brief History of Time. Stephen Hawking. Bantam amazon.uk
  2. Characterising People as Non-Linear, First-Order Components in Software Development. Alistair Cockburn. ACM 2001
  3. Getting IT Right for Government. A review of public sector IT projects by the Computing Services & Software Association.
  4. Peopleware: Productive Projects and Teams. Tom de Marco, Timothy Lister. Dorset House amazon.uk
  5. A Theory of Human Motivation. A. H. Maslow. Psychological Review, 50, 370-396 (1943)
  6. Who Dares Sells. Patrick Ellis. Thorsons, Harper Collins amazon.uk
  7. The Seven Habits of Highly Effective People. Stephen R Covey. Franklin Covey Co amazon.uk
  8. The 10-Day MBA. Steven Silbiger. William Morris/Piatkus. amazon.uk
Click here for the US book list
Click here for the UK book list