Chapter 14 - The way of the SA

Survival of the fittest

Long ago, when we used to say ugh and fight off dinosaurs with nothing more than a flaming stick swiftly dragged from the fire, the fittest survived. Not necessarily the ones with the best oxygen conversion rate and lowest at rest heart rate, but those who fitted the survival statistics best. Luck, although hugely important to an individual, meant little in the millions who made up the statistics.

You too can survive as a software architect if you are fit to do so. Survival may mean that you fit the statistical survival pattern so well that itís a breeze. The less fortunate of us must fight, or at least argue a little, to keep on top of our hill.

We can fight to win, or we can organise to win. We humans have learned that organising and politicing are just as effective as fighting. More so, if you can play the great games of democracy and debate.

Survival of the best organised

The best organised is part of an organisation. To be upheld in your state of software architectness by a professional organisation is likely to help you keep your hill more easily. If the organisation is supported by law, or charter, then you are part of the magnificent structure of civilisation itself.

What do others do?

Professional people are upheld by professional organisations. Architects are chartered by an organisation such as the Royal institute of British Architects (RIBA) or the American Institute of Architects (AIA). The people who build their architectures belong to other organisations, builders to the guild of master craftsmen, gas workers to the council of registered gas installers (CORGI). Whatever the profession, there is a professional organisation which you can support, and it will support you.

If you have a half decent organisation founded when your great-great-grandfather was still around, you get professional recognition by listing your membership in letters after your name. The letters mean you are a recognised professional in a recognised field, and even gives your level of belonging to that field: associate, member, fellow or founder etc.

There are a few software organisations, but as we are still in the formative years of software, it will be some time before these organisations can claim the same membership rewards and rites of initiation. 

An architect has the backing of formidable organisation. He has respect, as Architect is an instantly recognisable role with professional standing. The RIBA or AIA will define what you must know to use the title, help you if you have achieved the professional standing required and passed their exams, and hound you if you are not, yet claim to be, an architect.

If you tell someone you are a software architect it is likely they will tell you about their home extension, or the wonders of cathedrals. They wonít hear the software bit. It is, as yet, an unrecognised profession. For us there is no such qualification as Chartered Software Architect. At least not yet.

Yet I propose to you that the software industry itself is in the grips of a paradigm shift. Itís about time. We have collectively been creating substandard software and wasting our efforts for long enough. Software architects are emerging to lead the field of software development the same way that architects emerged as leaders of the building crafts.

We are not all Leonardos or Lloyd-Wrights. For most of us its a day job, just the same as everyone else's. We turn up in the morning, struggle through the day, and go home at night.  

The daily grind

There are many different types of software architect. Collectively, they are responsible for defining and defending their architectures. Some specialise in enterprise solutions, others in areas of technological excellence. There are those who create and defend the technological vision for their businesses, and others who are involved in the day to day running of projects.

Some are technologically carefree, some are technology evangelists. Some have a vague idea of process, others nail every action they do to it. We are an eclectic mix, but the core of our function is the same. That is, we create and uphold software excellence, or at least we do our best to.

Our working day is filled with tasks only loosely related to writing code, instead we operate in the stratosphere where ideas precipitate from vague clouds to rain down on the heads of development. We create architectures and defend them from attack. We argue our points of view and design choices. We prototype. We brainstorm. We communicate. We do our best to keep up to date with the ever growing mountain of technologies, leading people and visions of excellence, creating high level documents, reviewing others, and contributing to the act of ensuring that the code being written supports and delivers our architectures.

Creating architectures

Architectures are derived from many influences. Prominent among those influences is the business or system vision. If the business can state its long term objectives, then it is far easier to create and defend an architecture. Without it, there will be no clear vision, and nothing to hold up as a sword or shield when defending the architecture.

The stages of creating an architecture are the same as those in a project. First, the software architect must extract the architectural requirements from the business or domain. The requirements are then analysed and added to or clarified. Then comes design. The chin scratching part, when long hours pass in thought, and bosses wait nervously for a hint of output. Finally, there is prototyping to test the vision. When architectures are being created, they are continually measured against the architectural requirements. Middleware products are tested against their claims, and against each other. Suppliers are put under the microscope, their products examined and prices negotiated away. Relationships with suppliers are projected into the future to consider how the them and us relationship will progress once the initial purchase has been made. Finally, the architecture is broken down into deliverables and a programme timeline is created.

Putting forward arguments

Architectural arguments can range from the height of architectural style to the depth of technological choices. Whatever the argument, we must be able to clearly elucidate the requirements we are arguing for, and even argue to uphold the vision in the face of the ever chipping financial analysts.

Your arguments will concern software choices, design choices. You must sell your own choices, encourage others, or push back on those you deem incorrect. Incorrect will mean the price is too high, the technology will not deliver what it claims, a project will be the wrong thing to do, or being done at the wrong time. You may consider a supplier/customer relationship will not help you to defend your architecture. If a vendor comes in and sells the business their idea of a solution, you will have do do some mighty fine footwork to avoid it if you believe it will destroy your architecture.

All of this comes through argument. Not the shouting at each other kind of argument (though it may come to that on occasion), but the reasoned, thoughtful argument about what is best for the architecture and what is best for the domain or business. It may be that your architecture is holding up the business. You must accept this possibility, identify it to your leaders and then do something about it. The only way you will be able to manoeuvre around such problems is the respect you have built up previously through your reasoned arguments.

Design

Much of architecture is about careful design. Our knowledge of design is largely embodied in design patterns, which are the subject of the next chapter.

Patterns, however, are only part of the story. You must be able to design visually and technically, taking into account the wishes of the users and the opportunities and constraints of the platform and hardware you have chosen, as well as the constraints of the environment in which you work.

Defending the architecture from attack

As architect, you must review everything being created under the umbrella of your architecture. If you do not, its entropy will increase. All of the quick fixes begin to undermine your creation. It may be a seemingly innocuous change, or a slight addition to solve an operational problem, but it can signal the beginning of the end of the architecture. It may start as component interface changes, the hard coded data extracts in macros, the direct links to databases and assumed rather than looked up server names. Each one is minor on its own, but collectively, they increase the entropy of the system. They make it ever more difficult to upgrade applications, and each slight change is an attack. You must protect against them and fight for your architecture. You must explain why these attacks must not take place, and this is a very difficult thing to do when small attacks create quite an acceptable change for a business person in need of a quick solution. You need to turn the attack into a contribution towards the vision for the future.

If you do not do so, the long term effect is that each small request becomes ever more difficult to deliver. Finally, the architecture is so compromised that new technologies cannot be implemented cleanly, upgrades cannot be rolled out without severe changes, which means an even more compromised architecture. Thus begin the security holes, the ever increasing flow through the helpdesk. Old developers become frustrated, spending all their time helping the helpdesk, and frustrated new developers complain that it was never like this where they used to work.

Entropy is the slayer of architectures, and you are the knight valiant.

Creating documents

The best way to create documentation is from a set of templates. The problem is that architectures are often very different, and it would be foolish to try and squeeze them into a template. Certainly requirements fit well into a template, as does analysis, test plans, test scripts, release documents and implementation write ups. Designs fit less well, and it is best to structure a document in a top-down fashion so it follows an ever more detailed, but logical argument. 

Architecture documents are similar to design documents. They both show hardware topologies, software components and data models. The difference is that architecture documents detail systems and design documents detail single applications or components.

Reviewing documents

We may refer to it as a design review, requirements review etc, but more often than not, we are reviewing a document.

In doing so, we are addressing three issues: one, is this document complete and correct? Two, does it compromise the architecture? Three, can anything be extracted from it to extend the architecture?

Prototyping

Until software architectures are tested and proven, they are mere dreams. During prototyping, the software architect must carefully select the components and aspects which must be proven by a prototype. Too little prototyping will mean an important part of the system has not been proved, and too much means time is wasted and would be better spent during development.

Creating iterative timelines and programme plans

Architectures are usually delivered as programmes of work. Programmes are divided up into projects, and project managers then manage the delivery of those projects. However, there are cross project dependencies, and those dependencies must be a part of the managed timeline.

Leading or attending brainstorming sessions

Requirements and designs can be delivered by a Rodin like thinker, or by a gathering of ideas where the best are extracted out of people's heads, argued about and recorded. 

If you have the best architectural solution, and the person who disagrees with you is a better arguer, you will end up with a poorer architecture. You must learn to put forward your arguments. You must argue to preserve them, or accept that your solution was not the best and gratefully accept your opponent's suggestion.

Brainstorming sessions can be highly creative with the right people, or useless with the wrong people. The best sessions are where people largely agree with the way forward, and can decorate and embellish the path. The wrong people are those who dominate the meeting by asking for explanations at every point or have political agendas that have nothing to do with the brainstorming session and suffocate the life out of what could have been.

For an effective brainstorming session, get some like minded people and use an effective chairman to elucidate for the group and to lead the meetings.

Keeping up to date with the news, new methods and product releases

Strengthening the vision is as important as maintaining it. 

What may weaken the vision is a change in political situation, a change from outside suppliers, a change in internal personnel with their own ideas of what the vision for the business, domain, or architecture should be.

For these reasons, the software architect with a vision to maintain must keep abreast of the political situation surrounding their architecture. They must also keep up to date with new methods, new technologies and software upgrades.

You must have an opinion on any new technology or paradigm before it hits your inbox. You must know what it is, how it is used, and how to defend an attack by it should you consider it harmful. Each piece of new technology or new thinking promises the earth, and can deliver less than an earthworm.

Displaying the traits of leadership and success

Most of the leaders in our societies are people of unstoppable drive. They are people who relate well to people and can negotiate our myriad complexities to bring out the best in us all. True, some may have reached their pinnacles through intimidation and other unsavoury acts, but they are in the minority.

Leaders have certain traits that have allowed them to surface from the ever deepening layers of society. You must tap into that current if you are to lead as a software architect. Leaders show the traits and habits of success. They are discussed in [2] and [3].

Keeping up with the Joneses

Can you, as a software architect, remain in better contact with the leading edge of software development than your software developers?

Developers spend most of their time coding, buried in technical journals, struggling to make the plethora of fantastic technologies work together, and through the trial, become more skilled, more knowledgeable. Meanwhile, you are stuck in meetings, going over peopleís designs, reviewing their output, and worst of all, permanently detached from the path which led you to software architect. You were top banana once; you will not be when you are software architect. You will merely be the oil among the cogs.

To be useful oil, you must lead with vision. It is your vision that will make you stand out as a leader among software developers. Your vision will contain, and promote, the significant decisions made within a software system.

It is you who will define the composition of structural elements and interfaces above the level of individual projects or products. You will define the architectural style, the behaviour and collaboration required to support your organisation. You will be thinking of usability, functionality, performance, resilience and reuse. You will trade off economic against technological constraints, and guide the aesthetic whole of your systems.

You will be concerned with the architectural views which display facets of your architecture to developers, to the business, and to the individual concerns of the personalities within your business.

Your skills will be your proactivity, your experience of software development and the domain in which you are architecting. You will be goal oriented, a leader, an authority and personality, and your judgement in matters of development will be as fair and considerate as in a lawcourt.

If you are the architecture boss, and you have junior architects who answer to you, or designers who create the applications within your architecture, do your bit, then get out of the way and leave them to do theirs. Yes, reviews are good. Guidance is good; coaching in a formal or informal way is good. Interfering is not.

What you bring to architecturally led development is you. A single source of technical and artistic endeavour. A vision. A beacon around which the moths of development will gather, driven into some semblance of logic and reason by your light.

And still, we are discussing how a software architect will be, without a clear understanding of what software architecture is. The best claim for the blueprint of software architecture I have seen so far is from Grady Booch. Here it is:

Figure 14.1 Software Architecture metamodel

He states an architectural charter, which is:

He also says this:

The life of a software architect is a long
(and sometimes painful) succession
of suboptimal decisions made partly in the dark

Both are views into a model software architect.

I used the word blueprint. Booch calls it a metamodel, and uses the term Architectural Blueprint within his metamodel. The software blueprint is the holy grail of software architecture. As yet, no-one has managed to define it in the simple terms of first or third angle projection that an architect or draughtsman might recognise. Instead we struggle to constrain our blueprint by using views, and struggle again to define what those views are. It is like trying to constrain a sphere of water in zero gravity with only a pair of hands. No-one has managed to do it effectively and agreeably yet; the (software) alternatives are discussed in chapter 16.

Despite the high cost of living, it remains popular

Despite the high cost of technology, it remains popular. New technology, however, is part of the problem in that there is so much of it. Our attempts to integrate immature technologies using terms like early adopter and bleeding edge to pretend we're cool, mean we barely come to understand current technologies before new paradigms are thrust upon us.

What do our compatriots in building and naval architecture do? Do you think they would use a new material on a building or ship without it being fully tested?

Hi, says the electro-salesman (software people actually go out looking for trouble, enthused by a particular magazine article or presales blurb blunderbussed out as a sea of glistening electrons). Click here to download our untested beta. You do so. It will save a monthís programming. Youíre hooked.

Hi says the real salesman to a yacht designer. Hereís a material we havenít fully tested yet, but itís as strong as steel and only one hundredth of the weight. It's manufactured from silk spun using nano-technology exactly the way a spider weaves. We havenít finalised the bonding yet and it cracks sometimes, but that shouldnít be a problem.

Great, says the yacht designer. Iíll use it.

It is time to realise that those in other professions would not dream of delivering excellence in the environment (both controlled and physical) that we do it in.

The gulf between dreams and powers

You have dreams of magnificence. We all do. You have to deliver within a measured time, cost etc. This antagonism of concerns is what dilutes most dreams. It is what drives men to despair, artists to monoearism, and space shuttles to disaster.

We can only reach for excellence within our constraints. That may sound negative, but look at what NASA has achieved on an ever reducing budget. It has been taught to be more, not less, inventive, and gets spacecraft out to asteroids using plasma drives.

If NASA can test software and deliver it to a spacecraft on the edge of our solar system, why canít you? If they can control a lander on Mars where the round trip between signal and response is measured in hours, whatís to stop you?

The challenge for the software architect

According to Dana Bredemeyer[4], the software architect has five domains of competence.

Competency 1: Technology

What you know What you do What you are
In depth understanding of the domain and pertinent technologies
Understanding what technical issues are the key to success
Development methods and modelling techniques
Modelling
Tradeoff analysis
Prototype, experiment, simulate
Prepare architectural documents and presentations
Technology trend analysis and roadmaps
Take a system viewpoint
Creative
Insightful
Practical and pragmatic 
Insightful
Tolerant of ambiguity, willing to backtrack, seek multiple solutions
Good at working at an abstract level

Competency 2: Strategy

What you know What you do What you are
Your organisationís business strategy and rationale
Your competition (products, strategies and processes)
Your companyís business practices
Influence business strategy
Translate business strategy into technical vision and strategy
Understand customer and market trends
Capture customer, organisational and business requirements on the architecture
Visionary

Entrepreneurial

Competency 3: Organisational Politics

What you know What you do What you are
Who the key players are in the organisation
What they want, both personal and business
Communicate, communicate and communicate
Listen, network, influence
Sell the vision, keep the vision alive
Take and retake the pulse of all critical influences on the architecture project
Able to see from and sell to multiple viewpoints
Confident and articulate
Ambitious and driven
Patient and not
Resilient
Sensitive to where the power is and how it flows in your organisation.

Competency 4: Consulting

What you know What you do What you are
Elicitation techniques
Consulting frameworks
Build trusted advisor relationships
Understand what the developers want and need from the architecture
Help developers see the value of the architecture and understand how to use it successfully
Mentor junior architects
Committed to othersí success 
Empathic, approachable 
An effective change agent, process savvy 
A good mentor and teacher

Competency 5: Leadership

What you know What you do What you are

Yourself

Set team context and vision
Make decisions, and make them stick
Build teams
Motivate
You and others see you as a leader
Charismatic and credible
You believe it can and should be done, and that you can lead the effort
You are committed, dedicated and passionate
You see the entire effort in a broader business and personal context

This is quite an exhaustive list of personal and professional traits. It can be used as a guide to where you need to end up, and in the meantime, no-one can stop you calling yourself software architect. 

We have no body of knowledge. We are largely confused over our terms and methods. We have many gurus promoting alternatives.

We have the management trap. Senior software people end up doing no software. And we do not have enough software people to go around.

How to make your mark as a software architect?

The road to becoming a successful software architect is a long one. We have no clear statement of what we do, no common view or point of reference we can consult when we are considering one path or another. Thus, the area between software architect and software builder, and between software architect and project manager is difficult to define. The gray areas can be almost as wide as our areas of expertise.

Because of this lack of a central repository of information, or a software architect's book of knowledge, we are largely on our own. To succeed, we must balance many conflicting issues.

Balance

Sometime after birth, as the neurons in our brain begin to join together, and our synapses begin firing in what could be considered something of an order, we learn to balance. Eventually we stand, and our balance allows us to walk. Of course, this is only the beginning of our troubles. We must then learn to balance our food and fluid intake, our relationships, our work and home lives. We must balance our time between rushing forward and pausing for reflection, between earning and spending and between falling and getting up again.

Software architects spend a lot of time balancing.

Sell the vision or sell the process?

You want people to buy into your vision. You also want then to work within a defined process so you have some control over how, or if, your vision will be created. Your selling time and energy is limited, so you must balance the time you have between selling the vision, and selling the method by which you will get to the vision.

Why didn't you sell him the software clock?

If you cast your mind back to chapter 5, the software clock for the rather hopeless department of clock readers and announcers was abandoned. It could have been a tidy earner. The department, and more importantly, the sponsor, would have paid well and been glad to do so, but a different course of action was suggested that didn't earn a penny.

The reason is that it is better to build a relationship which will bring future work, than to fleece someone just because they are willing to pay. You must balance immediate income with building the relationships to bring future income.

Short termism against long termism

One of the recurring problems in software architecture is that is takes more time to deliver something the right way than it does to deliver it the wrong way.

You need stronger arguments than 'if we do it this way, there will be more support calls'. You must have watertight arguments for defending your architecture right down to the long term effects quick fixes will ultimately have on the architecture and on the domain or business.

This is the balance of short term thinking against long term thinking. If you are writing commercial software, the long term solution may mean the business goes bankrupt. You must then balance architectural defense against defense of the business.

Integrate new technology or deliver software?

Six months spent integrating new technology into a software system is six months not delivering anything. You must balance the promise of the new technology against the knowledge of current technology.

If technological advances do not allow us to meet the needs of our clients better, then what is the point? If it takes two months to integrate and costs the equivalent of four monthsí salaries, why should it be used when six months work could create its equivalent? You must decide if, and why?

Developing for the integration of future technologies

Similar to the previous example is the question of how much development effort should be put into paving the way for new technologies. I recommend none. Every time I have tried to do so, the new technology has not been quite what I expected, or supersedes the entire piece of software, making it obsolete.

You may have more luck. It is up to you to balance the up and coming promises against the realities of what is available today. 

Software fix or social fix?

When I write software systems, I do my best to make them secure. I don't create back door entry points. I demand well formed passwords mixing upper and lower case gobbledegook rather than allowing Username: George Password: George.

When I use software systems, I create security holes. If I need a 4 digit number, I use my date of birth. If I need a character string, I use the name that I am known by to my wife's family. In short, I am sloppy. I am lazy, forgetful, and really believe I cannot remember a new password and login for each software system, website, bike lock and burglar alarm I am faced with. Therefore, they are all the same. If I want to know someone's password for another site, I can create a dummy site on the internet, get them to create a username and password, and, as if by magic, I have their electronic keys to the web.

This is not a technological problem. It is a social problem.

Business often lumps software departments with social problems. You must push back social problems to the business or offer technological solutions.

An example

Emails are taking longer to arrive. Client server and thin client application response is poor. People are screaming. The same people have no conception of the fact that when they do a screen dump and paste it into an office application, it is faithfully rendered as a 24 bit bitmap. Thus, each picture in their presentation or document is around 2.4Mb from a 1024x768 screen grab. Most of the network users paste around ten pictures into a document and twenty into a presentation. The result is that each document is 25 to 50Mb. 

There are simple, but extra steps to be taken, to reduce this size to something less significant. For example, a screen shot can be reduced from 2.5Mb to around 50kb by converting the image format to a gif. Compressing a 25Mb document containing such images could make up to 100:1 difference.

You may have recently installed a document management system to alleviate the problems, but have found the users are still emailing documents between one another.

The social problem is one of education and persuasion. Using the document management system would help. Compressing documents or converting graphics to gifs rather than bitmaps would also help. Getting a faster network would help too. Which should you do?

Procrastination or haste?

Sometimes, putting the brakes on is the best course of action. At other times, rushing is the only feasible option. Both extremes will annoy someone. You must walk the thin line between the two to get software out of the door as effectively and efficiently as possible.

Resilient or agreeable?

The waves of wrath will break upon you, as you will be viewed as the sea wall. You will stand between the business and development, between the business and your architecture, and between development and your architecture. They will all rage against you, and you must be resilient if you and your architecture have any chance of survival. At other times, you must accept forces that may corrode your architecture if it is for the greater good, and then you must be agreeable to change.

Risk and return

There are many risks in developing software, most of which you must steer around carefully. Risk management is something the project manager will do for the project, and you will do for the technologies and architecture. 

On occasion, a risk will present itself with a return that you consider worthwhile.

Example 1

You need to get someone's attention badly to preserve your architecture. Suddenly, you - mild mannered Clark Kent - become a snorting, raging bull. This is guaranteed through millions of years of evolution to get someone's full attention. It can even work for a room full of people, but I wouldn't suggest trying it more than once in a blue moon.

Of course you may get to practice it only once per job. That is why you must balance its risk against the return you get.

Example 2

SuperWidgets inc has just released a database killing technology. It is based on a search engine and an object database and builds 3D indexes of all the data you hurl at it. There is no need to structure data, you merely pass it your class and it is saved. When it is saved the 3D index updates, and a cube view of it, and all related items is instantly available for searching and drilling. It is called Unstructured, Searchable Object Data.

It is in alpha test. It is expensive. The architecture needs a rejig to utilise it correctly which means a lot of work. The return is huge. Do you ignore it, and stick to your relational, difficult to maintain data model? 

Fluidity or immutability

In an early architecture, ideas must be allowed to flow. Offering and criticism must come from all stakeholders in the architecture and it must grow from amorphous lump to crystalline clarity. Once the crystal has formed, you, as defender of the architecture, must match its immutability. 

Buy or make?

It is usually cheaper to buy software than to write it yourself. When you can't get what you want out of a box, you must write it yourself. Somewhere between these two extremes is the buy and customise option. You must decide which is better.

Cost vs performance

Performance in this case does not just mean speed. It also means security, stability, fault tolerance etc. Well performing software runs on more expensive hardware and takes longer to write. You must balance acceptable performance against acceptable cost.

How was it for you?

The New Scientist ran an article entitled: Love it or loathe it: Why can't Architects figure out which buildings will be crowd pleasers[5].

Architects are hopeless when it comes to deciding whether the public will view their designs as marvels or monstrosities... [The public] say designers should go back to school to learn about ordinary people's tastes.

Many buildings that appeal to architects get the thumbs down from the public.

The report cites work from the Journal of Environmental Psychology[6] on a survey of 25 architects and 27 'ordinary' people. Each group was asked to rate a building from 10(excellent) to 0(awful). The architects gave the Disney HQ in Burbank CA a score of 4, the joe publics gave it 7. The Stockley Park Building B-3 got over 6 from the architects and less than 5 from the public, and the Old Bailey got 5 from the architects and 7.5 from the public.

Measuring the success of an architecture can only be done by testing it against its requirements. Peer review could provide some keen insights, but at present, understanding someone else's architecture is a fairly heavy undertaking. People and architects can both judge the architects' output by walking around a building and giving it a score based on their impression of it. We must determine a way to be able to document and consider one another's architectures.

References

  1. The Royal Institute of British Architecture (RIBA) www.architecture.com
  2. The Seven Habits of Highly Effective People. Stephen Covey. Franklin Covey Co amazon.uk
  3. The Power of Focus. Jack Canfield, Les Hewitt, Mark Victor Hansen. Health Communications amazon.uk
  4. The Role of the Architect in Software Development. Dana Bredemeyer, Ruth Malan. www.bredemeyer.com
  5. New Scientist, 21st April 2000
  6. The journal of Environmental Psychology  (Vol21 p93)
Click here for the US book list
Click here for the UK book list