Chapter 17 - Enterprise Architecture

The world of Enterprise architecture is one of wrestling with the network myths. It comprises the architectural concerns, how they are contained and delivered within frameworks and represented in views.

It is about architectural patterns, architecture documents and architectural insights.

You must understanding why the unit of release is the unit of reuse, how architectural vision is created and upheld, how to meld two disparate systems when companies merge, how to get from a legacy system to a new system without rewriting the whole lot at once, and how architecture will help a business reach for its goals.

The network myths or the Eight Fallacies of Distributed Computing

Deutsch[1] claims that everyone who first builds a distributed application makes the following assumptions, all of which turn out to be false.

The network is reliable

Usually, the network is reliable. It is, in modern terminology, up for a significant percentage of the time, usually measured as a percentage of recurring nines. If a network is up 99.99% of the time, it might appear to be highly adequate. The trouble is that 99.99% means the network will be down, on average, 8.64 seconds per day. 

In itself, 8.64 seconds a day is quite manageable. Applications will generally wait far longer than that before timing out. Unfortunately, networks are usually up for months on end, and when the down time comes, it makes up for each of those 8.64 seconds in one hit. This means a straight run of three months will be followed by network down for 13 minutes. During those 13 minutes, the network managers will run around panicking and pressing buttons, the helpdesk phones will reach a mutual cacophony matched only by repeating proclamations from helpdesk employees that the network is, indeed, down.

What terrible things happen during down time? Can you roll back the invoicing run and the bank update? Can those whose applications have hung get back to where they were before the dreaded down? Will they have to retype data they have not enjoyed typing once already? Will they be happier with their software systems?

Should you, as architect, demand 99.9999% uptime from the network people, even though each additional nine is increasing the cost of the network exponentially? Should you write your applications to be able to recover from a disappearing network, and if so, how long should they wait before doing something about a lost connection?

Latency is zero

Latency can be though of simply as the delay between a request and a reply. Across wide area networks, it is most often measured by a ping, or sending a small packet on a round trip to and from a remote server. A typical ping to the opposite side of the world across the internet on ground based lines takes around 250ms. Straighter wires or dedicated satellites may improve this number slightly, but until we master entanglement the measure is not likely to improve significantly.

The causes of latency are:

1.  Switches and bridges

Even within the local network, devices with store and forward mechanisms will slightly delay data packets.

2. Routers

Routers change header information. To do so, they must store, manipulate and then forward. A router is a gateway; there may be other gateways slowing the packet before it reaches the internet.

3. Transmission medium

Copper cannot transmit quite as fast as fibre-optics. Fat fibre-optics cannot transmit as fast as thin fibre-optics and require more signal boosting stations, each of which introduces delay.

4. Packet size

The larger the packet, the longer it takes to arrive. Smaller packets might arrive quicker, but you may need more of them. More than one packet usually takes longer to arrive than one packet. How do you recommend and control packet sizes when they are typically between one and four kilobytes. 

5. Propagation

The speed of light is finite. A trip to the other side of the world and back at the speed of light on straight, ground based lines, takes 100ms. Propagation alone can have quite an effect on a user in Hong Kong accessing a US web application.

Time in seconds What's happening
0
0.1
0.2
0.3
0.4
0.5
6.5 
User clicks link. Request travels to server
Server receives request, builds page and send out first packet
First packet arrives. User's computer sends out an acknowledgement
Acknowledgement arrives, next packet is sent out
Packet arrives at user, acknowledgement is sent
Acknowledgement is received. 
30 more packets are sent, each of them requiring an acknowledgement

This simplistic approach assumes an immediate response from the user and server machines, and no other network delays. The propagation delay means the Hong Kong user will have to wait 6.5 seconds for 32 packets whereas the US users will receive them instantaneously. 

In reality, the other latency and network effects magnify the propagation latency, and the Hong Kong user is more likely to wait far longer than 6.5 seconds for thirty packets of data.

6. Firewalls

The firewall stores, analyses and forwards. It analyses for malicious content, and receives many more hits than it lets through, so is not fully employed speeding through your packets.

7. Virus checkers

A virus checker has to scan through each incoming chunk of information and compare it to the signatures of thousands of viruses. This takes time.

The reason latency often escapes notice is because it is not usually present in development or test or demonstration environments. Only when an application rolls out to live does latency begin to take effect.

Bandwidth is infinite

The bandwidth of your network may be 100Mbit or 1GBit on paper, but when it comes down to moving messages around, you simply cannot use all of it. 

Data must be divided up into packets, and each packet has a header and a footer, each taking up part of your bandwidth. With the size of acknowledgements, and the collision detection/retransmit nature of most networks, you will be lucky to get 70Mbit/s of pure data through a 100Mbit network. This is the first problem.

The second problem is that if you are streaming data with a specific minimum data rate, you must allow for an overlap between incoming packets, and played or displayed packets. You must ensure that packets are transmitted fast and frequently enough not to break the stream. It's no good stopping a video or audio stream on a client while the next packet comes in. You will frustrate your viewer or listener beyond their ability to view and listen.

The network is secure

There is always a way to get into your network and onto your servers. It can be done by technological know-how, persistence and by detective work.

Technological know-how will exploit a security hole in your operating system and/or software.

Persistence means looping through endless variations of access such as login names and passwords until access is granted. Lophtcrack is one of the best known methods of persistence supported by data.

Finally, detective work will look in notepads, gather social data, search through rubbish, observe building entry and egress patterns and get into the building with a username and password.

Topology doesn't change

A piece of software might work fine one day, then not the next, because the bandwidth it requires to do its work has been reduced by a topology change. Suppose the Hong Kong office has been reduced in size and much of their work moved to the Beijing office. To avoid changing the Beijing line, Hong Kong is routed through Sydney as this is the cheapest option. Sydney has a slower line, and a different daily usage profile than the Hong Kong Office. Something has stopped working...

There is one administrator

Over the course of its life, a network will have many administrators. Each one will have their own way of doing things. If your software comes without an ikiwisi wysiwyg setup and relies on some obscure administrative tweak, then it is absolutely guaranteed that one day it will stop working.

The downtime while the magic tweak is found will quickly erode the availability below 99.99%.

Transport cost is zero

It costs a great deal of money to move information around the world. Bandwidth may be getting cheaper, but that just means we use more of it. Thus, we maintain the cost of running our applications.

Software designed with cost in mind must take into account operational costs, which are often far higher than development costs, and that includes large and/or increasing data rates.

The network is homogenous

Most large organisations have non-homogenous networks. They may use Windows and Linux desktops talking to IBM or Unix servers. Every step of non-homogeneity is a headache for someone. If your applications must work across these networks, then they will rely on both ends of the network being up, and the bridge in between them being up. If each of the networks and the bridge have individual uptimes of 99.99%, the total uptime for one application being able to talk to its server through both networks and across the bridge is 99.99 x 99.99 x 99.99%. Suddenly, your network is only 99.97% reliable. A continuous up of only one month will mean a down of 13 minutes.

Standard models

There are many models to pick from to help a viewer or designer to create or understand an architecture. Here is a list to hone your architecture from:

 

Roles

Each employee in an organisation has one or more roles. This model explains those roles

Services and Processes

 

Deployment

A list if servers and their geographical locations

Applications

A list of applications and their purpose.

Integration

The means of understanding and moving information between applications

Security

Digital signatures, intrusion detection, key management, firewalls, encyption (all from TOGAF's III-RM)

Network/Communications

Wires, leased lines, bandwidth, protocols

Technology

What technologies are used where?

Technology

 

Technology

 

Mappings

Maps may exist between any of the above models. The maps may be part of other models, and help to delineate areas of responsibility, or to specifically exclude parts of work.

The architectural concerns

The ongoing quality of service is the ultimate measure of the quality of the architecture. To maintain the architectural quality, each relevant architectural concern must be considered in the design of the software. The architectural concerns are also referred to as non-functional requirements or the -ilities.

What must it be capable of? What might it be capable of? Each of the following topics may add to the price and quality. It is your choice which concerns are addressed, and which are not.

Automatic distribution and installation

Servers are often tended by experienced people who have a knack of getting applications working. Client machines are usually tended by people who get frustrated with continual updates and keeping them running.

Creating an install mechanism for your software can help alleviate many client headaches and frustrated help desk conversations.

Availability

If you have specified 99.99% availability, how will you achieve it? Is that network or application availability? If you have offered 365 x 24 x 7, how will you achieve this?

Availability is usually achieved by building in redundancy to the system. This means that if one piece of it breaks, another can take over the function. Application availability must take into account the availability of the network, the database server, the web server, the component server, the email server and the client machine.

If your network is 99.99% available, your applications will be less available than that. Sometimes they will break when the network is fine. When the network starts up, there will be a finite delay before all the services are up and running, and your application can start.

Backup and restore

What is backed up and when? How is the system restored from its backup after failure?

If availability has been quoted, how have you predicted the increasing backup and restore times with increasing information in the system? How will the volume of backed up data affect availability when data or servers need to be restored?

Efficient backup is a balancing act, for backing up information requires energy. Using energy affects the speed and response of the system being backed up. It may even mean shutting down the application to avoid synchronisation and data locking problems.

Rotating tapes, full and partial backups, and backing up database logs and data at different frequencies can improve the overall drain on the system.

Beauty

The most forgotten of all, and the saddest omission.

Cost

There are three costs to a software system. One is how much it costs to develop. The second is how much it costs to deploy, i.e. for the hardware your software will run on. The third is how much it costs to run. Upgrade costs are simply development costs revisited.

The highest cost is the most often the running cost. The cost focused upon above all others is the development cost.

Common interface

If the system is made up from a number of applications, is the user interface consistent? This includes ensuring that keystrokes and mouse activity have the same effect in any part of the system.

Common interfaces must be defined, and if possible controlled through common views and controllers. These may take the form of compiled or script libraries and style sheets.

Compatibility

Is your system compatible with existing software and hardware? If not, why not?

Sometimes prudence dictates supporting past mistakes. Other times, a clean sweep is needed. Which prudence rules this software?

Component interchangeability

How easy is it to add, remove or replace components? 

A component can be as small as a single function or as large as a document management system. A component could also be increasing or decreasingly complex components that a customer may upgrade or downgrade to. Interchangeability is usually managed by carefully designed layered interfaces such as the proxy-stub or factory patterns.

Conceptual models

How does the software map to the users' conceptual models? How many conceptual models are supported?

Supporting more than one conceptual model can be achieved by applying the MVC or PAC patterns.

Concurrency

Occasionally, two execution paths must run concurrently to achieve an outcome which would not be realised if both paths ran sequentially. A simple example of this might be moving the arm and opening the hand to reach out for an object. In software, two disparate systems may need to be updated together to maintain two matching data sets for data redundancy. Data concurrency is usually managed within transactions. Robotic concurrency is managed in time-sliced threads

Consistency

How is consistency across applications, client and server operating systems, and thin/thick client interfaces maintained?

Control

Can the user gain control of your software? Can they work at different levels of control?

Coupling

Loose coupling minimises change and design dependency. However, it can make interfaces more complex.

Cultural Adaptability, Localization

How easy it is to change icons, menus, words and layout to suit different cultures.

Dependability

The measure of how reliable the predictability is.

Deployability, Installability

Does it require the click of a button, or a manual, fault-intolerant script to install the software? Is Configuration Management software a dream or a straightjacket?

Detail

Can the user move up and down through levels of detail or abstraction?

This is most easily achieved my marking items with a detail level and showing or hiding them as appropriate. As an example, a form may require only skeleton data to be posted, yet allow full detail to be entered. A selectable option can change between the skeleton and rich data views.

Dynamic composability

Plug and play was once the coolest kid on the block. Does your software allow for such dynamic composability, in both harware(additional servers) and software (additional software modules, components, users etc)?

Dynamic resourcing

Can your software decide which resource (human, hardware or software) to use at run time, based on system loads?

Dynamic negotiation

Can your software negotiate dynamically with external systems? Can it argue prices, conditions? Can it negotiate if an interface or function call doesn't quite match the expected parameters. Do different external systems get different levels of trust?

Dynamic negotiation can also take place to support dynamic quality of service.

Dynamic quality of service

A quality of service recorded in the software can be maintained by dynamic negotiation between running components as to who gets what resource to maintain what level of service.

Energy use

What does your system do to conserve energy? Does it have partial or complete shutdown? How does it restart itself? How will you ensure that the batteries are not depleted by the time the spacecraft reaches Mars?

Error handling

How does your software trap and handle errors. Does it have categories of error, and can it recover from some errors or categories? Are errors logged in a central location for analysis? Are bad errors automatically sent to a proactive helpdesk, where problems can be resolved even before the call comes in, or user becomes aware of it?

Extendability*

How easy is it to extend the system, or extend into the system for external systems?

Failure prevention

What will make your software fail, and how are those failures overcome?

Fault tolerance

Do you have levels of error or failure, and a means to overcome some levels of error without the user noticing?

Federation
Heterogeneity

A federated system appears as one seamless whole, despite being composed of separate systems

Feedback

Feedback can improve or hinder output. What does your software provide feedback on?

Flexibility, changeability, enhanceability

If there is a change required to the structure or part of the system, how have you ensured that it will be as easy as possible?

Globalisation

Is there a global view of geographically disparate systems? How easy it is to deploy and support globally.

Help

How is help provided, and what does it contain?

Installability

What will make your system easy to install and roll back?

Integrability

How easy is this to integrate with existing systems? If it is a replacement system, do the old and new run concurrently? 

If youíre offering 24x7x365, you must be able to swap over systems without the user being aware.

Interoperability

How does your system interoperate with others? What can be done to provide a clean and clear interoperability interface?

Learnability

How easy is it for the user to absorb the ways of the software?

Localisation

How the system (icons, words, ways of thinking) is targetted at a specific global area or belief system.

Logging

How, where and to what level of detail does the system log information, and how are those logs viewed or searched?

Longevity

How long will it last? What is the mean time between catastrophic failures? What will kill it?

Manageability

How easy is it to keep the system going from the point of view of operations staff? Are there concise and timely warnings to provide the operator with information enough to prevent failures? Can it be stopped and started easily?

Memorability

How easy is it for the user to remember the sequences of actions necessary to perform their work?

Memorability differs from learnability in that one is gaining the knowledge, and the other is retaining it.

Modularity

Modularity is the key to being able to swap in and out parts of the system.

Natural mapping

Is the process portrayed by the software the one that the user is familiar with?

Navigation

What visual aids guide the eye and mind to points of navigation? Is the map of the site or application visible or easily understood?

Obsolescence

When will the system become obsolete? What could advance the obsolescence?

Open

Can the executing objects run on different nodes, across different platforms? Can they interact across non-homogenous environments?

Performance

How fast does your system respond to users' actions? How many transactions can it process per minute or hour?

Portability

What base of code is in ANSI C or HTML, portable across different operating systems? How will you maximise the ratio of portable to OS specific code?

Predictability

How you can measure the expected response to all inputs.

Proactive support

A software system can report errors to a centralised area. Incoming problems can then be picked up by support personnel and solved proactively even before the user is aware there is a problem.

Reliability

What length of time is expected between failures? What are those failures? And how can those failures multiply to reduce availability?

Resource control

What resources are available to the software, and how does it make the best, and most timely use of them?

Responsiveness

How the system responds, under high and low loads, to user actions.

Responsiveness is slightly different to performance. Consider a user requesting a database write. In one system, the write is undertaken before control returns to the user. In another, the write request spawns another process or thread. Control returns to the user, and the created thread then manages the database write.

Both systems have the same performance, but the latter is more responsive.

Reportability

Does the system publish information about how well it is working against its predicted quality of service or service level agreements?

Return on investment

How long will the software take to pay for itself? This is a highly subjective area, and often very difficult to quantify reliably.

Reuse

How have you built in reuse? What have you reused that already exists? How does reuse minimise the cost?

Robustness

Robustness is the collation of many other architectural concerns. A robust system will fail well, and will not crumble under user pressure. It is the software's own internal defence against things going wrong.

Scalability

How does your system scale? What happens when the number of users or transaction throughput is doubled or squared?

Schedulability

How the system makes scheduling tasks easy

Security

What kind of attack is your software secure from? How does it manage this security?

Self configuring

Some software can configure itself by seeking out new hardware or software services it can make use of. For example, if a new component server or file server is added to the system, the software will find it and make us of it for load balancing, safety or backup.

Self tuning

Some software systems can configure themselves to work as efficiently as possible. A simple example of this is to ping a set of servers and use the one with the fastest response. A more complex example is measuring the incoming requests and managing pools of objects to service those requests based on what is being requested most.

Standards based

Standards flourish in IT. A standard is far preferable to inventing your own formats or methods, but can add complexity.

Style

How do you reuse common style elements, or enforce styles?

Substitutability,
Behavioural compatibility

how easy is it to replace one interface or object with another without the system being aware of the change?

Supportability

How is the system supported? What helps support staff to understand what is happening from the usersí point of view?

Synchronisation

Most enterprise systems require some form of data or document replication, or other types of synchronisation. One obvious example is that server clocks (of the 24 hour variety) must be synchronised so one event can be measured against another to find out which came first. Data is often replicated between data servers.

At a lower level, threads must be synchronised, for example to prevent multiple threads writing the same property to the same object at the same time.

System replacement
or system interchangeability

How easy will it be to unplug all or part of the system and replace it with something else?

Tailorability, Customisability

How easy is it to change your system for different viewers? Are all user interfaces user editable?

Task completion

Can tasks be stopped half way through and left until later, or resumed after errors

Task distribution

Can tasks be allocated to different servers for load balancing, or different individuals if one is on holiday or overloaded with other requests?

Technological options

What are the different technologies which can be used? How do they interoperate? What do they cost?

Testability

How does your system allow itself to be tested? What tests can be performed against what metrics?

Time to market

Of extreme importance in the commercial world, how does the architecture affect the time to market?

Traceability, Auditability

Who are what did what, and when

Transparent

The user does not have to know how the system works to be able to use it.

Understandability

How is your system easily understood?

Unity

How the parts of a system appear as a seamless whole. Not necessarily uniformity.

Usability

How easy is it to use? Can the user find what they want without a torturous search, then use it without hesitation and unnecessary input activity?

User Access

How is access granted and controlled? Must the user log in to gain access, or is it all done in the background? Do you use LDAP or have you defined your own access rights mechanism?

User satisfaction

How does the quality of the system improve the quality of the users' lives?

User's performance speed

How fast can a user perform their activities? A response of less than 1ms is largely irrelevant if it takes a minute to make all the entries.

*Extensibility is often used in error for extendability. It is not a word.

Frameworks

The architectural concerns need to be delivered by a shared or common set of methods. Those method are delivered in layered frameworks.

In its simplest form, a framework is a collection of layered or otherwise related components. 

On top of the distributed operating system will sit your middleware. Above that will be the runtime environment, such as the .net framework, MFC or the Java runtime. Above that sits your enterprise framework, and above that, the application frameworks.

Earlier, I introduced the concept of framework and tools. The framework is layered and distributed. The tools have no particular structure, and can be called from anywhere in the enterprise or application frameworks.

The term framework implies a structure which has certain properties, and can contain other objects. Each of the relevant architectural concerns must be allocated a place in the layered frameworks, and each framework must allow for incoming and outgoing calls, limiting the scope of its activity into specific areas of responsibility. A framework may be distributed or localised, depending on its requirements.

Figure 16.1 Enterprise and application frameworks

Frameworks are defined and communicated using views.

Views

Two definitions of the word view are:

1. That which is seen
2. A way of thinking

This is pretty much what an architectural view is. It is the part of a system which is seen, and the filter through which it is observed.

Just as a building can have a plan, front and side view, it can also have a plumbing view and an electrical view. Similarly, it can have an expandability view, or a storage view. They are not common, but could be relevant views for the sponsor or local building authority. 

In software architecture, almost everything we provide is a view. Views and their descriptions make up the architectural documentation.

Software processes also contain views, or viewpoints. Unfortunately, the differences in the terminology, and what exactly goes in those views, is often difficult to comprehend.

The problem is that all software processes or houses of thought are attempting to limit the views so they can be presented in a conveniently packaged way. But we are not dealing with anything that can be conveniently packaged, and so we have a huge difference in terminology and viewing options. There are as many views as there are architecturally interesting facets to a system, multiplied by the number of likely viewers of those facets. Even in a simple system, thatís a huge amount.

We are attempting to squeeze our views into a convenient blueprint based upon the 3rd angle projection and perspective drawing of a building, and they donít fit. Building plans or blueprints rarely show the electricity view, the plumbing view, the insulation view, the dynamic way the passages and doors can be used view, the how to get to the lavatory in case of emergency view, yet that is what is required to sufficiently define our software systems.

We are at a point in time similar to when the renaissance gave us the perspective. The Romans, and many other fallen civilisations, managed to build great and huge buildings without perspective. We have been able to build software systems without whatever it is we are missing. When perspective came, it became easier to see a building in one picture before it was built. Will anyone ever create a software perspective?

The difference between a view, a model and a diagram

A diagram is a complete or partial view into a model from the point of view of a stakeholder. A view can be contained in a single diagram, or may require more than one diagram and accompanying text. It is unlikely that a diagram will contain multiple views.

A view provides a complete or partial representation of a system, semantically consistent with other views.

A model is an abstract form of the software. As such a view is a model, but a prototype or mock up, or textual description is also a model. A single application can have one or more models, one or more views and one or more diagrams.

The Unified Process defines a model as a self-contained view, where a system will have use a case model, an analysis model, a design model, a test model, a deployment model and an implementation model.

Figure 16.2 Diagrams, views and models

In software, a view and a model can become somewhat intertwined. It is up to you to define what you mean by a model and a view. The relationship is one of hierarchy. Here is how the software processes define views: 

The 4+1 views of the Unified Process

4+1 is an odd but fitting description of the architectural views of the Unified Process.

The Unified Process contains a primary model set, comprising a process, logical, implementation and deployment view, plus the use case view. The process view is sometimes referred to as the dynamic or runtime view, and the implementation view is sometimes called the component view.

Figure 16.3 The 4+1 view of the unified process

The 4+1 views map directly to the UML diagrams:

The Use Case View comprises usually one, or maybe more, use case diagrams. It models a system as a set of high level functions.

The Process or Dynamic view contains Activity diagrams, State diagrams, Object diagrams, Collaboration diagrams and Sequence diagrams, and models control and intercommunication.

The Logical view contains Class diagrams and Package diagrams. It models packages, classes and their inter-relationships. 

The Implementation view contains Component diagrams and displays the component or modular structure of the software.

The Deployment view contains Deployment diagrams, and displays the mapping of software components or objects onto hardware.

At this point, the grey line between a model and a view becomes ever more murky. The problem is our use of the word model. In building and engineering, a model is something which can be tested, such as a 1:4 scale model of an aeroplane for aerodynamic testing, or a 1:100 model of a building, which can be admired and agreed before construction begins. The views are the plan or elevation views, or the perspective view. Views are on paper. Models are things you can hold. However, models are also mathematical static and dynamic representations in finite elements of 3D structures. The model is numbers in memory. It is rendered on a computer screen or printer as a view.

The viewpoints of the Zachman Framework

The Zachman framework comprises 30 views in a 6 x 5 (or 36 in a 6 x 6) matrix. It is primarily intended for developing business systems and is a framework to contain an enterprise architecture.

Zachmanís structure can be used to contain a set of views, allocating them to particular intersections of journalism and construction.

Figure 16.4 The Zachman framework

Zachman arouses strong feelings from its promoters and nay-sayers. It offers us a view of a software system from a slightly different mindset than our software development life cycle view so far discussed. The development moves down the rows of the framework, ensuring we are focusing on people, places and motivations as we add increasing detail to our definitions.

In all its glory, including the additional row for the users, or functioning enterprise:
  What
(Data)
How
(Function)
Where
(Network)
Who
(People)
When
(Time)
Why
(Motivation)
Scope
(Contextual)
PLANNER
Things important to the business What the business does Locations in which the business operates Organisations important to the business Events and cycles significant to the business Business goals and strengths
Business model
(Conceptual)
OWNER
Semantic model of entities and relationships Business process model Business logistics system of locations and links Work flow model, organisational charts, roles, skill sets and security issues Master schedule Business plan
System model
(logical)
DESIGNER
Logical data model (database E-R style) Application architecture Distributed system architecture Human interface architecture Processing structure of system events, dependencies and processing cycles Business rule model
Technology model
(physical)
BUILDER
Physical data model System design Technology architecture Presentation architecture Control structure Rule design
Detailed representations
(out-of-context)
SUBCONTRACTOR
Data definition Application design Network architecture Security architecture Timing definition Rule specification
Functioning enterprise
USER
Data (from the point of view of the user) Functions or applications to use Network, or communication facilities Organisation, i.e. skilled people, evangelists Schedule, or business events Strategy, or the enforced rules

For more on the Zachman Framework, see [2], or for an introduction, see [3].

The viewpoints of RM-ODP

An alternate set of five viewpoints is specified by the Reference Model for Open Distributed Computing (RM-ODP). They are the enterprise, information, computational, engineering and technology viewpoints.

Figure 16.5 The RM-ODP Viewpoints

The Enterprise viewpoint contains the business roles, objects, scope and policies. Policies include what the roles and objects are permitted or obliged to do, and also prohibited from doing.

The Information viewpoint shows what the system contains and deals with. It may contain static, invariant or dynamic schemas for containing chunks of related information.

The Computational viewpoint contains what the software does. It shows how the system is distributed by functional decomposition into large-grained objects and interfaces.

The Engineering viewpoint shows how the system is distributed. It focuses on the mechanisms and functions which will support the distributed interactions in the computational viewpoint.

The Technology viewpoint displays how the system is implemented. It shows what choices were made and how those choices support the above viewpoints.

Views or viewpoints?

The choice of the word view or viewpoint is largely semantic. Neither word means something specific, except in the context of the methodology which contains it.

As an example, consider the security view. Where is it in each of these methodologies?

For the 4+1 view of RUP, the answer is simply wherever you want it to be.

In Zachman, it would be divided up into relevant sections within the grid. There would be more than one, but less than thirty aspects to the security view.

In RM-ODP, security is not a view, but a function. The four functions of ODP are Management, Coordination, Repository and Security.

So we find that the apparent simplicity of expecting a view for all architecturally significant aspects is not delivered by any of these methods. Neither are they wrong not to contain a security view, for Security may be offered partly by software, partly by hardware technologies, and partly by business procedures. If locking the door at night is part of the security of your software system, you would contain that information in your hybrid security view in any of the methodologies.

RM-ODP

RM-ODP functions

Figure 16.6 RM-ODP Functions

The security function contains how the security of the system is managed, how material within the system is delivered only to those authorised to see it, how integrity of the information is maintained, audited and how access rights are granted to the system's users.

The management function details how the system is managed from nodes (servers) to objects running on those servers.

The coordination function details how the significant system events are coordinated.

The repository function details how information is organised and held.

RM-ODP Distribution Transparencies

RM-ODP also contains eight distribution transparencies

Figure 16.7 RM-ODP Distribution transparencies

Access transparency

Allows an object to be created or called, regardless of its physical location or the technological boundaries which exist between the caller and the object.

Failure transparency

Ensures that failures can occur in the system, yet be trapped and corrected, invisibly to the user.

Location transparency, or
Abstract Locality

Does it matter where the servers are located, or which servers are up or down at the time of use? If a user can call the help desk and manually change settings to point at another server while the local one is down, then there is a flaw in the architecture. The user must not be concerned with which server to connect to.

Abstract locality can be managed logically by recording a user's location and routing them to the most appropriate server, or by pinging a group of servers, then selecting the one with the fastest response.

Location transparency allows access to locations without knowing their name or physical address.

Migration transparency

Allows objects, servers or middleware applications to be moved yet still allowing access as before migration.

Persistence transparency, or
object pooling

If an object is deactivated, it is put in a pool to be reused rather than being deleted and recreated again when needed.

Relocation transparency

Allows objects and interfaces to be relocated even during use.

Replication transparency

Allows the seamless update of many copies of the same software entity. This can lead into fragmentation transparency if the software entities are fragmented or distributed in copied parts.

Transaction transparency

Allows the coordinated activity across non-heterogeneous networks of begin/commit/rollback transactions.

RM-ODP is the most complete of all the methodologies for presenting a specific set of views for a software architectures. It covers many of the items discussed in the architectural considerations directly, and the rest by implication. It has been accepted as an ISO standard in four parts X.901 to X.904, and its scope covers the many facets of distributed computing.

Figure 16.8 The scope of RM-ODP

Of the RM-ODP viewpoints, the Enterprise, Information and Computational viewpoints are distribution independent. These and the Engineering viewpoint are Implementation independent. Such an approach to architectural definition can help to separate out the concerns of your software system. 

RM-ODP is discussed more fully in [4] and [5]

Views, patterns and architectural considerations

It is one of my greater software hopes that one day we will have a cross reference between the architectural concerns and the patterns, and that we will be able to provide views by selecting those relevant from a three dimensional matrix of architectural concerns, viewpoints and viewer.

The chain of command pattern for help was discussed in the previous chapter. The technicalities of this are of importance to the software designer and programmer. The way it works is of interest primarily to the user. Therefore, to fully understand the system in sufficient detail, the user and the designer/programmer require different views of this one feature.

Architecture Documentation

Architectures need documenting. If they are not well documented they will be compromised, leading to ever increasing entropy. When working on a project within the architecture, the project documentation needs to be as light as possible (even non-existent if you XP), leaving the architecture documentation to address all architectural issues.

The architectural documentation is concerned with high level design, creating frameworks and tools which together support the creation of applications.

The Architecture, in the form of a document or electronic storage, will contain all facets of the enterprise system. Creating a structure to hold this information in a logical way can be arduous, or you can follow RM-ODP. If you choose the RM-ODP structure, you will have viewpoints, functions and transparencies. You must also think how best to introduce the applications which sit within the architecture, and how to provide views from each viewpoint to best provide the insight the viewer will require.

Introduction

The introduction should cover about half a page. Any more will lose your reader before they begin. It will cover at least the following points:

How the architecture supports the enterprise goals

This section is more relevant for enterprise systems, and explains how the long term enterprise goals are supported by the architecture.

The reason for it is twofold. First, it is a sales pitch to the sponsors of why the architecture is right for their given requirements. Second, it provides a defence against changes down the line. It's prominence forms a foundation in the mind of the sponsor and application designer that the architecture is not just a technical statement, but supports the wishes of the sponsors.

Viewpoints

Technology

What servers will be required, and where will they be located. What mechanism will be used to choose and introduce new locations, and what type of servers will be used. What storage is available on those servers (RAID, SAN?)

What client devices will be supported and what are the minimum requirements of each type of device?

What printing facilities will be required?

What operating systems, networking and protocols and directory servers are required? What data rates and response times must be supported?

What middleware is required? i.e. what database, email, internet access, document management system, collaboration tools, search engine, messaging?

What COTS (commercial, off the shelf) applications will be used? Which browsers will be supported?

Enterprise

What is the structure of the enterprise? What are the roles and goals, what are their guiding principles? What markets do they operate in?

What are the obligations, permissions and prohibitions of each role?

Where does the enterprise see itself in five years' time?

What are the enterprise's assets? Which of these assets are relevant to the software?

What earthly locations does the company operate in? Must the software support multiple spoken languages?

What quality of service do they expect?

What is their reporting hierarchy and who is responsible for what?

What are the enterprise objects? e.g. product, invoice, delivery note, supplier, customer, employee, timecard, expense claim. What are the relationships between these objects?

Information

How are the assets and enterprise objects created, persisted and destroyed?

How is the persisted information collated and summarised for efficient reporting?

What is the static, unchanging, or infrequently changing data the system uses as reference data?

Create a data dictionary and data model and define the basic lookup information such as countries, tax rates etc for the enterprise objects.

Engineering

How is bandwidth maintained for all users at critical times for critical applications?

How does the system pass messages among users who are not always connected to the network?

How does the system pass messages around the non-homogenous network, allowing for failure and transaction processing?

How is offline data maintained on client devices?

How are remote calls made across the network, eg COM+, Java RMI, CORBA, SOAP?

Computation

Define the object models, frameworks, layers, interfaces and shared tools of the software.

Functions

The relevant management, security, coordination and repository functions are documented.

Distribution transparency

The relevant transparencies are documented.

Applications

After defining the architecture according to RM-ODP, guidelines are given for the applications.

The architecture constrains and supports the applications. But that is not the end of the story. Applications must have a common look and feel. They must nestle comfortably in the frameworks defined in the computational view. This section introduces the applications contained within the architecture, and details how each will conform to common standards. 

It must not contain any application specific detail.

A final check through all of the architectural concerns will highlight any weaknesses in the architecture and/or documentation.

Other views

You will have a requirements document, which in itself is a view of what the software must do. Each document created for analysis, design, project planning, test, deployment and maintenance is a view from a particular perspective.

You may also create traceability views from requirements into analysis, design or test.

If your system has heavy data processing capabilities you may define data flows, or if user interaction is intense, you will provide user interface and navigation views. You must decide whether they are enterprise architecture or application architecture issues and document them accordingly. As a rule of thumb, if something (an object, a style, a sequence) is used in more than one application, it is an enterprise architecture issue. It can be created in one of the applications, then promoted to enterprise level for reuse.

All views must target the audience the view is being created for. They should contain all relevant information without overloading the viewer with unnecessary or unrelated information. Views can, and should, contain notes for succinct clarification. They must ensure that logic, appearance and methods are discovered, communicated and continually agreed as development progresses.

All of these views are supported by the project's eye view of the particular methodology you are following to deliver the software.

Personally, I find RM-ODP is better for documenting architectures, and RUP's 4+1 more useful at the application level.

Architectural insights

Architectural insights are a case of if I knew then what I know now... All of these problems are highly politically charged, and fraught with high risk from the outset. Nevertheless, it is beneficial to have an idea of what stepping stones are available among the crocodiles.

These are all simplistic offerings. It will never be as easy with live systems, but at least they are foundations to build upon.

Insight 1. How to get from a legacy application to a new one without big bang implementation.

There is a need to begin moving from a system of high entropy to one of low entropy. A legacy system is working and in continuous development. It is rather worn out, having survived operating system and development environment upgrades, and a high turnover of personnel. Many different coding standards, development approaches, aging technologies and unsupported controls exist within its firmament.

It needs to be rewritten, but times are tight and the business cannot afford the astronomical cost and long wait you have identified for their new system

A new system with a common architecture is required. The steps to get there are large scale refactors of isolation, extraction and rework.

Step 1. Understand the existing architecture and applications

This, in itself, is a huge undertaking. Most legacy systems are somewhat lacking in documentation. The architecture of the collected applications must be extracted and documented. Over time, alternative methods will have been created to do the same thing. Each method must be examined, and a successor to both chosen for the new architecture. At application level, it is better to extract common features and document each separately. It is not necessary to model the applications in any detail, for you may decide to scrap some applications, roll others into one, and extract a number of applications out of one current application. Therefore, full static and dynamic models would not be of much use. Use case models are probably sufficient.

Step 2. Extract common components and functionality

Identify areas where methods, objects, functions, files and data are shared or similar.

Step 3. Define the architecture

At this point, the architecture is everything that needs creating before work can begin on any of the applications. It should be as sparse as possible, yet inclusive enough not to allow deviation from the architectural vision.

Step 4. Define the applications

Take all of the use cases from the application models. Add in management and system functions. Gather all of the use cases into a logical set of applications.

Step 5. Create a programme plan

Create a programme plan for creating the architecture and the applications. Each step of the programme can then be broken down into project plans with careful mapping of dependencies, so work can continue at a measured and reportable pace.

The architecture will be begun as a separate project, then finished by adding in each shared component developed with the applications. The necessity of this is so sponsors can see continuous output rather than developing the architecture in its entirety before committing to developing the applications. IT budgets are expensive, and no-one likes to see their money disappearing into the IT cloud without a visible return.

Step 6. Follow the cycle of water

In its Water method, Taoist meditation teaches us to use full effort without strain or force. That may sound rather difficult to our western ears, well versed as we are in physics equations.

For a moment, try to suspend your disbelief, and we shall go on a journey from the sea to the clouds to the sea again. In the churning sea of data that contains our structured (data) and unstructured (documents) information, we have temporarily lost sight of its higher structure, and involve ourselves in the detail of tables, columns, keys and replications.

The problem with our data is that we have the seas, the lakes and the clouds, but we have lost the raindrops. If we could get back to the raindrops, our lives might be a little easier. 

A raindrop, in this case, is that piece of information held by a screen in an application, or by a section of a document. A raindrop might be an introduction. I could ask for all the introductions for wind farms and use the most suitable raindrops to create my own puddle of a document on wind farming. Otherwise, I am left wading through lakes, or delving down to the sentences (which we haven't stored). A raindrop of data might be an address, or a sales order. An address is not Customer name, company name, address line 1, address line 2, address line 3 (depending on whether or not it is empty), country and post code. That's data. 

So we allow our data to precipitate up and become a rather unstructured cloud. We then allow it to rain, and catch the raindrops as they fall. Once we have our raindrops, we can easily recreate our sea. Not only can we recreate it, but keeping the raindrops means less database calls. We are not short of storage at present. It does not matter if we store an invoice as HTML, SGML, PDF, relational data and summarised data in twenty-seven different forms. It probably mattered when the application was first written, but it is time to move on. We need the print render without having to create it from scratch every time. We need the relational data for searching and summarising. We need the summarised data for efficient reporting. There is no need to rebuild data every time it is queried right from the lowest level of relational data. Get creative with those raindrops. Create a sea with everything from atoms to huge waves. Your sponsor deserves it.

To rework the data in this way, a two way bridge between the old way and the new way is required. This is insight 2.

Insight 2. How do you redesign a database?

Redesigning data is not very difficult. What is difficult is to do so without being influenced by the demons of the past, and to create a method for switching between an old design and a new design without upsetting anyone. Data is always in a state of change. It can't be stopped, reworked then started again.

People are always trying to redesign databases, directory structures and other storage mechanisms, and they always come up against the same problem: they try to fit two eggs into the same eggshell.

The solution is one of steady migration.

Redesigning the database at the data level requires the creation of a new and separate database. Application by application, data is lifted from one database and written to the new one. It can be done two ways:

Method 1. Reporting first

All data is extracted from the old database and written to the new database, in its new formats. All reporting is then moved from the old database to the new database (hopefully dropping paper based reports and moving to browser based in the process).

Once reporting has been redirected, triggers in the old database write to the new database. The triggers continue working until each application is redirected to read from and write to the new database.

Method 2. Reporting last

Create the new database. Build a two-way bridge between the two databases so that any data written to either database is replicated to the other.

The next method for redesigning the database involves building a new object layer above the database. It is insight 3.

Insight 3. How to move from a data based system to an object based system

A wasp is a good model of an object based system. Its yellow and black striped abdomen is the data. Its magnificent multi-celled eyes, mandibles, proboscis and antennae are the interfaces to the outside world (applications). Somewhere in its thorax are a pile of objects, and the vanishingly thin wasp waist is a single component which writes data from objects to databases.

Options for the wasp-waist are:

  1. a single component taking in an object as a string of XML and persisting it.

  2. J2EE's entity beans using container or bean managed persistence

  3. a single component that can iterate through all object properties and read and write them from and to a database

The wasp is easy enough to understand. What is difficult is that you often have a bee. A bee has a fat waist, represented by many links from many applications at multiple levels. One application may read and write data directly using ODBC. Another might use stored procedures via JDBC or ADO.net. Yet another will write a text file which is imported by the DBAs directly into the database every Friday evening.

Getting from bee to wasp is difficult, and becomes ever more difficult as the freedom to improvise database solutions continues on an ad-hoc basis according to developers' preferences.

The only way is to create a layer in the architecture for managing all database activity, gradually including each application. Once the layer is in place, you are free to migrate to a new database. The data objects can write to both databases in an interim period.

I hope the message from insights 1, 2 and 3 is clear: Escape the straitjacket of thinking always about data at the relational level.

Insight 4. How to merge the architectures of two merging organisations

The first and foremost goal to reach is an understanding of how the businesses will work together. As long as their interwoven process remains undefined, so the software definition will be vague and unprogrammable. It is probably the worst environment in which to create software.

Mergers are cruel times. Many fail to provide a return on the investment. Heartache is high, egos are trampled, careful thought thrown to the wind. The business leaders set their date, empower IT departments to fight the good fight, and have many more things to worry about than their IT systems. They have mergers to do, processes can fall into place around the reappointed individual leaders.

The only way to define an architecture is to get into the heads of those running the merger. Who are the drivers of this merger? What are the reasons? What working practices must be upheld and which will be discouraged? Whose ideas will form the basis of the merged organisation?

Which if the two systems best matches, and is likely to be able to support, the new organisation. How much work is required to make the new system behave exactly as the old one did - that is always the request. No-one can possible operate without the reports they are used to seeing week after week. You will need to reassure them that the information will be available.

Whichever is the best (for the business, and architecturally), you must choose. You must then analyse what those who are losing their system cannot do without, and recreate it in the chosen system. A plan must be created for the switch off of one system, and full implementation of the other. You are unlikely to have any influence over the date, and it is equally likely to be too soon.

There will be a lot of firefighting to be done. People are already unhappy that their company has evaporated, and lots of newcomers, and new managers are abroad in their world. You can only do your best.

Insight 5. How to create an architecture from a collection of loosely related applications

The Paris Guide to IT Architecture[6] argues four steps to creating an architecture. They are:

1. Define a long term plan

In defining a plan, the important aspects of an architecture are gathered into zones. A stationer may have a zone for its products, a zone for how it holds customer information, and a zone mapping customers to pricing policies. It will have another zone for how it manages itself financially and manages its employees. Other zones will summarise sales data for analysis, and manage sales and invoicing.

Enveloping these zones will be the company's vision of where they see themselves in the near and distant future. And permeating the zones will be the quality of service, the style, and the clarity of view into the collected information which they expect from their software.

This information forms the long term plan.

2. Build a stable interface infrastructure

Build into the software the concept of services. Each service is a wasp's waist through which information passes, and is filtered or augmented.

A number of services provide a limited number - though fully functional set - of points through which data passes. This limited set of pathways is the stable interface structure.

3. Appoint a Zoning Board

The zoning board is responsible for what applications belong in which zones. They also manage requested deviations from the long term plan.

4. Make the most of what you have

If you have some core applications which require little change to fit the long term plan, plug them in. Modify others to fit where possible, and make discarding and rewriting the final option after all refactors and other alterations have been considered.

Insight 6. How to create the architectural vision

The vision is what the architecture is reaching for. It is not a business vision, but a vision of how the business vision can be can contained in a software architecture. It is gathered mostly by listening to people, and reading the mission statements and behavioural guidelines produced by a company.

The vision is created by weighing up where the business wants to go, what the users, customers and suppliers want from the software, and what is technologically feasible and affordable.

Often no-one in the business will have a full knowledge of what is possible, and their personal vision will be shaped by their experiences with other software and more strongly with what they want from the software to make their lives easier.

Balancing all of this input and extracting the vision is your architectural responsibility.

The architectural design is created from the architectural vision. The vision is the bringing together of the business and employees' visions, their ideals, their beliefs and desires. It will contain quality, style, a means of conforming to common ideals, a method for drilling into its taxonomy or ontology, and a strategic statement of global, numerical and technological intent.

Insight 7. How the architecture helps the business reach for its goals

When a business defines its high level goals, the architecture must be designed to support those goals. If the architecture then comes under attack, a misalignment with the goals of the business can be defended against.

Defending the architecture can often be a cerebral activity argued out as points of excellence in software creation. Businesses care little for such activities. However, when someone attacks an architecture there to support business goals, they are attacking the business goals. Then, less effort is required in defense. Everyone feels free to attack the architecture. Very few will attack the business goals.

Insight 8. Why is the unit of release, the unit of reuse[7]?

Most commercial systems or applications have specific release numbers, just as software components have version numbers. The components and systems have release dates and service pack updates.

Systems written in-house for in-house consumption often suffer from its applications becoming intertwined with one another, or separated from one another.

In an intertwined system, two applications share some functionality. The functionality requires a change for one application, but that will affect the other application, which must be reworked to use the new functionality.

In separated systems, two applications will contain the same functionality, but to change the functionality, it must be updated separately in each application.

The latter example has no reuse, the former has complicated reuse to the point that the reused component cannot be changed without changing both applications.

The reused component cannot be released on its own. It requires updated releases of both applications using the component. Hence, reuse and release are intertwined. 

Enterprise Architecture Case Study

In the last chapter, we looked at some example architectures. Mostly, they were software architectures combined with some high level business information. The term enterprise architecture gathers together the business structure, the wishes of the business leaders and future direction of the business, the technical architecture, and the IT infrastructure. To attempt to represent the enterprise architecture other than in the most abstract way will lead to ever increasing detail. Any attempt to represent it technically will probably be unfruitful. Here is an example of how I once chose to represent an enterprise architecture. I doubt it would do for any other enterprise, but it may give a flavour of the abstraction and focus required in comparison to the technical detail of pure software architectures.

This case study concerns one of the UK's largest insurers, where I worked for nine months on a Business Process Management initiative.

It soon became apparent that what we were attempting to do reached across the entire enterprise, encompassing the corporate goals, the architecture, the commercial vision and the process management, the latter of which was our original scope.

Only at the end of this nine month period, was I able to grasp the full picture, following conversations with many people who each supplied one part of the puzzle. It was a bottom-up process, rather than a top-down one [8], and looking back I'm not sure a top down approach would have supplied the eventual (though complicated) clarity, considering its complexity and scope.

For example, the insurer was the largest purchaser of white goods (fridges, washing machines etc) in the country; it would also like to be the largest seller. They also had a mind-boggling 1100 software systems and many other manual ones upon which their business ran.

The insurer's commercial vision, was a programme of work covering processes, people and systems, which reflected the desires of the business. It contained around 20 items of change, only four of which impinged upon more than one area of the enterprise.

Although these were part of the work stated as moving the insurer towards its goals, the actual goals were less forthcoming and existed in the hive mind rather than being set down in hard or soft copy. All of the business goals, requirements and fears must be included in an enterprise architecture.

The software architecture we had teased out at this point was as follows

From the business process management side, the following is an idealized future state, not all of which is currently available.

These, I felt, were the important facets of the enterprise architecture. Figure 16.9 details their interdependence. I called this the smoke chamber or the flower diagram.

Figure 16.9. Initiative collaboration

This, in itself is hardly an enterprise architecture, but an important starting point as it charts the interdependence of the initiatives and hopes of the business along with programmes already underway.

It is a highly complex, even complicated picture, and that is the role of the enterprise architect: to reduce the complicated to merely complex, while tying everything together.

References

  1. The Eight Fallacies of Distributed Computing. Peter Deutsch.
  2. The Zachman Framework. http://www.zifa.com
  3. The Zachman Framework: An introduction. David Hay. http://www.tdan.com/i001fe01.htm
  4. Architecting with RM-ODP. Janis R. Putman. Prentice Hall. amazon.uk
  5. ISO X.901, X.902, X.903, X.904. http://www.iso.org
  6. The Paris Guide to IT Architecture. Jurgen Laartz, Ernst Sonderegger, Johan Vinckier. The McKinsey Quarterly, 2000/3.
  7. Design Principles and Design Patterns. Robert C Martin. http://www.objectmentor.com
  8. Top Down Thinking for Enterprise Architects. Nigel Leeming. http://www.softwarearchitect.biz/articles/tdt/topdownthinking.pdf
Click here for the US book list
Click here for the UK book list