Chapter 13 - When can I have it?

When can I have it?

Someone may have tried recently to convince you that time doesnít really exist, that we humans have imposed our collective belief on a timeless universe. They may be correct, but it will not stop people asking you how long something will take. Consider this:

You have just spent a whole afternoon in a meeting with the top honchos in your, or your customerís, company. It has been a drawn out affair, and rather painful for all concerned. You didnít begin the meeting very well, because you tried to sell them the benefits of your new iterative, incremental process. The message came across rather plainly that they werenít interested in your techniques, just that you deliver on time. You then started waffling on about unchanging requirements, and they didnít sound too interested in that either. They want software to enable better customer and internal communication and they want it matching their business process. Their process, as they so patiently explained, changes from day to day. They work in a very competitive market.

The meeting has rounded off with the Chief Executive shaking your hand and looking you in the eye. He thanked you for your time, and then, as if it were a mere matter of counting how many blows it would take to clear a dandelion, asked ĎHow long will it take?í

It has just happened. It is you he is looking at. It is you, alone, who has to provide an answer. The contract may well depend on it, and on that contract, your business. What do you say?

Option 1: Be honest. Tell him you have no idea. Calculating project times is impossible, some things simply take a long time.

Option 2: Be forthright. Tell him it would be foolish if you mentioned a delivery date as it may be misleading. You say ĎAfter I have analysed our discussion and identified the salient points, I shall be able to give you an estimate based on our previous analysis and the outcome of this meeting.í Sharp customers will try to get a deal based on your original estimate if future estimates are longer. You also donít want to scare off your customer by overlong estimations. Your Sales Director has told you not to say anything about time and price. You have embarrassed him before. He is staring at you with that look on his face.

Option 3: Guess. Tell him anything to close the deal. You can change your mind later.

Option 4: Guess and multiply by your lucky number. It has worked in the past.

Option 5: Based on your experience of similar systems for similar customers, you reckon about eighteen months. Eighteen months, as you know, is far shorter than a year and a half. You answer, ĎAround eighteen months.í There follows a long, hollow silence while your customer waits for a few foundations upon which to accept your estimate. The Sales Director is glowing. He is frightened.

Option 6: During the meeting you have identified the main use cases of the applications, and carried out a function point analysis. You know the magic numbers for this type of client and your current development team, so you look him back in the eye and say: ĎMy initial analysis shows that this assignment will take us in the region of eighteen months. That includes all of the requirements we have identified previously and those added during this meeting. I believe your changing business needs will not impede our progress as we have factored in a 10% requirement change, and will focus in the early stages on your core business where changes are minimal. I expect your core functions of inventory, management information and invoicing will be seventy five percent of the required effort, and our analysts will keep the development in line with your business requirements as we have previously explained with our iterative approach.í You smile. You know what you have to do, and how to approach it. You know what management your customer needs. ĎWe will remain in daily contact with your people, so you can rest assured that weíll be right on top of any opportunities that come along during the development.í

Which one do you do?

Which one should you do?

Before we look at some estimation methods, letís look to the horizon of estimation. What will we end up with?

We will get a number representing the development effort for a project in person hours. But what is a person hour, and how many person hours can you get out of a person in a week?

The average person spends 5 days each week at work. Over those 5 days, they work, they chat and they read Dilbert cartoons.

The structure of their work day also changes with the external ups and downs of their lives, and with their age.

Figure 13.1 Worker activities over a working lifetime

Now, far be it from me to suggest such a model is true. For us, its more like: write code, report, meet and chat. Itís an accurate enough model upon which to begin.

Suppose a worker is contracted to work a 40 hour week. Many work more than this, as is the nature of IT and many other facets of business. Getting things done takes extra time. We shall ignore this heroic and generous behaviour for now.

If, on average, our workers are half way through their careers, they will spend some time prototyping, some time thinking, some coding, some reporting, some in meetings, and some chatting. They will also spend some time on holiday, some time working in support of existing software, going to the drinks machine, the toilet, looking forward to their holidays, and collectively, some time learning, leaving, and joining. All of these activities eat away at a 40 hour week.

A worker may spend 30 days of the year on holiday. If they work Monday to Friday, that will be 30 out of 260, or around half a day a week. Add sick to that, dreaming of holidays, daydreaming, chatting, and all of the other activities which fill a workers week, then youíre probably down to 3 or 4 days per week. Then one day of support per week, and another few hours of meetings and youíre down to 2 or 3 days, or 16 to 24 hours. Letís take the mid point and say that, on average, your workers will contribute around half their week to your project, and the other half doing things that are all part of their contribution to the environment.

In estimation, you may calculate the length of time it will take in person weeks, and multiply that by their productivity. The above discussion will mean your productivity factor is 0.5, i.e. you will get half a week out of a person week. Secondly, are you calculating development time, or development, testing and deployment time? Donít go quoting development time alone; your customer will expect delivery at the culmination of your development time. So you will then have to increase your estimate again by your factor for testing time. You may also have a range of factors for testing times on small to large projects. Larger projects require more time spent on communication between the developers, analysts, project managers etc.

In short, you will need the standard time estimation, multiplied by various correlation coefficients (fiddle factors) that are particular to your development effort.

Figure 13.2 Communication overhead increases with project size

The larger the project you are working on, the more effort must be put into communicating the various aspects of the project. You will also spend more time on the business processes you are programming, and how best to manage their intercommunication.

In a small business, say fifty people, you probably know everyone. You know their opinions, the personalities of the people at the top, and have a fairly good idea about the business products, processes and where it is headed. The business may get together every six months to sell you the vision. You already know it, so it doesnít take too long.

In a large business, say five thousand people, you will have to answer to layers of people. Project managers, analysts, managers, team leaders, the owner, who has told you his objectives, and all of the layers of management beneath him, who each have their own slightly different version of the truth.

You will spend time involved in organisational politics. You will also spend time finding out who people are, whose chains of command they belong to, and worry about how to recognise them from their picture on the intranet when you go looking for them.

Figure 13.3 Effect of business size on effort.

The larger the business, the more of your energy is likely to be expended in finding things out, and playing political games.

Productivity is also a function of the personís hours per week. Most of us are normal, moderately healthy human beings, and we have activities other than work. There may be some who relish all night code-ins and sixty hour weeks, but on average, you might get forty hours in the building. The sixty hour all-nighters are likely to be younger, more virile, more excitable and more likely to spend forty of their sixty hours maximising the work not needed to be done.

Figure 13.4 Worker productivity and their hours per week

When you have conquered the maze of person hours, itís still not as simple as it might seem. For example, if someone has just returned from a three week holiday touring Kenya, they will have been on safari. They will have ballooned across the savannah chasing herds of migrating wildebeest. They will have dived off the deserted white sandy beaches, and youíll be lucky to get an hour out of them in the first week. This is exacerbated in larger businesses; there are that many more people to tell.

Have your other workers been overdoing it lately on an urgent job? If so, donít expect them to dive in with 100% effort on this one. There may be other circumstances that slow them down: cancelled overtime, time off in lieu reduced, personal problems, bad feeling between team members, between management layers or company owners.

The point is that all of these things affect how many person hours come out of a person in a working week. If a worker is stuck on a particularly difficult support problem with a particularly difficult customer, you may even get zero hours. If you ask them why you got zero this week, you will likely get zero next week as well.

In short, here is an equation for the calculation of person hour per week.

Hours on job = Hours per week x Productivity ratio x Project size factor x Business communication factor

You may prefer a single fiddle factor to replace the three factors above, while you improve your accuracy.

Hours on job = Hours per week * Productivity Coefficient

If you have no idea where to start, try a productivity coefficient of 0.5. You can improve on it later.

You may have spotted the obvious implications in the first equation. It belies two important facts

  1. You can have an optimal business size

  2. You can have an optimal project size

The optimal business size is probably as small as you can get it. Less people means less politics, less communication problems and fewer conflicting ideas on the best solution. 

The optimal project size seems to have been agreed at around three to twelve months. Any shorter, and the project team spend too long understanding the problem. Any longer, and you have goal setting problems, developer apathy and high turnover.

Now you have an understanding of how much time you can squeeze out of a developer each week, it is time to estimate the time required for a project.

Time estimation

You can only start estimating time accurately when you have been estimating it inaccurately for a while.

There are two rather different styles of estimation. One style involves estimation based on lines of source code in the resultant application and libraries. The other style involves breaking down the complexity of the project into manageable chunks and allotting time to each chunk. The result is then modified by taking into account technical, team and environmental factors.

You may want to try both methods to find which most accurately measures your development effort.

Time estimation method 1 Ė Gut feeling

What is your first stab at how long it will take? If you canít come up with a guess, or your guess is greater than a few months, then you need to divide your work into smaller pieces and estimate for each piece. Gut feeling is often based on summing up the difficulties, and wondering how long it might take to code it up. Donít forget to factor in testing and release, and how much developer time you are likely to get in a week

Gut feeling is an OK method. It gives you an order of magnitude estimate, which is better than not having an estimate. It should be measured alongside a more logical method, or at least past history. You might want to write down your estimates as you go, and record against them how long things actually took. After a few projects, you should be able to come up with a fairly good fit between actual and estimated time.

Figure 13.5 Project time: estimates vs actuals

What this example data shows is that things are always taking longer than your gut feeling tells you. It also shows that your estimations for projects that take longer is increasingly unreliable, although not too inaccurate overall.

Gut feeling may be a good enough method for you. If you are fairly accurate, you may promote yourself from gut feeler to expert assessor. Others may want different models to rely on.

Time assessment method 2 Ė The Putnam/COCOMO Method

I have linked these together as they are both based on the assumption that you can estimate the project size in lines of source code. The Putnam and original Cocomo (Constructive Cost Modelling) equations are the same.

Project Time = KLOC x Multiplier

KLOC = Thousand lines of source code. Take that and multiply it by a number to get your time estimation. Putnam recommends a number of 3 to begin with to give a time in person months. You can further refine this based on your growing experience.

As an estimation method, you wonít have many headaches understanding it. All you need is an estimate of the lines of code likely to be produced, then multiply. The problem you are faced with is coming up with the right number for lines of source code.

If you are a programmer who likes to write source code comments before writing the code, you may be half way to this method already. You might write some pseudo code like this

Public String getPersonnelNumber(String username) {

//this function looks up personnel numbers from database A
//from the username

//initialise variables
//check the database is available, and that the userís machine is logged in
//try a direct match between the name passed in and that held in the database
//if the more than one match is returned, allow user to choose between them
//if no match, try a more fuzzy approach based on surname + abbreviated first name

}

Before you pass the coding on, or write it yourself, you count the number of comment lines. On average from experience, you know that you end up with between 4 and 5 lines of source code per comment line. Your time estimation would therefore be

lower estimate: 4 * (Comment lines/1000) * 3 person months

upper estimate: 5 * (Comment lines/1000) * 3 person months

If you have 500 comment lines, then your application, according to your previous projects, will take between 6 and 7.5 person months. Bringing in your efficiency factor and multiplying this estimate by 2, gives 12 person months, or three to four months for a team of four. If you are opening your mouth, always go for the higher estimate.

Getting to grips with this estimation method means recording time against source code lines. You may have a source code control or change management system (youíd better have) from which you can check out projects and count the lines of code. If you have a record of the time taken, you may be able to reproduce your own version of the graph in figure 13.5 for use in future estimations.

You can use the Cocomo method to predict defects, or bugs:

Number of bugs = KLOC x multiplier

This will give you an estimate of how many bugs, on average, you can expect to find in your code. In turn, you will be able to spot the less defective programmers, and estimate the likely iterations through test using similar techniques.

Cocomo has been around for a long time, and has grown into a rather formidable set of equations:

where A = Multiplative constant and Size is in KLOC. There are 17 effort multipliers and 5 scale factors. In this, it is similar to function point analysis as it recognises that many other factors influence development time.

As an estimation method, it is OK. However, creating lots of lines of code is a disease of those unable to educate themselves. I have seen many occasions where 1KLOC is 900LOCs too many, and could have been done using a better method. Basing anything on lines of code can imply to developers that lots of lines is a good thing, and can lead, quite logically, to:

Revenue = A x KLOC

Then writing lots of lines of code becomes a goal for project managers to attain.

Time assessment method 3 Ė Function Point Analysis

Function Point Analysis is claimed to be able to

For it to be able to do these things for you, you must first begin to use it to estimate project time. There is an international user group of function points, IFPUG[1] if you want more information. ISO is currently trying to ratify a standard which will be called Functional Size Measurement Standards, based on the work of the IFPUG. We shall be using it as a software development time estimation technique.

The function point metric began at IBM as a means of measuring software size and productivity. It uses functional, logical entities such as inputs, outputs, and inquiries that tend to relate more closely to the functions performed by the software as compared to estimating lines of source code. It has been further developed since to try to pin down its accuracy.

Function point analysis can take your analysis models, and come up with an estimation of time. It works like this:

  1. Identify the complexity of your actors (system interfaces)
  2. Identify the complexity of your use cases
  3. Assign a measurement for technical factors
  4. Assign a measurement for environmental factors
  5. Add up all of the project influences, and you have your estimation.

In detail, here are the steps you need to take.

  1. Calculate your points based purely on functionality. This is known as the unadjusted function point value.
  2. Adjust the function points for technical and environmental complexity
  3. Multiply your adjusted function points by how long it takes to deliver a function point's worth of software.

Identify the complexity of your actors

Take each actor in turn, and decide whether their interaction with your system is simple, average or complex.

Actor type Description Points
Simple Procedural text interface, e.g. ATM 1
Average Interactive text or protocol communication (http, ftp) or interactive text (VT) 2
Complex Graphical Interface, e.g. Windows user 3

Identify the complexity of your use cases

Take each use case in turn, and as with the actors, decide if they are simple, average or complex. This complexity can be estimated from the number of transactions a use case performs, or from the number of analysis classes identified in the analysis model.

Use case type Transaction based estimate or Class based estimate Points
Simple 3 or less transactions or operations Fewer than 5 analysis classes 5
Average 4-7 transactions or operations 5-10 analysis classes 10
Complex 7-10 transactions or operations More than 10 analysis classes 15

There are two schools of thought on having reached this point. You can calculate the unadjusted use case points by the following formula:

UFP = UUCP = Sum(Actor points) + Sum(Use Case points)

Or, for more complex applications, there may be a need to drill into the transactions to detail your unadjusted function point count.

Figure 13.6 Routes to the unadjusted function point value

What is a transaction?

Transactions are defined as external inputs, external outputs and external inquiries. The word external, in this instance, applies to the use case. 

External input

When an actor moves data or control information into the use case, it is an external input. For example, when a user completes a form and submits it to a server, this is an external input.

External output 

When the use case delivers derived data or control information to an actor, it is an external output. It may be a form or report, or output file. 

External inquiry 

A simple input and output, when combined together to request and return basic data held by the use case, is an external inquiry. This does not apply when data must be changed, or derived from the data held by the use case. Then it becomes a separate external input, and external output.

The information being passed around by these inputs and outputs is held either within the use case or outside the use case. If it is held within the use case, for example within an object, it is called an Internal Logical File. If it is held outside, in a database or file, then it is called an External Interface File.

The inputs, outputs and enquiries are given a simple, average or complex rating according to how many file types they reference, and how many data elements they pass to those file types. For those programming databases, a file type can be thought of synonymous with a database table, and a data element as synonymous with a field.

External inputs
Data elements
file types 1-4 5-15 15+
1 low low average
2 low average high
3+ average high high
 
External outputs, External enquiries
Data elements
file types 1-5 6-19 20+
1 low low average
2-3 low average high
4+ average high high

Similarly, the Internal logical files and external interface files are given low, average or high ratings based on the number of data elements being passed, and also the number of repositories for those data elements. For database programmers, the Record Element Types is essentially the number of tables.

Internal logical files, External interface files
Data elements
record types 1-19 20-49 50+
1 low low average
2-4 low average high
5+ average high high

We now apply the function point values to each of these low, average or high ratings for the inputs, outputs and files, and add them up to give an unadjusted function point count.

Function point values
Rating External input External output External inquiry Internal file External file
low 3 4 3 7 5
average 4 5 4 10 7
high 6 7 6 15 10

If a single use case has more than ten transactions or 50 function points, try to break it down into two or more use cases. If you have many use cases with more than ten transactions, or many single transaction use cases, it may mean you are not defining them at the correct granularity.

The unadjusted use case points or unadjusted function points are then multiplied by further factors which introduce technical and environmental diffculties.

Identify the technical difficulty

The unadjusted function points are then adjusted by technical and environmental factors. The multiplication factor is known as the value adjustment factor.

The answer to each question is a value from 0 which means irrelevant, to 5 which means essential.

 

Factor Scores
1 Data communications 0 batch processing or standalone PC
1 batch with remote data entry or printing
2 batch with remote data entry and printing
3 online data entry, batch UI, query system
4 Application with only one comms protocol
5 Application with more than one comms protocol
2 Distributed data processing 0 Application does not distribute data
1 Application prepares data only
2 Data is prepared and transferred
3 Data processing and posting
4 Data processing, posting and retrieval
5 Data processing, dynamic updates
3 Performance 0 No performance requirements
1 Performance requirements reviewed, no action necessary
2 Response critical during peak loads
3 Response critical at all times
4 Performance analysis required
5 Performance tools required
4 Heavily used configuration 0 No operational restrictions
1 No effort required to meet required restrictions
2 Some security or timing considerations
3 Specific processor requirements for part of application
4 Special constraints on application within system
5 Special constraints on application in distributed components
5 Transaction rate 0 no peak transaction period
1 peak transaction period anticipated
2 weekly transaction peak
3 daily transaction peak
4 high rates required, must design for performance
5 high performance rate in SLA, must use performance analysis tools
6 Online data entry 0 All transactions processed in batch mode
1 1-7% interactive
2 8-15% interactive
3 16-23% interactive
4 24-30% interactive
5 30%+ interactive
7 End user efficiency Human factors taken into consideration. A typical windows application gets 5.
8 Online update Applies to the update of Internal Logical Files
0 None
1 Volume updating of a few files is low and recovery easy
2 Online updating of 4 or more ILFs
3 Online updating of many ILFs
4 As 3 with protection from data loss
5 As 4 with highly automated recovery
9 Complex processing A count of the following:
 Sensitive control or specific security processing
 Extensive logical processing
 Extensive mathematical processing
 Exception processing for incomplete transactions
 Complex processing for multiple i/o devices
10 Reusability 0 No code reusable in other applications
1 Little reusable code
2 <10% of application reusable elsewhere
3 10%+ of application can be used elsewhere
4 Specific design for reuse
5 Specific parameterised design for reuse
11 Installation ease 0 No special setup
1 Special setup required, but not specified
2 Conversion and installation requirements, but not considered important
3 Conversion and installation considered important
4 Automated conversion and installation
5 Automated conversion and installation considered important
12 Operational ease 0 nothing beyond normal backup
The following all count as 1
_ Effective startup backup and recovery
_ As 1, no operator intervention required
_ Minimise need for backup
_ Minimise need for paper
5 Design for unattended operation
13 Multiple sites 0 One user
1 Multiple users on identical hardware
2 Multiple users similar hardware
3 Multiple users, different hardware and OS
4 Documentation and support plan provided and tested for 2
5 Documentation and support plan provided and tested for 3
14 Facilitate change A count of the following:
 Flexible query and reporting one one ILF
 Average reporting from more than one ILF
 Complex requests, multipe ILFs
 Business control data online, but written daily
 Business control data online with immediate update

Add up the influences to give your value adjustment factor (VAF), a value between 0 and 70, then put the number into this formula to give your adjusted function point count.

Function Points = UFP * VAF

That is the IFPUG's version of the story. I mentioned two schools of thought earlier. Here is the other, as recorded in Applying Use Cases[2]. This time there are only thirteen technical factors, and they are not all weighted the same.

Factor Weighting
T1 Distributed system 2
T2 Performance requirements 1
T3 End user efficiency requirements 1
T4 Complex internal processes 1
T5 Reusable code modules or components 1
T6 Easy to install .5
T7 Easy to use .5
T8 Portable (across operating systems) 2
T9 Easy to change 1
T10 Concurrent 1
T11 Special security features 1
T12 Provides direct 3rd party access 1
T13 Special user training facilities required 1

Somewhat similar to before, each T factor is assigned a value of between 1 and 5, and each value is multiplied by its weighting before being added up. The maximum total, as before, is still 70.

Technical difficulty multiplier = 0.6 + (0.01 * sum(T-Factors))

As you may be able to see, the technical difficulty factor will be between 0.6 and 1.3. This means your function point count will be diminished if it is a simple projects, and inflated if it is difficult.

Function points can be calculated by anyone, and it is hoped that two skilled people would come up with roughly the same number of function points for a given piece of work. What it does not give you, is an estimation of how long it will take. To get there, we will use the method described in Applying Use Cases[2].

Identify the environmental factors

Environment is that which makes one team deliver quicker than another.

Environmental Factor Weighting Value
E1 Familiar with development process 1.5 0 No experience
5 Expert
E2 Experienced application developers 0.5 0 No experience
5 Expert
E3 Object oriented experience 1 0 No experience
5 Expert
E4 Lead analyst capability 0.5 0 No experience
5 Expert
E5 Motivation 1 0 None
5 Highly motivated
E6 Stable requirements 2 0 Unstable
5 Fixed
E7 Part time workers or contractors -1 0 No part timers
5 All part timers
E8 Programming language difficulty -1 1 Basic
2 Visual Basic
3 Java
4 C++
5 Assembler

As with the technical factors, each environmental factor is given a value between 1 and 5. Each value is multiplied by its weighting and added up. The environmental factor is then calculated:

Environmental difficulty multiplier = 1.4 + (-.03 * sum(E-Factors))

The maximum value for the sum of the environmental factors is 32.5. This represents the best team. The minimum summed value is -10. The range of the Environmental difficulty multiplier is therefore 0.425 to 1.7 or 1:4 difference between the best and worst prepared teams. This largely agrees with the Peopleware claim of a 1:3 difference in coding output observed between teams in different environments in the code wars.

To calculate the amount of time it will take, the unadjusted function points have the technical and environmental factors applied, then the whole thing is multiplied by a person hour value. Initially, we shall take 20 hours per point. This includes development and initial testing and should take you up to the point of release.

Time = 20 x UFP x Technical Difficulty x Environmental Difficulty

In Applying Use Cases, we are recommended to count up the number of points E1 to E6 which score below 3, and one each if E7 and E8 are below 3. A total of 3 or 4 indicates experience problems and a multiplier of 28 man hours is recommended. If the number is 5 or more, then you have a serious experience problem, and should get some more experienced team members.

I have a spreadsheet based on the above discussion, It was fairly easy to create, removes keying errors, and can be kept for future reference or to recalculate when requirements change.

Figure 13.7 Function point estimation

Does it really work?

The function point technique is very general. I would suggest having your own fiddle factor to apply to the final result. Begin with a fiddle factor (i.e. a multiplier) of 1.5, and as you gain experience with function points you may be able to refine it.

Notice there is nothing in there about business politics, size or communication. If your software is being defined in the US, coded in India and tested in the Ukraine, that would add a few difficulties and certainly extend the project. Yet there is no mechanism for addressing such problems.

The greatest danger with the function point technique occurs when you calculate 20 weeks for four developers for a project. The developers plod on merrily, and lo and behold, you find it takes four of them 20 weeks. When you realise later on that you entered the wrong number somewhere and it should have been 10 weeks for two developers, what will that tell you?

Any estimation technique is useful for predicting approximate times. Developers will live up to your estimations, but it is them meeting the expectation rather than the estimation method being accurate. You are then into monitoring people to see if they are over-stressed trying to meet the demands you have placed on them, or are plodding when you have overestimated.

So, estimation techniques are good. They can give you a good idea of how long something is likely to take. Despite the high precision appearance of the formulae and factors, there is little point in the accuracy. What you are dealing with is people. If a developer is expected to meet the expectation of the company president by getting a spreadsheet macro out by next Tuesday, it will be done. If a developer has to meet the expectation of a Project Manager shouting words like urgent, under pressure, must have, then the result is entirely dependent on personal factors rather than any estimation by function point analysis.

Good relationships and good communication deliver software. Good estimation is extremely useful, but guarantees nothing.

In Peopleware, the coding wars showed a three to one difference between developers in an organisation, and a ten to one difference in developers overall. That makes rather a mockery of any generic measurement technique.

Even so, if you are not trying to do it, you are not improving your software capability. NASA lists the elements of good estimating practice, and the evidence for maturity in that practice on its website, in a Parametric Cost Estimating Handbook. IFPUG also has a function point counting manual.

And finally, do not be afraid to turn the question on its head. Ask the business how long they would like it to take, then tell them what they will get for their money.

References

  1. The International Function Point User Group (IFPUG). www.ifpug.com
  2. Applying Use Cases. Geri Schneider and Jason P Winters. Addison Wesley amazon.uk
Click here for the US book list
Click here for the UK book list