Estimating and Scheduling
Suppose you are taking a car trip... What are three planning activities that you can do (other than the driving)?
1) Estimating the time needed.
2) Planning the route. Maybe making plans to share drivers, if it is a long trip.
3) As you drive, tracking your progress as you go and possibly making changes.
These correspond to 3 planning activities for software development -- estimating, scheduling (who will do the work and what will they do), and tracking/adjusting (how are we actually doing).
Estimating is notoriously difficult in software development. I believe it is because software projects tend to be "new", whereas other kinds of engineering tend to repeat known projects. Explain, with examples.
Role playing of manager/engineer scenarios...
I will be the manager. Take volunteers to be engineer, with class helping with responses.
Background. (draw on board) Worcester is 50 miles away, Hartford 100, Bridgeport 150, NYC 200. The time estimates for these trips are 1, 2, 3, and 4 hours.
Boss: Let's go to Worcester for a meeting in 1 hour. OK?
It is now 55 minutes later, you have fought through downtown Worcester, into a parking garage, and are getting out of the car.
Boss: "Oh, the meeting has changed to Hartford, but you said that was a 2 hour trip, and we have 1 hour left, so we are fine."
Is this OK?
Moral -- Development times do not always add and subtract cleanly. The total may be more, or less, then the sum of the parts.
Boss: We have to get to Bridgeport in 2 hours. Can you do it? (Assume that this is not possible.)
"We can skip lunch and drive fast."
"Be a team player."
"At least show that you are trying."
Moral -- All of these reasons are beside the point. Trying to do something that is impossible, still makes it impossible.
How would your moral be at the start of the trip (and throughout it)?
Side note -- What would be the worst thing that could happen about trying to get to Bridgeport in 2 hours? (Other than a car crash.) You actually get there! Your manager will remember this for years and expect you to do it again.
Boss: We have to get to NYC in 2 hours.
"Of course, this is not possible with our current transportation. I'm a good manager and I know that. So, I want you to rent a Lear jet and we'll fly in that."
"Can you do it?"
"Why not, Lear jets are fast?"
Moral -- New tools (or extra employees) are good. They can save you time on future projects. But there is a startup cost associated with them. You have to spend real time now to save time later.
What is a better approach to planning car trips? What do experienced drivers do?
Make accurate, realistic estimates, based on experience, allowing time for lunch, gas stops, rest breaks, traffic jams. (without artificial padding)
Execute the plan efficiently, and get places on time.
The same is true for software. Good software estimates are realistic and allow for things to go wrong.
Why estimates fail... (my experience and readings)
1) Scope creep.
Expanding the project as it progresses.
This is like changing the route to Hartford, when you are almost to Worcester.
Scope creep is one of the biggest problems in software development.
What to do about it -- Try harder to understand the full scope at the beginning. Don't listen to one person. Interview everyone involved and see if they really agree with the limited scope. Especially, talk to the final users to see if they will use the system as it is planned.
Also -- Make a realistic assessment of whether scope creep will occur. It is not good enough to say "We won't allow the functional specification to change." You need to be realistic about whether this project can be nailed down at the start. If not, add generous time to the schedule -- 50% or more.
2) Basing estimate only on an external deadline.
Examples -- The Christmas selling season, the big trade show in December, beating competitors to market.
What to do about it -- Try not to base estimates solely on external deadlines.
And -- If the external deadline is truly important (and sometimes it is) modify the functional specification to fit the deadline.
3) New problem domain or technology.
What I mean by this.
Example of Frank Lloyd Wright buildings. His projects were often over-budget and overdue. Why? He was trying new methods.
New technologies are a particular problem because you don't know what will go wrong. You don't know what you don't know. All estimates are a guess.
What to do about it -- If the project involves new technology, add substantially to the schedule -- 100% or more.
And -- Be realistic about whether the project is new or well-known. Has THIS group of engineers really done something VERY SIMILAR to this SEVERAL times? If so, it is a known problem. If not, allow time for new learning.
4) Assuming all will go well.
Suppose there are 20 tasks in a software project and that each has an 90% chance of going well (which is very high).
How likely is it that everything will go well? -- 12% (0.9**20).
How about if each task has an 80% chance of going well (still high)? -- 1%.
Remember these numbers when you are making a schedule at work.
What to do about it -- Plan on several significant things going wrong. If this is a big project (50+ people, for 20+ months), plan on many things going wrong.
5) Wanting to get the work.
Each person does not want to say (at the start) what they secretly know to be true.
Example of me bidding against someone else for a consulting job. How I could lose the job by telling the truth. (I tell the truth anyway.)
This also occurs within organizations -- people don't want to tell their boss the bad news.
This happens all the way up and down the ladder, and the problem multiplies as it goes.
What to do about it -- Tell the truth; it will help you in the long run.
And -- When you are a manager, look for evidence that people are telling you the truth (rather than what you want to hear).
6) The project was deliberately scheduled too aggressively, as a way to get people to work hard.
The theory is that an unrealistic estimate will get everyone moving as fast as possible.
Management knows that the schedule will probably slip, but believes that the final result will be faster than giving people a more relaxed schedule in the beginning.
Reality -- Unrealistic schedule makes the project later. See chart on RD page 219.
What to do about -- When you become a manager, don't schedule this way. As an engineer, try to avoid these kinds of schedules (and managers).
(Or, making a concrete plan to run the project to your estimate.)
Assume you are all college freshmen... You go into a room and write a plan for your life:
- You will graduate from college with straight A's.
- You will get a full scholarship to the grad school of your choice.
- You will finish your Ph.D. in two years.
- Your starting job will be as vice-president of a Fortune 100 company for $500,000/year.
- You will fall in love with the man (or woman) of your dreams by age 25, and you will never argue.
- You'll have 3 smiling children, who always listen to you.
- And you'll live happily ever after in Greenwich CT.
You emerge from the room, hold up the plan, and announce "I'm all set. I have it all written down."
What is good about this kind of planning? (It is good to have goals and work toward a plan.)
What is bad about it? (It is probably not realistic.)
Moving this story to software development, here is a true story...
A VP of software development was concerned that a software project was slipping. Every week, the finish date was slipping farther into the future. He decided to have a summit meeting, bring in the finish date, and nail down the schedule once and for all.
The VP called all the important managers for the project into a meeting. They shut the door and decided not to emerge until they had shortened the schedule.
One of the managers laid a rock on the table, turned it over, and said that they would "leave no stone unturned" in their effort to shave time off the schedule.
They emerged several hours later with a tightened schedule. Many tasks were shortened a little, some tasks were made concurrent, and some time was eliminated.
All the managers were happy: they had accomplished their goal of shortening the schedule.
Can anyone guess what happened? (The managers' effort was a waste of time. It did not change the pace of development at all.)
What could the managers have done that WOULD have made a difference?
What is the moral of the above stories?
Making a plan that looks good on paper does not get you anywhere.
You must make realistic plans, and have a realistic way to accomplish your plan.
Copyright 2001 by Charles H. Connell Jr.