Team #2 Presentation

Quality -- Introduction

Software quality is a complete approach, not just something you do after the code is written.

SW quality = compliant to spec + absence of bugs + on time and budget + benefit to user + good internal structure.

Errors are easiest to fix at the beginning of the software development cycle, and progressively more expensive the later they are fixed.

This is the central, most important, idea about creating high-quality software.

Draw graph of development stage (X) versus time to fix an error (Y).

Example: Poor performance of a data comms software related to poor design -- basically impossible to fix.

Example: Logical error in functional specification (user allowed to do contradictory operations). Can be fixed later, but may involve changing LOTS of code.

Defect amplification

Explain what it is.

Draw picture on board showing this. Box for each phase. Number of errors as output.

Assume perfect software to start. Show effect of one error at halfway point. Show effect of one error at the beginning.

Technical reviews / inspections

What are they?

Can be used for functional spec, or design, or coding.

How often are they really used?

I find review most useful for technical design. Catching errors here saves A LOT of time later, so reviews are worth the time and effort.

CC Chapter 24 has useful information about this topic.

Read checklist on 581.

Statistical measures of software quality... I will not discuss them, but would make a good extra credit or presentation topic.

Some useful quality measures

MTTF -- mean time to failure. Average time between software failures.

MTTR -- mean time to repair. How long does it take to fix the problem and get the system back up.

MTBF -- mean time between failures. Sum of the above. Average time from the start of one failure to the start of the next.

Availability -- Percentage of time that system is up and running.

Draw example on board of one system, its up and downs, showing each item above.

Of course, the definition of "failure" is murky. Give some examples.

ISO 9000 approach to software quality. We will not discuss it but this is a good extra credit / presentation topic. I have some material you can borrow on this.

Testing, Part 1

Validation and Verification

Can be confusing.

Validation -- Checking that the functional spec is what the customer really wants. Give some examples.

Verification -- Checking that the software matches the functional spec. Give some examples.

How to remember this? Validation comes first (customer says what they want) and A comes before E.

Examples: Validate = a report has the right rows and columns of info as specified in the functional spec. Verify = numbers in a spreadsheet add correctly.

There is certainly overlap between these two.

How do we know when we are done testing?...

All known and probable major bugs are gone.

Count of minor bugs is acceptable and decreasing.

Performance and usability are good.

Testing effectiveness.

Draw ideal graph of time (X) versus # of outstanding known bugs (Y).

What does it mean. (The longer we test, the fewer bugs we find.)

Is this graph reality? (Sometimes.)

Sometimes, real testing is a saw tooth graph. (Draw it.)

This kind of debugging can be very frustrating for new engineers and managers. You think everything is going great, at the bottom of the first dip. Then after a couple spikes, you don't know what is happening.

Testing, Part 2

Stubs and drivers.

Driver -- Simple code that creates input for a function you are testing. Give some examples.

Stub -- Simple function that can be called by a higher-level routine you are testing. Give some examples.

Is it always easy to write drivers and stubs? No, they can be hard to create for complex situations.

Usually it is worth the effort to write these, especially for complex systems.

Example -- For programs that take input from a file, a small data file can act as a driver. You can construct a sample input file by hand, then use it as the input case. Can someone think of an example? (A program that reads daily NYSE stock prices and prints the names of companies that have lost 20% value in one day. Create a small, simple input file with the proper structure, rather than testing on the giant, real input file.)

Unit testing. What is it?

White box testing... Explain what it is. Uses knowledge of code details. Uses this knowledge to design tests, such as: exercises all code paths, test all error conditions, test code that you are unsure about.

Black box testing. Explain what it is. Tests that a given set of inputs produces the correct outputs.

Basis-path testing. Summarize. This is a good EC / presentation topic. Read this, plus find other material and teach the class about it.

What do I expect in a good design spec?

Enough detail that another programmer could pick up the spec and write the code that you want them to write.

Must explain key algorithms, overall structure of the code, major classes, important data structures and file formats.

Do not need to explain every tiny low-level routine.

Do not need to explain code that is part of the operating system of runtime libraries, such as Windows Foundation classes, java libraries, etc.