Software QA and Testing Resource Center

© 1996-2000 by Rick Hower

|Home/TOC |FAQ 1 |FAQ 2 |Other Resources |Tools |Web Tools |Bookstore |Index |About |

Software QA and Testing Frequently-Asked-Questions, Part 1

What is 'Software Quality Assurance'?
What is 'Software Testing'?
Are there still major computer system failures caused by software bugs?
Why is it often hard for management to get serious about quality assurance?
Why does software have bugs?
How can new Software QA processes be introduced in an existing organization?
What is verification? validation?
What is a 'walkthrough'?
What's an 'inspection'?
What kinds of testing should be considered?
What are 5 common problems in the software development process?
What are 5 common solutions to software development problems?
What is software 'quality'?
What is 'good code'?
What is 'good design'?
What is SEI? CMM? ISO? Will it help?
What is the 'software life cycle'?
Will automated testing tools make testing easier?


What is 'Software Quality Assurance'?
Software QA involves the entire software development PROCESS - monitoring and improving the process, making sure that any agreed-upon standards and procedures are followed, and ensuring that problems are found and dealt with. It is oriented to 'prevention'. (See the Bookstore section's 'Software QA' category for a list of useful books on Software Quality Assurance.)

Return to top of this page's FAQ list

What is 'Software Testing'?
Testing involves operation of a system or application under controlled conditions and evaluating the results (eg, 'if the user is in interface A of the application while using hardware B, and does C, then D should happen'). The controlled conditions should include both normal and abnormal conditions. Testing should intentionally attempt to make things go wrong to determine if things happen when they shouldn't or things don't happen when they should. It is oriented to 'detection'. (See the Bookstore section's 'Software Testing' category for a list of useful books on Software Testing.)

Return to top of this page's FAQ list

Are there still major computer system failures caused by software bugs?

Return to top of this page's FAQ list

Why is it often hard for management to get serious about quality assurance?
Solving problems is a high-visibility process; preventing problems is low-visibility. This is illustrated by an old parable:
In ancient China there was a family of healers, one of whom was known throughout the land and employed as a physician to a great lord. The physician was asked which of his family was the most skillful healer. He replied,
"I tend to the sick and dying with drastic and dramatic treatments, and on occasion someone is cured and my name gets out among the lords."
"My elder brother cures sickness when it just begins to take root, and his skills are known among the local peasants and neighbors."
"My eldest brother is able to sense the spirit of sickness and eradicate it before it takes form. His name is unknown outside our home."

Return to top of this page's FAQ list

Why does software have bugs?

Return to top of this page's FAQ list

How can new Software QA processes be introduced in an existing organization?

(See the Bookstore section's 'Software QA', 'Software Engineering', and 'Project Management' categories for useful books with more information.)

Return to top of this page's FAQ list

What is verification? validation?
Verification typically involves reviews and meetings to evaluate documents, plans, code, requirements, and specifications. This can be done with checklists, issues lists, walkthroughs, and inspection meetings. Validation typically involves actual testing and takes place after verifications are completed. The term 'IV & V' refers to Independent Verification and Validation.

Return to top of this page's FAQ list

What is a 'walkthrough'?
A 'walkthrough' is an informal meeting for evaluation or informational purposes. Little or no preparation is usually required.

Return to top of this page's FAQ list

What's an 'inspection'?
An inspection is more formalized than a 'walkthrough', typically with 3-8 people including a moderator, reader (the author of whatever is being reviewed), and a recorder to take notes. The subject of the inspection is typically a document such as a requirements spec or a test plan, and the purpose is to find problems and see what's missing, not to fix anything. Attendees should prepare for this type of meeting by reading thru the document; most problems will be found during this preparation. The result of the inspection meeting should be a written report. Thorough preparation for inspections is difficult, painstaking work, but is one of the most cost effective methods of ensuring quality. Employees who are most skilled at inspections are like the 'eldest brother' in the parable in 'Why is it often hard for management to get serious about quality assurance?'. Their skill may have low visibility but they are extremely valuable to any software development organization, since bug prevention is far more cost effective than bug detection.

Return to top of this page's FAQ list

What kinds of testing should be considered?

(See the Bookstore section's 'Software Testing' category for useful books on Software Testing.)

Return to top of this page's FAQ list

What are 5 common problems in the software development process?

(See the Bookstore section's 'Software QA', 'Software Engineering', and 'Project Management' categories for useful books with more information.)

Return to top of this page's FAQ list

What are 5 common solutions to software development problems?

(See the Bookstore section's 'Software QA', 'Software Engineering', and 'Project Management' categories for useful books with more information.)

Return to top of this page's FAQ list

What is software 'quality'?
Quality software is reasonably bug-free, delivered on time and within budget, meets requirements and/or expectations, and is maintainable. However, quality is obviously a subjective term. It will depend on who the 'customer' is and their overall influence in the scheme of things. A wide-angle view of the 'customers' of a software development project might include end-users, customer acceptance testers, customer contract officers, customer management, the development organization's management/accountants/testers/salespeople, future software maintenance engineers, stockholders, magazine columnists, etc. Each type of 'customer' will have their own slant on 'quality' - the accounting department might define quality in terms of profits while an end-user might define quality as user-friendly and bug-free. (See the Bookstore section's 'Software QA' category for useful books with more information.)

Return to top of this page's FAQ list

What is 'good code'?
'Good code' is code that works, is bug free, and is readable and maintainable. Some organizations have coding 'standards' that all developers are supposed to adhere to, but everyone has different ideas about what's best, or what is too many or too few rules. There are also various theories and metrics, such as McCabe Complexity metrics. It should be kept in mind that excessive use of standards and rules can stifle productivity and creativity. 'Peer reviews', 'buddy checks' code analysis tools, etc. can be used to check for problems and enforce standards.
For C and C++ coding, here are some typical ideas to consider in setting rules/standards; these may or may not apply to a particular situation:

Return to top of this page's FAQ list

What is 'good design'?
'Design' could refer to many things, but often refers to 'functional design' or 'internal design'. Good internal design is indicated by software code whose overall structure is clear, understandable, easily modifiable, and maintainable; is robust with sufficient error-handling and status logging capability; and works correctly when implemented. Good functional design is indicated by an application whose functionality can be traced back to customer and end-user requirements. (See further discussion of functional and internal design in 'What's the big deal about requirements?' in FAQ #2.) For programs that have a user interface, it's often a good idea to assume that the end user will have little computer knowledge and may not read a user manual or even the on-line help; some common rules-of-thumb include:

Return to top of this page's FAQ list

What is SEI? CMM? ISO? IEEE? ANSI? Will it help?

       Level 1 - characterized by chaos, periodic panics, and heroic
                 efforts required by individuals to successfully
                 complete projects.  Few if any processes in place;
                 successes may not be repeatable.

       Level 2 - software project tracking, requirements management,
                 realistic planning, and configuration management
                 processes are in place; successful practices can
                 be repeated.

       Level 3 - standard software development and maintenance processes
                 are integrated throughout an organization; a Software
                 Engineering Process Group is is in place to oversee
                 software processes, and training programs are used to
                 ensure understanding and compliance.

       Level 4 - metrics are used to track productivity, processes,
                 and products.  Project performance is predictable,
                 and quality is consistently high.

       Level 5 - the focus is on continouous process improvement. The
                 impact of new processes and technologies can be
                 predicted and effectively implemented when required.


      (Perspective on CMM ratings:  During 1992-1996 533 organizations
      were assessed.  Of those, 62% were rated at Level 1, 23% at 2,
      13% at 3, 2% at 4, and  0.4% at 5.  The median size of
      organizations was 100 software engineering/maintenance personnel;
      31% of organizations were U.S. federal contractors.  For those
      rated at Level 1, the most problematical key process area was
      in Software Quality Assurance.)

Return to top of this page's FAQ list

What is the 'software life cycle'?
The life cycle begins when an application is first conceived and ends when it is no longer in use. It includes aspects such as initial concept, requirements analysis, functional design, internal design, documentation planning, test planning, coding, document preparation, integration, testing, maintenance, updates, retesting, phase-out, and other aspects. (See the Bookstore section's 'Software QA', 'Software Engineering', and 'Project Management' categories for useful books with more information.)

Return to top of this page's FAQ list

Will automated testing tools make testing easier?

See the 'Tools' section for test tool listings and the 'Web Tools' section for web site testing tools.

Return to top of this page's FAQ list



|Home/TOC |FAQ 1 |FAQ 2 |Other Resources |Tools |Web Tools |Bookstore |Index |About |

About the Software QA and Testing Resource Center and its author

Send any comments/suggestions/ideas to: Rick Hower

© 1996-2000 by Rick Hower
Last revised: 1/2/2000