Software-Quality Discussion List
Digest # 018

Tenberry Home
Software-Quality Home
Reducing Defects
Automated QA
Site Index

           Software-Quality Discussion List

      S O F T W A R E - Q U A L I T Y   D I G E S T

      "Cost-Effective Quality Techniques that Work"
List Moderator:                      Supported by:
Terry Colligan                       Tenberry Software, Inc.     
June 8, 1998                         Digest # 018



    Back in Business, part II

    ==== CONTINUING ====

    Re: Software-Quality Digest # 017
      John Cameron 

    Job Postings
      "West Jaton" 

    Job postings
      "Phillip Senn" 

    Fewer Comments
      "Phillip Senn" 

    your comments in #17
      Jon Chappell 

    RE: Software-Quality Digest # 017
      David Bennett 

    Software-Quality Digest # 017
      Julie Zachman 

    ===== NEW POSTS =====

    (Hopefully) Controversial statements
      "Phillip Senn" 

      "Phillip Senn" 

    Contacting organizations with "Formal Inspection Process"
      "N.J. Hillary" 

      "Phillip Senn" 

    Help wanted.
      Jesus Munoz 

    The List

    Book for QA
      Val Stone 

    ==== ANNOUNCEMENTS ====

    Third international multidisciplinary congress in Quality and
    Reliability in Paris (March 25th/26th).


  Back in Business

  You may have noted another long delay between #018 and #017.
  This issue was supposed to be sent while I was on vacation
  in Scottsdale, Arizona.  Unfortunately, the computer/ISP
  connection which worked flawlessly in testing failed miserably
  when put into production on the road.

  We are back from the vacation, and have mostly gotten caught
  up on all the email which didn't get answered while in the email

  In any case, I apologize for the delay.  I hope that this long delay
  can be recovered from.

  Thanks to John Cameron, David Bennett and all the others who wished
  me good health.  I appreciate it!

==== CONTINUING ====

++++ New Post, New Topic ++++

From: John Cameron 
Subject: Re: Software-Quality Digest # 017

Terry, I hope that you are OK.  I wish you good health,

>P.S John Cameron, how old is dirt? If it's in tabloids, it's fresh
>that morning!
Erik, how you carry on.  I have been accused of being fresh.  John

And for something completely different.

>I'm an electrical engineer who develops software for embedded systems
>and I'm looking for DOS/WIN95/WINNT testing tools for the C (or C++)
>programing language. Most tools I know aren't suitable for embedded
>systems development and I'd be grateful if you could give me any
>advice about tools or information sources you know.
>Rodolfo Moeller

The place to start is with emulators.  I would recommend Lauterbach.
See  I use them, I have no stock  in the company.

Using the debugger to walk through your code on the embedded system is
the very best thing you can do.  A couple of days ago I found that our
sophisticated disable interrupt routine left our system in a state
where interrupts were never disabled.  We may never have found it
before our customers found it, if I didn't watch every line of code
execute.  The people I work with think I am nuts, walking through all
the code, but my code is almost bug free.  [I am a developer, not a QA
person, by the way]

Terry is also right about building the testing into the product.  I am
madly putting analysis code into the product.  I should be able to test
our entire library automatically in a week or so, including operating
system calls and interrupt handling.  We run pSOS which is quite small
so the task isn't quite as daunting as some of you might imagine.

Finally be sure the hardware is designed to facilitate output of the
analysis information.  An rs232 serial port to hook up to a PC is a
necessity.  Even if it's only soldered on for development.

Here's to a bug free world,

++++ Moderator Comment ++++

  John is doing great work.  He is also proof that even *really* old
  programmers like him (and me) are never too old to learn how to
  create high quality code!

  John has always been a very good programmer, but he didn't gain the
  quality concerns that he expresses here until relatively late in his

  A question for you, John:  Do the people who think you are nuts value
  the results you are producing?

++++ New Post, New Topic ++++

From: "West Jaton" 
Subject: RE: Software-Quality Digest # 017

agree w/ the "no job postings" decision

++++ Moderator Comment ++++

  So far, no one has wanted job postings.  I'll add it to the help
  and charter messages.

++++ New Post, Same Topic ++++

From: "Phillip Senn" 
Subject: Job postings

If people want to advertise that they are available for hire, then let
them do so with a modest signature line after posting their thoughts
about something relevant to the newsgroup.

++++ New Post, New Topic ++++

From: "Phillip Senn" 
Subject: Fewer Comments

I disagree with the request for 'fewer comments from the moderator'.
The Software-Quality listserv is YOUR (Terry's) listserv, and the
comments that you make are part of what makes it what it is. I want to
hear what other people may think, sure.  But you are the cement (or the
glue) that holds the listserv together.  I give your comments more
weight than others.

++++ Moderator Comment ++++

  Okay, noted.  (I couldn't really bring myself to stop commenting
  anyway! ;-)

++++ New Post, Same Topic ++++

From: Jon Chappell 
Subject: your comments in #17


Re: "Fewer Comments

  At least one person suggested that I'm getting carried away with my
  comments, serving more as an advocate than a moderator. I think the
  complaint has validity, so I will try to "tone it down" a bit...

My $0.02:

One trick I have learned (I hope) is that when I feel like preaching
(again!), I sometimes will stop myself and instead as the audience a
question and see what happens next.

At this point, you would be more than justified in asking "Well, if
that's the case, why didn't he instead ask me if I had any ideas for
"toning it down"? :-)

At any rate, sometimes "asking the right question" is more valuable
than providing the answers. Tends to keep you from being viewed
as a zealot.



++++ Moderator Comment ++++

  But what if I *am* a zealot?  ;-)

++++ New Post, New Topic ++++

From: David Bennett 
Subject: RE: Software-Quality Digest # 017

Hi Terry

* Several of you may have noted a long delay between #017 and #016.
The delay was caused by a series of medical problems, which have turned
out to not be serious (or so we think.)

I really hope everything is OK.  I've missed you!  I'm looking forward
to getting to meet you when I'm next in that neck of the woods (unless
you happen to make it Downunder before then).  I sure hope you're
looking after yourself at least until then!

* ++++ Moderator Comment ++++
* Good idea!  Will do!  (Also, I always indent my comments 2 spaces.)
* +++++++++++++++++++++++++++

Not here it didn't (see above).  But that could be Word auto-formatting
the text.

* Can we use the terms behavioral and structural instead of black box
* and white box?  In a recent post to swtest, Boris Bezier said these
* terms predate black and white box, so why do we hold on to jargon that
* few people outside of the testing arena can understand?

You can, but would I understand them?  I think people from all walks of
life understand "black box".  It's used in electronics, all types of
engineering, algorithm design, lots of places.

White box is already a problem.  (What about grey box?  Light grey box?
Almost black box?)  I think it implies transparency rather than
reflectivity or pallor, but that one certainly is open to
misunderstanding.  Is "structural" any better?

* 3.    If I were using Eiffel, the only thing QA'd would be the
* optimized versions squirted out.  I see no conflict here.  How you get
* to the executable to be tested is kind of interesting, but not very
* relevant to the points I was making.

I think it's very relevant to the points.  Perhaps I misunderstood, but
I thought your push was for the programmers to work on the very same
executable as the customers and QA staff.  With Eiffel, that's
virtually impossible.

Furthermore, Eiffel is predicated on the assumption that you will
remove pre- and post-conditions from the shipping product, so there are
no limits on the size and complexity of debug code.

I said in an earlier post that one should budget (say) 5-20% of size
and speed on debug/trace/logging/QA code in shipping software.  I now

1. Plan to expend *at least* 20% of size and speed on
debugging/tracing/logging/QA in your debug builds.  If the figure is
100% or even higher be happy!  You probably have very safe software.

2. Trim that to between 5 and 20% in your shipping versions (more in
the betas; less in stable product), but never below 5% or so.

Who's going to try and convince me I'm wrong?  Tune the figures by all
means, but the concept makes a helluva lot of sense to me.

* I have one suggestion for your situation:  Make it a requirement that
* the software system be fully automatically testable, so that QA time is
* spent on creating new tests, rather than trying to rerun old tests.
* This should be fairly easy to specify and will have a huge impact on
* overall quality.

* Others?  Agree?  Disagree?

Agree 100%.  If there is one thing we've done right over and over again
it is to build automated testing in from the beginning.  It saves time
and disasters downstream in a big way!

David Bennett
[ POWERflex Corporation     Developers of PFXplus ]
[ Tel:  +61-3-9888-5833     Fax:  +61-3-9888-5451 ]
[ E-mail: ]
[ Web home:   me: ]

++++ Moderator Comment ++++

  My push, as you put it, is for the programmers to work on the very
  same *code* as the customers and the QA staff.  I don't consider
  #if-#endif's to be the same code.  Although I occasionally add
  debugging traces/printouts to my code, I generally think that these
  are not the most effective way to go.

  I'm arguing for making the code be present all the time, and just
  run-time disabling/enabling, rather than going the #if-route.

  I still don't see why Eiffel precludes doing this.  It seems exactly
  analogous to a screen designer which lets you pull down menus.
  Hopefully the programmers spend some time with the real code, and not
  just simulating it in the design tool.

  I would rephrase your 2. to use "Disable" rather than "Trim".

++++ New Post, New Topic ++++

From: Julie Zachman 
Subject: Software-Quality Digest # 017

*  Thanks for the nice feedback!  Can I quote you when I start to
*  promote the list?


* I have one suggestion for your situation:  Make it a requirement that
*  the software system be fully automatically testable, so that QA time
*  is spent on creating new tests, rather than trying to rerun old
*  tests.  This should be fairly easy to specify and will have a huge
*  impact on overall quality.

Thanks for the suggestions.  I'm all ears for more.


Julie Clare Zachman
University of Wisconsin
Department of Medical Physics
(608) 262-3425

++++ Moderator Comment ++++

  What other suggestions do people have for Julie?  (original question
  in SQ 017 (available at

===== NEW POST(S) =====

++++ New Post, New Topic ++++

From: "Phillip Senn" 
Subject: (Hopefully) Controversial statements

David Bennett spoke of:
"If you design or write code, you MUST devise a way to test it."

I agree that the code that you put out should be bug-free as far as you
know. However, there have been numerous times that I've said (on the
day of installation) "I didn't think about that".

So my statement is: You can't test for conditions that you didn't know
you were supposed to program for in the first place.

This is usually what happens:
1. User tells boss he wants feature A.
2. Boss tells pgmr to program for feature A.
3. Pgmr tells computer to provide feature A.
4. Pgmr installs feature A.
5. User tells pgmr:
   A. What he really wanted was feature AB.
   B. Now that it does feature A, can it do feature AB?
   C. Why is it doing feature C? (also known as a bug).

++++ Moderator Comment ++++

  Isn't it the responsibility of software professionals to try to think
  of everything?

  To clarify, I don't expect to have every possible feature someone
  might ask for, but I do expect that, no matter what input the user
  supplies, a program A) doesn't crash and either B) does what was
  requested or C) says that it can't.

++++ New Post, New Topic ++++

From: "Phillip Senn" 
Subject: timeliness

I think one important aspect of quality is dependability, or

Which would you prefer A) The computer going down 5 minutes per day
randomly B) The computer going down for 1 hour every day at precisely

The answer is obvious.  If you can predict when the computer will be
undependable, then you can work around it.

The same can be true for digests.  If you know that once-a-week, you're
going to get something from an ezine, even if it's not polished, that's
still better than a hit-and-miss publication.  Whatdya say?

++++ Moderator Comment ++++

  Phillip, although it may not seem like it, I agree with you!  Let's
  see if I can do better in the future.

++++ New Post, New Topic ++++

From: "N.J. Hillary" 
Subject: Contacting organizations with "Formal Inspection Process"

I am heading a SEPG effort to define and document a formal inspection
process in one of the bureaus in the US Department of Treasury.  Our
past SEPG efforts have had many false starts.

I would like to make contacts with those organizations that have
developed and implemented formal inspection processes.

I wonder if anyone can help me to point in the right direction.

Josi Hillary

++++ Moderator Comment ++++

  What's a SEPG?  (guess: Software Engineering Program Group?)

  I'm more of a fan of informal inspections.  Will any of you who use
  formal inspections offer Josi some of your wisdom?

++++ New Post, New Topic ++++

From: "Phillip Senn" 
Subject: Testing

1. Given unlimited computing power, along with unlimited amount of time
to test, what would you do?
A) Make the person requesting the program provide test data
B) Make the programmer provide test data
C) Have a separate testing department provide it's own test data
D) Send it out as 'beta', when it really should be labelled 'Alpha'

2. Who should be the one with the authority to delay the ship-date?
A) The owner of the company
B) The project manager
C) The programmer
D) The testing department

++++ Moderator Comment ++++

  1. We generally do A, B and C, although getting the customer to
     provide useful tests (A) often doesn't work very well.

  2. I find the way you have phrased the question to be very adversarial
     in nature.  Aren't all these people working together?
     In any case, why should the person to make this decision be any
     different than the person(s) who made the ship date decision in
     the first place?

  I think these are very good questions -- they are essentially:

    What is the role of a testing/quality department?

  To this I add:

    Should there be a separate testing/quality department at all?
    (Or should this function be part of the development team?)

++++ New Post, New Topic ++++

From: Jesus Munoz 
Subject: Help wanted.

I just wanted to know if you, or any other people, could help me with
the following question:

I am the responsible for builds and automated testing in a Software
company, and I am looking for the best technique to get the most and
the better level of quality assurance automation.
We know about some tools, like Visual Test, but we want tot gather some
more in deep information to see what can we get about QA automation. We
have to consider if is better for our purposes to buy an standard
system or develop our own tools.

Please, just tell me if you could help me on this purpose or, if not,
where can I get more information.
Let me give you my congratulations for a good job with the forum, and
thank you in advance.

Jes£s Mu¤oz

Tel.: +34 3 240 30 38
Fax: +34 3 240 30 56
Visit our web site

++++ Moderator Comment ++++

  My experience is that build versus buy is not an either-or decision.
  There are good tools available which can help in certain parts of a
  thorough automated testing program.  However, if you want to approach
  full testing automation, you are going to have to build some stuff
  and have changes made to the application (to build-in other stuff.)

  To everyone: do you build or buy your automated testing tools?

++++ New Post, New Topic ++++

Subject: The List

Is the list still up?

++++ Moderator Comment ++++

  Yes, although it hasn't seemed much like it lately!  Sorry! :(

++++ New Post, New Topic ++++

From: Val Stone 
Subject: Book for QA


Does anybody suggest me the best book for QA?

Thanks in advance,
Val Stone

Project Manager
Software Factory International Ltd
1223 Peoples Avenue
Troy, NY 12180-3554
Phone:     518.276.2077
Fax:       518.276.6380
Free Call: 800.336.5904

++++ Moderator Comment ++++

  This seems like a great question!

  Can it be done with one book?

  What would be the books you would recomment for a QA start-up?


++++ New Post, New Topic ++++

From: Rufereq 
Subject: Third international multidisciplinary congress in Quality and
  Reliability in Paris (March 25th/26th).

Subject: Call for papers - Third International Congress in Quality and
Reliability .
 Paris - France
March 25-26 1999

Today, every company must take into account two essential values of our
society: Quality and Reliability .The congress takes place at the
forefront of progress, as it emphasizes on the latest results of
research, and their implementation in Quality and Reliability fields.
It is a prime opportunity for professionals and academics to share
views. This year, it also focuses on international participation. To
contribute to the transfer of research toward industry, we ask you to
communicate with us, by  proposing an article to the scientific
committee on one of the following topics:

- Optimization and control of industrial processes
- Quality management and control, human factors
- Systems reliability (product, software...)
- Quality, health and safety, and environment
- Quality and Reliability: Case studies
    This list can be completed

This congress is organized by:

L'Ecole Nationale Sup‚rieure d'Arts et M‚tiers
Laboratoire Conception de Produits Nouveaux
Le R‚seau Universitaire Fran‡ais pour l'Enseignement et la Recherche en

Dates and important informations:

Deadline for proposals                          June 15th, 1998
Author's acceptation notification               July 17th, 1998

Dates and location of the congress:

It will be held in Paris, the 25th - 26th 1999.

Concerned as you are, by Quality and Reliability we thank you for
distributing this information as widely as possible. To propose a
communication, or for further informations, you can also surf on our

You can also send a e-mail to :

The Software-Quality Digest is edited by:
Terry Colligan, Moderator.

And published by:
Tenberry Software, Inc.     

Information about this list is at our web site,
maintained by Tenberry Software:

To post a message

To subscribe

To unsubscribe

Suggestions and comments

End of Software-Quality Digest

[Tenberry] * [Software-Quality] * [Zero Defects] * [Automated QA] * [Consulting] * [Site Map]
Last modified 1998.6.9. Your questions, comments, and feedback are welcome.