Quality Assurance Procesures

There are two major factors which influence the applicability of a given piece of software: the validity of the initial program, and the care taken in its maintenance. When a new feature is added to the program, it is normally checked by examples which can be verified by manual computations. The major question is how to assure that during the maintenance process, something is not done to upset the initial results within the proven range of validity. This section describes a set of procedures which minimize the introduction of unintentional errors. One should notice that the goal of these procedures is to maintain a status quo in the results of the programs, not to insure the applicability of the results to any given situation.

Company Structure:

As a small scientific software vendor, Ultramarine has a corporate structure with the integrity of its software as its basis. The corporate president has direct control of every step of the development, maintenance, and distribution of all software. All of the personnel involved with these activities are graduate engineers with degrees in the area for which they develop software. Thus, each are allowed considerable discretion in development and maintenance of our products. This freedom, however, is controlled by the Release Administrator who is responsible for the integrity of each release of Ultramarine's software. It is primarily the duties of this individual which are discussed below.

Programming Standards:

Research has shown that many errors can be eliminated by a set of programming standards, and that the particular ones adopted are not as important as the existence of a standard. We have adopted most of the well accepted standards. While it is difficult to quantify, we feel that the most important factor in software integrity is the overall architecture of the product. To use currently popular jargon, systems should be designed "top down" and written with "structure". The two elements of our standard which we feel are next in importance are the typing of all variables and the initialization of all data in only one location. The first of these, along with a good compiler, eliminates most of the typographical errors. The second mitigates the possibility of forgetting to change it in all places.

Corrective Action:

To allow us flexibility in making major changes and enhancements we maintain two complete copies of all of our software. One copy is considered a "development" copy, while the second is a "release" copy. Major changes are made only in the development copy, and consequently the only changes made to the release copy are those necessary to correct reported problems. The Release Administrator is responsible for responding to all reported problems. If corrective action is necessary, the action is undertaken with his direction. It is his responsibility to document all changes in the release copy of the software. This documentation includes:

The discoverer of the problem
A description of the problem
The programs affected by the problem
The changes in the code to fix the problem

The revision number of all programs is incremented in the last digit whenever a change is made in the release copy. Two types of verification are performed for problem fixes. The first is to execute a problem which exhibited the error to ascertain if the problem is, in fact, fixed. For most problems, this is all that is done. In some cases, however, the entire verification tests for a given program may be run to ensure that the fix has no other impact.

New Versions :

Whenever all of the major modifications have been made for a release, the development copy of the source is "migrated" to the release world. This migration is performed by the Release Administrator. When a migration occurs, the revision number is changed in the first digit past the decimal, or if the changes in a release are particularly dramatic, the number to the left of the decimal is incremented. It is during this migration process that quality control procedures become increasingly important, as one must be sure that all problems fixed in the previous release copy will be fixed in the new version.

The first step in this procedure is to archive a tape containing all of the data for the old release. The next step is that the source of the current release is machine compared with the original source of that release. Since any discrepancies between these two sources should have been documented as a "problem fix", the list of problem fixes is compared to the true source changes to ensure that all alterations have been documented. Next, the source code of the new release version is compared, by machine, with the source code of the previous version. The results of this comparison are checked against the documented changes to make sure that all changes in the previous release have been made in the new one. This machine comparison is then archived.

Verification :

The final step in the migration procedure is to execute a set of standard verification problems. These verification data sets were designed to test each major option of the application, and currently there are a total of over one hundred problems which comprise the verification suite. The results of the verification runs are compared, by machine, with the results of the same test executed on the previous release. Any differences in the results are justified considering the changes in the release, and the comparison of the results is filed for reference.

After any discrepancies between the results of the verification runs have been resolved, the process repeats.

Documentation:

The documentation for each application is maintained in a manner similar to the source code. At the end of a release cycle, the documentation is revised so that it accurately reflects the new version of the program. After the manuals have been revised, a copy of each one is proofread, and any errors are corrected. The manuals are then printed for the release. Since the only changes which can be made during a release cycle are to fix errors, a set of documentation is valid for any version of the program which has the same revision numbers to the left of the decimal, and for the first digit to the right of the decimal. The revision number for the documentation is printed on the bottom of each page of the documentation.

Releases:

Our software is actually supplied to a site on a "release tape", the format of which varies with the type of computer at the site. In fact, for some installations, the release will be supplied on a media other than tape. Each tape will contain, however: All of the modules necessary to install the programs at the site,

A new manual for each program on the tape
Our verification data sets for each program on the tape

Along with the tape, a document describing the changes made in this release is enclosed.

The actual process of writing release tapes has been automated. One of our products, URSULA, keeps a database of which customers have which programs, and when instructed, a tape tailored for that site will be written. URSULA also keeps a database of the date the tape was sent and the revision number of the products on the tape.

Ultramarine, Inc. | Suite 325 | 3100 S. Gessner | Houston, Texas 77063 | (713)975-8146

Site Map

Copyright(C) 1996-2008 Ultramarine,Inc.