Software Quality according to ISO 25010

According to the ISO 25010 standard [ISO 25010], software quality can be classified in two major ways⁷:

• The Quality in Use Model, and
• The Product Quality Model

The quality in use model comprises the following five characteristics:

• Effectiveness
• Efficiency
• Satisfaction
• Freedom from risk
• Context coverage

The product quality model comprises eight characteristics:

• Functional suitability
• Performance efficiency
• Compatibility
• Usability
• Reliability
• Security
• Maintainability
• Portability

The product quality model has the most similarities with the previous ISO 9126 standard. Details of the Data Quality Model can be found in the ISO 25012 standard [ISO 25012].

In order to effectively judge the quality of a software system, all of these characteristics and quality criteria need to be considered during testing. The level of quality that each characteristic of the test object is intended to fulfill has to be defined in advance in the quality requirements. The fulfillment of these requirements and their criteria then has to be checked using appropriate tests.

Forty (sub-)characteristics

ISO 25010 breaks down the 13 quality characteristics listed above into a total of 40 further sub–characteristics. It is beyond the scope of this text to go into detail on all 40 sub–characteristics of the quality in use and product quality models. More details are available online at [ISO 25010]. Some of the more important characteristics are summarized below:

Functional suitability/functionality

The functional suitability (or, more simply, functionality) of the product quality model covers all the characteristics involved in describing the planned functionality of the system.

A quality characteristic is divided into three sub–characteristics:

• Functional completeness
  Does the function set cover all specified tasks and user objectives?
• Functional correctness
  Does the product/system deliver correct results with the required degree of accuracy?
• Functional appropriateness
  To what degree do the available functions fulfill the required tasks and specified objectives?

Appropriate tests can be used to check whether specified and implicit requirements are mirrored in the available functionality, thus answering the questions posed above.

Functionality is usually described in terms of specified input/output behavior and/or a specific system reaction to specified input. Tests are designed to demonstrate that each required functionality has been implemented in such a way that the specified input/output behavior or system behavior is complied with.

Reliability

The reliability aspect of the product quality model describes a system’s ability to perform at a specific level under specified circumstances for a specified period of time.

This quality characteristic has four sub–characteristics:

• Maturity
  To what degree does a system, product, or component provide the required reliability under normal operating conditions?
• Availability
  Is the system, product, or component always ready for use, and how easily is it available when it is required?
• Fault tolerance
  How well does the system, product, or component function in spite of the presence of known hard- or software faults?
• Recoverability
  How long does it take to recover specific data and normal service following a system or product failure or crash?

Satisfaction

The satisfaction aspect of the quality in use model addresses the degree to which user needs are fulfilled when the product or system is used under specified circumstances. This quality characteristic has four sub–characteristics:

• Usefulness
  How happy is the user with the perceived fulfillment of pragmatic objectives, including the results and consequences of using the system?
• Trust
  How certain is the user (or other stakeholder) that the product or system will behave as intended?
• Pleasure
  How much pleasure does the user experience when using the system to fulfill his/her personal requirements?
• Comfort
  How comfortable does the user find the system—also in terms of physical well-being?

Most of the characteristics of the quality in use model have a strong personal element and can only be objectively viewed and precisely evaluated under exceptional circumstances. In order to test this quality characteristic you will need to refer to multiple users (or user groups) in order to obtain usable test results.

A software system cannot fulfill all quality characteristics to the same degree. Fulfilling one characteristic often means not fulfilling another. A highly efficient software system is not easily portable, as its developers will have designed it to utilize specific attributes of the platform it runs on.

Prioritizing quality characteristics

It is therefore necessary to prioritize these characteristics. The resulting priorities will also act as a guideline for the testing thoroughness for each characteristic.