When building a product such as an anti-lock braking system for an automobile, or a railroad, or process control safety system, making sure that the product works as specified is a big part of functional safety. When it is time for the system to engage to prevent an accident, you need to know that the system is going to work almost all of the time. One area that is often overlooked in the effort to ensure that the system works properly is the effects of software tools used in the development process. Examples of such tools include:
- Static checkers
- Verification tools
- Test automation tools
So if we can agree that these tools must be analyzed, what can be done? At exida, we have been facing this issue for many years and have come up with a standard methodology that has worked well. Additionally the methodology is compliant with multiple safety standards including IEC 61508 (General), ISO 26262 (Automotive), and EN 50128 (Rail) and to some extent IEC 60880 (Nuclear). The methodology includes the following six steps:
Step 1: Description of the assumed workflow(s), tool functions, and properties used in the workflow
- Define exactly how to tool will be used including its relationship to other related tools.
- Determine the criticality of each tool function. This analysis is based on the effect a tool failure could have on the final product. Steps (3) through (6) only need to be implemented for the tools that have been classified as critical tools that can affect the functional safety of the product.
- Tool qualification is the step that shows the tool conforms to its specification or user manual. Some of the requirements for how to do this require some analysis and tailoring of other standards. For example, ISO 26262 states that one technique that can be used to show tool qualification is conformance to a safety standard. However, ISO 26262 readily admits that not all of the requirements of most safety standards will apply to tools, so these standards must be analyzed and tailored to the requirements that are applicable.
- This analyzes how well the tool development conformed to the tailored requirements. As we are focusing on the suitability and safety of the tools for use in functional safety projects, this “standard approach” is extended by (5) and (6).
- This is analysis of potential failure modes of the tool along with a definition of mitigating measures that make it likely that such failures will be discovered so that they can be corrected.
Five of the six steps defined in this methodology can be done by the user of the tool. However, Step 4 requires that the tool manufacturer participate. This is not always easy, but it is a necessary step for critical tools. There are some alternatives for vendor participation, such as confidence from use that can be used in some cases. Confidence from use can serve as validation of tools that have been successfully used on many previous projects. However, this approach is not applicable to tools that have not been used before, or for the higher Automotive Safety Integrity Levels (ASIL). In these cases the tool development process must be analyzed.