Gopalakrishnan M
10. October 2023 Categories: Technology,

Embedded systems have become integral to various aspects of modern life. They power everything from our smartphones and cars to the smart thermostats that regulate our home's temperature. Despite their hidden presence, embedded systems play a crucial role in keeping our technology-driven world running smoothly. However, it's vital to conduct structured testing to guarantee the dependability, performance, safety, and security of these systems. Else, as discussed in our earlier blog “Crucial Need for High Quality in Embedded systems”, there might be a major impact on the product success and brand.

To ensure that the right product is developed in the right way, there are set of process and methodologies that are broadly grouped into the Verification and Validation process. In this article, we will delve into the idea behind Verification and Validation (V & V) for embedded products and the process to be covered during each phase of the product development life cycle.

Verification in embedded product development

Verification is essentially the process of checking whether the development done meets the requirements established for the given phase. It encompasses a range of activities including examining business requirements, system requirements, conducting design reviews, and going through the code during the product development process.

This type of testing is often referred to as static testing, which essentially means making sure that we are creating the correct product by ensuring the product is being developed according to the defined standards and guidelines.

Validation of embedded products

Validation, on the other hand, is the process of evaluating a system or component during or at the end of the development process to determine whether it meets the defined requirements. It involves checking that the product fulfills its intended purpose in the real-world application and environment.

In embedded product development scenario, it mostly involves working with the real physical product and checking for functionalities using one or more of well-defined testing methodologies.

Need for Verification and Validation

Both the verification and validation processes are crucial in delivering a high-quality product. They are expected to be done in tandem throughout the product development life cycle. But in today's competitive market, in many cases, some of the Verification and Validation (V & V) activities often occur in the later stages of the product development cycle. This can lead to longer lead times in identifying problems and higher costs associated with fixing them.

V & V - Cost Impact

V & V - Cost Impact

When the V & V process is delayed or not carried out correctly, it can result in security and performance issues being discovered late in the program. Fixing these issues at this stage can be challenging and expensive. Additionally, because the issue has persisted for a longer period during development, there may be a temptation to opt for a quick workaround rather than implementing a thorough and proper solution. To prevent unexpected issues and costly mistakes in the program, it's crucial to conduct verification and validation at relevant stages of the development cycle such as Planning, Design, Development, Testing & Deployment.

Verification Methodologies

There are many standards that define the verification and validation methodologies to be followed, based on the use cases, such as ASME V&V 40 for medical devices, ISO 26262 Functional safety for automotive applications. The common thread among them involves a careful analysis and review at each stage with varying levels of details and documentation.

The below diagram captures the standard V model for Verification and Validation.

V Model In Verification And Validation

V Model In Verification And Validation

In general, the following documentations are needed to evaluate the verification of the embedded product.

  • Business Requirements Documents
  • Software/System Requirements Document
  • High Level Design Document
  • Low level design document

Some of the recommended approaches to be followed at each stage of Embedded product development are tabulated below.

Stage Approach
Planning Requirement review
Traceability matrix
Integration Plan review
Design Design and Specification analysis
Use case review
Test plan review
Test case review
Traceability matrix
Development Schematic review
Mechanical design review
Code review
Traceability matrix

Being an internal process, it is important to carefully document the activities done along with the results, without missing anything.

Validation Methodologies

Validation is perhaps the most visible part as it concerns the external stake holder and serves as the proof of the developed product conforming to the requirements dictated by him. It involves various testings as listed below and their need determined by the underlying business needs and complexity.

  • Unit Testing
  • Integration Testing
  • System/Functional Testing
  • Acceptance Testing
  • Regression testing
  • Performance testing
  • Security Testing

Let us take a look at each of the testing methods.

Unit Testing :

In the V-Model, Unit Test Plans (UTPs) are formulated during the module development phase. These plans are executed to pinpoint and rectify errors in the development stage. A unit or module represents the smallest independent entity, such as a program module. Unit testing confirms that this compact entity functions accurately when isolated from the rest of the code or units.

Tools that can be used in unit testing are Googletest, JUnit etc.

Integration Testing :

Integration testing takes place as the second step in the software testing process, following the completion of unit testing in the development life cycle. Integration Test Plans are devised during the Architectural Design Phase. These tests are conducted to validate that independently created and tested groups can effectively coexist and communicate with each other. This phase of testing holds significant importance in guaranteeing seamless interactions among various units or components, assessing their functionalities, and evaluating how well they can operate collectively as a single entity.

Some of the tools used in integration testing is Vector tool, LDRA Suite, etc.

System/Functional Testing :

System testing is the third phase in the testing process, following integration testing. As the name suggests, this involves testing all the components of the system together to verify if the overall product meets the specified functional requirements. System testing holds significant importance as it occurs when the product is nearly ready to be shipped. This phase allows for testing in an environment that closely mirrors the user's experience once the product is deployed.

Some of the recommended tools to be used for functional testing are Embunit, TestBot etc.

Acceptance testing :

Acceptance testing is closely tied to the analysis of business requirements. In this phase, essential positive test cases are selected from the pool of system test cases for use in acceptance testing. This involves evaluating the product in the actual user environment. It's a test carried out by the customers or their representatives to determine whether they are willing to accept the system that has been developed. Some of the tools used in Acceptance testing are Robot Framework, TestRail etc.

Regression testing :

Regression testing involves re-running a subset of unit, integration, and system tests after a change has been made to the code. The primary goal is to verify that the recent code modification hasn't negatively affected the security, performance, or functionality of the product.

This process includes identifying the alterations in the code and the associated workflows to guarantee that the latest changes do not introduce any adverse effects. It also encompasses updating the product while ensuring that previous configurations and data points are preserved in the embedded product. Some of the tools used in Regression testing are Robot Framework, TestBot, etc.

Embedded product performance testing involves evaluating the speed, responsiveness, and efficiency of an embedded system or product under various conditions and workloads. This type of testing assesses how well the product performs in terms of processing speed, memory usage, and resource utilization. It aims to ensure that the embedded system meets specified performance requirements and can handle its intended tasks without experiencing significant slowdowns or failures. This testing is crucial to identify potential performance bottlenecks and optimize the system for optimal operation in real-world scenarios.

Key differences between verification and validation

Some of the key differences between the verification and validation are captured below.

Verification Validation
Purpose To ensure that the software conforms to its specified requirements and that it has been developed correctly. To determine whether the software meets the customer's needs and expectations and whether it fulfills its intended purpose in the real-world environment.
When It Occurs It followed throughout the development process. It involves activities like code reviews, design inspections, and unit testing. It is performed at the end of the development process or during the testing phase. It involves activities like system testing and acceptance testing.
Focus Check whether software is built right, according to the specified requirements and design. Check whether right product has been built, by ensuring it meets the customer's needs and expectations.
Methods Uses static techniques such as reviews, inspections, and walkthroughs. It does not involve the actual execution of the code. Uses dynamic techniques such as testing (e.g., unit testing, system testing) to evaluate the software's behavior by executing it.
Testing Level It may not require a full product for testing It requires a full product for testing

Conclusion

As can be seen, in an embedded product development life cycle, verification and validation are important. Verification ensures that the product is built the right way, following all the rules. Validation makes sure the product works well in the real world and meets what customers want. Following the procedure, help catch mistakes early, thereby making it cheaper to fix them. They also make the product stable and strong. So, by carefully checking and testing, we can make sure embedded products work great, make customers happy, and save money in the long run.

Subscribe to our Blog