Software development in automotive industry has bestowed greater comforts and conveniences to mankind. A remarkable progress in this field often faces a setback due to minor defects in the software. So there is recurring need for standardization and implementation of testing strategies. But the process of creation of test scripts to check if the software created complies with its specifications and requirements is both time- and resource-consuming. Generating a short but effective test suite usually requires a lot of manual work and expert knowledge. Patronizing research work in this field is the need of the hour. This paper solves the problem by using Model-Based Testing where test harness and evaluation are performed economically through automation. Simulink Design Verifier and Reactis are the tools used to carry out this purpose in Adaptive Front Light System. The resulting outputs obtained from Simulink Design Verifier and Reactis using Model-Based Testing prove that short test suites can be generated for the model where full model coverage can be achieved easily through automation. The outputs of these test cases when compared with the expected outputs confirm that the model developed is working as per the requirements.
In this modern era, automotive industries are investing their money, energy and technological talent into the quest to develop the best software solutions that will give transport an edge. Government also puts pressure on automobile industries to reduce congestion and pollution and to follow the safety standards. As automation technology unfolds its wings, the manufacturers have to innovate and update their software. Even a small error in the software developed may lead to accidents which may be fatal. Several cars of different manufacturers have been globally recalled due to faulty airbags and faulty software that can briefly shut down the engine. Never before there were so many recalls and fixes that are directly the result of software issues. So the manufacturers recognize the need for an adaptable and comprehensive software quality strategy for the automotive software development. This is only achievable through a truly integrated testing ecosystem that focuses on risks, with complete transparency of all testing-related activities. Failures can be prevented only when a system is tested before it is marketed. In the software testing process a new technique called Model-Based Testing has become quite popular in the recent years. Model-Based Testing is an art of improving the development process by cutting down the time consumption and costs during the development of an automotive embedded system [
In this paper, Model-Based Testing is performed using the following tools―Simulink Design Verifier, Reactis and SystemTest. By using these tools the requirements for the model are checked, model coverage is performed, and the test analysis is done. By doing this process using Model-Based Testing, the model is verified to be without any errors. Thus the errors if present in the design can be easily identified using this method during the early development stage of automotive software which reduces the total cost of testing.
The fixing of a defect found in the design stage itself costs much less when compared to the fixing up of defects found in the final testing process before production [
Simulink Design Verifier (SLDV) and Reactis are the tools used in the Model-Based Testing procedure. Simulink Design Verifier is a tool that is capable of automatic test case generation and for proving behavioral properties. An important feature in SLDV is the detection of overflows, division by zero errors as well as dead code present in the system during the design error detection process. SLDV is also used to automate requirement tracing. Thus verification and validation techniques can be used in the earlier developing stage itself. Reactis is an automatic test generation tool which can be used to test the Simulink models and to generate test cases which aims maximum model coverage. In Reactis, when an error occurs it is easier to trace an error back to its original source by backward stepping. SystemTest is a software tool that helps in the analysis of test results. The simulation outputs can be checked with the expected outputs to check for any deviations present. These results can be stored in the form of html document so that the tester can use this information for further testing.
The Adaptive Front Light System as shown in
Model Advisor checks for the requirements for MISRA-C [
To test the system completely, first model coverage must be performed [
This test case has resulted in the decision coverage of 32% and condition coverage of 19% for the model developed where decision coverage is a metric which measures the decisions paths covered in the model and condition coverage is a metric which measures the transitions covered in the model [
>>slvnvmakeharness (‘dssensorafls’)
The test harness is developed as shown in
To save the already generated test case and to use it in the further testing process the following command is used.
>>dssensoraflslog1 = sldvlogsignals (‘dssensorafls_harness’)
>>save existingtestcaseforafls. mat dssensoraflslog1
Thus an extended harness can be created for the model during the test harness step where the extra test cases generated help to increase the decision coverage of the model as seen in
The model is tested in Design Error Detection mode to check whether there are any errors like division by zero or integer overflow errors present in the model developed and it took 17 s to do this analysis and no errors were found. When the Simulink Design Verifier is used in test generation mode, the transitions, conditions and state coverage are performed. The analysis time taken for this purpose is 11 s. Only one objective is proven unsatisfiable for this model out of 47 objectives and it is shown in
The objective proven unsatisfiable may be due to the following reasons.
1) This often indicates the presence of dead-code in the model.
2) Other possible reasons can be inactive blocks in the model.
3) In rare cases, the approximations performed by Simulink Design Verifier can make objectives impossible to achieve.
Using the test cases generated from the test harness, model coverage is performed. Since all the decision paths have been covered for the model during this analysis 100% decision coverage is obtained. Condition coverage of 96% and Modified Condition/Decision Coverage of 92% is obtained. Each transition or input must be changed separately while keeping the remaining inputs fixed to achieve full Modified Condition/Decision Coverage. Thus the coverage report indicates how the behavior of the model can be exercised using the generated test cases. The statistical quality of the generated test suite and the parts of the model that were not tested properly can be identified. If a particular path through the model has no test generated for it, some of the test generation parameters can be changed and the test generation step can be repeated. This is where it can be useful to use an animation tool to investigate the behavior of the path in the model and decide whether the lack of tests is due to an error in the model, a normal feature of the model, or inadequate test generation. So to improve the coverage, an abstract test can be added for this path manually or give hints to the tool to get the desired results. This test harness coverage can also be done using another tool called Reactis.
In Reactis automatic test case generation and the coverage analysis can be performed [
The window as seen in
# | Type | Model Item | Description | Test Case |
---|---|---|---|---|
43 | Condition | Chart. “[ux > 5 & uz ≥ 1]” | Transition: Condition 2, “uz ≥ 1” F | n/a |
Reactis has generated five test cases where the first test case contains a single sample point input, the second test case has 3 sample points input, the third test case contains 7 sample points and the remaining two test cases contain 8 sample points. All these test cases produce correct results. When the tests developed using Reactis is made to run in Simulink, there is no error when interpolation is turned off and there will be some differences in the outputs generated in Simulink when compared to Reactis outputs when interpolation is turned on because in this case Simulink will sample inputs slightly differently than the original Reactis inputs which are shown in
The SystemTest software provides a framework that integrates software, hardware, simulation, and other types of testing in a single environment [
test results. In SystemTest the final process of test evaluation is performed where the test outputs will be compared with the expected outputs. During iteration 1 the input test vector is [0.01 9 0.3; 0.02 5 0.2; 0.03 1 0.6] and for iteration 2 [10 × 3] vector is taken as input and their results are seen in
Manual work has been pushed to the background and day by day software is taking complete control in automotive industry. Model-Based Testing will be a blessing to rectify the errors in the software with less time consumption and yield products of high quality. This paper deals with the identification and removal of faults in automotive software development using Simulink Design Verifier and Reactis. Using these tools 100% decision
coverage is achieved by the developed model. Reactis shows a slight improvement when compared to Simulink Design Verifier with condition coverage of 97% and modified condition/decision coverage of 93%. Additional test cases can be added to the test suite by adjusting some parameters in the tools used to achieve full condition coverage and MC/DC coverage. SystemTest has been used to display the outputs obtained from Adaptive Front Light System in the form of graphs during the final test analysis procedure in Model-Based Testing. Thus the proposed method helps in doing the testing process at a faster pace. Today, software is being developed to regulate the speed of the vehicles upon seeing the unexpected obstacles while driving up or down in a mountainous terrain. Testing methods combined with Model-Based Testing are to be developed for this type of software which will be of an important value for a safer ride and upgrading the testing skills is needed.
P. Sivakumar,B. Vinod,R. S. Sandhya Devi,R. Divya, (2016) Deployment of Effective Testing Methodology in Automotive Software Development. Circuits and Systems,07,2568-2577. doi: 10.4236/cs.2016.79222