Part 1: What is an Automotive SPICE® assessment?
An Automotive Software Process Improvement and Capability Determination (Automotive SPICE) assessment applies to developers of automotive electronics. Their automotive customers expect the applied processes to meet the criteria of Automotive SPICE.
Criteria empower developers to design processes in a purposeful way, which is meaningful because the quality of processes can affect the reliability of the product. For example, let’s say you can’t control the configuration of a major release. Then what happens if your customer fails to integrate your system into the car? As a result of the missing system, the car manufacturer has to postpone the start of production. Therefore, reliable processes are mission critical. Automotive SPICE provides developers with elementary process criteria that is based on industry experience.
Hence, the assessment is designed to evaluate whether processes reflect these criteria. To do this, an assessment checks the implementation of a process against the process requirements specified by Automotive SPICE.
On the surface level, a supplier may initially question the value of an assessment. The benefits have proven themselves to be clear: Third-party evaluation of processes better informs the customer organization of the supplier’s level of capabilities in the assessed project. Additionally, the supplier benefits from an Automotive SPICE Assessment report that provides guidance for process improvement.
Insight: An Automotive SPICE assessment is the evaluation of the capabilities that exist within your research and development (R&D) processes.
What is the scope of an Automotive SPICE assessment?
Automotive SPICE is derived from the acronym for SPICE: Software Process Improvement and Capability Determination. So, the last character stands for dEtermination – the capability level reached by your processes. Ultimately, Automotive SPICE is a process assessment model for determining the capabilities of development processes.
Since customers initiate the requests for assessments, the assessment scope contains a selection of processes that are performed in the customer’s project. The capability level therefore refers to these processes. Automotive SPICE assessments usually do not cover the entire R&D. As an example: If you work for car maker A, the assessment only refers to processes applied to projects for car maker A. Processes for car makers B and C are not considered. With Automotive SPICE, there are only assessments, no audits. In other domains, there are exhausting debates about their distinction: An audit is against the process and refers to a management system. For this, the evaluation follows a binary logic, determining only whether the requirements have been met or not.
An Automotive SPICE assessment examines processes based on actual evidence: The assessor must use work products to prove that processes meet the requirements. In the Automotive SPICE assessment, one is asked only about the Automotive SPICE requirements, not about additional internal process requirements.
In doing so, Automotive SPICE takes a much more fine-grained approach than an audit: instead of simply determining whether you passed, Automotive SPICE looks at many parameters. This starts with what are called base and generic practices: These practices reflect good industry practice. Here, you list the process requirements necessary to achieve the process goal (process outcomes). You use work products to demonstrate the implementation of these practices. This gives the assessor an indication of process capability. Thus, they can apply a process attribute to the process and evaluate it.
The assessment then reveals the extent to which certain processes, or their attributes, meet the Automotive SPICE requirements. For example, you’ll get one level statement per assessed process. Based on a matrix, this statement aggregates the ratings of base practices, process attributes and capability levels to an instance. In the end, a detailed picture emerges of how goal-oriented the project’s R&D processes are, and where there is potential for improvement. This makes an Automotive SPICE assessment a valuable asset.
Insight: An assessment covers a set of processes in a particular project. There, it extensively determines the potential for process improvements.
What are the different types of Automotive SPICE assessments?
Consider these questions: What is the purpose of your Automotive SPICE assessment? Do you want to provide a basis for process improvement, or do you want to determine process risk that could have an impact on a specific release? An Automotive SPICE assessment supports both goals, but the approaches differ.
- Process improvement assessments identify strengths and weaknesses of the processes: In other words, are the processes fit for purpose? With this information, one can redesign processes to mitigate weaknesses. Use this type of assessment in the process improvement phase and to prepare preparation for a product risk assessment.
- Can quality risks be encountered effectively? Finding an answer to this question is the intent of a product risk assessment. In a release, the assessment represents the customer’s view of getting a reliable product on time.
Here is a practical example for further explanation: SWE.6 is the Software Qualification Test process. In a process improvement assessment, the implementation of the process is reviewed against the Automotive SPICE process requirements. For this, appropriate examples are inspected. One-fifth of the requirements are sufficient for a test here.
On the other hand, in a product risk assessment, if only 20% of the requirements had been tested for a release, then there is a significant risk that the software will not function properly.
Insight #3: Whether your assessment scope is on a product improvement project or on product risks, the assessment is about the capabilities associated with the processes.
Part 2: What is the assessment process?
Since independent individuals evaluate a set of processes in a specific project of teams, the assessment structure reflects the interaction between the third party and team members.
Some activities are the responsibility of the assessor, other tasks are the purview of the supplier team and organization. Activities can also be distinguished between those that are performed on location from those that are performed remotely.
There are three phases of the assessment process:
Assessment preparation phases
The assessment preparation is mainly dealing with planning the assessment. The assessment scope is defined as well as the objectives and other factors that influence the assessment. The result is an assessment plan with all necessary data for the assessment as well as an assessment agenda, which is coordinated between the lead assessor, the sponsor and the organization to be assessed.
An important part of this is the assessment scope definition which contains mainly processes, capability levels, instances, parts of the product, teams and locations.
An important planning decision is also to determine the assessment use case: Do we speak about process improvement assessments or process-related product risk assessments?
The assessor moderates the initialization meeting with the sponsor and the representative of the project being assessed. Planning the assessment process and reviewing documents are among the important preparations of the lead assessor.
Assessment execution phase
An assessment is most efficient when performed on site and in person. In the opening briefing, the assessment typically begins with a short introductory session. The lead assessor would provide a presentation to briefly introduce the concept of Automotive SPICE, the assessment process and the scope of the assessment. A representative from the supplier company generally follows with a presentation of the product, the project and the organization.
The interviews and documentation reviews are then carried out according to the agenda. The actual interviews on the processes start with
- Project management,
- Quality assurance and
- Configuration management.
Through these crucial processes the assessors can receive insights into the project. This is followed by the engineering processes according to their natural sequence. This sequence is important because these processes build on each other.
For example, to check the traceability of the system tests to the system requirements, one needs to have previously understood the structure of the system requirements.
After the engineering processes, remaining support processes and supplier monitoring will follow.
Between the interviews, the assessment team performs consolidation sessions. There, the recorded evidence is reviewed and consolidated, and the practices and process attributes are rated. These consolidations and their results are confidential. The main reason is that ratings may need to be revised later due to new evidence.
At the end of the assessment the assessors present the results.
Assessment conclusion phase
The final phase gives an assessment report, which provides teams with information about:
- The capability levels. Processes are rated individually due to certain parameters. The assessor calculates them with a rating formula.
- The rationale why the assessor made the rating decision: a justification of the impact – or absence – of relevant process outcomes.
- The statement of how the evidence provided relates to process requirements of Automotive SPICE. This also includes a detailed description of the identified strengths, weaknesses, and observations per practice.
- The reference of the seen evidence to the documented observation.
- A management summary to identify the most crucial aspects of the assessment.
Assessment roles
As in any process, there are different roles in an assessment. Let’s review three major roles: lead assessor, co-assessor and sponsor.
The lead assessor is responsible for the formally correct execution of the assessment. The required qualification for a lead assessor is the Automotive SPICE Competent or Principal Assessor level. Responsibilities include
- Communicates with the assessment sponsor.
- Plans the assessment.
- Confirms that the assessment team is qualified and covers all aspects necessary to consider as assessment context, e.g., cybersecurity or functional safety.
- Confirms that the assessment is conducted according to the requirements of the assessment process.
- Ensures the correct assessment documentation.
An assessment team consists of at least two assessors, the lead assessor and one or more co-assessors. These must be certified with the VDA Quality Management Center (QMC), otherwise the assessment will not be recognized by most original equipment manufacturers (OEMs). For an independent assessment, at least the lead assessor must be from another organization. Responsibilities include:
- Confirms that the assessors have a valid certification.
- Confirms that the assessment team has access to all necessary resources.
The sponsor commissions the assessment, receives the assessment report and decides on next steps for sharing the information. Think of the sponsor as a sparring partner: they are an outsider to a project, but highly interested in its success. If the sponsor comes from within the supplier company, they are usually a member of senior management. If a manufacturer commissions an assessment from a supplier, the supplier’s representative becomes the sponsor.
Furthermore, it has proven useful to appoint an assessment facilitator for each site who takes care of the organization of the assessment.
To summarize: An assessment has at least three important roles: the lead assessor, one or more co-assessors and the sponsor from supplier’s senior management
Part 3: How to prepare for an Automotive SPICE assessment
The result of an assessment is of great importance to projects: The team gains valuable insights for improving workflows to reduce risks. Think of the assessment as an opportunity that will provide some useful insights.
However, part of the right attitude is to prepare thoroughly. For example, this is the optimal way to familiarize yourself again with tasks and processes that have been in the past for a long time. There is a lot of time pressure in the assessment itself, but thorough preparation allows you to answer clearly and precisely. In this way, you are in control of the assessment to a certain extent.
For some, it will be important to practice the terminology of Automotive SPICE again. This way, one can better anticipate the assessor’s expectations when they specifically ask about a work product.
In this video, we review the expectations during an assessment and discuss an ideal preparation in three phases: 2-3 months in advance of the assessment, 2-3 weeks in advance of the assessment and expectations during the assessment.
Preparation phases
2-3 months in advance:
- Finalize the assessment agenda in time. This is a prerequisite to identify the right people as interviewees.
- Understand the status quo of the project regarding Automotive SPICE: Consider which areas in the project are neuralgic, areas in which teams have particularly struggled.
- Map Automotive SPICE practices to the real-life implementation in a project. Do they fit?
- If they do not fit, identify remaining gaps between formal expectation and the implementation.
- Create an improvement plan for closing the gaps.
Insight: Perhaps you now have an idea of what good preparation comes down to: Deal intensively with what you and your team have accomplished. With this, you can make up for minor omissions – nobody is perfect. In this way, assessments are performed honestly and with rigor for everyone’s benefit.
2-3 weeks in advance:
- In a first step, clean up the project folder. Ensure there are no issues with naming and versioning. Remove artifacts that are not relevant anymore.
- Make sure all teammates know where to find all the important documents.
- Prepare or select work products to present as evidence.
- Ensure that the interviewees understand the requirements by Automotive SPICE. If they need a little coaching, recommend a short video: Automotive SPICE Oversimplified.
- Teach teammates about their role in the assessment.
- Brief teammates on how to explain the ways in which their tasks contribute to meeting ASPCE requirements.
- The last point is on soft skills: Coach teammates and stakeholders intensively on expectations in the interview situation.
Insight: If you keep this list in mind, you will see the importance of intensive preparation. You don’t want to do anything wrong, but you also don’t need to sell your performance short.
During the assessment:
For soft skills, the best preparation for an interviewee is to become familiar with the content of the processes relevant to their role and responsibilities.
- Do teammates know the related process descriptions and process steps?
- Do they know the generic practices? These address such aspects of capability level 2 and 3 as skill management or resource management.
What about the project binder structure?
- Do teammates understand the overall structure?
- Do they know where the work products are stored?
- Do they find them quickly and easily?
- Do they know which work products and selected samples are being presented? Consider a work product list as part of preparation.
Does everyone involved understand Automotive SPICE?
- Does the team know the essential terms of the standard?
- Is there an understanding of the underlying logic of Automotive SPICE?
- Has the team read through the best practices and generic practices of all related processes?
This briefing is essential. This is because interviewees are often unaware of how their work and performance fit into the model. As a result, ratings can be worse than they should be. Avoid this trap by preparing thoroughly for the interview. Ask yourself: Do you have an answer ready for a possible closing question: What are the topics that have not yet been addressed by the assessors? What is particularly good, what are you proud of?
Automotive SPICE assessment: key takeaway
Take assessment preparation seriously – if your processes show too many weaknesses, you may feel the displeasure of your customer. And it is likely you will have more trouble eliminating the weaknesses than you will have to invest in preparation.
Learn more about Automotive SPICE.
If you want to learn more about Automotive SPICE®, check out our Automotive SPICE Training.
Read our related content
Services for Automotive SPICE and Extensions
Automotive SPICE® Assessments guide