ITEA (Integrated Test Evaluation and Acceptance) is the EUREKA Cluster programme supporting innovative, industry-driven, pre-competitive R&D projects in the area of Software-intensive Systems & Services (SiSS). SiSS is a key driver of innovation in Europe’s most competitive industries, such as automotive, communications, healthcare and aerospace. Now also being taken into the defence sector.
ITEA is a process for continuous development and follow up of a product or system, preferably through the whole system lifetime - typically for between fifteen to twenty-five years. The process supports all types of system upgrades to include so called midlife-updates. The process goes in cycles during the whole system lifetime and when used, ensures the possibility to:
- Follow the technological development and update the system accordingly.
- Gain experience from operational use and operational tests during the entire lifecycle.
- Adjust doctrines and tactical procedures in accordance with new experiences.
An ITEA regime must include all type of system users, support personnel and all required system interfaces. Co-ordination with the organisation’s Research and Development (R&D)-, Concept Development, and Experimentation (CD&E) activities or programs is preferable. It is a prerequisite that the contractor is involved from the outset of an ITEA process. If possible, the contractor should be incorporated already from the initial development of user requirements based on the internal defined performance criteria of the organisation.
The figure below describes an ITEA loop, which is also called an iteration. The loop consists of the following stages: user feedback, requirements update, implementation, test, evaluation and acceptance and roll-out.
In the case of an upgrade of an existing system, then the process starts with the legacy system and an initial selection of prioritised new requirements. One should then prioritise and initiate a limited development according to the selected requirements. The implemented results should be tested and evaluated accordingly and thus one has started the first of many ITEA loops.
To ensure success, the developed requirements must be verifiable. Requirements such as: "The system must be robust", "The system should be as small and light as possible", or "The system should use as little power as possible", must be avoided at all costs. This also applies for all tests in the test hierarchy. There must be defined methods to measure performance and desired targets. For example "The system should not weigh more than 2 kg" and "The system should be able to start from off to operational in 3 minutes”.
ITEA Loop Requirement Development
When deriving measurable requirements for a system, it is recommended to start with a user unit’s measurable performance criteria. Based on the unit performance criteria, user requirements may be derived. Then based on the user requirements, system requirements may be derived and lastly, based on these, the technical requirements.
Performance criteria are derived by first defining a unit’s measurable performance. I.e. how much better the unit shall perform in a defined scenario and how to measure it. It can be according to timeliness, the ability to move and navigate in darkness, the ability to intercept a defined enemy or such.
User and unit requirements
The next step is to define the requirements that single users must satisfy in order to enable the unit to reach the desired unit requirement. Again, these requirements must be measurable. I.e. information exchange possibilities, time units or navigation performance.
User requirements are then broken further down into simple measurable system requirements that will enable users to perform functions that fulfil their user requirements. I.e. the ability to share situation picture updates, message types, types of supported communications bearers, communications distances, etc.
When the system requirements are defined, one must together with the contractor, further break these down into simple system requirements that can be measured in defined technical tests.
Evaluation and acceptance
After a delivery, the result should be run through a test regime. The contractor performs the defined technical tests locally, and then the system tests together with the customer. Upon approval of the technical delivery, the update should go through a small-scale operational testing and if this also is approved, the update should be run on a unit level to see if the update provides the desired increased operational performance of the unit. It is essential that the contractor participates in all these tests, to gain the necessary experience for further development.
[In the case of less than satisfactory performance ]If experiencing lack of- or less performance than expected, one can at any time choose to stop during the process. If so, one may decide to adjust and start a new ITEA iteration with redefined requirements. That way the development process is adjusted as one continuously learn during the process. This applies to both the technical, operational and doctrinal sides. It might also result in a need to change the current tactical procedures by which the functionality is updated and provide increased operational performance.
One describes and manages such processes through an ITEA plan (ITEAP), following the system through the whole system lifetime. An example process is outlined in the next figure.
To ensure success using ITEA, one must develop a continuous updated ITEA plan with regular, e. g. annual audits for coordination and management of:
- User Involvement
- Concept development
- Contractor Cooperation
- New releases and deployments
Benefits of using ITEA as a development framework
The ITEA process fits like a hand-in-glove with the Lean and Agile improvement philosophy. The process is ideal for national operational programs, for the size of an operational defence unit, a police force or a non-government organisation. ITEA supports all phases of a system’s lifetime; from the requirement phase through initial procurement, via operationalisation to a continuous improvement phase, until the end of the system’s lifetime.
For further background information on ITEA organisation, bodies and tasks, please also see: https://itea3.org/about-itea.html