In complex systems development, integration is often a difficult step where different codebases are put together. This critical milestone comes often late in the project and can lead to unpredictable delays to delivery.
Continuous Integration (CI) is an agile software practice that tackles the issue of integration by enforcing repeated partial integration as early as possible. This gives the team an early warning and early bug-fixing capability. The final integration is then less stressful, more predictable, and thus more efficient.
At Watt&Well, we decided to adapt this mainstream software practice to the hardware world through continuous FPGA integration. This allows us to spot bugs earlier, to nip them in the bud, and to monitor progress consistently throughout the development.
Our Agile electronics solution provided
Most hardware projects follow the waterfall or “V” model. The first steps aim at improving the product definition and design (“Left side of the V”), and the following steps aim at improving the confidence in the product through Verification & Validation (“Right side of the V”). Product maturity increases as time goes by, and the end of each step is usually formalized by a review with all stakeholders.
However, this model is not particularly representative of the reality of development: When issues are identified, a rollback is made through previous phases to correct them, resulting in code, architectural, or requirement changes. The later these issues are identified, the more consequences they have on the project timeline: a requirement change, for instance, is less costly when implemented at the beginning that at the end of a project.
When applying strictly the V-model, trust in the design’s validity is gradually built-up during the verification operation, first at the module level, then at integration. What Continuous Integration allows is to build up trust earlier and more gradually, integrating iteratively developed modules during development. Tools aim at making this process as easy as possible by automating the building and testing process. The outcome is earlier bug detection and continuous monitoring of the codebase (code quality, verification progress, critical resource use, etc.…) throughout the development. Designers, testers, and the project team have then a better view of global progress.
Adapting Agile To Reliable Hardware Design
Aerospace products require a high level of confidence in the electronics design, including FPGAs. This is reached through strict verification procedures requiring a high level of functional coverage, code coverage, and code quality.
To achieve these goals, we decided to implement two iterative loops throughout our FPGA development: one to assess code quality continuously, the other to track the functional coverage progress. Both are an insight into our development: are we building the product right, and are we building the right product?
The main idea of Continuous integration is: Integrate consistently through automation: as new RTL code is developed; it is pushed to a centralized source-control software. This ensures every actor of the team works with a known baseline.
The CI system then launches the code quality pipeline, which runs an automated test suite on the whole codebase at the unit level. Coding rules compatibility, code coverage metrics, and unit test results are obtained through a combination of simulation and linting.
Periodically, the CI system launches the integration pipeline, to put together all the modules, to run functional simulations and extract functional coverage, to implement the FPGA design on the hardware target, and to track critical metrics such as area/frequency performance.
Then, the version on which all these tests have been run is kept for review. If this version is deemed fit for release (internally or as final delivery), it is configured in the source control tool. Otherwise, the test results summary is kept for reference.
What Makes Our Response Unique
Continuous Integration of FPGA design
Shorter integration time
Incremental delivery to customer
Development following ECSS Quality Assurance standards