SensorNet package CI Release

After the late release of Raspberry Pie Pico W, I wanted to rework my last design around the new feature that this hardware allows:

The new design is going to work around a git flow to support CD/CI. To ensure deployment quality, an integration stack is deployed before the production one, and a docker container will run tests to ensure everything works as expected. Once validated, the production deployment set will be deployed.

Let’s get in details on how the gitflow is defined and set:

SensorNet Package

This package is the main event, it contains the logic behind reading the sensor endpoints and loading the data into the database.

This package have provides a daemon that will run and periodically query all sensor configured, I’ll get into more details on how I’ve decided to work on this package to support delivery feature deliveries.

The package is designed is such a way that integration of new feature is done by creating pull requests, on dev branches.

Each push triggers a workflow that runs unit tests, coverage and code style. This trigger is defined in the validate workflow. Tests are run by pytest, coverage is validated by Codecov, this allows to enforce not only a minimal coverage value, but also a rule to enforce each new commit to provide an equal of greater coverage rate.

Once a branch is ready to be merged, the developer will create a pull request, pull request will trigger this workflow once more, to ensure no new issues was created by concurrent merges. It will also run codeQL workflow to check for security issues. More workflows can be added later on to review outdated dependencies, etc….

Then the branch is merged, and this triggers the last workflow that publish the package in PyPI. This step is crucial, for the next part of the CDCI pipeline. Note that if for some reason more developers joins the project, moving the trigger from on merge to nightly build will start to make sense.

Docker images

We have to create 2 docker image, the first one will be generated from SensorNet releases, the package will be private, and I will create a sensor image with configuration that will run SensorNet on daemon mode.

The image will then be published for being used by in the deployment sets.

The second docker package will create a image that will host all test endpoints and run integration tests.

Deployment Sets

Once all docker images are ready, the integ deployment set will be rollout, integration test will run, and once the integration tests are OK production deployment set is rollout.