I know have a bottom of the server ready to run, an empty metrics API is running. Everything for this basic implementation is up and running.

Defining how to interact with data

At the end of the day, this app will have to manipulate data, store and read it.

The easiest way to make this work would be to use Flask database interface, but I want to secure the future, and it’s a personal project, so I’ll use it to extend my design abstraction beyond reasonable to experiment and learn.

So I’ve create a basic abstract implementation that will connect Data to Data Interfaces. my goal is to allow:

• 0 Coupling between my data and how they are stored.
• Being able to add handles easily in the future to allow other processes to subscribe when certain threshold are met.
• Being able to use SQL and NoSQL back-end the exact same way.
• Being able to change technology/endpoint with a single change in the configuration
• Being able to update data structure with out need to write and SQL code.

I don’t think the current design meet all those requirements, and some might be dropped at the end of the day.

All the commit code is abstract, so there is no much implementation to discuss, but let’s see a few interesting things:

Implementation Notes:

Context Manager Decorator:

A context manager is a design that ensure that action are taken before (opening a file, creating a live connection, etc…) and after (closing the file, closing connections, etc…) certain operations.

In python this can be done easily with a with statement:

with open("file", "r") as fp:
    do_stuff

When using this syntax python will silently call the __enter__ method of the object and assign its return to a local variable (that will exists only within the with context)

Once exiting, normally or because an exception occurs, the __exit__ method will be called.

Generic implementation are quite similar:

1. Create the object that will be worked with
2. Cleanup the object after use

I created a class that can be used to create __enter__ and __exit__ when used as a decorator to another class.

This is really useful to reduce the implementation of any DataInterface, as the protocol defines connect and close on a standardized way. This mean that implementing __enter__ and __close__ will most likely be very similar.

Why not using abc.ABC to provide a generic __enter__ and __exit__? Well ABC works with inheritance that is a bit heavier than protocol. This also removes the need of importing the module if __enter__ and __exit__ are to be implemented in an other way.

This will reduce the risk of creating import cycles when the implementation gets more complex.

Design choices

A lot of choices have been made for the design here:

Data conversion limitation:

I chose to limit the built-in conversion to a List of basic types, an object is a mapping of attributes names to any of these basic types. This reduced the range of the data that can be stored, but that allows me to implement normalize the data that will be read/written making complete decoupling with the back-end possible.

Interfaces Register:

Data object might end up stored in different endpoints depending of the configuration, but all they need to know about the implementation is the name of the interface, this will heavily reduce implementation on the data, as almost no code will be required.

Unit Test:

I’m still quite unhappy with the readability of the unit tests. So I’m trying to use setUp and tearDown methods in my test cases. And merge this to splitting the tests cases to the setup they requires. This should makes tests more readable.

In addition to those changes, I try to explain the test content in the method documentation string.

I still need to do more tests like this to see how it helps reading the tests.