I am Dolf Andringa, a geek and software engineer who loves diving into new challenges.

For a project I am currently working on, I need to develop a web and mobile client. In short, it is a data-management system for a government agency where people in the field will report environmental data, while regional and national offices validate the data and use it in management reports.

Of course, I am using existing (open source) products as much as possible, not to re-invent the wheel. But by now I have a set of tools available that allow me to rapidly develop this stack.

So what am I using for this? To create nice looking reports/dashboard for policy makers and managers, I decided to use Apache Superset. This is an awesome tool that allows not-too-technical people (depends on your point of view I guess) to create dashboards that are great for policy makers. It can connect to a whole host of databases and pull the data in, and provides drag-and-drop interfaces to create graphs, pivot tables, filters, maps, etc. Basically anyone with decent MS Excel skills can also use superset.

The nice thing for me personally, is that superset is developed using Flask App Builder (FAB). I have been using Flask for a long time already so it allows me to extend Superset if needed. Also, it allows me to develop additional functionality in FAB and integrate the two in one application.

So the next element in the stack is a web-application built using FAB. The web-application allows government officials to see all the reported data and verify it to ensure data quality. Once verified, data will be available for the reports in Superset. Additionally, the web-application allows authentication and authorisation management. Different government agencies will report using different datasets. In the web-application we can configure who has access to which datasets. The datasets are defined using SQLAlchemy.

Aside from the security/credentials/authentication functionality, FAB also provides views functionality, which allow you to define views very easily based on your SQLAlchemy data models, providing nice looking CRUD functionality out of the box. So certain government officials can use the web-application to edit reports (for validation) or add new data based on paper reports (yes that still happens). FAB uses WTForms for the form generation. Since location is an important part of the data being collected, I also use GeoAlchemy to add the geographic component to the data models using PostGIS. This is where the first custom development comes in since FAB doesn’t support GeoAlchemy. Guess there will be a contribution from me coming to FAB soon :)

The next step is a mobile app that will be used by people in the field to collect data. Data will be collected on- or offline, using the mobile camera and GPS and data will be sent to the central server when internet is available. Ionic will be used for the mobile app development, which will basically just display forms for the datasets that the user has access to as defined on the server. Forms and validation are easy in Angular/Ionic. So I’ll create an app that gets a form definition from the server and dynamically generates the form from it. So how do I send the form/dataset definitions from the server to the mobile client? Using JSON Schema. This allows me to define the datasets, data types and restrictions like number limits, string formats (email, date time, length, etc) and turn that into a form in Angular. How do I generate the JSON Schema, I will use a library to convert the WTForms forms as generated by the FAB views into JSON Schema.

And how do the mobile and web-applications communicate? Using websockets. I will add those to FAB and add JSON Web Token functionality to the FAB security manager so the mobile app can store the JWT locally after the user logged-in once and use that for authentication.

I will write a number of articles next covering the non-standard parts of this whole stack. The topics covered will be the JWT authentication addition to FAB, the GeoAlchemy addition to FAB, the generation of the JSON Schema from WTForms and maybe a few other things. The standard parts won’t be covered in articles, just use the links above to find the docs of the respective packages. They all have excellent documentation.