VOIS library: pushing data science dashboards to the limits

<!--HTML-->The Joint Research Centre (JRC) of the European Commission has set up the JRC Big Data Analytics Platform (BDAP) as a multi-petabyte scale infrastructure to enable EC researchers to process and analyse big data in support to EU policy needs [1]. One of the service layers of the platform is based on Jupyter notebooks and the Python programming language to enable exploratory visualization and interactive analysis of big geospatial and non-geospatial datasets [2]. In this context, we have gained a lot of expertise in the design, development and deploy in production of many complexes Voilà dashboards [3] that enable JRC scientists and research groups to achieve a better communication of their scientific results and policy relevant insights to a non-technical audience as well as the public. Although Voilà Jupyter plugin [4] automatically transforms a notebook into a dashboard, creating an impactful dashboard is still a hard task. Beyond the classical communication issues (regarding, for instance, the story to tell, the message to convey, the graphic elements to use), from a pure web-development point of view, the designers and developers have to clearly define the single-page or multi-page style of the application they want, how to position elements on the page and how to intercept user inputs. One can tackle all these aspects using the standard tools available in the Jupyter world, as for instance the ipywidgets library [5] that provides a basic set of input widget and that is widely used in the data science community. Nevertheless, when a dashboard intends to create a strong impact, we found that it needs to exploit more advanced UI (User-Interface) frameworks. One open-source library recently gained a lot of interest for creating rich and engaging user experiences: ipyvuetify [6]. It is a widget library based on Vuetify/Vue JavaScript library [7] for making modern looking GUI’s in Jupyter notebooks and dashboards. It implements the Google material design philosophy [8] best known from the Android user interface and provides a large set of widgets with multiple variants, all highly customizable. The usage of ipyvuetify is not easy at all given that, for creating non-trivial components, one needs to dig into the details of the Vuetify widgets and the JavaScript API syntax. For this reason, we started to develop a library with the aim to simplify the complex tasks involved in the creation of modern dashboards and to provide easy-to-use and reusable components: the VOIS library [9]. This pure Python library provides many ready-to-use widgets and exposes an “app” class that can serve as the base for creating the dashboard structure. With few lines of code, the “app” can be customised using styles, colours, fonts, images, and all the graphic elements that contribute to its unicity. The VOIS library has many functions for the easy creation of complex geospatial visualisations (like bi-variate and tri-variate choropleth maps for vector data, or fast display of multi-terabytes raster datasets). It contains several custom-made SVG interactive charts that allow for modern user interaction and widgets for the display of hierarchical and tabular data. Among the functions of the VOIS library, we can cite: file uploads (to enable users to send their local input data to the web application); file downloads (to have a local copy of charts, tables, CSVs, reports in PDF or Word .docx format); management of parameters passed into the dashboard URL (in reading and writing mode); easy creation of dialog-boxes; support for responsive application development; etc. This presentation will illustrate the concepts that are at the base of the development of the VOIS library, as well as demonstrate some of the main dashboards that we created using the library for impactful policy support. It is interesting to note that the recently introduced VaaS service (Voilà as a service) enables BDAP users to autonomously create and deploy their dashboard in production, through an automated procedure based on GitLab repositories. This new service, together with the intensive training on the usage of the VOIS library, is contributing to the spreading of Voilà dashboards usage by many research groups in the JRC. The VOIS library will soon be available as a fully open source project at https://code.europa.eu/, the main repository of OSS for the European Commission, with the aim to create a community of users and, possibly, of interested contributors. These developments were partially funded by the H2020 project CS3MESH4EOSC, led by CERN and to which JRC participates providing support in the Earth Observation use case. The JRC Big Data Analytics Platform is a living demonstration of a complex ecosystem of cloud applications and services that allows data scientists’ navigation inside a multi-petabyte scale world. In particular, the exploratory visualization and interactive analysis tools and the Voilà/VaaS services are widely used to create GUI applications to communicate scientific research results to end-users ranging from policy makers to citizens. [1] P. Soille, A. Burger, D. De Marchi, P. Kempeneers, D. Rodriguez, V. Syrris, and V. Vasilev. “A Versatile Data-Intensive Computing Platform for Information Retrieval from Big Geospatial Data”. Future Generation Computer Systems 81.4 (Apr. 2018), pp. 30-40. https://doi.org/10.1016/j.future.2017.11.007. [2] D. De Marchi, A. Burger, P. Kempeneers, and P. Soille. “Interactive visualisation and analysis of geospatial data with Jupyter”. In: Proc. of the BiDS'17. 2017, pp. 71-74. https://zenodo.org/record/3248741 [3] https://jeodpp.jrc.ec.europa.eu/bdap/voila/ [4] https://github.com/voila-dashboards/voila [5] https://ipywidgets.readthedocs.io/en/latest/ [6] https://ipyvuetify.readthedocs.io/en/latest/ [7] https://vuetifyjs.com/en/ [8] https://m2.material.io/ [9] https://jeodpp.jrc.ec.europa.eu/services/shared/vois/1_intro.html