Overview and Portfolio of the eStation

Overview

Introduction

Due to the advent of free and open geospatial technologies and data, in the last decade, many software tools and cloud processing services of Earth Observation data have been developed and made available. Although these services and tools make geospatial analysis much easier to do, a system that offers automated data acquisition and processing was still missing. The eStation is such a web-processing system for automating the acquisition and processing of key environmental indicators derived from remotely sensed data.

In addition to the acquisition and processing services, the eStation offers a highly customized web application for analysis and visualization of the key environmental indicators. With as main features, the creation and saving of user workspaces that can contain multiple floating map windows and various types of time series graphs, linkable in time and space, and the creation of reusable user map and graph templates. Future developments will make all processing steps easily configurable, allowing the user to modify the generated environmental indicators and to implement new ones for computing ad-hoc thematic products.

Its open source configuration encourages the community of end-users to customize the tools to their needs. It is worthwhile to stress that capacity building activities are an essential component of the project supporting the distribution and use of the eStation in Africa, continued in the transition to the GMES&Africa phase 2 support program (2022-2025), that builds on GMES&Africa phase 1 and MESA achievements and aims at extending the Copernicus services to the African Institutions.

Text version

Text version

The eStation has been distributed to more than 130 Institutions in all 48 sub-Saharan African countries involved in the MESA project (Monitoring of Environment and Security in Africa, 2013-2017). Implemented by the African Union Commission in partnership with the African Caribbean Pacific (ACP) Secretariat and the five African Regional Economic Groups, that are supported by the European Development Fund.

Text version

System concept

The Station is meant to be a processing server for EO and Climate datasets, rather than a stand-alone GIS platform (like QGIS or similar solutions). The functioning of the application is organized around a number of Services, as displayed in figure below, namely:

• Data Acquisition: systematically acquire EO and Climate data from various sources, through FTP, HTTP protocols, APIs and locally available file systems, as a EUMETCast Receiving Station. Therefore, there are several ‘get’ services that can be configured and controlled independently:
  • Get EUMETCast
  • Get Internet (data from remote FTP and HTTP servers)
  • Get Data Store
• Data Ingestion: the ‘Ingest’ Service converts the files from the format under which they are acquired in (the so called ‘native’ format) into GeoTiff or NetCDF, also called ‘pivot’ formats. Optionally, this operation includes geographical re-projection and clipping to a specific region of interest;
• Processing Service: to derive from the input data additional products, like long term statistics, anomalies, and other added-value indicators;
• System Service: to run a number of ‘house-keeping’ and background tasks, including managing the local file system and cleaning temporary directories.

System structure and configurations 

The Station’s services and the processing components are written in python; the visualisation and GUI components are written in JavaScript (using Ext JS library). A PostgreSQL database stores the EO and Climate products definition, settings for accessing server storing data through the ‘get’ services and all relevant user configurations.

Several software and libraries are required, including the GDAL library and its python wrapper for geo-processing, Mapserver for maps rendering, and a series of non-standard python modules, such as Ruffus, a computation pipeline library, for the processing engine.

All the components of the eStation, such as database, GUI, IMPACT tool and  Mapserver, are containerized and deployed in the server using Docker.

Application overview 

A web interface allows controlling the Station. It folds different controls in separate pages, namely:

• Dashboard: it presents the overall status of the Station, and offers the control of all the enabled services.
• Portfolio: to configure the list of datasets in the Station, mainly to activate products for acquisition and processing.
• Acquisition: to view and control the status of the services for retrieving and ingesting the EO and Climate data; it represents and gives access to the ‘Get’ (internet and datastore) and ‘Ingestion’ services, represented in Figure 1: Overview of the Services running on the Station.
• Processing: to start and stop the processing of new products, i.e. to control the ‘Processing’ Service.
• Data Management: it lists all available data sets, both those acquired and processed by the eStation, showing the completeness of time series (and thus listing missing data).
• Analysis: to perform the data analysis and generate images for bulletins/reports.
• Fitness for Purposes: it offers the capability to perform comparison among different products. The comparison can be conducted in terms of statistical analysis and/or using global (or regional) maps.
• IMPACT toolbox: it offers a combination of remote sensing, photo interpretation and processing technologies in a GIS environment, allowing non specialist users to easily accomplish all necessary pre-processing steps while giving a fast and user-friendly environment for visual editing and map validation.
• Jupyter Notebooks: it proposes a working environment of python (ipykernel notebook) where users can code their process to create the indicators from the existing data and visualize them.
• System Settings: to control the application settings (including the region, log level, and working directories paths), and to generate system reports.
• Help: to read system help files: allows downloading the pdf files and access reference web sites.

Portfolio