Developing iterative, near-term ecological forecasts requires workflows that aggregate a heterogeneous set of hardware and software, resulting in a complex, distributed cyber-infrastructure of sensors, networks, compute/storage servers, software and applications. Here, we outline a workflow we developed for a water quality forecasting system in a drinking supply reservoir. At the reservoir, environmental sensors are deployed and connect wirelessly to the Internet via gateways. Data collected from sensors are then processed and aggregated to drive the execution of forecasting simulation models in high-performance cloud computing resources to support the execution of long-running ensembles of models. As the number, variety, and volume of data produced by sensors continue to increase, the need to configure and aggregate sensors dynamically, and dispatch and execute software near the sensors – at the “edge” of the network – becomes increasingly important. Furthermore, it is key to preserve integrity and privacy of data and prevent malicious attacks on such distributed cyber-infrastructure to provide trustworthy forecasts. This work addresses the question: what kind of distributed system architectures support the interconnection of sensor gateways, edge and cloud computing devices, while preserving privacy, integrity, and supporting existing applications and software for end-to-end ecological forecasting workflows?
Results/Conclusions
Our cyber-infrastructure linking sensor gateways and cloud computing resources via an overlay virtual network – IPOP (IP-over-P2P) – has been successfully deployed for water quality forecasting at the Falling Creek Reservoir in Roanoke, VA, USA. Sensor data are collected on a daily basis, transmitted securely via the IPOP overlay to servers at the University of Florida using the git protocol, where ensemble forecasting with the General Lake Model (GLM) is triggered to produce 16-day forecasts automatically every day. The system has been operational within the first year of an NSF-funded Smart and Connected Communities (SCC) project, and has now been generating forecasts since summer 2018. The presentation will overview the major components of the end-to-end workflow, and explore lessons learned and results from the operation of the system.