Resource Management of Data Stream Processing in Geo-Distributed Environments

The deployment of Data Stream Processing (DSP) frameworks in geo-distributed computing infrastructures can bridge the gap between Cloud and edge devices and reduce data transfers over long distances, which is a critical challenge for new emerging IoT applications where the data sources are located far from Cloud servers.

However, due to the heterogeneous network latencies experienced by the resources and unpredictable workload variations experienced by the applications, optimal resource usage in these environments in a way that meets certain QoS requirements when running DSP applications remains a challenge. 

In this thesis, we addressed this problem over three contributions. First, we proposed a performance model to capture DSP performance in geo-distributed environments. Second, we designed a model-based DSP auto-scaler to deal with non-stationary workloads of new IoT application scenarios. Finally, we developed a generic experimental Fog computing testbed customized to support various DSP experimentations.