MYRIADS - PhD Defense of Mehdi BELKHIRIA : "Contributions to the Decentralization of Stream Processing : Algorithms and Architectures"

Stream Processing has become the \emph{de facto} standard way of supporting real-time data analytics. With the advent of new geographically dispersed computing platforms such as Edge and Fog computing paradigms, where distribution and locality is the norm, revising stream processing mechanisms towards decentralization appears necessary, as centralized management is no longer an option.Decentralization is the main axe around which this thesis revolves. In this dissertation, we introduce three contributions targeting the decentralizing of the stream processing. Firstly, we inject decentralization into scaling by presenting a new fully decentralized autoscaling algorithm for stream processing applications. Secondly, we give the foundations to design and build a software prototype of a decentralized stream processing engine. Throughout decentralized autoscaling decisions, nodes must always remain somewhat in synchronisation with each others. In the first two contributions we made an ad-hoc solution which is specific for our algorithm. However, in the third contribution, we revised the group mutual exclusion problem which is an algorithm based on classical primitives of distributed systems, so as to make it usable in our particular context of decentralized stream processing.