Abstract

Stream processing is an increasingly popular model for online data processing that can be partitioned into streams of elements. It is commonly used in real-time data analytics services, such as processing Twitter tweets and Internet of Things (IoT) device feeds. Current stream processing frameworks boast high throughput and low average latency. However, users of these frameworks may desire lower tail latencies and better real-time performance for their applications. In practice, there are a number of errors that can affect the performance of stream processing applications, such as garbage collection and resource contention. For some applications, these errors may cause unacceptable violations of real-time constraints. In this paper we propose applying redundancy in the data processing pipeline to increase the resiliency of stream processing applications to timing errors. This results in better real-time …

Date

December 17, 2018

Authors

Geoffrey Phi Tran, John Paul Walters, Stephen Crago

Conference

2018 IEEE/ACM 11th International Conference on Utility and Cloud Computing (UCC)

Pages

194-203

Publisher

IEEE