Low Latency Execution Guarantee Under Uncertainty in Serverless Platforms
2022 (English)In: Parallel and Distributed Computing, Applications and Technologies. PDCAT 2021 / [ed] Hong Shen; Yingpeng Sang; Yong Zhang; Nong Xiao; Hamid R. Arabnia; Geoffrey Fox; Ajay Gupta; Manu Malek, Springer, 2022, p. 324-335Conference paper, Published paper (Refereed)
Abstract [en]
Serverless computing recently emerged as a new run-time paradigm to disentangle the client from the burden of provisioning physical computing resources, leaving such difficulty on the service provider's side. However, an unsolved problem in such an environment is how to cope with the challenges of executing several co-running applications while fulfilling the requested Quality of Service (QoS) level requested by all application owners. In practice, developing an efficient mechanism to reach the requested performance level (such as p-99 latency and throughput) is limited to the awareness (resource availability, performance interference among consolidation workloads, etc.) of the controller about the dynamics of the underlying platforms. In this paper, we develop an adaptive feedback controller for coping with the buffer instability of serverless platforms when several collocated applications are run in a shared environment. The goal is to support a low-latency execution by managing the arrival event rate of each application when shared resource contention causes a significant throughput degradation among workloads with different priorities. The key component of the proposed architecture is a continues management of server-side internal buffers for each application to provide a low-latency feedback control mechanism based on the requested QoS level of each application (e.g., buffer information) and the worker nodes throughput. The empirical results confirm the response stability for high priority workloads when a dynamic condition is caused by low priority applications. We evaluate the performance of the proposed solution with respect to the response time and the QoS violation rate for high priority applications in a serverless platform with four worker nodes set up in our in-house virtualized cluster. We compare the proposed architecture against the default resource management policy in Apache OpenWhisk which is extensively used in commercial serverless platforms. The results show that our approach achieves a very low overhead (less than 0.7%) while it can improve the p-99 latency of high priority applications by 64%, on average, in the presence of dynamic high traffic conditions.
Place, publisher, year, edition, pages
Springer, 2022. p. 324-335
Series
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349 ; 13148
Keywords [en]
Dynamic controller of computer systems, Serverless computing, Virtualized platforms, Quality of Service (QoS)
National Category
Computer and Information Sciences
Research subject
Computer Science
Identifiers
URN: urn:nbn:se:kau:diva-89819DOI: 10.1007/978-3-030-96772-7_30ISI: 000787768500030Scopus ID: 2-s2.0-85127679307ISBN: 978-3-030-96772-7 (electronic)ISBN: 978-3-030-96771-0 (print)OAI: oai:DiVA.org:kau-89819DiVA, id: diva2:1659150
Conference
22nd International Conference on Parallel and Distributed Computing, Applications and Technologies (PDCAT 2021), DEC 17-19, 2021, Sun Yat Sen Univ, Guangzhou, China
2022-05-192022-05-192022-11-28Bibliographically approved