Abstract

Scheduling distributed machine learning pipelines in edge environments is a growing area of research as developers work to bring large, high-accuracy models to relatively low-powered devices. Edge environment dynamics, such as device availability and connectivity, make distributed scheduling a more challenging problem than in traditional cloud environments. Existing approaches usually require significant a priori knowledge of the environment and make assumptions about model availability, both of which are impractical in real edge deployments. We address this problem by proposing a simple and efficient reverse auction algorithm, where a device that wants to distribute a large machine learning workload requests bids from available resources in the environment to construct connected pipelines. We implement our reverse auction scheduling on an existing distributed machine learning pipeline framework …

Date

September 18, 2023

Authors

Connor Imes, David W King, John Paul Walters

Conference

2023 Eighth International Conference on Fog and Mobile Edge Computing (FMEC)

Pages

196-203

Publisher

IEEE