Hadjer BENMEZIANE's thesis defense: Automatic Optimization of Deep Learning Applications on Edges Material Platforms

Composition of the proposed jury

Mr SMAIL NIAR, Université Polytechnique Hauts de France, Thesis Director
. Mr Muhammad SHAFIQUE, New-York Univ, Abu-Dhabi, Rapporteur
M. Gilles SASSATELI, Université Montpellier, France, Rapporteur
Mrs Liliana CUCU-GROSJEAN, INRIA ; Rocquencourt, France, Examiner
Mrs Kaoutar EL MAGHRA, University of Montpellier, France, Rapporteur
Mrs Kaoutar EL MAGHRA, University of Montpellier, France, Rapporteur Mme Kaoutar EL MAGHRAOUI, IBM Research AI, NY USA and Adjunct Professor, Columbia University, USA, Co-Dissertation Supervisor
M. Hamza OUARNOUI, IBM Research AI, NY USA Mr. Hamza OUARNOUGHI, LAMIH, UPHF/INSA, Thesis co-supervisor
. Mr. Brett MEYER, Mc Gill University, Invited

Keywords

Optimization,Deep Learning,Material constraints,Edge systems,

Abstract

Inference models based on deep neural networks (eng, Deep Neural Networks (DNN)) are widely used in many edge platforms for several reasons. Firstly, DNNs have demonstrated outstanding performance in a variety of complex tasks such as image recognition, natural language processing and speech synthesis. Their ability to extract meaningful features from large datasets enables them to achieve high levels of accuracy and predictive power, making them indispensable for a wide range of applications. Secondly, deploying these models directly on edge platforms offers several advantages. Running the inference process locally on edge devices reduces reliance on cloud-based computing, thereby reducing network latency and ensuring real-time responsiveness. However, edge platforms often operate in resource-constrained environments, characterized by limited computing power, energy constraints and intermittent connectivity. DNN models are unsuitable for such platforms. This has encouraged research into the automatic design of neural architectures adapted to such devices. This method is called Hardware-aware Neural Architecture Search (HW-NAS). HW-NAS is the cornerstone of this thesis. HW-NAS can provide models that are both efficient and accurate. This thesis aims to accelerate and generalize the applicability of HW-NAS to multiple platforms and multiple tasks. In particular, this thesis proposes innovative solutions for rapidly estimating the efficiency of a DNN deployed on a target hardware platform. Our proposed HW-NAS approach encompasses multi-objective optimization techniques, significantly accelerating the search process in both supernetwork- and cell-based search spaces. In the multi-objective context of HW-NAS, conflicting objectives, such as task-specific performance (e.g. accuracy) and hardware efficiency (e.g. latency and power consumption), need to be optimized simultaneously. To meet this challenge, we define a new Pareto rank objective. By incorporating multiple objectives and Pareto optimization principles, our approach enables the exploration of trade-offs between task-specific performance and hardware efficiency. We also examine the human bias induced by current search spaces and propose a non-restrictive search space for finding new operators suitable for a target hardware platform. These methods have been validated on image classification repositories. In the second part of the thesis, we demonstrate the usefulness of our methods in real-life scenarios. Firstly, how to apply HW-NAS to new hardware architectures, notably in-memory analog computing hardware, or in-memory analog devices. We therefore propose an HW-NAS dedicated to these platforms, and deduce the characteristics that differentiate a neural network deployed on these platforms from one deployed on conventional platforms. Finally, we build a searchable reference of neural architectures for medical imaging that includes architectures for 11 tasks, including tumor detection, liver segmentation and hippocampal volume estimation. Using this reference, we propose a new HW-NAS that not only includes accuracy and latency as goals, but also seeks a generalizable architecture that can be refined for unknown medical tasks.