Hubert wins the lottery PARP(-P) pruning the HuBERT model for downstream tasks

van Heerwaarden, Floor M (2023) Hubert wins the lottery PARP(-P) pruning the HuBERT model for downstream tasks. Master thesis, Voice Technology (VT).

PDF MA_5189438_FM_vanHeerwaarden.pdf Restricted to Repository staff only Download (1MB)

Abstract

Automatic Speech Recognition (ASR) models have been growing in parameter size to a point where one needs a GPU of formidable size to be able to use it. The Lottery Ticket Hypothesis has been suggested as a way to prune models to extremely low sparsity without losing, or even gaining, performance. To address the challenge of ASR being inaccessible due to large and computationally complex models, a Lottery Ticket Hypothesis based method Prune-Adjust re-Prune (PARP) was used to prune a HuBERT based model that is used used for tasks such as phoneme recognition and ASR. The experiments have demonstrated that the HuBERT model can be pruned by up to 70% sparsity without sacrificing performance. This significant reduction in model size opens up possibilities for improving computational efficiency and resource utilization, opening up paths for inclusive and accessible ASR. Experiments on inference time and memory usage, however, show that the PARP method is not sufficient in reducing the computational complexity of a model, most likely due to the use of unstructured pruning.

Item Type:	Thesis (Master)
Name supervisor:	Coler, M.L. and Hopwood, F.J.
Date Deposited:	12 Sep 2023 11:10
Last Modified:	12 Sep 2023 11:10
URI:	https://campus-fryslan.studenttheses.ub.rug.nl/id/eprint/372

Actions (login required)

View Item