Superando o esquecimento catastrófico em Redes Neurais Convolucionais para o aprendizado incremental de tarefas com Adaptadores Low-Rank

Abstract

With the advancement of artificial intelligence, particularly deep neural networks, several daily tasks are now being performed by these models with accuracy comparable to or even surpassing that of humans. However, such networks face a critical limitation: their inability to learn continuously without degrading performance on previously learned tasks—a pheno- menon known as catastrophic forgetting, first identified in 1989. Several strategies have been proposed to mitigate this problem, generally classified into four groups: (i) rehearsal-based; (ii) distance-based; (iii) sub-network-based; and (iv) dynamic networks. Among these, sub- network and dynamic approaches stand out, with the latter showing greater promise due to their ability to expand the model over time. In this thesis, we propose the use of ConvLoRA for continuous learning in convolutional neural networks, inspired by Low-Rank Adapters (LoRA) originally applied in natural language processing models. ConvLoRA introduces task-specific trainable adapters into every convolutional layer, enabling incremental learning without overwriting previous parameters and without storing data from past tasks, thereby adhering to data privacy constraints. When combined with LoRA in fully connected layers, the proposed approach results in only a 2.2% increase in total parameters per task. In scenarios focusing solely on convolutional layers, using Con- vLoRA exclusively leads to an 85.2% reduction in model growth compared to traditional methods such as SeNA-CNN. Experiments were conducted on established benchmark datasets for continual learning in image classification, including CIFAR-100 and CUB-200. Results demonstrate that the ConvLoRA + LoRA combination maintains or approaches the average accuracy of state-of- the-art methods while delivering significantly greater parameter efficiency. These findings confirm that the proposed method is effective, scalable, and suitable for real-world applica- tions that require continuous learning under memory and privacy constraints.