Institutional-Repository, University of Moratuwa.  

Topological pruner a neural network pruner using topological data analysis

Show simple item record

dc.contributor.advisor Fernando S
dc.contributor.advisor Amarasinghe A
dc.contributor.author Perera WMMJU
dc.date.accessioned 2022
dc.date.available 2022
dc.date.issued 2022
dc.identifier.citation Perera, W. M. M. J .U. (2022). Topological pruner a neural network pruner using topological data analysis [Master's theses, University of Moratuwa]. Institutional Repository University of Moratuwa. http://dl.lib.uom.lk/handle/123/21476
dc.identifier.uri http://dl.lib.uom.lk/handle/123/21476
dc.description.abstract Architectural damage due to neural network pruning has been a research problem. To recover the accuracy loss, after pruning, pruned neural network needed to be trained further for a certain time period to gain the accuracy back. If the damage done by the pruning process is severe, some layers can collapse and at worse, the entire model may become untrainable. Therefore, pruning process needs to be done carefully to prevent any significant damage to the neural network. Although some existing approaches have been used to overcome this issue by identifying the salience of a neuron with respect to the overall architecture, it is not computationally efficient. Further, the exiting solutions do not count the topological meaning of the neural network architecture during the pruning process. We believe that identifying the salience of neuron with respect to the layer is sufficient to avoid severe damages to the overall architecture. Topology, the champion of mathematical shapes, has been introduced to solve the aforesaid problem. We introduce ‘Topological Pruner’, a novel pruner that uses a genetic algorithm powered by a topological fitness function to identify removable neurons of each layer of a pre trained neural network. After pruning is done, the model is retrained so that the parameters of the remaining neuron can be readjusted to recover the model. As per to our knowledge this is the first ever attempt to use persistence homology, a topological tool for pruning. Number of parameters, FLOPs and recovery time of the new pruner is evaluated on CIFAR10 dataset on VGG-16 architecture against L1Filter Pruner, L2Filter Pruner and FPGM Pruner. Evaluation results show that the new pruner competes well with the existing pruners. We conclude that, topological data analysis can be used to explain the recoverability and mitigate damage cause by neural network pruning. en_US
dc.language.iso en en_US
dc.subject TOPOLOGICAL PRUNER en_US
dc.subject TOPOLOGICAL DATA ANALYSIS en_US
dc.subject TOPOLOGY BASED NEURAL NETWORK PRUNING en_US
dc.subject NEURAL NETWORK PRUNER en_US
dc.subject INFORMATION TECHNOLOGY -Dissertation en_US
dc.subject COMPUTATIONAL MATHEMATICS -Dissertation en_US
dc.subject ARTIFICIAL INTELLIGENCE -Dissertation en_US
dc.title Topological pruner a neural network pruner using topological data analysis en_US
dc.type Thesis-Abstract en_US
dc.identifier.faculty IT en_US
dc.identifier.degree MSc in Artificial Intelligence en_US
dc.identifier.department Department of Computational Mathematics en_US
dc.date.accept 2022
dc.identifier.accno TH5010 en_US


Files in this item

This item appears in the following Collection(s)

Show simple item record