• Home
  • Neural Networks
    • List of Articles Neural Networks

      • Open Access Article

        1 - Identify and assess the relative importance of knowledge management strategies by using ANN (Case study Knowledge base Software Companies)
        Saeedeh khabbazkar Mohsen Shafiei Nikabadi مائده  دهقان
        Abstract: Knowledge management is an important resource for any organization. Organizations to implement knowledge management strategies, improve innovation in processes, activities, products and services. The aim of this study is to identify the key strategies of knowl More
        Abstract: Knowledge management is an important resource for any organization. Organizations to implement knowledge management strategies, improve innovation in processes, activities, products and services. The aim of this study is to identify the key strategies of knowledge management by ANN .The innovative aspect of the research is, the use of artificial neural networks (ANN) to rank the strategies of knowledge management. The population consists of the all employees of the   knowledge based software companies in Tehran, that the total questionnaires were distributed, only 123 were usable. This study is practical from the objective aspect, and descriptive-survey from data collection method aspect. Data from the surveys and questionnaires obtained and then by using the ANN techniques h as been investigated the research objectives.  Results and ANN outputs indicated that sequencely, explicit knowledge startegy is the most important criteria of Knowledge management strategy and tacit khowledge, internal and external strategy are the next priorities  knowledge based  software companies  are located in Tehran.  Manuscript profile
      • Open Access Article

        2 - Comparing A Hybridization of Fuzzy Inference System and Particle Swarm Optimization Algorithm with Deep Learning to Predict Stock Prices
        Majid Abdolrazzagh-Nezhad mahdi kherad
        Predicting stock prices by data analysts have created a great business opportunity for a wide range of investors in the stock markets. But the fact is difficulte, because there are many affective economic factors in the stock markets that they are too dynamic and compl More
        Predicting stock prices by data analysts have created a great business opportunity for a wide range of investors in the stock markets. But the fact is difficulte, because there are many affective economic factors in the stock markets that they are too dynamic and complex. In this paper, two models are designed and implemented to identify the complex relationship between 10 economic factors on the stock prices of companies operating in the Tehran stock market. First, a Mamdani Fuzzy Inference System (MFIS) is designed that the fuzzy rules set of its inference engine is found by the Particle Swarm Optimization Algorithm (PSO). Then a Deep Learning model consisting of 26 neurons is designed wiht 5 hidden layers. The designed models are implemented to predict the stock prices of nine companies operating on the Tehran Stock Exchange. The experimental results show that the designed deep learning model can obtain better results than the hybridization of MFIS-PSO, the neural network and SVM, although the interperative ability of the obtained patterns, more consistent behavior with much less variance, as well as higher convergence speed than other models can be mentioned as significant competitive advantages of the MFIS-PSO model Manuscript profile
      • Open Access Article

        3 - Improvement in Accuracy and Speed of Image Semantic Segmentation via Convolution Neural Network Encoder-Decoder
        Hanieh Zamanian Hassan Farsi Sajad Mohammadzadeh
        Recent researches on pixel-wise semantic segmentation use deep neural networks to improve accuracy and speed of these networks in order to increase the efficiency in practical applications such as automatic driving. These approaches have used deep architecture to predic More
        Recent researches on pixel-wise semantic segmentation use deep neural networks to improve accuracy and speed of these networks in order to increase the efficiency in practical applications such as automatic driving. These approaches have used deep architecture to predict pixel tags, but the obtained results seem to be undesirable. The reason for these unacceptable results is mainly due to the existence of max pooling operators, which reduces the resolution of the feature maps. In this paper, we present a convolutional neural network composed of encoder-decoder segments based on successful SegNet network. The encoder section has a depth of 2, which in the first part has 5 convolutional layers, in which each layer has 64 filters with dimensions of 3×3. In the decoding section, the dimensions of the decoding filters are adjusted according to the convolutions used at each step of the encoding. So, at each step, 64 filters with the size of 3×3 are used for coding where the weights of these filters are adjusted by network training and adapted to the educational data. Due to having the low depth of 2, and the low number of parameters in proposed network, the speed and the accuracy improve compared to the popular networks such as SegNet and DeepLab. For the CamVid dataset, after a total of 60,000 iterations, we obtain the 91% for global accuracy, which indicates improvements in the efficiency of proposed method. Manuscript profile
      • Open Access Article

        4 - DeepSumm: A Novel Deep Learning-Based Multi-Lingual Multi-Documents Summarization System
        Shima Mehrabi Seyed Abolghassem Mirroshandel Hamidreza  Ahmadifar
        With the increasing amount of accessible textual information via the internet, it seems necessary to have a summarization system that can generate a summary of information for user demands. Since a long time ago, summarization has been considered by natural language pro More
        With the increasing amount of accessible textual information via the internet, it seems necessary to have a summarization system that can generate a summary of information for user demands. Since a long time ago, summarization has been considered by natural language processing researchers. Today, with improvement in processing power and the development of computational tools, efforts to improve the performance of the summarization system is continued, especially with utilizing more powerful learning algorithms such as deep learning method. In this paper, a novel multi-lingual multi-document summarization system is proposed that works based on deep learning techniques, and it is amongst the first Persian summarization system by use of deep learning. The proposed system ranks the sentences based on some predefined features and by using a deep artificial neural network. A comprehensive study about the effect of different features was also done to achieve the best possible features combination. The performance of the proposed system is evaluated on the standard baseline datasets in Persian and English. The result of evaluations demonstrates the effectiveness and success of the proposed summarization system in both languages. It can be said that the proposed method has achieve the state of the art performance in Persian and English. Manuscript profile
      • Open Access Article

        5 - Evaluation of Pattern Recognition Techniques in Response to Cardiac Resynchronization Therapy (CRT)
        Mohammad Nejadeh Peyman Bayat Jalal Kheirkhah Hassan Moladoust
        Cardiac resynchronization therapy (CRT) improves cardiac function in patients with heart failure (HF), and the result of this treatment is decrease in death rate and improving quality of life for patients. This research is aimed at predicting CRT response for the progno More
        Cardiac resynchronization therapy (CRT) improves cardiac function in patients with heart failure (HF), and the result of this treatment is decrease in death rate and improving quality of life for patients. This research is aimed at predicting CRT response for the prognosis of patients with heart failure under CRT. According to international instructions, in the case of approval of QRS prolongation and decrease in ejection fraction (EF), the patient is recognized as a candidate of implanting recognition device. However, regarding many intervening and effective factors, decision making can be done based on more variables. Computer-based decision-making systems especially machine learning (ML) are considered as a promising method regarding their significant background in medical prediction. Collective intelligence approaches such as particles swarm optimization (PSO) algorithm are used for determining the priorities of medical decision-making variables. This investigation was done on 209 patients and the data was collected over 12 months. In HESHMAT CRT center, 17.7% of patients did not respond to treatment. Recognizing the dominant parameters through combining machine recognition and physician’s viewpoint, and introducing back-propagation of error neural network algorithm in order to decrease classification error are the most important achievements of this research. In this research, an analytical set of individual, clinical, and laboratory variables, echocardiography, and electrocardiography (ECG) are proposed with patients’ response to CRT. Prediction of the response after CRT becomes possible by the support of a set of tools, algorithms, and variables. Manuscript profile
      • Open Access Article

        6 - Deep Learning Approach for Cardiac MRI Images
        Afshin Sandooghdar Farzin Yaghmaee
        Deep Learning (DL) is the most widely used image-analysis process, especially in medical image processing. Though DL has entered image processing to solve Machine Learning (ML) problems, identifying the most suitable model based on evaluation of the epochs is still an o More
        Deep Learning (DL) is the most widely used image-analysis process, especially in medical image processing. Though DL has entered image processing to solve Machine Learning (ML) problems, identifying the most suitable model based on evaluation of the epochs is still an open question for scholars in the field. There are so many types of function approximators like Decision Tree, Gaussian Processes and Deep Learning, used in multi-layered Neural Networks (NNs), which should be evaluated to determine their effectiveness. Therefore, this study aimed to assess an approach based on DL techniques for modern medical imaging methods according to Magnetic Resonance Imaging (MRI) segmentation. To do so, an experiment with a random sampling approach was conducted. One hundred patient cases were used in this study for training, validation, and testing. The method used in this study was based on full automatic processing of segmentation and disease classification based on MRI images. U-Net structure was used for the segmentation process, with the use of cardiac Right Ventricular Cavity (RVC), Left Ventricular Cavity (LVC), Left Ventricular Myocardium (LVM), and information extracted from the segmentation step. With train and using random forest classifier, and Multilayer Perceptron (MLP), the task of predicting the pathologic target class was conducted. Segmentation extracted information was in the form of comprehensive features handcrafted to reflect demonstrative clinical strategies. Our study suggests 92% test accuracy for cardiac MRI image segmentation and classification. As for the MLP ensemble, and for the random forest, test accuracy was equal to 91% and 90%, respectively. This study has implications for scholars in the field of medical image processing. Manuscript profile
      • Open Access Article

        7 - Recognition of Attention Deficit/Hyperactivity Disorder (ADHD) Based on Electroencephalographic Signals Using Convolutional Neural Networks (CNNs)
        Sara Motamed Elham Askari
        Impulsive / hyperactive disorder is a neuro-developmental disorder that usually occurs in childhood, and in most cases parents find that the child is more active than usual and have problems such as lack of attention and concentration control. Because this problem might More
        Impulsive / hyperactive disorder is a neuro-developmental disorder that usually occurs in childhood, and in most cases parents find that the child is more active than usual and have problems such as lack of attention and concentration control. Because this problem might interfere with your own learning, work, and communication with others, it could be controlled by early diagnosis and treatment. Because the automatic recognition and classification of electroencephalography (EEG) signals is challenging due to the large variation in time features and signal frequency, the present study attempts to provide an efficient method for diagnosing hyperactive patients. The proposed method is that first, the recorded brain signals of hyperactive subjects are read from the input and in order to the signals to be converted from time range to frequency range, Fast Fourier Transform (FFT) is used. Also, to select an effective feature to check hyperactive subjects from healthy ones, the peak frequency (PF) is applied. Then, to select the features, principal component analysis and without principal component analysis will be used. In the final step, convolutional neural networks (CNNs) will be utilized to calculate the recognition rate of individuals with hyperactivity. For model efficiency, this model is compared to the models of K- nearest neighbors (KNN), and multilayer perceptron (MLP). The results show that the best method is to use feature selection by principal component analysis and classification of CNNs and the recognition rate of individuals with ADHD from healthy ones is equal to 91%. Manuscript profile
      • Open Access Article

        8 - Determining of Classifiers Behavior Using Hidden Markov Model Based Decision Template
        H. Sadoghi Yazdi
        Studying of classifier behavior is interested from viewpoint of error checking and presentation of suitable solution for decreasing error rates and decreasing performance. Weakness operation of recognition system is because of small number of training samples, noisy sam More
        Studying of classifier behavior is interested from viewpoint of error checking and presentation of suitable solution for decreasing error rates and decreasing performance. Weakness operation of recognition system is because of small number of training samples, noisy samples, unsuitable extracted features, method of determining of system response. Presentation of suitable model for behavior or response of recognition system, we can improve operation of recognition system. In this paper, a new hidden Markov model based decision template is generated for modeling of neurons behavior in neural network. In existing methods, relation of neurons and interaction between them is not studied whereas; response of neural network includes response value of all neurons. So, relations of neurons are modeled using new hidden Markov decision templates. This method is used into three applications include recognition of Farsi number images, normal traffic in internet network, and recognition of types of vehicles. Increasing performance of neural network indicates to superiority of the proposed system. Manuscript profile
      • Open Access Article

        9 - Numeric Polarity Detection based on Employing Recursive Deep Neural Networks and Supervised Learning on Persian Reviews of E-Commerce Users in Opinion Mining Domain
        Sepideh Jamshidinejad Fatemeh Ahmadi-Abkenari Peiman Bayat
        Opinion mining as a sub domain of data mining is highly dependent on natural language processing filed. Due to the emerging role of e-commerce, opinion mining becomes one of the interesting fields of study in information retrieval scope. This domain focuses on various s More
        Opinion mining as a sub domain of data mining is highly dependent on natural language processing filed. Due to the emerging role of e-commerce, opinion mining becomes one of the interesting fields of study in information retrieval scope. This domain focuses on various sub areas such as polarity detection, aspect elicitation and spam opinion detection. Although there is an internal dependency among these sub sets, but designing a thorough framework including all of the mentioned areas is a highly demanding and challenging task. Most of the literatures in this area have been conducted on English language and focused on one orbit with a binary outcome for polarity detection. Although the employment of supervised learning approaches is among the common utilizations in this area, but the application of deep neural networks has been concentrated with various objectives in recent years so far. Since the absence of a trustworthy and a complete framework with special focuses on each impacting sub domains is highly observed in opinion mining, hence this paper concentrates on this matter. So, through the usage of opinion mining and natural language processing approaches on Persian language, the deep neural network-based framework called RSAD that was previously suggested and developed by the authors of this paper is optimized here to include the binary and numeric polarity detection output of sentences on aspect level. Our evaluation on RSAD performance in comparison with other approaches proves its robustness. Manuscript profile
      • Open Access Article

        10 - An Intelligent Vision System for Automatic Forest Fire Surveillance
        Mohammad Sadegh  Kayhanpanah Behrooz Koohestani
        Fighting forest fires to avoid their potential dangers as well as protect natural resources is a challenge for researchers. The goal of this research is to identify the features of fire and smoke from the unmanned aerial vehicle (UAV) visual images for classification, o More
        Fighting forest fires to avoid their potential dangers as well as protect natural resources is a challenge for researchers. The goal of this research is to identify the features of fire and smoke from the unmanned aerial vehicle (UAV) visual images for classification, object detection, and image segmentation. Because forests are highly complex and nonstructured environments, the use of the vision system is still having problems such as the analogues of flame characteristics to sunlight, plants, and animals, or the smoke blocking the images of the fire, which causes false alarms. The proposed method in this research is the use of convolutional neural networks (CNNs) as a deep learning method that can automatically extract or generate features in different layers. First, we collect data and increase them according to data augmentation methods, and then, the use of a 12-layer network for classification as well as transfer learning method for segmentation of images is proposed. The results show that the data augmentation method used due to resizing and processing the input images to the network to prevent the drastic reduction of the features in the original images and also the CNNs used can extract the fire and smoke features in the images well and finally detect and localize them. Manuscript profile
      • Open Access Article

        11 - A Step towards All-Optical Deep Neural Networks: Utilizing Nonlinear Optical Element
        Aida Ebrahimi Dehghan Pour S. K.
        In recent years, optical neural networks have received a lot of attention due to their high speed and low power consumption. However, these networks still have many limitations. One of these limitations is implementing their nonlinear layer. In this paper, the implement More
        In recent years, optical neural networks have received a lot of attention due to their high speed and low power consumption. However, these networks still have many limitations. One of these limitations is implementing their nonlinear layer. In this paper, the implementation of nonlinear unit for an optical convolutional neural network is investigated, so that using this nonlinear unit, we can realize an all-optical convolutional neural network with the same accuracy as the electrical networks, while providing higher speed and lower power consumption. In this regard, first of all, different methods of implementing optical nonlinear unit are reviewed. Then, the impact of utilizing saturable absorber, as the nonlinear unit in different layers of CNN, on the network’s accuracy is investigated, and finally, a new and simple method is proposed to preserve the accuracy of the optical neural networks utilizing saturable absorber as the nonlinear activating function. Manuscript profile
      • Open Access Article

        12 - Compilation of artificial neural networks and the thinned Fault likelihood auto-tracking algorithm, for identification, interpretation and extraction of faults
        Alireza Ghazanfari Hoseyn Mohammadrezaei Hamidreza Ansari
        Fault identification and investigating their evolution is of special importance in the exploration and development of hydrocarbon resources. Success in exploration and development of hydrocarbon fields, need to recognition of petroleum systems and in this regard one of More
        Fault identification and investigating their evolution is of special importance in the exploration and development of hydrocarbon resources. Success in exploration and development of hydrocarbon fields, need to recognition of petroleum systems and in this regard one of the most important topics is identifying faults and their extension condition as a main fluid migration path, specially in deeper zones. Faults and fractures have crucial role in making high permeable and porous segments and cut reservoir and cap rock in the fluid migration path. In addition, for maximizing the production of hydrocarbon from reservoirs and also for reducing the risk of drilling, it is necessary to gain information about geometry and nature of faults of reservoirs. In this paper, the purpose is investigating the performance of combination of neural networks and Fault Likelihood auto-tracking algorithm for identification and interpretation of faults in seismic data. At first using the Dip-steering feature of software, the early filter for accurate identification of dip of structures in the data, have been designed and applied. Then with designing and applying the appropriate filters, the seismic data have been improved. After that proper seismic attributes for fault identification have been calculated from seismic data. With picking fault and non-fault points from data, a supervised neural network using the selected attributes was formed and after training the network, the appropriate output achieved. Then the output of neural network has been used as a input for Thinned Fault Likelihood auto-tracking algorithm. The output of this part contains a volume of tracked faults. Finally using sub-tools of TFL and optimal setting of parameters, 3D fault planes has been interpreted and extracted. Manuscript profile
      • Open Access Article

        13 - Convolutional Neural Networks for Medical Image Segmentation and Classification: A Review
        Jenifer S Carmel Mary Belinda M J
        Medical imaging refers to the process of obtaining images of internal organs for therapeutic purposes such as discovering or studying diseases. The primary objective of medical image analysis is to improve the efficacy of clinical research and treatment options. Deep le More
        Medical imaging refers to the process of obtaining images of internal organs for therapeutic purposes such as discovering or studying diseases. The primary objective of medical image analysis is to improve the efficacy of clinical research and treatment options. Deep learning has revamped medical image analysis, yielding excellent results in image processing tasks such as registration, segmentation, feature extraction, and classification. The prime motivations for this are the availability of computational resources and the resurgence of deep Convolutional Neural Networks. Deep learning techniques are good at observing hidden patterns in images and supporting clinicians in achieving diagnostic perfection. It has proven to be the most effective method for organ segmentation, cancer detection, disease categorization, and computer-assisted diagnosis. Many deep learning approaches have been published to analyze medical images for various diagnostic purposes. In this paper, we review the works exploiting current state-of-the-art deep learning approaches in medical image processing. We begin the survey by providing a synopsis of research works in medical imaging based on convolutional neural networks. Second, we discuss popular pre-trained models and General Adversarial Networks that aid in improving convolutional networks’ performance. Finally, to ease direct evaluation, we compile the performance metrics of deep learning models focusing on covid-19 detection and child bone age prediction. Manuscript profile
      • Open Access Article

        14 - Multi-level ternary quantization for improving sparsity and computation in embedded deep neural networks
        Hosna Manavi Mofrad Seyed Ali ansarmohammadi Mostafa Salehi
        Deep neural networks (DNNs) have achieved great interest due to their success in various applications. However, the computation complexity and memory size are considered to be the main obstacles for implementing such models on embedded devices with limited memory and co More
        Deep neural networks (DNNs) have achieved great interest due to their success in various applications. However, the computation complexity and memory size are considered to be the main obstacles for implementing such models on embedded devices with limited memory and computational resources. Network compression techniques can overcome these challenges. Quantization and pruning methods are the most important compression techniques among them. One of the famous quantization methods in DNNs is the multi-level binary quantization, which not only exploits simple bit-wise logical operations, but also reduces the accuracy gap between binary neural networks and full precision DNNs. Since, multi-level binary can’t represent the zero value, this quantization does’nt take advantage of sparsity. On the other hand, it has been shown that DNNs are sparse, and by pruning the parameters of the DNNs, the amount of data storage in memory is reduced while computation speedup is also achieved. In this paper, we propose a pruning and quantization-aware training method for multi-level ternary quantization that takes advantage of both multi-level quantization and data sparsity. In addition to increasing the accuracy of the network compared to the binary multi-level networks, it gives the network the ability to be sparse. To save memory size and computation complexity, we increase the sparsity in the quantized network by pruning until the accuracy loss is negligible. The results show that the potential speedup of computation for our model at the bit and word-level sparsity can be increased by 15x and 45x compared to the basic multi-level binary networks. Manuscript profile
      • Open Access Article

        15 - Proposing an FCM-MCOA Clustering Approach Stacked with Convolutional Neural Networks for Analysis of Customers in Insurance Company
        Motahareh Ghavidel meisam Yadollahzadeh tabari Mehdi Golsorkhtabaramiri
        To create a customer-based marketing strategy, it is necessary to perform a proper analysis of customer data so that customers can be separated from each other or predict their future behavior. The datasets related to customers in any business usually are high-dimension More
        To create a customer-based marketing strategy, it is necessary to perform a proper analysis of customer data so that customers can be separated from each other or predict their future behavior. The datasets related to customers in any business usually are high-dimensional with too many instances and include both supervised and unsupervised ones. For this reason, companies today are trying to satisfy their customers as much as possible. This issue requires careful consideration of customers from several aspects. Data mining algorithms are one of the practical methods in businesses to find the required knowledge from customer’s both demographic and behavioral. This paper presents a hybrid clustering algorithm using the Fuzzy C-Means (FCM) method and the Modified Cuckoo Optimization Algorithm (MCOA). Since customer data analysis has a key role in ensuring a company's profitability, The Insurance Company (TIC) dataset is utilized for the experiments and performance evaluation. We compare the convergence of the proposed FCM-MCOA approach with some conventional optimization methods, such as Genetic Algorithm (GA) and Invasive Weed Optimization (IWO). Moreover, we suggest a customer classifier using the Convolutional Neural Networks (CNNs). Simulation results reveal that the FCM-MCOA converges faster than conventional clustering methods. In addition, the results indicate that the accuracy of the CNN-based classifier is more than 98%. CNN-based classifier converges after some couples of iterations, which shows a fast convergence in comparison with the conventional classifiers, such as Decision Tree (DT), Support Vector Machine (SVM), K-Nearest Neighborhood (KNN), and Naive Bayes (NB) classifiers. Manuscript profile
      • Open Access Article

        16 - Improvement of intrusion detection system on Industrial Internet of Things based on deep learning using metaheuristic algorithms
        mohammadreza zeraatkarmoghaddam majid ghayori
        Due to the increasing use of industrial Internet of Things (IIoT) systems, one of the most widely used security mechanisms is intrusion detection system (IDS) in the IIoT. In these systems, deep learning techniques are increasingly used to detect attacks, anomalies or i More
        Due to the increasing use of industrial Internet of Things (IIoT) systems, one of the most widely used security mechanisms is intrusion detection system (IDS) in the IIoT. In these systems, deep learning techniques are increasingly used to detect attacks, anomalies or intrusions. In deep learning, the most important challenge for training neural networks is determining the hyperparameters in these networks. To overcome this challenge, we have presented a hybrid approach to automate hyperparameter tuning in deep learning architecture by eliminating the human factor. In this article, an IDS in IIoT based on convolutional neural networks (CNN) and recurrent neural network based on short-term memory (LSTM) using metaheuristic algorithms of particle swarm optimization (PSO) and Whale (WOA) is used. This system uses a hybrid method based on neural networks and metaheuristic algorithms to improve neural network performance and increase detection rate and reduce neural network training time. In our method, considering the PSO-WOA algorithm, the hyperparameters of the neural network are determined automatically without the intervention of human agent. In this paper, UNSW-NB15 dataset is used for training and testing. In this research, the PSO-WOA algorithm has use optimized the hyperparameters of the neural network by limiting the search space, and the CNN-LSTM neural network has been trained with this the determined hyperparameters. The results of the implementation indicate that in addition to automating the determination of hyperparameters of the neural network, the detection rate of are method improve 98.5, which is a good improvement compared to other methods. Manuscript profile
      • Open Access Article

        17 - Multi-Level Ternary Quantization for Improving Sparsity and Computation in Embedded Deep Neural Networks
        Hosna Manavi Mofrad ali ansarmohammadi Mostafa Salehi
        Deep neural networks (DNNs) have achieved great interest due to their success in various applications. However, the computation complexity and memory size are considered to be the main obstacles for implementing such models on embedded devices with limited memory and co More
        Deep neural networks (DNNs) have achieved great interest due to their success in various applications. However, the computation complexity and memory size are considered to be the main obstacles for implementing such models on embedded devices with limited memory and computational resources. Network compression techniques can overcome these challenges. Quantization and pruning methods are the most important compression techniques among them. One of the famous quantization methods in DNNs is the multi-level binary quantization, which not only exploits simple bit-wise logical operations, but also reduces the accuracy gap between binary neural networks and full precision DNNs. Since, multi-level binary can’t represent the zero value, this quantization does not take advantage of sparsity. On the other hand, it has been shown that DNNs are sparse, and by pruning the parameters of the DNNs, the amount of data storage in memory is reduced while computation speedup is also achieved. In this paper, we propose a pruning and quantization-aware training method for multi-level ternary quantization that takes advantage of both multi-level quantization and data sparsity. In addition to increasing the accuracy of the network compared to the binary multi-level networks, it gives the network the ability to be sparse. To save memory size and computation complexity, we increase the sparsity in the quantized network by pruning until the accuracy loss is negligible. The results show that the potential speedup of computation for our model at the bit and word-level sparsity can be increased by 15x and 45x compared to the basic multi-level binary networks. Manuscript profile