2021, issue 3, p. 65-73
Received 28.07.2021; Revised 08.08.2021; Accepted 28.09.2021
Published 30.09.2021; First Online 25.10.2021
https://doi.org/10.34229/2707-451X.21.3.6
Comparative Analysis of the Application of Multilayer and Convolutional Neural Networks for Recognition of Handwritten Letters of the Azerbaijani Alphabet
Elshan Mustafayev *
,
Rustam Azimov
Institute of Control Systems of the NAS of Azerbaijan, Baku
* Correspondence: This email address is being protected from spambots. You need JavaScript enabled to view it.
Introduction. The implementation of information technologies in various spheres of public life dictates the creation of efficient and productive systems for entering information into computer systems. In such systems it is important to build an effective recognition module. At the moment, the most effective method for solving this problem is the use of artificial multilayer neural and convolutional networks.
The purpose of the paper. This paper is devoted to a comparative analysis of the recognition results of handwritten characters of the Azerbaijani alphabet using neural and convolutional neural networks.
Results. The analysis of the dependence of the recognition results on the following parameters is carried out: the architecture of neural networks, the size of the training base, the choice of the subsampling algorithm, the use of the feature extraction algorithm. To increase the training sample, the image augmentation technique was used. Based on the real base of 14000 characters, the bases of 28000, 42000 and 72000 characters were formed. The description of the feature extraction algorithm is given.
Conclusions. Analysis of recognition results on the test sample showed:
· as expected, convolutional neural networks showed higher results than multilayer neural networks;
· the classical convolutional network LeNet-5 showed the highest results among all types of neural networks. However, the multi-layer 3-layer network, which was input by the feature extraction results; showed rather high results comparable with convolutional networks;
· there is no definite advantage in the choice of the method in the subsampling layer. The choice of the subsampling method (max-pooling or average-pooling) for a particular model can be selected experimentally;
· increasing the training database for this task did not give a tangible improvement in recognition results for convolutional networks and networks with preliminary feature extraction. However, for networks learning without feature extraction, an increase in the size of the database led to a noticeable improvement in performance.
Keywords: neural networks, feature extraction, OCR.
Cite as: Mustafayev E., Azimov R. Comparative Analysis of the Application of Multilayer and Convolutional Neural Networks for Recognition of Handwritten Letters of the Azerbaijani Alphabet. Cybernetics and Computer Technologies. 2021. 3. P. 65–73. https://doi.org/10.34229/2707-451X.21.3.6
References
1. Lecun Y., Bottou L., Bengio Y. Haffner P. Gradient-based learning applied to document recognition. Proc. of the IEEE. 1998. 86 (11). P. 2278–2324. https://doi.org/10.1109/5.726791
2. Golovko V.A., Krasnoproshin V.V. Neural network data processing technologies. Minsk: BSU, 2017. 263 p. (in Russian)
3. Osovsky S. Neural Networks for Information Processing. Finance and statistics. 2002. 344 p. (in Russian)
4. Mustafayev E.E. Handwritten text recognition methods. Fuyuzat, 2020. 189 p. (in Russian)
5. Aida-zade K.R., Mustafayev E.E. Intelligent recognition system of Azerbaijani handwritten forms. Proc. The Scientific Conference “Modern problems of Cybernetics and Information Technologies”. Vol. III. Baku. 2006. P. 85–88. (in Russian)
6. Aida-zade K.R., Mustafayev E.E. About one hierarchical handwritten recognition system on the bases neural networks. Transactions of the NAS of Azerbaijan, series of PTMS. 2–3. 2002. P. 94–98. (in Russian)
7. Aida-zade K.R., Mustafayev E.E., Hasanov J.Z. About knowledge base usage for increasing intellectuality of recognition systems, Proc. the 11th Russian Conference “Mathematical Methods of Pattern Recognition”. 2003. Moscow. P. 6–8. (in Russian)
8. Mustafayev E.E. Hierarchical Multilevel Form Recognition System. Proceedings of scientific conference “Modern problems of applied mathematics”. Baku. 2002. P. 154–157.
9. Aida-zade K.R., Mustafayev E.E. Intelligent handwritten form recognition system based on artificial neural networks. Proceedings of the Intern. Conf. on Modeling and Simulation, 2006, 28-30 August, Konya, Turkey. P. 609–613.
10. Arif A.F., Takahashi H., Iwata A., Tsutsumida T. Handwritten postal code recognition by neural network – a comparative study. IEICE Trans.Inf.&Syst. 1996. E79-D (5). P. 443–449.
11. Francois Ch. Deep Learning with Python. Manning Publications, Shelter Island, NY. 362 p.
12. Eldan R., Shamir O. The power of depth for feedforward neural networks. Conference on Learning Theory. 2016. 49. P. 907–940.
13. https://towardsdatascience.com/metrics-to-evaluate-your-machine-learning-algorithm-f10ba6e38234 (accessed: 26.07.2021)
ISSN 2707-451X (Online)
ISSN 2707-4501 (Print)