2021, issue 3, p. 74-85
Received 14.09.2021; Revised 24.09.2021; Accepted 28.09.2021
Published 30.09.2021; First Online 25.10.2021
https://doi.org/10.34229/2707-451X.21.3.7
Previous | FULL TEXT (in Ukrainian) | Next
Experience of OpenStack Test Deployment and Comparison of Virtual and Real Cluster Environments
Tamara Bardadym, Oleksandr Lefterov, Sergiy Osypenko
V.M. Glushkov Institute of Cybernetics of the NAS of Ukraine, Kyiv
Correspondence: This email address is being protected from spambots. You need JavaScript enabled to view it.
Introduction. A brief overview of the properties and architecture of one of the components of the National Cloud of Open Science prototype – the cloud platform OpenStack is given. The list of software and hardware components of the OpenStack test cloud environment and the sequence of actions required for the deployment of both OpenStack itself and the Slurm virtual cluster environment for portable, scalable, reproducible scientific biomedical computing are presented.
The purpose of the paper is a description of the experience of test deployment of OpenStack to create a scalable computing environment for reproducible scientific computing using modern technological solutions, which can be applied to both cloud (OpenStack, AWS, Google) and cluster platforms (Slurm).
Results. The structure of the created test containerized (using Singularity technology) biomedical application, which contains modern software and libraries and can be used in conventional and cloud virtual cluster environments is briefly described. The results of a comparative test of this application in the virtual cluster environment Slurm under the control of OpenStack and in the node of cluster SKIT-4.5 in the V.M. Glushkov Institute of Cybernetics of the NAS of Ukraine are given. Information on solving the problem of finding the optimal in terms of saving resources scaling parameters for the developed application in two comparable cluster environments is given. Some features of the use of these cluster environments are clarified, in particular, a comparison of the dependence of the application speed on the number of parallel processes for two cluster environments is presented. Empirical data are presented in graphical form, which illustrate the nature of the load on the OpenStack server and the use of RAM on the number of parallel processes. Possibilities of portability between the specified cluster environments, scaling of calculations and maintenance of reproducibility of calculations for the offered test application are demonstrated. The advantages of using OpenStack technology for scientific biomedical calculations are pointed out.
Conclusions. The described example of test deployment and use of OpenStack gives an idea of the requirements for the necessary technical base to ensure the reproducibility of scientific biomedical calculations in cloud and cluster environments.
Keywords: cloud technologies, reproducible calculations, cluster platform.
Cite as: Bardadym T., Lefterov O., Osypenko S. Experience of OpenStack Test Deployment and Comparison of Virtual and Real Cluster Environments. Cybernetics and Computer Technologies. 2021. 3. P. 74–85. (in Ukrainian) https://doi.org/10.34229/2707-451X.21.3.7
References
1. Zagorodniy A.G. European cloud of open science as a global research tool. Svit (newspaper). 2020. № 25, 26. Р. 1–3. (in Ukrainian)
2. Gorbaschuk V., Gavrilenko S., Dunaevskiy M. To Ukraine's participation in the European cloud of open science. Global and Regional Problems of Informatization in Society and Nature Using. 2021. Р. 169–171. (in Ukrainian)
3. Sefraoui O., Aissaoui M., Eleuldj M. OpenStack: toward an open-source solution for cloud computing. International Journal of Computer Applications. 2012. 55 (3). P. 38–42. https://doi.org/10.5120/8738-2991
4. Bell T., Bompastor B., Bukowiec S., Leon J.C., Denis M., van Eldik J., Lobo M.F., Alvarez L.F., Rodriguez D.F., Marino A. Scaling the CERN OpenStack cloud. Journal of Physics: Conference Series. 664. IOP Publishing 2015. P. 022003. https://doi.org/10.1088/1742-6596/664/2/022003
5. Andrade P., Bell T., Van Eldik J., McCance G., Panzer-Steindel B., dos Santos M.C., Traylen S., Schwickerath U. Review of CERN data centre infrastructure. Journal of Physics: Conference Series. 396. IOP Publishing. 2012. P. 042002. https://doi.org/10.1088/1742-6596/396/4/042002
6. OpenStack. Hands-on familiarity with the cloud operating system. / Markelov A .: M .: DMK Press, 2016. (in Russian)
7. Strozzi F., Janssen R., Wurmus R., Crusoe M. R., Githinji G., Di Tommaso P., Belhachemi D., Möller S., Smant G., de Ligt J. Scalable workflows and reproducible data analysis for genomics. Evolutionary Genomics. Springer, 2019. P. 723–745. https://doi.org/10.1007/978-1-4939-9074-0_24
8. Shor N.Z. Nondifferentiable Optimization and Polynomial Problems. London: Kluwer Acad. Publ, 1998. 381 p. https://doi.org/10.1007/978-1-4757-6015-6
9. Shor N.Z., Zhurbenko N.G. A minimization method using the operation of extension of the space in the direction of the difference of two successive gradients. Cybernetics and Systems Analysis. 1971. 7 (3). P. 450–459. https://doi.org/10.1007/BF01070454
10. Shor N.Z. Methods for Minimization of Nondifferentiable Functions and Applications. Kyiv: Nauk. dumka, 1979. 199 p. (in Russian)
11. Laptin Y.P., Bardadym T.A. Problems of calculation the coefficients of exact penalty functions. Cybernetics and Systems Analysis. 2019. 3. P. 64–79. https://doi.org/10.1007/s10559-019-00147-2
12. Zhuravlev Y.I., Laptin Y.P., Vinogradov A.P., Zhurbenko N.G., Lykhovyd O.P., Berezovskyi O.A. Linear classifiers and selection of informative features. Pattern Recognition and Image Analysis. 2017. 27 (3). Р. 426–432. https://doi.org/10.1134/S1054661817030336
13. Novoselova N.A., Tom I.E. Trait ranking algorithm for detecting biomarkers in gene expression data. Artificial intelligence. 2013. 3. P. 58–68. http://dspace.nbuv.gov.ua/handle/123456789/84980
14. Novoselova N.A., Skobtsov V.Y., Bardadym T.A., Gorbaschuk V.M., Osypenko S.P. Modern possibilities of development and organization of intelligent analytical systems. Proc. IX All-Ukrainian scientific conference “Glushkov’s Readings”, December, 18, 2020. Kyiv: Ministry of education and science of Ukraine, Kyiv National Taras Shevchenko University, 2020. P. 113–116. (in Ukrainian)
15. Bardadym T.O., Gorbachuk V.M., Novoselova N.A., Osypenko S.P., Skobtsov V.Yu., Intelligent analytical system as a tool to ensure the reproducibility of biomedical calculations. Artificial Intelligence. 2020. 3. P. 65–78.
16. Ritz C., Strebig J.C., Ritz M.C. Package ‘drc’. Creative Commons: Mountain View, CA, USA. 2016.
17. DeLean A., Munson P., Rodbard D. Simultaneous analysis of families of sigmoidal curves: application to bioassay, radioligand assay, and physiological dose-response curves. American Journal of Physiology-Endocrinology and Metabolism. 1978. 235 (2). P. E97. https://doi.org/10.1152/ajpendo.1978.235.2.E97
18. Sørensen T. A method of establishing groups of equal amplitude in plant sociology based on similarity of species and its application to analyses of the vegetation on Danish commons. Biologiske Skrifter. 1948. 5. P. 1–34.
ISSN 2707-451X (Online)
ISSN 2707-4501 (Print)
Previous | FULL TEXT (in Ukrainian) | Next