68259

Автор(ы): 

Автор(ов): 

2

Параметры публикации

Тип публикации: 

Доклад

Название: 

Architecture of a distributed system for parallel execution of numerical experiments on large data sets

ISBN/ISSN: 

978-5-209-10732-3

Наименование конференции: 

  • Всероссийская конференция с международным участием "Информационно-телекоммуникационные технологии и математическое моделирование высокотехнологичных систем" (Москва, 2021)

Наименование источника: 

  • Материалы Всероссийской конференции с международным участием "Информационно-телекоммуникационные технологии и математическое моделирование высокотехнологичных систем" (Москва, 2021)

Город: 

  • Москва

Издательство: 

  • РУДН

Год издания: 

2021

Страницы: 

474-480
Аннотация
One of the key problems in computer modeling is getting numerical results from computational heavy models for large input parameters sets. In particular, this problem arises when researchers need to get accurate results for data visualization or machine learning. For instance, execution of a simulation model on thousands of input parameters combinations can take days or weeks. Other examples of tasks include estimation of open queuing networks properties, solving of optimization problems and various physics and economics numerical models. In this article we describe an architecture of a distributed system that allows to run computations in parallel on large input data sets. The application includes a web server, SQL and Redis databases, supervisor and a pool of worker threads. Any input problem is automatically split into smaller chunks (tasks) that are computed by workers in parallel. Workers pool can be created or adjusted dynamically based on the current load. User interacts with the system using web interface or REST API. For problem execution user should provide the computational algorithm packed into Docker container that accepts and returns JSON (JavaScript Object Notation). The system can be deployed in any public cloud. To show the performance gain, we consider two models: simulation model of a wireless network implemented in NS-3 and analytical model of a complex queuing system written in Python, and demonstrate the increase of computation speed.

Библиографическая ссылка: 

Соколов А.М., Ларионов А.А. Architecture of a distributed system for parallel execution of numerical experiments on large data sets / Материалы Всероссийской конференции с международным участием "Информационно-телекоммуникационные технологии и математическое моделирование высокотехнологичных систем" (Москва, 2021). М.: РУДН, 2021. С. 474-480.