daGui: A DataFlow Graphical User Interface

Thumbnail Image

Files

master_Uhlir_Adam_2017.pdf (1.67 MB)

URL

Journal Title

Journal ISSN

Volume Title

Perustieteiden korkeakoulu | Master's thesis
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.

Authors

Date

2017-08-28

Department

Major/Subject

Cloud Computing and Services

Mcode

SCI3081

Degree programme

Master's Programme in ICT Innovation

Language

en

Pages

8+50

Series

Abstract

Big Data is a growing trend. It focuses on storing and processing a vast amount of data in a distributed environment. There are many frameworks and tools which can be used to work with this data. Many of them utilise Directed Acyclic Graphs (DAGs) in some way. A DAG is often used for expressing the dataflow of computation as it offers the possibility to optimise the execution, because it contains the overview of the whole computation. This thesis aims to create an Integrated Development Environment (IDE) like software, which is user-friendly, interactive and easily extendable. The software enables to draw a DAG which represents the dataflow of a program. The DAG is then transformed into launchable source code. Moreover, the software offers a simple way to execute the generated source code. It compiles the code (if necessary), and launches it based on the user's configuration, either on localhost or cluster. The software primarily aims to help beginners learn these technologies, but experts can also use it as visualisation for their workflow or as a prototyping tool. The software has been implemented using Electron and Web technologies, which ensure its platform independence. Its main features are code generation (i.e. translation of a DAG into source code) and code execution. It is created with extensibility in mind, to be able to plug-in support for more frameworks and tools in the future.

Description

Supervisor

Heljanko, Keijo

Thesis advisor

Dowling, Jim

Keywords

big data, apache spark, DAG, dataflow, GUI

Other note

Citation

Permanent link to this item

https://urn.fi/URN:NBN:fi:aalto-201709046800

Collections

[dipl] Perustieteiden korkeakoulu / SCI