Browsing by Author "Pajarinen, Joni"

Filter results by typing the first few letters
Now showing 1 - 20 of 50
  • Automated Test Case Generation for Web Applications Using Machine Learning
    (2024-01-22) Lehtonen, Riku
     Sähkötekniikan korkeakoulu | Master's thesis
    In recent years, web software development has gained significant prevalence. Consequently, the resources and costs required for verifying the functionality of web applications have also increased substantially. Thus, automation in the testing process, such as test case generation, enhances testing efficiency and reduces testing costs. Automatic test case generation employs gathered knowledge of the software to create test steps without direct control by a tester. In web software development, test generation is challenging as applications frequently consist of multiple complex systems. Therefore, machine learning algorithms have been implemented in test case generation to replicate the manual testing traditionally performed by humans. Recent research has created test cases by exploring the application using search algorithms and directly converting the source code to test cases utilizing language processing. However, previous work has not suggested a generation framework for widely used test automation libraries and machine learning algorithms. This thesis proposes a framework for transmitting information, such as visible elements and actions, between the machine learning algorithm and the software. For the framework, two machine learning algorithms, Proximal Policy Optimization (PPO) and Online Decision Transformer (ODT), are implemented to benchmark search-based test generation performance. The algorithms optimize test steps for achieving user-provided test objectives, such as logging into a website. Results indicate that the framework can support the algorithms for exploration-based test generation for web applications. The PPO can optimize the test steps towards various test objectives. The ODT efficiently clones the behavior from collected trajectories, for example, previously created test cases. This thesis also analyzes solutions to address potential scalability challenges in the algorithms used, particularly as the number of available actions increases in larger applications. Furthermore, the future aim for these algorithms is simultaneous and rapid test case generation across multiple applications.
  • Automatic real time phishing email detection with explainable decision making
    (2021-08-23) Maanonen, Joonas
     Sähkötekniikan korkeakoulu | Master's thesis
  • Autonomous solutions for harvesting Typha latifolia seed fluff
    (2023-01-23) Viljanen, Hermanni
     Sähkötekniikan korkeakoulu | Master's thesis
    In order to mitigate climate change, greenhouse gas (GHG) emissions that originate from agriculture should be reduced as they account for 30-40 percent of the total GHG emissions in the world. Paludiculture is the sustainable production of biomass on wet peatlands, and compared to agricultural farming methods based on drained peatlands, it reduces the total GHG emissions by 8-35 tons of CO2 equivalents per hectare of farmed land. On wet peatlands, paludicrops such as Typha latifolia, the common cattail, can be cultivated for profit. The seed fluff of Typha latifolia can be produced to be used as textile filling, however the harvesting that is currently done manually is ineffective and no agricultural robots have been developed for the task. In an effort to effectivize the harvesting of Typha latifolia seed fluff, we developed and tested two harvesting prototypes based on suction. Additionally, we reviewed agricultural robot systems and their feasibility in paludiculture. It was concluded that a tracked mobile robot or a drone could realistically traverse and navigate the waterlogged environment of a Typha latifolia plantation while detecting the plants' flower pistons containing the harvested seed fluff. It could be used to harvest the seed fluff, provided that a feasible fluff harvesting system was available. We compared the harvesting speed and material loss of the two prototypes to manually harvesting the seed fluff. Prototype 2 achieved similar speeds and material loss as harvesting the fluff by hand. Many environmental variables, such as the plant’s flower piston ripeness and limited movement in the testing terrain affected the test results and further research should be made to verify them. The suction harvesting method could also be evaluated and compared to other methods of harvesting such as cutting. We concluded that harvesting based on suction needs to be made simpler in order to be a feasible method for autonomous harvesting of Typha latifolia seed fluff. Harvesting with the prototypes causes the plant stalks to bend and move, moving the flower pistons. This creates additional requirements for the detection of the flower pistons as well as the harvesting of the seed fluff.
  • Autonomous underwater vehicle link alignment control in unknown environments using reinforcement learning
    (2024-09) Weng, Yang; Chun, Sehwa; Ohashi, Masaki; Matsuda, Takumi; Sekimori, Yuki; Pajarinen, Joni; Peters, Jan; Maki, Toshihiro
     A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
    High-speed underwater wireless optical communication holds immense promise in ocean monitoring and surveys, providing crucial support for the real-time sharing of observational data collected by autonomous underwater vehicles (AUVs). However, due to inaccurate target information and external interference in unknown environments, link alignment is challenging and needs to be addressed. In response to these challenges, we propose a reinforcement learning-based alignment method to control the AUV to establish an optical link and maintain alignment. Our alignment control system utilizes a combination of sensors, including a depth sensor, Doppler velocity log (DVL), gyroscope, ultra-short baseline device, and acoustic modem. These sensors are used in conjunction with a particle filter to observe the environment and estimate the AUV's state accurately. The soft actor-critic algorithm is used to train a reinforcement learning-based controller in a simulated environment to reduce pointing errors and energy consumption in alignment. After experimental validation in simulation, we deployed the controller on an actual AUV called Tri-TON. In experiments at sea, Tri-TON maintained the link and angular pointing errors within 1 m and (Formula presented.), respectively. Experimental results demonstrate that the proposed alignment control method can establish underwater optical communication between AUV fleets, thus improving the efficiency of marine surveys.
  • Conceptualization of an automated material flow and optimized production process in a paint shop
    (2023-01-23) Knappe, Henrik
     Sähkötekniikan korkeakoulu | Master's thesis
  • Connection Tracking Functionality for a Personal Firewall
    (2003) Pajarinen, Joni
     Helsinki University of Technology | Master's thesis
    Nykypäivinä henkilökohtaiset palomuurit ovat tärkeitä ohjelmistoja tietokoneissa, jotka ovat yhteydessä sekä Internettiin että yksityisiin verkkoihin. Ne voivat estää tunnettuja ja tuntemattomia uhkia, joita hyökkääjät julkisessa verkossa edustavat. Monet henkilökohtaiset palomuurit seuraavat yhteyksiä ulos- ja sisäänpäin tietokoneista, joihin ne ovat asennettuina. Tässä työssä kehitetään yhteyksien seurantaa toteuttava kirjasto henkilökohtaisia palomuureja varten. Työ alkaa kertomalla palomuurien taustoja ja kertomalla miten yhteyksien seurantaa käytetään niissä. Tarpeet yhteyksien seurantaan niissä selvitetään. Esitetään uhkia, jotka koskevat palomuureja joissa käytetään yhteyksien seurantaa ja uhkia, jotka koskevat palomuureja joissa ei käytetä yhteyksien seurantaa. Kerrotaan ongelmat joita yhteyksien seuranta yrittää ratkaista ja mitä täytyy huomioida ratkaistessa näitä ongelmia. Yhteyksien seurannalle asetetaan toiminnalliset, suorituskyvylliset ja hallinnalliset vaatimukset. Ratkaisu, joka tyydyttää asetetut vaatimukset muodostetaan. Ratkaisu koostuu yhteyksien seuranta- kirjaston yleisestä arkkitehtuurista, tietorakenteista, algoritmeistä joita käytetään tietorakenteiden käsittelyyn ja protokollien tarkastelusta. Kun yhteyksien seuranta- kirjasto on suunniteltu ja toteutettu, se testataan. Testeistä saatuja tuloksia verrataan työn alussa asetettuihin vaatimuksiin. Johtopäätökset toteutetusta ratkaisusta ovat, että se tyydyttää kaikki sille asetetut vaatimukset ja tarjoaa ne palvelut joita voi odottaa henkilökohtaisen palomuurin yhteyksien seurannalta. Lopuksi esitetään mahdollisia tulevia ominaisuuksia yhteyksien seurantaan liittyen ja yksityiskohtia joita pitää ottaa huomioon ominaisuuksia arvioitaessa.
  • Continuous Monte Carlo Graph Search
    (2024) Kujanpää, Kalle; Kannala, Juho; Babadi, Amin; Ilin, Alexander; Zhao, Yi; Pajarinen, Joni
     A4 Artikkeli konferenssijulkaisussa
    Online planning is crucial for high performance in many complex sequential decision-making tasks. Monte Carlo Tree Search (MCTS) employs a principled mechanism for trading off exploration for exploitation for efficient online planning, and it outperforms comparison methods in many discrete decision-making domains such as Go, Chess, and Shogi. Subsequently, extensions of MCTS to continuous domains have been developed. However, the inherent high branching factor and the resulting explosion of the search tree size are limiting the existing methods. To address this problem, we propose Continuous Monte Carlo Graph Search (CMCGS), an extension of MCTS to online planning in environments with continuous state and action spaces. CMCGS takes advantage of the insight that, during planning, sharing the same action policy between several states can yield high performance. To implement this idea, at each time step, CMCGS clusters similar states into a limited number of stochastic action bandit nodes, which produce a layered directed graph instead of an MCTS search tree. Experimental evaluation shows that CMCGS outperforms comparable planning methods in several complex continuous DeepMind Control Suite benchmarks and 2D navigation and exploration tasks with limited sample budgets. Furthermore, CMCGS can be scaled up through parallelization, and it outperforms the Cross-Entropy Method (CEM) in continuous control with learned dynamics models.
  • Convex Regularization in Monte-Carlo Tree Search
    (2021) Dam, Tuan; D'Eramo, Carlo; Peters, Jan; Pajarinen, Joni
     A4 Artikkeli konferenssijulkaisussa
    Monte-Carlo planning and Reinforcement Learning (RL) are essential to sequential decision making. The recent AlphaGo and AlphaZero algorithms have shown how to successfully combine these two paradigms to solve large-scale sequential decision problems. These methodologies exploit a variant of the well-known UCT algorithm to trade off the exploitation of good actions and the exploration of unvisited states, but their empirical success comes at the cost of poor sample-efficiency and high computation time. In this paper, we overcome these limitations by introducing the use of convex regularization in Monte-Carlo Tree Search (MCTS) to drive exploration efficiently and to improve policy updates. First, we introduce a unifying theory on the use of generic convex regularizers in MCTS, deriving the first regret analysis of regularized MCTS and showing that it guarantees an exponential convergence rate. Second, we exploit our theoretical framework to introduce novel regularized backup operators for MCTS, based on the relative entropy of the policy update and, more importantly, on the Tsallis entropy of the policy, for which we prove superior theoretical guarantees. We empirically verify the consequence of our theoretical results on a toy problem. Finally, we show how our framework can easily be incorporated in AlphaGo and we empirically show the superiority of convex regularization, w.r.t. representative baselines, on well-known RL problems across several Atari games.
  • Curriculum reinforcement learning via constrained optimal transport
    (2022) Klink, Pascal; Yang, Haoyi; D'Eramo, Carlo; Pajarinen, Joni; Peters, Jan
     A4 Artikkeli konferenssijulkaisussa
    Curriculum reinforcement learning (CRL) allows solving complex tasks by generating a tailored sequence of learning tasks, starting from easy ones and subsequently increasing their difficulty. Although the potential of curricula in RL has been clearly shown in a variety of works, it is less clear how to generate them for a given learning environment, resulting in a variety of methods aiming to automate this task. In this work, we focus on the idea of framing curricula as interpolations between task distributions, which has previously been shown to be a viable approach to CRL. Identifying key issues of existing methods, we frame the generation of a curriculum as a constrained optimal transport problem between task distributions. Benchmarks show that this way of curriculum generation can improve upon existing CRL methods, yielding high performance in a variety of tasks with different characteristics.
  • Deep Learning for Efficient Retail Shelf Stock Monitoring and Analysis
    (2023-10-09) Lachhab, Walid
     Sähkötekniikan korkeakoulu | Master's thesis
    This thesis explores the automation of stock management in retail stores, with a specific focus on stores specializing in the sale of fruits and vegetables. Traditionally, these stores have relied on manual stock management methods, involving periodic inspections to maintain product availability. In response, this study proposes the application of Deep Learning techniques, particularly object counting models, to automate stock management. The automation process comprises two key steps. Initially, a camera positioned above a shelf of fruits and vegetables captures an image, which is processed to identify boxes containing fruits and vegetables, along with their respective categories. Afterward, a Deep Learning counting model is employed to provide an estimation of the number of objects present within each box. These estimations can then be continuously monitored or subjected to analysis to optimize store operations. The research encompasses four distinct data scenarios: supervised learning, semi-supervised learning, few-shot learning, and zero-shot learning. Within each scenario, existing object counting methods are evaluated using object detection and density estimation methodologies. The primary goals of this research are to establish an experimental setup for assessing object counting models across different learning frameworks, evaluate their performance in various scenarios, and analyze the practical strengths and limitations of these techniques in retail store environments. Key findings from the study highlight the superior performance of YOLO models, especially YOLOv5, in supervised learning scenarios, striking a balance between speed and model size. In semi-supervised learning, the application of the Efficient-Teacher approach to YOLO models enhances performance with limited labeled data. Zero-shot learning, specifically the CLIP-Count method offering a balance between speed and acceptable error rates, is recommended for data-scarce environments with sufficient computational resources. While few-shot learning, represented by the SAFECount approach, remains as the last option due to its relatively higher error, and it is suggested for situations with limited data and computational resources. Furthermore, our study reveals that improving the counting model's performance can be achieved through the removal of certain complex-shaped categories that present counting difficulties, such as grapes and hot peppers. Additionally, merging categories of fruits and vegetables with similar appearances emerges as a viable strategy for optimization. Overall, this thesis offers practical insights into automating stock tracking in retail stores. It emphasizes the importance of selecting the right learning framework and model based on specific operational needs and constraints such as data availability, providing valuable guidance to improve stock management efficiency in diverse data scenarios.
  • Development of an automated sanding solution for auto body repairs
    (2022-08-22) Wargh, David
     Sähkötekniikan korkeakoulu | Master's thesis
    Due to the increased interest in a circular economy and declining labor force in the auto body repair industry it is essential to start automating the repair processes. Robotics provide an excellent opportunity to automate collision repair processes. However, industrial robots are mostly used in continuous applications where most of the external factors stay unchanged. This is a problem for automating collision repair processes, where the vehicle shape, damage location and extent of the damage varies on a case-to-case basis. Furthermore, since the cases are unique it is beneficial to be able to find damages without the need of CAD models. A potential approach for overcoming this problem is to use a 3D camera or 3D scanner to localize and outline the damaged part. Such solutions include “Scan & Sand” (Gray Matter Robotics, Ltd.) and “Craftmate” (Nordbo Robotics). These systems help bridge the gap toward adaptive robotic sanding systems. However, the systems need an operator to manually select the part or region to process. This thesis develops a software solution as a proof-of-concept. The solution is able to scan a part, automatically detect damages and prepare the damages for filling and painting by sanding them. The hardware consists of a 3D camera, a sanding system, and a collaborative robot. The software utilizes standard image processing algorithms to find visible defects like scratches and point cloud processing to find non-visible damages such as dents. After the defects are found the software generates a trajectory for the robot and sends commands to the robot to execute the sanding sequence. The approach was evaluated by processing six painted metal sheets and visually inspecting the result. Some of the cases were also evaluated by comparing the detected damages to the actual damages. The approach proves that it is possible to use machine vision to automate robotic sanding. The developed solution finds most of both visible and non-visible defects through the respective algorithms. The generated trajectories were mostly appropriate, except for some cases where an unnecessarily large area was sanded. The results can be translated to car parts which are flat or slightly curved. For example, the hood or roof of most modern cars.
  • Establishment of line-of-sight optical links between autonomous underwater vehicles: Field experiment and performance validation
    (2022-12) Weng, Yang; Matsuda, Takumi; Sekimori, Yuki; Pajarinen, Joni; Peters, Jan; Maki, Toshihiro
     A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
    Establishing a line-of-sight link between autonomous underwater vehicles (AUVs) is an unavoidable challenge for realizing high data rate optical communication in ocean exploration. We propose a method for link establishment by maintaining the relative position and orientation between AUVs. Using a reinforcement learning algorithm, we search for the policy that can suppress external disturbances and optimize the link establishment efficiency. To evaluate the performance of the proposed method, we prepared a hovering AUV to conduct the link establishment experiments. The reinforcement learning policy trained in a simulation environment was deployed on the AUV in real environments. In field experiments, our approach successfully performed the link establishment from the hovering AUV to an autonomous surface vehicle. Based on the experimental results, we evaluate the performance of the AUV in executing the link establishment policy. Comparisons with existing optical search-based link establishment methods are presented.
  • Heterogeneous Robot Coordination in Unstructured and Space Environments
    (2024-09-29) Döhmer, Marc-Lorenz Alexander
     School of Electrical Engineering | Master's thesis
    Space environments are becoming increasingly more relevant as they hold considerable resources. Past missions onto planets such as Mars used very sophisticated single rovers, these however provide a single point of mission failure. It is therefore valuable to look at Multi Robot Systems (MRS) as an alternative. Especially teams of heterogeneous robots are relevant as they provide a good trade-off between robustness and specialised capabilities. In this paper research about heterogeneous robot coordination in unstructured and space environments was conducted. The main goal was to determine the optimal team composition for a chosen mission scenario to achieve a short mission time and low workload for individual robots in a team. A framework was developed to simulate different mission scenarios with heterogeneous teams comprised of different robots. A series of experiments were carried out, which analysed the effect of increasing the number of robots in a robotic team based on the Weakest Link Assumption introduced in this work. It was found that the reduction in mission time when adding more robots to a team where the tasks are split does not seem to behave linear and might suddenly flatten off or that he mission time would increase drastically due to an increased number of incidents between robots. Future work could focus on understanding this behavior in more detail.
  • Hierarchical Imitation Learning with Vector Quantized Models
    (2023-07) Kujanpää, Kalle; Pajarinen, Joni; Ilin, Alexander
     A4 Artikkeli konferenssijulkaisussa
    The ability to plan actions on multiple levels of abstraction enables intelligent agents to solve complex tasks effectively. However, learning the models for both low and high-level planning from demonstrations has proven challenging, especially with higher-dimensional inputs. To address this issue, we propose to use reinforcement learning to identify subgoals in expert trajectories by associating the magnitude of the rewards with the predictability of low-level actions given the state and the chosen subgoal. We build a vector-quantized generative model for the identified subgoals to perform subgoal-level planning. In experiments, the algorithm excels at solving complex, long-horizon decision-making problems outperforming state-of-the-art. Because of its ability to plan, our algorithm can find better trajectories than the ones in the training set.
  • Implementation of an automatic log analysis tool for health care applications
    (2024-03-11) Turunen, Arttu
     Sähkötekniikan korkeakoulu | Master's thesis
    Large software applications generate extensive amounts of log data to monitor behavior and diagnose software issues. While log data can be examined manually, the large volume of log data makes manual examination error-prone and time-consuming. Log data analysis software and methods have been developed to automate log data analysis, thereby enhancing software reliability and development. Commercial and open-source log analysis software exists, but this cannot always be used due to various restrictions, such as data security. In this case, an automatic log data analysis program has to be implemented from scratch. In this thesis, we designed and implemented an automatic log analysis program for the healthcare software Effector by seeking inspiration from studies and existing implementations. We tailored the implementation to the requirements of the Effector and implemented anomaly detection using statistical methods based on Z-score and sliding window. We tested the implementation against real data from Effector in real use cases and detected genuine anomalies in the production data. During the analysis of six months of log data, the log analysis program detected between 10 and 40 anomalies per week, depending on the size of the source data examined. For instance, the log analysis program detected a critical bug in the automatic import of patient data after an Effector version update. This bug would have gone unnoticed for a longer period without the assistance of the program.
  • Keinolihakset robotiikassa
    (2015-05-06) Rauhio, Kalle
     Sähkötekniikan korkeakoulu | Bachelor's thesis
  • Latent Derivative Bayesian Last Layer Networks
    (2021) Watson, Joe; Lin, Jihao Andreas; Klink, Pascal; Pajarinen, Joni; Peters, Jan
     A4 Artikkeli konferenssijulkaisussa
    Bayesian neural networks (BNN) are powerful parametric models for nonlinear regression with uncertainty quantification. However, the approximate inference techniques for weight space priors suffer from several drawbacks. The 'Bayesian last layer' (BLL) is an alternative BNN approach that learns the feature space for an exact Bayesian linear model with explicit predictive distributions. However, its predictions outside of the data distribution (OOD) are typically overconfident, as the marginal likelihood objective results in a learned feature space that overfits to the data. We overcome this weakness by introducing a functional prior on the model's derivatives w.r.t. the inputs. Treating these Jacobians as latent variables, we incorporate the prior into the objective to influence the smoothness and diversity of the features, which enables greater predictive uncertainty. For the BLL, the Jacobians can be computed directly using forward mode automatic differentiation, and the distribution over Jacobians may be obtained in closed-form. We demonstrate this method enhances the BLL to Gaussian process-like performance on tasks where calibrated uncertainty is critical: OOD regression, Bayesian optimization and active learning, which include high-dimensional real-world datasets.
  • Learning Assembly Tasks from Human Demonstration
    (2016-05-09) Ikkala, Aleksi
     Sähkötekniikan korkeakoulu | Master's thesis
    This thesis presents a method for learning and reproducing assembly tasks using Learning from Demonstration paradigm and a graph representation of assembly parts and their spatial relations. We show that this graph representation combined with inexact graph matching techniques provide a framework capable of learning assembly tasks, even with uncertain information of assembly operations. In this thesis our method replicated observed assembly tasks, where Lego Quatro bricks were manipulated with pick-and-place operations. We tested our proposed method through a series of experiments. In the experiments, a robot first observed a human teacher demonstrate an assembly task in front of a Kinect sensor. Then, the robot generated a simulation that depicted the learned assembly product. We also introduced uncertainty into the experiments by changing some of the assembly parts, or by not showing the intermediate assembly operations to the robot. In these events, the robot generated a simulated structure that was similar to the observed one. We used inexact graph matching techniques to measure the similarity between assembly structures. In the experiments our method successfully replicated a learned task when the robot was provided with a complete set of assembly parts. Also, the task was repeated relatively well when only one or two assembly parts were replaced with another type of Lego. We conclude that our method provides a convenient platform for a more general assembly method. Also, our method is capable of ``improvising'' in unanticipated situations, where the robot is supplied with imperfect knowledge of the task.
  • Learning-Based Propulsion Control for Amphibious Quadruped Robots With Dynamic Adaptation to Changing Environment
    (2023-12-01) Yao, Qingfeng; Meng, Linghan; Zhang, Qifeng; Zhao, Jing; Pajarinen, Joni; Wang, Xiaohui; Li, Zhibin; Wang, Cong
     A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
    This letter proposes a learning-based adaptive propulsion control (APC) method for a quadruped robot integrated with thrusters in amphibious environments, allowing it to move efficiently in water while maintaining its ground locomotion capabilities. We designed the specific reinforcement learning method to train the neural network to perform the vector propulsion control. Our approach coordinates the legs and propeller, enabling the robot to achieve speed and trajectory tracking tasks in the presence of actuator failures and unknown disturbances. Our simulated validations of the robot in water demonstrate the effectiveness of the trained neural network to predict the disturbances and actuator failures based on historical information, showing that the framework is adaptable to changing environments and is suitable for use in dynamically changing situations. Our proposed approach is suited to the hardware augmentation of quadruped robots to create avenues in the field of amphibious robotics and expand the use of quadruped robots in various applications.
  • LIDAR-based Semantic Segmentation for Navigation in Semi-Dynamic Environments
    (2024-03-11) Tuomisto, Jaakko
     Sähkötekniikan korkeakoulu | Master's thesis
    The adoption of autonomous mobile robots across industries has increased the demand for sophisticated navigation systems capable of operating in complex and dynamic environments. To address the need, this thesis presents the development and evaluation of a navigation system for mobile robots. In particular, the focus is on managing the semi-dynamic obstacles of the environment, which are obstacles that can be static for long periods of time but will still occasionally move. Examples of semi-dynamic obstacles include temporary construction barriers or parked vehicles. These semi-dynamic obstacles pose a challenge for autonomous navigation, as they are often erroneously mapped as static obstacles. This causes navigation systems to operate based on incorrect assumptions about obstacle locations, which can cause the robot to take unnecessary detours or become trapped. To address the challenge, this thesis proposes utilizing deep-learning semantic segmentation applied to LIDAR data to differentiate between temporarily and permanently blocked paths. The semantic segmentation model employed in this system is trained on a custom synthetic dataset, and operates in real time as the robot explores the environment. The acquired semantic information is combined with three-dimensional occupancy mapping, A*-based path planning, and recovery behaviors for reacting to movements of semi-dynamic obstacles. The system's proficiency in managing semi-dynamic obstacles is validated through simulated experiments. Furthermore, the system is tested using the real-world Boston Dynamics Spot robot.