Reinforcement Learning Based Underwater Wireless Optical Communication Alignment for Autonomous Underwater Vehicles

Thumbnail Image

Access rights

openAccess
acceptedVersion

URL

Journal Title

Journal ISSN

Volume Title

A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)

Authors

Date

2022-10

Department

Department of Electrical Engineering and Automation

Major/Subject

Mcode

Degree programme

Language

en

Pages

15

Series

IEEE Journal of Oceanic Engineering, Volume 47, issue 4, pp. 1231-1245

Abstract

With the developments in underwater wireless optical communication (UWOC) technology, UWOC can be used in conjunction with autonomous underwater vehicles (AUVs) for high-speed data sharing among the vehicle formation during underwater exploration. A beam alignment problem arises during communication due to the transmission range, external disturbances and noise, and uncertainties in the AUV dynamic model. In this article, we propose an acoustic navigation method to guide the alignment process without requiring beam directors, light intensity sensors, and/or scanning algorithms as used in previous research. The AUVs need stably maintain a specific relative position and orientation for establishing an optical link. We model the alignment problem as a partially observable Markov decision process (POMDP) that takes manipulation, navigation, and energy consumption of underwater vehicles into account. However, finding an efficient policy for the POMDP under high partial observability and environmental variability is challenging. Therefore, for successful policy optimization, we utilize the soft actor–critic reinforcement learning algorithm together with AUV-specific belief updates and reward shaping based curriculum learning. Our approach outperformed baseline approaches in a simulation environment and successfully performed the beam alignment process from one AUV to another on the real AUV Tri-TON 2.

Description

Publisher Copyright: IEEE

Keywords

Acoustic beams, Acoustics, Adaptive optics, Autonomous underwater vehicle (AUV), Optical beams, partially observable Markov decision process (POMDP), Photonics, reinforcement learning (RL), Scattering, soft actor–critic (SAC), Task analysis, underwater wireless optical communication (UWOC)

Other note

DOI

10.1109/JOE.2022.3165805

Citation

Weng, Y, Pajarinen, J, Akrour, R, Matsuda, T, Peters, J & Maki, T 2022, 'Reinforcement Learning Based Underwater Wireless Optical Communication Alignment for Autonomous Underwater Vehicles', IEEE Journal of Oceanic Engineering, vol. 47, no. 4, pp. 1231-1245. https://doi.org/10.1109/JOE.2022.3165805

Permanent link to this item

https://urn.fi/URN:NBN:fi:aalto-202210196049

Collections

[article-cris] Sähkötekniikan korkeakoulu / ELEC