A groundbreaking research team recently published a paper in Science Robotics, demonstrating how reinforcement learning can empower autonomous underwater robots and vehicles to locate and track marine objects and animals effectively. By using neural networks that learn the best actions based on rewards, these robots can achieve specific goals, even outperforming traditional methods in certain situations.
This technology is vital for exploring the oceans, providing valuable in-situ data to complement satellite observations and study essential phenomena like CO2 capture by marine organisms, contributing to climate change regulation. Ivan Masmitjà, the study’s lead author from Institut de Ciències del Mar (ICM-CSIC) and Monterey Bay Aquarium Research Institute (MBARI), emphasizes the significance of this learning approach in optimizing complex tasks that were previously challenging to accomplish.
Researchers, including Joan Navarro from ICM-CSIC, have made significant strides in using reinforcement learning to study ecological phenomena in marine environments. Autonomous robots equipped with neural networks can now track and monitor marine species’ migration and movement, both at small and large scales. This advancement enables real-time monitoring through a network of robots, where some operate on the surface, transmitting actions performed by others on the seabed via satellite.
The team utilized range acoustic techniques to estimate object positions, but the accuracy depended on where the measurements were taken. By applying artificial intelligence, particularly reinforcement learning, the robots could identify optimal trajectories based on the best data points.
Training the neural networks involved using the supercomputer at the Barcelona Supercomputing Center (BSC-CNS), speeding up parameter adjustments for various algorithms.
Experimental missions were conducted using different autonomous vehicles, including the AUV Sparus II developed by VICOROB. These missions took place in the port of Sant Feliu de Guíxols and Monterey Bay (California), collaborating with Kakani Katija from MBARI’s Bioinspiration Lab.
The team plans to explore applying the same algorithms for more complex missions, such as using multiple vehicles for object detection, locating fronts and thermoclines, and achieving cooperative algae upwelling through multi-platform reinforcement learning techniques.