Ingrid Utseth

This presentation provides an overview of our recent work on artificial intelligence (AI) methods for fisheries acoustic data. Our initial work focused on acoustic target classification of Sandeel using CNNs. These models achieved strong performance. However, they required precise labels for training and most survey series lack the precise labels available for Sandeel. Often, the data is weak-ly labelled, where portions of energy in large areas are assigned to an acoustic category. We have developed methods for processing these weak labels to make them suitable for train-ing deep learning models. We are also developing and training a foundation model (a large, general-purpose model that can be adapted to many tasks) for acoustic data using a self-supervised approach. The model will be pre-trained on a large, unlabelled dataset. During this pre-training, the mod-el learns general patterns and structures in the data. Once pre-trained, the model can be adapted to a variety of tasks, such as acoustic target classification of multiple species, using only small amounts of labelled data. Key challenges in this work include the computational demands and large dataset required for effective pre-training, and the critical role of data sampling. We are developing sam-pling strategies using existing large-scale AI models and statistical features of the data, to ensure the model receives enough samples containing acoustic targets during pre-training. Finally, inference code for trained models is containerized and made available as separate tasks for data processing flows, such as the Kongsberg Blue Insight system and the IMR data platforms.

In this study we have investigated how vision foundation models, pretrained on different domains, compete with a specialized model for classification as a function of the size of the labeled training set of medical images. Furthermore, we have looked into the different models' ability to generalize to difficult cases. Our experiments are conducted for cardiac ultrasound images and the downstream task of view recognition. Still, this classification task is meant to serve as a demonstrative example, where we think that the findings should be transferable to other classification tasks and other domains. Through these experiments we found that the foundation models were able to beat the performance of our task-specific supervised model when labelled training data were limited. This was true even for models trained on natural images and when using the simple linear probing method to create a classifier. We observed that more domain-specific foundation models achieved an even higher performance with limited data. On the other hand, the more general models showed a greater ability to generalize and perform well on difficult, out-of-distribution cases. Still, for typical in-domain cases with sufficient labeled data, a task-specific ResNet model was competitive with the foundation models, while also being both smaller and faster.

In this study we have investigated how vision foundation models, pretrained on different domains, compete with a specialized model for classification as a function of the size of the labeled training set of medical images. Furthermore, we have looked into the different models' ability to generalize to difficult cases. Our experiments are conducted for cardiac ultrasound images and the downstream task of view recognition. Still, this classification task is meant to serve as a demonstrative example, where we think that the findings should be transferable to other classification tasks and other domains. Through these experiments we found that the foundation models were able to beat the performance of our task-specific supervised model when labelled training data were limited. This was true even for models trained on natural images and when using the simple linear probing method to create a classifier. We observed that more domain-specific foundation models achieved an even higher performance with limited data. On the other hand, the more general models showed a greater ability to generalize and perform well on difficult, out-of-distribution cases. Still, for typical in-domain cases with sufficient labeled data, a task-specific ResNet model was competitive with the foundation models, while also being both smaller and faster.

Scientific acoustic-trawl surveys collect data that are used to track fish and zooplankton populations over time. Most rely on manual annotation during acoustic target classification, but automated methods have been proposed. Here, we report on a framework for testing deep learning-based acoustic classification models and integrating them into the survey estimation process. The approach was applied to North Sea lesser sandeel (Ammodytes marinus) surveys from 2009 to 2024. Three U-Net-based models were tested: a baseline model, a depth-aware model, and a model trained with similarity-based sampling for the foreground class. A threshold based on the training years was applied to the models’ SoftMax outputs. The official sandeel estimation process was used as a starting point, replacing input data with model predictions. The biomass estimates were generally similar between manual annotations and model-based estimates, but variation existed across years. The baseline model misclassified a surface layer as sandeel and was prone to bottom contamination, causing larger deviations from official estimates. Discrepancies between the similarity-based model and the official estimates resulted from an incorrectly applied SoftMax threshold, leading to missing school interiors and indicating threshold sensitivity. Unlike traditional F1 score evaluations commonly used in image-based classification, our comparison assessed predictions in a survey-relevant context. The evaluation indicated that full automation was not yet feasible, but the predictions could be used as starting points for manual scrutiny. Annotating a subset of the data to refine thresholds or employing more advanced active learning approaches could enhance efficiency. These methods could enable faster, more consistent survey annotation.

We present a NAO robot that plays rock, paper, scissors with participants. The robot was an attempt to show how different aspects of computer science can be combined to make a fun and interesting interaction. We document the idea, how the different components work together, some events where we used the robot, and directions for future work.

We present a NAO robot that plays rock, paper, scissors with participants. The robot was an attempt to show how different aspects of computer science can be combined to make a fun and interesting interaction. We document the idea, how the different components work together, some events where we used the robot, and directions for future work.

The overarching objective of the survey is to collect data to support the CRIMAC activities and to collect data for the LoVe observatory. CRIMAC is a center of research-based innovation funded by the research council of Norway through their center for research-based innovation program (SFI). Sustainable, healthy food production and clean energy production for a growing population are important global goals, and CRIMAC will contribute to these by obtaining accurate underwater observations of gas, fish, nekton and other targets. The data will be used in conjunction with CRIMAC data from other surveys to build a reference data set for optical and acoustic target classification. The classification libraries will be used for developing methods and products toward the fishing industry and marine science. The survey was divided into two legs where leg one mainly focused on trawl instrumentation and data collection for behavioural studies on fish-trawl interactions. The main objectives of this part were to test in-trawl camera systems and data processing from such systems, test and develop trawl instrumentation and acoustic and optic monitoring of herring behaviour in relation to the trawl. The second leg of the survey focused mainly on broad band acoustic data, including sizing of fish using broad banded acoustics, noise estimation, calibration, time series consistency when changing to broad band acoustics, gas seep detection as well as performing the standard IMR LoVe transect.

Ingrid Utseth fra NR har vært på akustikktokt med Havforskningsinstituttet. Etter toktet sitter Ingrid igjen med et laaaangt bilde som kommer fra ekkoloddet til forskningsfartøyet Ingrid var på. Vi snakker om hvorfor og hvordan hun jobber med automatisk tolkning av slike data fra ekkolodd. Av Norsk Regnesentral med Anders Løland i studio. Produsent: Elin Ruhlin Gjuvsland.

Objectives The primary objective of the SFI is to advance the frontiers in fisheries acoustic methodology and associated optical methods, and to apply such methods to 1) surveys for marine organisms, 2) fisheries, 3) aquaculture and 4) the energy sector.

Acoustic surveys provide important data for fisheries management. During the surveys, ship-mounted echo sounders send acoustic signals into the water and measure the strength of the reflection, so-called backscatter. Acoustic target classification (ATC) aims to identify backscatter signals by categorizing them into specific groups, e.g. sandeel, mackerel, and background (as bottom and plankton). Convolutional neural networks typically perform well for ATC but fail in cases where the background class is similar to the foreground class. In this study, we discuss how to address the challenge of class imbalance in the sampling of training and validation data for deep convolutional neural networks. The proposed strategy seeks to equally sample areas containing all different classes while prioritizing background data that have similar characteristics to the foreground class. We investigate the performance of the proposed sampling methodology for ATC using a previously published deep convolutional neural network architecture on sandeel data. Our results demonstrate that utilizing this approach enables accurate target classification even when dealing with imbalanced data. This is particularly relevant for pixel-wise semantic segmentation tasks conducted on extensive datasets. The proposed methodology utilizes state-of-the-art deep learning techniques and ensures a systematic approach to data balancing, avoiding ad hoc methods.

Fish stock assessment and management requires accurate estimates of fish abundance, which are typically derived from echosounder observations using acoustic target classification (ATC). Skilled operators are regularly assisted in classifying acoustic targets by software and there has been an increasing interest toward using machine learning to create improved tools. Recent studies have applied deep learning approaches to acoustic data, however, algorithm data-preparation strategies (influencing model output) are presently poorly understood and standardization is needed to enable collaborative research and management. For example, a common pre-processing technique is to resample backscatter data coming from echosounder measurements from the original resolution to a coarser resolution in the horizontal (time) and vertical (range) directions. Using data values derived from the volume backscattering coefficient obtained during the Norwegian sandeel survey, we investigate which resampling resolutions are suitable for ATC using a convolutional neural network trained to classify single values of backscatter data. This process is known as pixel-level semantic segmentation. Our results indicate that it is possible to downsample the data if important information related to acoustic characteristics is not smoothed out. We also show that the classification performance is improved when providing the network with contextual information relating to range. These findings will provide input to fisheries acoustic data standards and contribute to the on-going development of automated ATC methods.

The aim of the study was to investigate whether context information related to acoustic data resolution could improve automatic acoustic classification of species using echosounder observations.