Academic Journal
MARINE-Transformer: A General-purpose framework for multivariate ocean time series analysis.
| Title: | MARINE-Transformer: A General-purpose framework for multivariate ocean time series analysis. |
|---|---|
| Authors: | Wang H; Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan, China; Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science, Jinan, China., Li X; Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan, China; Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science, Jinan, China. Electronic address: xiangli@sdas.org., Fu X; Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan, China; Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science, Jinan, China., Yang M; Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan, China; Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science, Jinan, China., Zhao Z; Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan, China; Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science, Jinan, China., Wu X; Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan, China; Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science, Jinan, China., Wang Y; Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan, China; Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science, Jinan, China., Tiwari P; School of Information Technology, Halmstad University, Halmstad, Sweden. |
| Source: | Neural networks : the official journal of the International Neural Network Society [Neural Netw] 2026 Jul; Vol. 199, pp. 108706. Date of Electronic Publication: 2026 Feb 11. |
| Publication Type: | Journal Article |
| Language: | English |
| Journal Info: | Publisher: Pergamon Press Country of Publication: United States NLM ID: 8805018 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1879-2782 (Electronic) Linking ISSN: 08936080 NLM ISO Abbreviation: Neural Netw Subsets: MEDLINE |
| Imprint Name(s): | Original Publication: New York : Pergamon Press, [c1988- |
| MeSH Terms: | Oceanography*/methods , Oceans and Seas*, Multivariate Analysis ; Autoencoder ; Time Factors |
| Abstract: | Despite the success of self-supervised pre-training in texts and images, its application to multivariate time series from oceanographic IoT devices lags behind tailored methods for critical tasks like forecasting, data imputation, and anomaly detection. To address this, we propose a general-purpose framework, named MARINE-Transformer. It conducts task-agnostic pre-training on large-scale unlabeled data to learn general representations, which are then adapted to specific downstream tasks through a parameter-efficient fine-tuning process. We identify two fundamentally heterogeneous dependencies in ocean data that demand separate modeling strategies: universal temporal dynamics, which are shared across different variables, and specific cross-variable physics, which govern their interactions. Our novel univariate-to-multivariate paradigm is devised to tackle this dichotomy, addressing each type of dependency in a dedicated stage. In the univariate pre-training stage, a Masked AutoEncoder (MAE) learns the intrinsic temporal dynamics of individual ocean variables by deliberately discarding their interdependencies. Subsequently, for multivariate fine-tuning, downstream tasks are formulated as specific mask-reconstruction problems, and the pre-trained encoder is leveraged to construct a dependency graph that explicitly models the complex interactions between variables. Experiments on multiple real-world oceanographic datasets show our framework achieves state-of-the-art performance in forecasting, imputation and anomaly detection. (Copyright © 2026 Elsevier Ltd. All rights reserved.) |
| Competing Interests: | Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. |
| Contributed Indexing: | Keywords: Anomaly detection; Data imputation; Masked autoencoder; Multivariate time series; Oceanography; Pre-training and fine-tuning; Self-supervised learning; Time series forecasting |
| Entry Date(s): | Date Created: 20260224 Date Completed: 20260708 Latest Revision: 20260708 |
| Update Code: | 20260708 |
| DOI: | 10.1016/j.neunet.2026.108706 |
| PMID: | 41734442 |
| Database: | MEDLINE |
| ISSN: | 1879-2782 |
|---|---|
| DOI: | 10.1016/j.neunet.2026.108706 |