Academic Journal
Προβολή σε EDS

Selecting synthetic data for successful simulation-based transfer learning in dynamical biological systems.

Λεπτομέρειες βιβλιογραφικής εγγραφής
Τίτλος: Selecting synthetic data for successful simulation-based transfer learning in dynamical biological systems.
Συγγραφείς: Witzke S; Digital Engineering Faculty, Hasso Plattner Institute, University of Potsdam, Potsdam, Germany., Zabbarov J; Digital Engineering Faculty, Hasso Plattner Institute, University of Potsdam, Potsdam, Germany., Kleissl M; Digital Engineering Faculty, Hasso Plattner Institute, University of Potsdam, Potsdam, Germany., Iversen P; Digital Engineering Faculty, Hasso Plattner Institute, University of Potsdam, Potsdam, Germany.; Department of Mathematics and Computer Science, Freie Universität Berlin, Berlin, Germany., Renard BY; Digital Engineering Faculty, Hasso Plattner Institute, University of Potsdam, Potsdam, Germany., Baum K; Digital Engineering Faculty, Hasso Plattner Institute, University of Potsdam, Potsdam, Germany. katharina.baum@fu-berlin.de.; Department of Mathematics and Computer Science, Freie Universität Berlin, Berlin, Germany. katharina.baum@fu-berlin.de.
Πηγή: BMC bioinformatics [BMC Bioinformatics] 2026 May 18; Vol. 27 (1). Date of Electronic Publication: 2026 May 18.
Τύπος έκδοσης: Journal Article
Γλώσσα: English
Στοιχεία περιοδικού: Publisher: BioMed Central Country of Publication: England NLM ID: 100965194 Publication Model: Electronic Cited Medium: Internet ISSN: 1471-2105 (Electronic) Linking ISSN: 14712105 NLM ISO Abbreviation: BMC Bioinformatics Subsets: MEDLINE
Imprint Name(s): Original Publication: [London] : BioMed Central, 2000-
Ιατρικοί όροι (MeSH): Computer Simulation* , Models, Biological* , Systems Biology* , Machine Learning*
Περίληψη: Background: Accurate prediction of the temporal dynamics of biological systems is crucial for informing timely and effective interventions, e.g., in ecological or epidemiological contexts, or for treatment adjustments in therapy. While machine learning has proven its capabilities in generalizing the underlying non-linear dynamics of such systems, unlocking its predictive power is often restrained by the limited availability of large, curated datasets. To supplement real-world data, informing machine learning by transfer learning with synthetic data derived from simulations using ordinary differential equations has emerged as a promising solution. However, the success of this approach highly depends on the designed characteristics of the synthetic data.
Results: We suggest scrutinizing these characteristics, such as size, diversity, and noise, of ordinary differential equation-based synthetic time series datasets. Here, we demonstrate how to systematically evaluate the influence of such design choices on transfer learning performance. We conduct a proof-of-concept study on three simple, but widely used systems and four real-world datasets. We find a strong interdependency between synthetic dataset size and diversity effects. Good transfer learning settings heavily rely on real-world data characteristics as well as the data's coherence with the dynamics of the model underlying the synthetic data. We achieve a performance improvement of up to 95% in mean absolute error for simulation-based transfer learning compared to non-informed deep learning.
Conclusions: Our work emphasizes the relevance of carefully selecting properties of synthetic data for leveraging the valuable domain knowledge contained in ordinary differential equation models for machine-learning based predictions. The code is available at https://github.com/DILiS-lab/opt-synthdata-4tl .
(© 2026. The Author(s).)
Competing Interests: Declarations. Ethics approval and consent to participate: Not applicable. Consent for publication: Not applicable. Competing interests: The authors declare that they have no conflict of interest.
References: Box GEP, Jenkins GM, Reinsel GC, Ljung GM. Time series analysis: forecasting and control. Hoboken: Wiley; 2015.
Siami-Namini S, Tavakoli N, Siami Namin A. A comparison of arima and lstm in forecasting time series. In: 2018 17th IEEE International conference on machine learning and applications (ICMLA), 2018. pp. 1394–1401. https://doi.org/10.1109/ICMLA.2018.00227.
Benzekry S. Artificial intelligence and mechanistic modeling for clinical decision making in oncology. Clin Pharmacol Ther. 2020;108(3):471–86. (PMID: 3255759810.1002/cpt.1951)
Martin L, Hutchens M, Hawkins C, Radnov A. How much do clinical trials cost? Nat Rev Drug Discov. 2017;16(6):381–2. (PMID: 2852931710.1038/nrd.2017.70)
Sapoval N, Aghazadeh A, Nute MG, Antunes DA, Balaji A, Baraniuk R, et al. Current progress and open challenges for applying deep learning across the biosciences. Nat Commun. 2022;13(1):1728. (PMID: 35365602897601210.1038/s41467-022-29268-7)
Walonoski J, Kramer M, Nichols J, Quina A, Moesel C, Hall D, et al. Synthea: an approach, method, and software mechanism for generating synthetic patients and the synthetic electronic health care record. J Am Med Inform Assoc. 2018;25(3):230–8. https://doi.org/10.1093/jamia/ocx079 . (PMID: 10.1093/jamia/ocx079290251447651916)
Pietz T, Gupta S, Schlaffner CN, Ahmed S, Steen H, Renard BY, et al. PEPerMINT: peptide abundance imputation in mass spectrometry-based proteomics using graph neural networks. Bioinformatics. 2024;40(Supplement-2):70–8. https://doi.org/10.1093/bioinformatics/btae389 . (PMID: 10.1093/bioinformatics/btae389)
Carré C, Mas A, Krouk G. Reverse engineering highlights potential principles of large gene regulatory network design and learning. npj Syst Biol Appl. 2017;3(1):1–15. https://doi.org/10.1038/s41540-017-0019-y . (PMID: 10.1038/s41540-017-0019-y)
Matsunaga Y, Sugita Y. Linking time-series of single-molecule experiments with molecular dynamics simulations by machine learning. Elife. 2018;7:32668. https://doi.org/10.7554/eLife.32668 . (PMID: 10.7554/eLife.32668)
Wang R, Kashinath K, Mustafa M, Albert A, Yu R. Towards physics-informed deep learning for turbulent flow prediction. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, KDD ’20, New York: Association for Computing Machinery; 2020. pp. 1457–1466.
Goncalves A, Ray P, Soper B, Stevens J, Coyle L, Sales AP. Generation and evaluation of synthetic patient data. BMC Med Res Methodol. 2020;20(1):108. https://doi.org/10.1186/s12874-020-00977-1 . (PMID: 10.1186/s12874-020-00977-1323810397204018)
Zhang J, Petersen SD, Radivojevic T, Ramirez A, Pérez-Manríquez A, Abeliuk E, et al. Combining mechanistic and machine learning models for predictive engineering and optimization of tryptophan metabolism. Nat Commun. 2020;11(1):4880. https://doi.org/10.1038/s41467-020-17910-1 . (PMID: 10.1038/s41467-020-17910-1329783757519671)
Le Novère N, Bornstein B, Broicher A, Courtot M, Donizelli M, Dharuri H, et al. BioModels database: a free, centralized database of curated, published, quantitative kinetic models of biochemical and cellular systems. Nucleic Acids Res. 2006;34(Database issue):689–91. (PMID: 10.1093/nar/gkj092)
Yazdani A, Lu L, Raissi M, Karniadakis GE. Systems biology informed deep learning for inferring parameters and hidden dynamics. PLoS Comput Biol. 2020;16(11):1007575. https://doi.org/10.1371/journal.pcbi.1007575 . (PMID: 10.1371/journal.pcbi.1007575)
Costello Z, Martin HG. A machine learning approach to predict metabolic pathway dynamics from time-series multiomics data. NPJ Syst Biol Appl. 2018;4:19. (PMID: 29872542597430810.1038/s41540-018-0054-3)
Bulcke T, Van Leemput K, Naudts B, Remortel P, Ma H, Verschoren A, et al. SynTReN: a generator of synthetic gene expression data for design and analysis of structure learning algorithms. BMC Bioinform. 2006;7(1):43. https://doi.org/10.1186/1471-2105-7-43 . (PMID: 10.1186/1471-2105-7-43)
Schaffter T, Marbach D, Floreano D. GeneNetWeaver: in silico benchmark generation and performance profiling of network inference methods. Bioinformatics. 2011;27(16):2263–70. https://doi.org/10.1093/bioinformatics/btr373 . (PMID: 10.1093/bioinformatics/btr37321697125)
Sherman TD, Kagohara LT, Cao R, Cheng R, Satriano M, Considine M, et al. CancerInSilico: an R/Bioconductor package for combining mathematical and statistical modeling to simulate time course bulk and single cell gene expression data in cancer. PLoS Comput Biol. 2019;14(4):1–12. https://doi.org/10.1371/journal.pcbi.1006935 . (PMID: 10.1371/journal.pcbi.1006935)
Von Rueden L, Mayer S, Beckh K, Georgiev B, Giesselbach S, Heese R, et al. Others: Informed machine learning-a taxonomy and survey of integrating prior knowledge into learning systems. IEEE TKDE. 2021;35(1):614–33.
Procopio A, Cesarelli G, Donisi L, Merola A, Amato F, Cosentino C. Combined mechanistic modeling and machine-learning approaches in systems biology - a systematic literature review. Comput Methods Programs Biomed. 2023;240:107681. (PMID: 3738514210.1016/j.cmpb.2023.107681)
Chen RJ, Lu MY, Chen TY, Williamson DF, Mahmood F. Synthetic data in machine learning for medicine and healthcare. Nat Biomed Eng. 2021;5(6):493–7. https://doi.org/10.1038/s41551-021-00751-8 . (PMID: 10.1038/s41551-021-00751-8341313249353344)
Selvarajoo K, Maurer-Stroh S. Towards multi-omics synthetic data integration. Brief Bioinform. 2024;25(3):213. https://doi.org/10.1093/bib/bbae213 . (PMID: 10.1093/bib/bbae213)
Lancaster MC, Sobie EA. Improved prediction of drug-induced torsades de pointes through simulations of dynamics and machine learning algorithms. Clin Pharmacol Ther. 2016;100(4):371–9. (PMID: 26950176637529810.1002/cpt.367)
Deist TM, Patti A, Wang Z, Krane D, Sorenson T, Craft D. Simulation-assisted machine learning. Bioinformatics. 2019;35(20):4072–80. (PMID: 30903692679206410.1093/bioinformatics/btz199)
Pan SJ, Yang Q. A survey on transfer learning. IEEE Trans Knowl Data Eng. 2010;22(10):1345–59. (PMID: 10.1109/TKDE.2009.191)
Weber M, Auch M, Doblander C, Mandl P, Jacobsen H-A. Transfer learning with time series data: a systematic mapping study. IEEE Access. 2021;9:165409–32. (PMID: 10.1109/ACCESS.2021.3134628)
Mignone P, Pio G, Ceci M. Distributed heterogeneous transfer learning. Big Data Res. 2024;37:100456. https://doi.org/10.1016/j.bdr.2024.100456 . (PMID: 10.1016/j.bdr.2024.100456)
Roster K, Connaughton C, Rodrigues FA. Forecasting new diseases in low-data settings using transfer learning. Chaos, Solitons Fractals. 2022;161:112306. (PMID: 35765601922234810.1016/j.chaos.2022.112306)
Panagopoulos G, Nikolentzos G, Vazirgiannis M. Transfer graph neural networks for pandemic forecasting. AAAI. 2021;35(6):4838–45. (PMID: 10.1609/aaai.v35i6.16616)
Gautam Y. Transfer learning for COVID-19 cases and deaths forecast using LSTM network. ISA Trans. 2022;124:41–56. (PMID: 3342233010.1016/j.isatra.2020.12.057)
Sharifi-Noghabi H, Peng S, Zolotareva O, Collins CC, Ester M. AITL: adversarial inductive transfer learning with input and output space adaptation for pharmacogenomics. Bioinformatics. 2020;36:380–8. (PMID: 10.1093/bioinformatics/btaa442)
Geroski T, Ranković V, Pavić O, Dašić L, Petrović M, Milovanović D, et al. Enhancing covid-19 disease severity classification through advanced transfer learning techniques and optimal weight initialization schemes. Biomed Signal Process Control. 2025;100:107103. https://doi.org/10.1016/j.bspc.2024.107103 . (PMID: 10.1016/j.bspc.2024.107103)
Fenoy E, Edera AA, Stegmayer G. Transfer learning in proteins: evaluating novel protein learned representations for bioinformatics tasks. Brief Bioinform. 2022;23(4):232. https://doi.org/10.1093/bib/bbac232 . (PMID: 10.1093/bib/bbac232)
Höll S, Humer C. Deep predictions and transfer learning for simulation-driven structural health monitoring based on guided waves. Expert Syst Appl. 2024;238:122133. https://doi.org/10.1016/j.eswa.2023.122133 . (PMID: 10.1016/j.eswa.2023.122133)
Tercan H, Guajardo A, Heinisch J, Thiele T, Hopmann C, Meisen T. Transfer-learning: Bridging the gap between real and simulation data for machine learning in injection molding. Procedia CIRP. 2018;72:185–90. https://doi.org/10.1016/j.procir.2018.03.087 . (PMID: 10.1016/j.procir.2018.03.087)
Daw A, Karpatne A, Watkins W, Read J, Kumar V. Physics-guided neural networks (PGNN): an application in lake temperature modeling 2017. arXiv:1710.11431 [cs.LG].
Steinacker M, Kheifetz Y, Scholz M. Individual modelling of haematotoxicity with NARX neural networks: a knowledge transfer approach. Heliyon. 2023;9(7):17890. (PMID: 10.1016/j.heliyon.2023.e17890)
Steinacker M, Kheifetz Y, Scholz M. Predicting chemotherapy-induced thrombotoxicity by NARX neural networks and transfer learning. J Cancer Res Clin Oncol. 2024;150(10):457. https://doi.org/10.1007/s00432-024-05985-y . (PMID: 10.1007/s00432-024-05985-y3939711111471701)
Kleissl M, Drews L, Heyder BB, Zabbarov J, Iversen P, Witzke S, Renard BY, Baum K. SimbaML: Connecting mechanistic models and machine learning with augmented data. In: ICLR Tiny Paper 2023. https://openreview.net/forum?id=1wtUadpmVzu.
Fröhlich F, Kessler T, Weindl D, Shadrin A, Schmiester L, Hache H, et al. Efficient parameter estimation enables the prediction of drug response using a mechanistic Pan-Cancer pathway model. Cell Syst. 2018;7(6):567–5796. (PMID: 3050364710.1016/j.cels.2018.10.013)
Cranmer K, Brehmer J, Louppe G. The frontier of simulation-based inference. Proc Natl Acad Sci. 2020;117(48):30055–62. https://doi.org/10.1073/pnas.1912789117 . (PMID: 10.1073/pnas.1912789117324719487720103)
Roth M, Hendeby G, Fritsche C, Gustafsson F. The Ensemble Kalman filter: a signal processing perspective. EURASIP J Adv Signal Process. 2017;2017(1):56. https://doi.org/10.1186/s13634-017-0492-x . (PMID: 10.1186/s13634-017-0492-x)
Fawaz HI, Forestier G, Weber J, Idoumghar L, Muller P-A. Transfer learning for time series classification. In: 2018 IEEE international conference on big data (Big Data), pp. 1367–1376. https://doi.org/10.1109/BigData.2018.8621990.
Bishop CM. Neural networks for pattern recognition. New York: Oxford University Press; 1995. (PMID: 10.1093/oso/9780198538493.001.0001)
Fuchs C. Inference for diffusion processes: with applications in life sciences. Berlin Heidelberg: Springer; 2013. (PMID: 10.1007/978-3-642-25969-2)
Willmott CJ, Matsuura K. Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Climate Res. 2005;30(1):79–82. (PMID: 10.3354/cr030079)
Blaskowitz O, Herwartz H. On economic evaluation of directional forecasts. Int J Forecast. 2011;27(4):1058–65. (PMID: 10.1016/j.ijforecast.2010.07.002)
Kunsch HR. The jackknife and the bootstrap for general stationary observations. Ann Stat. 1989;17(3):1217–41. https://doi.org/10.1214/aos/1176347265 . (PMID: 10.1214/aos/1176347265)
Berndt DJ, Clifford J. Using dynamic time warping to find patterns in time series. Proc. 3rd KDD, 1994; pp .359–370.
Keogh EJ, Pazzani MJ. Derivative dynamic time warping. Proceedings of the 2001 SIAM international conference on data mining (SDM), 2001; pp. 1–11. https://doi.org/10.1137/1.9781611972719.1.
Kermack WO, McKendrick AG. A contribution to the mathematical theory of epidemics. Proc R Soc Lond A Math Phys Sci. 1927;115(772):700–21.
Rosenbaum B, Raatz M, Weithoff G, Fussmann GF, Gaedke U. Estimating parameters from multiple time series of population dynamics using Bayesian inference. Front Ecol Evol. 2019;6:234. (PMID: 10.3389/fevo.2018.00234)
Lotka AJ. Analytical note on certain rhythmic relations in organic systems. Proc Natl Acad Sci U S A. 1920;6(7):410–5. (PMID: 16576509108456210.1073/pnas.6.7.410)
Volterra V. Fluctuations in the abundance of a species considered mathematically. Nature. 1926;118:558–60. (PMID: 10.1038/118558a0)
Robert Koch-Institut: SARS-CoV-2 Infektionen in Deutschland. Zenodo. Version 2022-09-02 2022. https://doi.org/10.5281/zenodo.7042186.
Blasius B, Rudolf L, Weithoff G, Gaedke U, Fussmann GF. Long-term cyclic persistence in an experimental predator-prey system. Nature. 2019;577(7789):226–30. (PMID: 3185306410.1038/s41586-019-1857-0)
MacLulich DA. Fluctuations in the numbers of the varying hare (Lepus Americanus). Toronto: University of Toronto Press; 1937. (PMID: 10.3138/9781487583064)
Hewitt CG. The conservation of the wild life of Canada. New York: C. Scribner; 1921.
Gilpin ME. Do hares eat lynx? Am Nat. 1973;107(957):727–30. (PMID: 10.1086/282870)
Dehning J, Zierenberg J, Spitzner FP, Wibral M, Neto JP, Wilczek M, et al. Inferring change points in the spread of COVID-19 reveals the effectiveness of interventions. Science. 2020;369(6500):eabb9789. (PMID: 32414780723933110.1126/science.abb9789)
Carpenter B. Predator-Prey Population Dynamics: the Lotka-Volterra model in Stan. https://mc-stan.org/users/documentation/case-studies/lotka-volterra-predator-prey.html . (2018). Accessed 05 July 2023.
Tyson JJ, Chen KC, Novak B. Sniffers, buzzers, toggles and blinkers: dynamics of regulatory and signaling pathways in the cell. Curr Opin Cell Biol. 2003;15(2):221–31. (PMID: 1264867910.1016/S0955-0674(03)00017-6)
Liu C, Lu J, Li P, Feng S, Guo Y, Li K, et al. A comparative study on epidemiological characteristics, transmissibility, and pathogenicity of three covid-19 outbreaks caused by different variants. Int J Infect Dis. 2023;134:78–87. (PMID: 36736993989080610.1016/j.ijid.2023.01.039)
De Visser JAG, Krug J. Empirical fitness landscapes and the predictability of evolution. Nat Rev Genet. 2014;15(7):480–90. https://doi.org/10.1038/nrg3744 . (PMID: 10.1038/nrg374424913663)
Mohammadi Z, Cojocaru MG, Thommes EW. Human behaviour, NPI and mobility reduction effects on COVID-19 transmission in different countries of the world. BMC Public Health. 2022;22(1):1594. https://doi.org/10.1186/s12889-022-13921-3 . (PMID: 10.1186/s12889-022-13921-3359961329394048)
Shorten C, Khoshgoftaar TM. A survey on image data augmentation for deep learning. J Big Data. 2019;6(1):1. (PMID: 10.1186/s40537-019-0197-0)
Bishop CM. Training with noise is equivalent to tikhonov regularization. Neural Comput. 1995;7(1):108–16. https://doi.org/10.1162/neco.1995.7.1.108 . (PMID: 10.1162/neco.1995.7.1.108)
Gleeson JP. Binary-state dynamics on complex networks: Pair approximation and beyond. Phys Rev X. 2013;3:021004. https://doi.org/10.1103/PhysRevX.3.021004 . (PMID: 10.1103/PhysRevX.3.021004)
Witzke S, Danz N, Baum K, Renard BY. Mobility data improve forecasting of COVID-19 incidence trends using graph neural networks. In: epiDAMIK 6.0 at KDD 2023.
Qiu X, Wang H, Tan X, Jin Y. Cvdllm: Automated cardiovascular disease diagnosis with large-language-model-assisted graph attentive feature interaction. IEEE Trans Artif Intell. 2025;6(6):1575–90. https://doi.org/10.1109/TAI.2025.3527401 . (PMID: 10.1109/TAI.2025.3527401)
Chen Y, Céspedes N, Barnaghi P. A closer look at transformers for time series forecasting: Understanding why they work and where they struggle. In: Forty-second international conference on machine learning 2025. https://openreview.net/forum?id=kHEVCfES4Q.
Chen RTQ, Rubanova Y, Bettencourt J, Duvenaud D. Neural ordinary differential equations. In: Proceedings of the 32nd international conference on neural information processing systems. NIPS’18, pp. 6572–6583. Curran Associates Inc., Red Hook, NY, USA 2018.
Jin M, Wang S, Ma L, Chu Z, Zhang JY, Shi X, Chen P-Y, Liang Y, Li Y-F, Pan S, Wen Q. Time-LLM: Time series forecasting by reprogramming large language models. In: The twelfth international conference on learning representations 2024. https://openreview.net/forum?id=Unb5CVPtae.
Hass H, Loos C, Raimúndez-Álvarez E, Timmer J, Hasenauer J, Kreutz C. Benchmark problems for dynamic modeling of intracellular processes. Bioinformatics. 2019;35(17):3073–82. (PMID: 30624608673586910.1093/bioinformatics/btz020)
Brockmann D, Helbing D. The hidden geometry of complex, network-driven contagion phenomena. Science. 2013;342(6164):1337–42. (PMID: 2433728910.1126/science.1245200)
Rolnick D, Donti PL, Kaack LH, Kochanski K, Lacoste A, Sankaran K, et al. Tackling climate change with machine learning. ACM Comput Surv. 2022;55(2):1–96. (PMID: 10.1145/3485128)
Karniadakis GE, Kevrekidis IG, Lu L, Perdikaris P, Wang S, Yang L. Physics-informed machine learning. Nat Rev Phys. 2021;3(6):422–40. (PMID: 10.1038/s42254-021-00314-5)
Lagergren JH, Nardini JT, Baker RE, Simpson MJ, Flores KB. Biologically-informed neural networks guide mechanistic modeling from sparse experimental data. PLoS Comput Biol. 2020;16(12):1008462. (PMID: 10.1371/journal.pcbi.1008462)
Gao J, Sharma R, Qian C, Glass LM, Spaeder J, Romberg J, et al. STAN: spatio-temporal attention network for pandemic prediction using real-world evidence. J Am Med Inform Assoc. 2021;28(4):733–43. (PMID: 33486527792893510.1093/jamia/ocaa322)
Rodríguez A, Cui J, Ramakrishnan N, Adhikari B, Aditya PB. EINNs: Epidemiologically-Informed neural networks. AAAI. 2023;37(12):14453–60. (PMID: 10.1609/aaai.v37i12.26690)
Wang H, Qiu X, Yang J, Li Q, Tan X, Huang J. Neural-seir: A flexible data-driven framework for precise prediction of epidemic disease. Math Biosci Eng. 2023;20(9):16807–23. https://doi.org/10.3934/mbe.2023749 . (PMID: 10.3934/mbe.202374937920035)
Contributed Indexing: Keywords: Epidemiology; Informed machine learning; Ordinary differential equations; Predator–prey; Time series; Transfer learning
Entry Date(s): Date Created: 20260519 Date Completed: 20260717 Latest Revision: 20260717
Update Code: 20260717
PubMed Central ID: PMC13188243
DOI: 10.1186/s12859-026-06469-1
PMID: 42151772
Βάση Δεδομένων: MEDLINE
Περιγραφή
ISSN:1471-2105
DOI:10.1186/s12859-026-06469-1