Academic Journal
Large Language Model-Based Text-to-Cypher Generation for Question-Answering on Biomedical Knowledge Graphs.
| Title: | Large Language Model-Based Text-to-Cypher Generation for Question-Answering on Biomedical Knowledge Graphs. |
|---|---|
| Authors: | Seeha S; Technical University of Munich, TUM School of Medicine and Health, Institute for AI and Informatics in Medicine (AIIM), TUM University Hospital., Abdelmoula A; Technical University of Munich, TUM School of Medicine and Health, Institute for AI and Informatics in Medicine (AIIM), TUM University Hospital., Boeker M; Technical University of Munich, TUM School of Medicine and Health, Institute for AI and Informatics in Medicine (AIIM), TUM University Hospital., Modersohn L; Technical University of Munich, TUM School of Medicine and Health, Institute for AI and Informatics in Medicine (AIIM), TUM University Hospital., Pallaoro P; Technical University of Munich, TUM School of Medicine and Health, Institute for AI and Informatics in Medicine (AIIM), TUM University Hospital.; Technical University of Munich, TUM School of Medicine and Health, Data Integration Center, TUM University Hospital. |
| Source: | Studies in health technology and informatics [Stud Health Technol Inform] 2026 Sep 17; Vol. 340, pp. 179-186. |
| Publication Type: | Journal Article |
| Language: | English |
| Journal Info: | Publisher: IOS Press Country of Publication: Netherlands NLM ID: 9214582 Publication Model: Print Cited Medium: Internet ISSN: 1879-8365 (Electronic) Linking ISSN: 09269630 NLM ISO Abbreviation: Stud Health Technol Inform Subsets: MEDLINE |
| Imprint Name(s): | Original Publication: Amsterdam ; Washington, DC : IOS Press, 1991- |
| MeSH Terms: | Information Storage and Retrieval*/methods , Large Language Models* , Natural Language Processing*, Humans |
| Abstract: | Introduction: Knowledge graphs are widely adopted due to their flexible structure and ability to represent large-scale, heterogeneous data. However, as these graphs grow, the queries used to retrieve information become more complex, making efficient information retrieval and question answering more challenging. Objective: This work investigates (1) the impact of few-shot versus zero-shot prompting on Cypher query generation, and (2) performance differences between small (∼7B) and mid-size (∼32B) large language models (LLMs) on both of these approaches. Methods: We propose a system that translates natural language questions into Cypher queries for execution on a Neo4j knowledge graph and evaluates answer quality on the PrimeKG and BioHopR datasets. To address data protection and cost constraints, we focus on locally deployable LLMs that require no additional training, using zero-shot and few-shot prompting strategies. Results: 32B models generate syntactically valid Cypher queries using only the knowledge graph schema information. Few-shot prompting with four examples improves F1 score by 35.7 percentage points (∼62% relative improvement) compared to zero-shot prompting. While 7B models struggle to generate syntactically valid queries in the zero-shot setting, few-shot prompting substantially improves their performance, enabling high query syntax validity and F1 scores above 80%. Conclusion: Few-shot prompting heavily improves query generation quality and enables effective use of smaller models, supporting the practical deployment of locally hosted LLMs for knowledge graph question answering. |
| Contributed Indexing: | Keywords: Database Management Systems; Factual Database; Information Storage and Retrieval; Large Language Models; Natural Language Processing |
| Entry Date(s): | Date Created: 20260917 Date Completed: 20260917 Latest Revision: 20260917 |
| Update Code: | 20260918 |
| DOI: | 10.3233/SHTI261000 |
| PMID: | 42752482 |
| Database: | MEDLINE |
| ISSN: | 1879-8365 |
|---|---|
| DOI: | 10.3233/SHTI261000 |