Academic Journal
Search at Scale: Quantifying the Performance Trade-offs of ANNS on Milvus.
| Title: | Search at Scale: Quantifying the Performance Trade-offs of ANNS on Milvus. |
|---|---|
| Authors: | Turan, Melek1 (AUTHOR) melek.turan@ozdilek.com.tr, Peker, Musa1 (AUTHOR) musa.peker@ozdilek.com.tr, Kocakır, Nadir1 (AUTHOR) nadir.kocakir@ozdilek.com.tr |
| Source: | International Journal of Software Engineering & Knowledge Engineering. 9/30/2026, Vol. 36 Issue 12, p1673-1699. 27p. |
| Subject Terms: | *Distributed computing, Computer performance, Benchmark problems (Computer science) |
| Abstract: | Efficient similarity search over high-dimensional embeddings is a core requirement of modern AI systems, yet most evaluations of Approximate Nearest Neighbor Search (ANNS) methods overlook the system-level dynamics that govern real-world performance. This study presents a comprehensive benchmark of six widely used ANNS algorithms — FLAT, IVF-FLAT, IVF-SQ8, HNSW, DiskANN, and ScaNN — integrated into Milvus and evaluated across both standalone and distributed (k3s-based) deployments. To capture modality-dependent behavior, the analysis spans two medium-size datasets, each consisting of 1.6 million embeddings: a large Turkish text corpus and a visual dataset generated using a unified 768-dimensional representation space. Each algorithm is assessed using recall@10, latency, throughput, and tail-latency (tp50–tp99), alongside detailed profiling of CPU utilization, RAM consumption, and disk I/O during indexing and querying. The results reveal distinct operational signatures: ScaNN and IVF-SQ8 achieve the highest throughput — particularly for image embeddings; HNSW offers superior recall but imposes greater memory pressure; and DiskANN provides stable, memory-efficient performance with a bounded recall ceiling for text. Distributed experiments further show that graph-based indexes benefit from multi-node parallelism, while disk-aware methods maintain consistent performance. Overall, the findings demonstrate that ANNS selection requires a holistic evaluation that accounts for both retrieval quality and system-level resource behavior under realistic workloads. [ABSTRACT FROM AUTHOR] |
| Copyright of International Journal of Software Engineering & Knowledge Engineering is the property of World Scientific Publishing Company and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Business Source Index |
| FullText | Text: Availability: 0 |
|---|---|
| Header | DbId: bsx DbLabel: Business Source Index An: 193950458 RelevancyScore: 1435 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 1435 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Search at Scale: Quantifying the Performance Trade-offs of ANNS on Milvus. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Turan%2C+Melek%22">Turan, Melek</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> melek.turan@ozdilek.com.tr</i><br /><searchLink fieldCode="AR" term="%22Peker%2C+Musa%22">Peker, Musa</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> musa.peker@ozdilek.com.tr</i><br /><searchLink fieldCode="AR" term="%22Kocakır%2C+Nadir%22">Kocakır, Nadir</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> nadir.kocakir@ozdilek.com.tr</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22International+Journal+of+Software+Engineering+%26+Knowledge+Engineering%22">International Journal of Software Engineering & Knowledge Engineering</searchLink>. 9/30/2026, Vol. 36 Issue 12, p1673-1699. 27p. – Name: Subject Label: Subject Terms Group: Su Data: *<searchLink fieldCode="DE" term="%22Distributed+computing%22">Distributed computing</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+performance%22">Computer performance</searchLink><br /><searchLink fieldCode="DE" term="%22Benchmark+problems+%28Computer+science%29%22">Benchmark problems (Computer science)</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Efficient similarity search over high-dimensional embeddings is a core requirement of modern AI systems, yet most evaluations of Approximate Nearest Neighbor Search (ANNS) methods overlook the system-level dynamics that govern real-world performance. This study presents a comprehensive benchmark of six widely used ANNS algorithms — FLAT, IVF-FLAT, IVF-SQ8, HNSW, DiskANN, and ScaNN — integrated into Milvus and evaluated across both standalone and distributed (k3s-based) deployments. To capture modality-dependent behavior, the analysis spans two medium-size datasets, each consisting of 1.6 million embeddings: a large Turkish text corpus and a visual dataset generated using a unified 768-dimensional representation space. Each algorithm is assessed using recall@10, latency, throughput, and tail-latency (tp50–tp99), alongside detailed profiling of CPU utilization, RAM consumption, and disk I/O during indexing and querying. The results reveal distinct operational signatures: ScaNN and IVF-SQ8 achieve the highest throughput — particularly for image embeddings; HNSW offers superior recall but imposes greater memory pressure; and DiskANN provides stable, memory-efficient performance with a bounded recall ceiling for text. Distributed experiments further show that graph-based indexes benefit from multi-node parallelism, while disk-aware methods maintain consistent performance. Overall, the findings demonstrate that ANNS selection requires a holistic evaluation that accounts for both retrieval quality and system-level resource behavior under realistic workloads. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of International Journal of Software Engineering & Knowledge Engineering is the property of World Scientific Publishing Company and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=bsx&AN=193950458 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1142/S0218194026500130 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 27 StartPage: 1673 Subjects: – SubjectFull: Distributed computing Type: general – SubjectFull: Computer performance Type: general – SubjectFull: Benchmark problems (Computer science) Type: general Titles: – TitleFull: Search at Scale: Quantifying the Performance Trade-offs of ANNS on Milvus. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Turan, Melek – PersonEntity: Name: NameFull: Peker, Musa – PersonEntity: Name: NameFull: Kocakır, Nadir IsPartOfRelationships: – BibEntity: Dates: – D: 30 M: 09 Text: 9/30/2026 Type: published Y: 2026 Identifiers: – Type: issn-print Value: 02181940 Numbering: – Type: volume Value: 36 – Type: issue Value: 12 Titles: – TitleFull: International Journal of Software Engineering & Knowledge Engineering Type: main |
| ResultId | 1 |