Enhancing Differential Testing with LLMs for Testing Deep Learning Libraries

Authors: Meiziniu Li, Dongze Li, Jianmeng Liu, Jialun Cao, Yongqiang Tian, Shing-Chi Cheung

Source: ACM Transactions on Software Engineering and Methodology. 35:1-39

Subject Terms: Software Engineering (cs.SE), FOS: Computer and information sciences, Computer Science - Software Engineering, Artificial Intelligence (cs.AI), I.2.5, Computer Science - Artificial Intelligence, D.2.5

Access URL: http://arxiv.org/abs/2406.07944

Automated Unit Test Generation via Chain-of-Thought Prompt and Reinforcement Learning from Coverage Feedback

Authors: Junwei Zhang, Xing Hu, Xin Xia, Shing-Chi Cheung, Shanping Li

Source: ACM Transactions on Software Engineering and Methodology. 35:1-30

LspFuzz: Hunting Bugs in Language Servers

Authors: Hengcheng Zhu 0001, Songqiang Chen, Valerio Terragni, Lili Wei 0001, Yepang Liu 0001, Jiarong Wu, Shing-Chi Cheung

Source: 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE). :2209-2221

Subject Terms: Software Engineering (cs.SE), FOS: Computer and information sciences, Cryptography and Security, D.2.5, Software Engineering, Cryptography and Security (cs.CR)

Access URL: http://arxiv.org/abs/2510.00532

Cross2OH: Enabling Seamless Porting of C/C++ Software Libraries to OpenHarmony

Authors: Qian Zhang, Tsz-On Li, Ying Wang 0038, Li Li 0029, Shing-Chi Cheung

Source: 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE). :1744-1755

Demystifying Cross-Language C/C++ Binaries: A Robust Software Component Analysis Approach

Authors: Meiqiu Xu, Ying Wang 0038, Wei Tang, Xian Zhan, Shing-Chi Cheung, Hai Yu 0001, Zhiliang Zhu 0001

Source: 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE). :1768-1780

CodeCleaner: Mitigating Data Contamination for LLM Benchmarking

Authors: Jialun Cao, Songqiang Chen, Wuqi Zhang, Hau Ching Lo, Yeting Li, Shing-Chi Cheung

Source: Proceedings of the 16th International Conference on Internetware. :71-83

A Tale of Two DL Cities: When Library Tests Meet Compiler

Authors: Qingchao Shen, Yongqiang Tian 0001, Haoyang Ma, Junjie Chen 0003, Lili Huang, Ruifeng Fu, Shing-Chi Cheung, Zan Wang

Source: 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE). :2201-2212

Subject Terms: Software Engineering (cs.SE), FOS: Computer and information sciences, Computer Science - Software Engineering

Access URL: http://arxiv.org/abs/2407.16626

DOMAINEVAL: An Auto-Constructed Benchmark for Multi-Domain Code Generation

Authors: Qiming Zhu, Jialun Cao, Yaojie Lu 0001, Hongyu Lin, Xianpei Han, Le Sun 0001, Shing-Chi Cheung

Source: Proceedings of the AAAI Conference on Artificial Intelligence. 39:26148-26156

Subject Terms: Software Engineering (cs.SE), FOS: Computer and information sciences, Computer Science - Software Engineering, Artificial Intelligence (cs.AI), Computer Science - Artificial Intelligence

Access URL: http://arxiv.org/abs/2408.13204

Differential Testing of Concurrent Classes

Authors: Valerio Terragni, Shing-Chi Cheung

Source: 2025 IEEE Conference on Software Testing, Verification and Validation (ICST). :255-266

An Empirical Study of Bugs in Data Visualization Libraries

Authors: Weiqi Lu, Yongqiang Tian 0001, Xiaohan Zhong, Haoyang Ma, Zhenyang Xu, Shing-Chi Cheung, Chengnian Sun

Source: Proceedings of the ACM on Software Engineering. 2:2075-2098

Subject Terms: Human-Computer Interaction, Software Engineering (cs.SE), FOS: Computer and information sciences, Computer Vision and Pattern Recognition (cs.CV), Software Engineering, Computer Vision and Pattern Recognition, Human-Computer Interaction (cs.HC)

Access URL: http://arxiv.org/abs/2506.15084

SemBIC: Semantic-Aware Identification of Bug-Inducing Commits

Authors: Xiao Chen 0026, Hengcheng Zhu 0001, Jialun Cao, Ming Wen 0001, Shing-Chi Cheung

Source: Proceedings of the ACM on Software Engineering. 2:1363-1385

Understanding and Characterizing Mock Assertions in Unit Tests

Authors: Hengcheng Zhu 0001, Valerio Terragni, Lili Wei 0001, Shing-Chi Cheung, Jiarong Wu, Yepang Liu 0001

Source: Proceedings of the ACM on Software Engineering. 2:554-575

Subject Terms: Software Engineering (cs.SE), FOS: Computer and information sciences, Computer Science - Software Engineering

Access URL: http://arxiv.org/abs/2503.19284

Question Selection for Multimodal Code Search Synthesis Using Probabilistic Version Spaces

Authors: Jiarong Wu, Yanyan Jiang 0001, Lili Wei 0001, Congying Xu, Shing-Chi Cheung, Chang Xu 0001

Source: IEEE Transactions on Software Engineering. 51:1724-1744

Less Is More: Failing Test Generation with Large Language Models

Authors: Tsz-On Li, Xiaochuan Yan, Wenxi Zong, Yibo Wang, Haoye Tian, Congying Xu, Ying Wang, Shing-Chi Cheung, Jeff Kramer

Source: ACM Transactions on Software Engineering and Methodology.

From Informal to Formal – Incorporating and Evaluating LLMs on Natural Language Requirements to Verifiable Formal Proofs

Authors: Jialun Cao, Yaojie Lu 0001, Meiziniu Li, Haoyang Ma, Haokun Li, Mengda He, Cheng Wen 0002, Le Sun 0001, Hongyu Zhang 0002, Shengchao Qin, Shing-Chi Cheung, Cong Tian 0001

Source: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). :26984-27003

Subject Terms: FOS: Computer and information sciences, Computer Science - Computation and Language, Computer Science - Programming Languages, Artificial Intelligence (cs.AI), Computer Science - Artificial Intelligence, Computation and Language (cs.CL), Programming Languages (cs.PL)

Access URL: http://arxiv.org/abs/2501.16207
https://aclanthology.org/2025.acl-long.1310/

CRUXEVAL-X: A Benchmark for Multilingual Code Reasoning, Understanding and Execution

Authors: Ruiyang Xu, Jialun Cao, Yaojie Lu 0001, Ming Wen 0001, Hongyu Lin, Xianpei Han, Ben He 0001, Shing-Chi Cheung, Le Sun 0001

Source: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). :23762-23779

Subject Terms: FOS: Computer and information sciences, Artificial Intelligence (cs.AI), Computer Science - Artificial Intelligence

Access URL: http://arxiv.org/abs/2408.13001
https://aclanthology.org/2025.acl-long.1158/

JavaBench: A Benchmark of Object-Oriented Code Generation for Evaluating Large Language Models

Authors: Jialun Cao, Zhiyong Chen, Jiarong Wu, Shing-Chi Cheung, Chang Xu 0001

Source: Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering. :870-882

Subject Terms: Software Engineering (cs.SE), FOS: Computer and information sciences, Computer Science - Machine Learning, Computer Science - Software Engineering, Computer Science - Programming Languages, Artificial Intelligence (cs.AI), Computer Science - Artificial Intelligence, Machine Learning (cs.LG), Programming Languages (cs.PL)

Access URL: http://arxiv.org/abs/2406.12902

MR-Adopt: Automatic Deduction of Input Transformation Function for Metamorphic Testing

Authors: Congying Xu, Songqiang Chen, Jiarong Wu, Shing-Chi Cheung, Valerio Terragni, Hengcheng Zhu, Jialun Cao

Source: Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering. :557-569

Subject Terms: Software Engineering (cs.SE), FOS: Computer and information sciences, Software Engineering

Access URL: https://arxiv.org/abs/2408.15815
http://arxiv.org/abs/2408.15815

Towards Understanding the Bugs in Solidity Compiler

Authors: Haoyang Ma, Wuqi Zhang, Qingchao Shen, Yongqiang Tian 0001, Junjie Chen 0003, Shing-Chi Cheung

Source: Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. :1312-1324

Subject Terms: Software Engineering (cs.SE), FOS: Computer and information sciences, Computer Science - Software Engineering

Access URL: http://arxiv.org/abs/2407.05981

Can Emulating Semantic Translation Help LLMs with Code Translation? A Study Based on Pseudocode

Authors: Songqiang Chen, Congying Xu, Jingyi Chen, Jialun Cao, Jiarong Wu, Shing-Chi Cheung

Source: ACM Transactions on Software Engineering and Methodology.

Subject Terms: Software Engineering (cs.SE), FOS: Computer and information sciences, Software Engineering

Access URL: http://arxiv.org/abs/2510.00920