An Enhanced Retrieval Scheme for a Large Language Model with a Joint Strategy of Probabilistic Relevance and Semantic Association in the Vertical Domain

文献类型： 外文期刊

第一作者： Chen, Qi

作者： Chen, Qi;Cheng, Jian;Yang, Ji;Chen, Qi;Zhou, Weifeng;Cheng, Jian;Yang, Ji

作者机构：

关键词： large language model; information retrieval; BM25; retrieval-augmented generation

期刊名称：APPLIED SCIENCES-BASEL （ 影响因子：2.5； 五年影响因子：2.7 ）

ISSN：

年卷期： 2024 年 14 卷 24 期

页码：

收录情况： SCI

摘要： Large language model (LLM) processing, with natural language as its core, carries out information retrieval through intelligent Q&A. It has a wide range of application scenarios and is commonly considered a kind of generative AI. However, when LLMs handle generation tasks, the results generated by fundamental LLMs with an insufficient comprehensive performance, specifically in the vertical domain, are often inaccurate due to a poor generalization ability, resulting in the so-called "hallucination" phenomenon. To solve these problems, in this study, an enhanced retrieval scheme for LLM processing was developed, named the BM-RAGAM (BM25 retrieval-augmented generation attention mechanism), by constructing a vectorized knowledge base, utilizing a hybrid joint retrieval strategy of keyword matching through searching and a semantic-enhanced association with an attention mechanism and taking ocean-front- and eddy-related knowledge in oceanography as an example. This scheme realized accurate word-based matching with the BM25 algorithm and text generation through a semantic-enhanced association using RAG, and it was used to construct a vector database of the text knowledge on ocean fronts and eddies. The output was compared and analyzed with the fundamental LLM of Qwen2-72B using the proposed scheme, and an ablation experiment was conducted. The results show that the proposed scheme greatly reduced hallucination generation in the process of text generation, making its outputs more interpretable.

分类号：

  • 相关文献
作者其他论文 更多>>

微信小程序