Jadwal Sholat

Memuat jadwal sholatโ€ฆ

Kedokteran & Kesehatan editorial

Open AccessOA2022

A comparative study of pre-trained language models for named entity recognition in clinical trial eligibility criteria from multiple corpora

This study fine-tunes six pre-trained language models for named entity recognition (NER) on clinical trial eligibility criteria across three corpora. Domain-specific models, especially PubMedBERT, outperform general models, and data augmentation from additional corpora improves NER performance.
Jianfu Li; Qiang Wei; O. Ghiasvand; Miao Chen; V. Lobanov; Chun-Hua Weng; Hua Xuยท BMC Medical Informatics and Decision Makingยท 2022ยท DOI 10.1186/s12911-022-01967-7

The core problem

Clinical trial protocols are essential for advancing medical sciences, but extracting accurate information from their complex and unstructured texts is challenging. Named entity recognition (NER) is a fundamental step to process and standardize such text using natural language processing (NLP). This study investigates the effectiveness of pre-trained language models for NER on clinical trial eligibility criteria. The authors systematically compare four biomedical domain-specific models (BioBERT, BlueBERT, PubMedBERT, SciBERT) and two general-domain models (BERT, SpanBERT) across three existing corpora. Additionally, they explore data augmentation approaches to enhance performance. The research question centers on whether domain-specific contextual embeddings yield better NER results than general embeddings, and whether leveraging multiple corpora can mitigate data scarcity.

Innovation

The study fine-tunes six pre-trained transformer models for token classification on three clinical trial eligibility criteria corpora. The corpora are not named in the abstract but are existing datasets. The models include:

- **Domain-specific**: BioBERT, BlueBERT, PubMedBERT, SciBERT
- **General**: BERT, SpanBERT

Fine-tuning is performed with standard NER setup, and evaluation uses tenfold cross-validation. Performance is measured by -score. Data augmentation is investigated by combining training data from multiple corpora to assess feasibility of leveraging additional sources. The exact augmentation techniques (e.g., oversampling, synonym replacement) are not detailed in the abstract, but the results indicate that using extra corpora improves NER performance.

Introduction
Clinical trial protocols are essential for advancing medical sciences, but extracting accurate information from their complex and unstructured texts is challenging. Named entity recognition (NER) is a fundamental step to process and standardize such text using natural language processing (NLP). This study investigates the effectiveness of pre-trained language models for NER on clinical trial eligibility criteria. The authors systematically compare four biomedical domain-specific models (BioBERT, BlueBERT, PubMedBERT, SciBERT) and two general-domain models (BERT, SpanBERT) across three existing corpora. Additionally, they explore data augmentation approaches to enhance performance. The research question centers on whether domain-specific contextual embeddings yield better NER results than general embeddings, and whether leveraging multiple corpora can mitigate data scarcity.

Why it matters

The findings underscore the importance of contextual embeddings trained from domain-specific corpora for biomedical NER tasks. PubMedBERT's superior performance likely stems from its pre-training on PubMed abstracts and full-text articles, which closely match the language of clinical trial eligibility criteria. The results also highlight the benefits of leveraging multiple data sources to address the challenging NER task in clinical trial text. The study provides insights for practitioners: when annotated data is scarce, domain-specific models and data augmentation can significantly boost performance. Limitations include the abstract's lack of detail on corpora specifics and augmentation methods, but the consistent trend across three corpora strengthens the conclusions.

Who should read this

Pembaca Research Digest dan praktisi rumpun ini.

Opening member contentโ€ฆ