Extracting Materials Science Knowledge: A Comprehensive Guide to Named Entity Recognition (NER)

Abstract

This article provides a comprehensive overview of Named Entity Recognition (NER) applications in materials science and related drug development fields. It explores the fundamental role of NER in automating the extraction of structured knowledge from vast scientific literature, covering key methodologies from traditional machine learning to advanced deep learning and large language models. The content details practical applications, including drug repurposing and material property database construction, addresses common challenges like data scarcity and semantic ambiguity, and offers comparative analyses of model performance. Designed for researchers, scientists, and drug development professionals, this guide serves as a vital resource for leveraging NER to accelerate discovery and innovation.

What is Materials Science NER and Why is it a Game-Changer for Research?

The rapid growth of scientific literature presents a formidable challenge for researchers, professionals, and institutions aiming to stay current with advancements in their fields. In materials science and drug development, this information overload creates significant bottlenecks in knowledge discovery and utilization. The majority of valuable scientific knowledge about specialized domains, such as solid-state materials, remains scattered across the text, tables, and figures of millions of academic research papers, making comprehensive understanding and effective leveraging of existing knowledge exceptionally difficult [1].

Named Entity Recognition (NER)—a subfield of natural language processing (NLP) that identifies and classifies entities in unstructured text into predefined categories—has emerged as a critical technological solution to this challenge [2]. By transforming unstructured text into structured, machine-readable data, NER enables the construction of knowledge graphs, facilitates information retrieval, and supports advanced analytics. The development of NER has evolved from early rule-based systems to modern approaches utilizing deep learning and large language models (LLMs), dramatically improving its capability to handle complex scientific information [2] [3].

The Scale of the Problem: Quantitative Analysis of Information Overload

The challenge of information overload is substantiated by quantitative data illustrating the exponential growth of scientific literature and the limitations of manual processing methods.

Table 1: Literature Growth and Processing Challenges in Scientific Domains

Metric	Findings	Implications for Research
General NER Publication Volume	Substantial growth in NER research publications over recent decades, with an explosion during 2018-2024 driven by Transformer-based models [2]	Indicates both the maturity of the field and the increasing recognition of its importance for managing textual data.
Materials Science Knowledge Dispersion	Majority of solid-state materials knowledge is scattered across millions of research papers [1]	Difficult for researchers to grasp the full body of past work and effectively leverage existing knowledge in experimental design.
Manual Extraction Limitations	Manual information extraction from vast text data is time-consuming and error-prone [3]	Creates scalability issues and potential for missed connections in scientific discovery.
Machine Learning Data Limitations	Machine learning models for property prediction are limited by available tabulated training data [1]	Restricts the potential of AI-driven materials discovery and design workflows.

The Evolution of NER: Methodologies and Technological Advances

Named Entity Recognition has undergone significant technological evolution, enhancing its capacity to address information overload in scientific domains.

Historical Development of NER Approaches

The progression of NER methodologies reflects a journey toward greater automation, accuracy, and contextual understanding.

Table 2: Evolution of NER Methodologies and Their Characteristics

Methodology	Time Period	Key Features	Limitations
Rule-Based Systems [2] [3]	Early-Mid 1990s	Hand-crafted rules, lexicons, and spelling features; high accuracy for specific domains	Labor-intensive, poor generalization, lack of flexibility and scalability
Machine Learning Approaches [2] [3]	~2000 onward	Statistical models (HMM, CRF), sequence labeling, more adaptable and data-driven	Requires large annotated corpora; limited ability to capture complex context
Deep Learning & Neural Networks [2] [4] [3]	2010s onward	Sophisticated pattern recognition (CNN, RNN), non-linear feature discovery	Computationally intensive; still requires significant labeled data
Transformer-Based Models & LLMs [2] [1]	2018-Present	Transfer learning, context-aware representations, few-shot learning capabilities	High computational requirements; complexity in fine-tuning and deployment

Advanced NER Architectures for Scientific Information Extraction

Modern NER systems increasingly employ sophisticated architectures specifically designed to handle the complexities of scientific text. The transition from pipeline approaches to joint named entity recognition and relation extraction (NERRE) represents a significant advancement. Pipeline methods process NER as a separate step before relation extraction, often leading to error propagation and loss of contextual information [1]. In contrast, joint NERRE approaches use a single model to simultaneously identify entities and their relationships, preserving critical scientific context [1].

More recently, Large Language Models (LLMs) have demonstrated remarkable capabilities in complex scientific information extraction. By fine-tuning pretrained LLMs on domain-specific text, researchers can create systems that extract structured knowledge hierarchies without requiring exhaustive enumeration of all possible entity relations [1]. This approach is particularly valuable for materials science, where knowledge is often inherently intertwined and hierarchical [1].

Experimental Protocols for NER Implementation in Materials Science

Implementing effective NER solutions for materials science research requires systematic methodologies tailored to domain-specific challenges.

Protocol: Fine-Tuning LLMs for Structured Information Extraction

This protocol outlines the process for adapting large language models to extract structured materials science knowledge from unstructured text [1].

Objective: To fine-tune a pretrained LLM for joint named entity recognition and relation extraction from materials science literature.

Materials and Reagents:

• Hardware: GPU-accelerated computing environment (minimum 16GB VRAM)
• Software: Python 3.8+, PyTorch or TensorFlow, Hugging Face Transformers library
• Data: Collection of materials science texts (abstracts, full papers) with target entities
• Model: Pretrained LLM (e.g., Llama-2, GPT-3) [1]

Procedure:

• Define Output Schema: Determine the structured representation for extracted knowledge (e.g., JSON format with specific keys for material, composition, morphology, application).
• Annotation Set Creation:
  • Select 100-500 representative text passages from the target domain
  • Manually annotate each passage according to the defined output schema
  • For efficiency, use intermediate models to pre-suggest entities for annotation
• Model Configuration:
  • Initialize with pretrained LLM weights
  • Set training parameters (learning rate: 2e-5, batch size: 16, epochs: 3-5)
  • Configure tokenization to preserve scientific terminology
• Fine-Tuning Process:
  • Convert annotated examples into prompt-completion pairs
  • Train model to generate structured completions from text prompts
  • Validate on held-out dataset after each epoch
• Evaluation and Validation:
  • Assess extraction accuracy using precision, recall, and F1-score
  • Perform manual validation by domain experts on sample extractions
  • Iterate on annotation and training based on performance feedback

Troubleshooting Tips:

• For poor performance on specific entity types, add targeted examples to training data
• If model generates incorrect formats, increase the number of format examples in training
• For memory limitations, reduce batch size or use gradient accumulation

Protocol: Annotation and Corpus Development for Materials NER

Creating high-quality annotated data is fundamental to effective NER implementation in specialized domains.

Objective: To create a domain-specific annotated corpus for training and evaluating materials science NER systems.

Materials:

• Text Sources: Domain-specific scientific literature (PDF or plain text format)
• Annotation Tools: brat, Prodigy, or custom annotation interface
• Guideline Documentation: Detailed annotation specifications and examples

Procedure:

• Entity Definition: Clearly define entity types relevant to materials science (e.g., material, composition, morphology, synthesis parameter, application).
• Annotation Guideline Development:
  • Create comprehensive guidelines with positive and negative examples
  • Address domain-specific challenges (nested entities, formula variations)
  • Establish conventions for ambiguous cases and boundary decisions
• Annotation Process:
  • Train multiple annotators on guidelines and practice examples
  • Conduct independent double-annotation on subset for quality control
  • Calculate inter-annotator agreement (Fleiss' kappa) to ensure consistency
  • Resolve disagreements through adjudication with domain experts
• Corpus Splitting: Divide annotated corpus into training (70%), development (15%), and test (15%) sets, maintaining document integrity.

Successful implementation of NER solutions requires a combination of computational tools, data resources, and methodological frameworks.

Table 3: Research Reagent Solutions for Scientific NER Implementation

Tool/Resource	Type	Function	Application Context
Pre-trained LLMs (LLaMA-2, GPT) [1]	Algorithm	Base models for fine-tuning on domain-specific tasks	Foundation for transfer learning to materials science domain
Transformer Architectures [2]	Algorithm Framework	Context-aware text processing using self-attention mechanisms	Handling complex syntactic and semantic relationships in scientific text
Annotation Platforms (brat, Prodigy)	Software	Efficient manual annotation of training data	Creating gold-standard corpora for model training and evaluation
Conditional Random Fields (CRF) [3]	Statistical Model	Probabilistic sequence labeling for entity recognition	Traditional machine learning approach for NER
BiLSTM-CRF Models [1]	Neural Architecture	Combining bidirectional context with sequence constraints	Hybrid approach balancing context capture and structural constraints
Fine-Tuning Datasets [1]	Data Resource	Domain-specific annotated examples	Adapting general models to specialized scientific subdomains

Advanced Applications and Future Directions

The implementation of NER systems in materials science research enables several advanced applications that directly address the information overload challenge.

Knowledge Graph Construction

Structured information extracted via NER can be utilized to build comprehensive knowledge graphs that integrate entities and relationships across the materials science literature. These graphs enable sophisticated querying, relationship discovery, and hypothesis generation that would be impossible through manual literature review alone [1].

Automated Database Population

NER systems can automatically populate structured databases with materials properties, synthesis parameters, and application information extracted from research papers. This addresses the critical limitation of machine learning models that suffer from insufficient tabulated training data [1].

Research Trend Analysis

By analyzing the occurrence and co-occurrence of specific entities over time, NER systems can identify emerging research trends, material combinations, and methodological shifts in the scientific literature, providing valuable insights for research direction and funding allocation [3].

What is Named Entity Recognition? A Core NLP Task for Science

Named Entity Recognition (NER) is a fundamental Natural Language Processing (NLP) task focused on identifying and classifying specific, real-world objects mentioned in unstructured text into predefined categories [5] [6]. These entities, often proper names, can include persons, organizations, locations, dates, and, critically for scientific domains, specialized terms like materials names, properties, and synthesis methods [7].

In the context of materials science, where the vast majority of knowledge is published as peer-reviewed literature, NER provides a powerful tool to automatically construct large-scale, structured databases from text, thereby accelerating data-driven research and materials discovery [8].

NER Performance: Model Comparison on Scientific Datasets

The performance of an NER system is typically measured using Precision, Recall, and the F1-score, which is the harmonic mean of precision and recall [9]. The table below summarizes the quantitative performance of various models on materials science datasets, demonstrating the advantage of domain-adapted models.

Table 1: Performance Comparison (F1-Score) of NER Models on Scientific Text

Model Name	Dataset / Domain	Reported F1-Score	Key Characteristics
MatBERT-CNN-CRF [9]	Perovskite Materials	90.8%	MatBERT embeddings + CNN for feature extraction + CRF for sequence labeling.
MatBERT [9]	General Materials Science	~85% (inferred)	BERT model pre-trained on a large corpus of materials science literature.
SciBERT [9]	Scientific Multidomain	Lower than MatBERT	BERT model pre-trained on a broad corpus of scientific publications.
BERT (Base) [9]	General Domain	Lower than SciBERT/MatBERT	General-purpose BERT model, lacking scientific domain knowledge.

Experimental Protocols: Implementing NER for Materials Science

The following section details the standard methodology for developing and applying an NER system, with specific examples from recent materials science research.

Protocol 1: Standard NER Model Training and Evaluation

This protocol outlines the end-to-end process for creating a functional NER model, from data collection to deployment [6].

• Data Collection and Annotation

  • Objective: Assemble a high-quality, labeled dataset for training and evaluation.
  • Procedure: a. Acquisition: Collect raw text data relevant to the target domain (e.g., using publisher APIs to gather scientific abstracts on perovskite materials) [9]. b. Annotation: Human experts label the text, marking the spans of text corresponding to entities and assigning them to predefined categories (e.g., MAT, PROP, DSP) [9] [10]. A common labeling scheme is IOBES (Inside, Outside, Beginning, End, Single) [9]. c. Guidelines: Develop clear, consistent annotation guidelines with examples to ensure label consistency across annotators [10].

• Data Preprocessing

  • Objective: Clean and prepare the raw text for model ingestion.
  • Procedure: a. Cleaning: Remove unnecessary characters, HTML tags, or formatting artifacts. b. Tokenization: Split the text into smaller units (tokens), which can be words or subwords. This is a critical step for models like BERT [6].

• Feature Extraction & Model Training

  • Objective: Train a machine learning model to recognize patterns associated with named entities.
  • Procedure: a. Feature Extraction: Convert tokens into numerical representations. Modern systems use pre-trained language models (e.g., MatBERT, BERT) to generate contextualized word embeddings that capture semantic meaning [8] [9]. b. Model Architecture: The embeddings are fed into a downstream model architecture. A common and effective approach combines: * CNN or BiLSTM: To capture local contextual relationships and feature information [9]. * CRF Layer: As the final output layer to model the dependencies between subsequent entity labels and ensure globally optimal tag sequences [9] [6]. c. Training: The model is trained on the annotated dataset, learning to associate specific word patterns and contexts with entity tags.

• Model Evaluation and Fine-tuning

  • Objective: Assess and improve model performance.
  • Procedure: a. Evaluation: The model's predictions on a held-out test set are compared to the ground-truth labels. Performance is quantified using Precision, Recall, and F1-score [9] [7]. b. Fine-tuning: Based on the evaluation, the model's hyperparameters may be adjusted, or it may be retrained with additional data to improve performance [6].

• Inference and Post-processing

  • Objective: Apply the trained model to new, unseen text.
  • Procedure: a. Inference: The trained model processes new text and predicts named entity labels for each token. b. Post-processing: The raw output may be refined, which can include grouping consecutive tokens into full entity names or linking entities to entries in a knowledge base for enrichment [6].

Protocol 2: Domain-Specific NER for Perovskite Materials

This protocol exemplifies a real-world application, detailing the specific model architecture from a recent study that achieved state-of-the-art results on a perovskite literature dataset [9].

• Model Architecture (MatBERT-CNN-CRF)

  • Objective: Achieve high-accuracy entity recognition on specialized materials science text.
  • Procedure: a. Embedding Layer: Input text is passed through MatBERT, a BERT model pre-trained on a massive corpus of materials science literature. This provides domain-aware, contextualized word vector representations [9]. b. Feature Extraction Layer: The MatBERT embeddings are fed into a 1D Convolutional Neural Network (CNN). The CNN's role is to further extract salient local features and contextual relationships from the sequence of word embeddings [9]. c. Sequence Labeling Layer: The feature maps from the CNN are passed to a Conditional Random Field (CRF) layer. The CRF layer considers the dependencies between adjacent labels to produce the final, globally optimal sequence of entity tags (e.g., B-MAT, I-PROP, O) [9].

• Knowledge Extraction and Analysis

  • Objective: Use the trained model to extract insights from a large corpus.
  • Procedure: The study used the trained MatBERT-CNN-CRF model to process 2,389 perovskite abstracts. This automated extraction identified 24,280 data points, allowing the researchers to quantitatively analyze trends, such as the most frequently discussed material components and applications within the field [9].

Workflow Visualization: NER in Materials Science

The following diagram illustrates the complete workflow for applying Named Entity Recognition to accelerate materials science research, from data collection to knowledge application.

Table 2: Key Research Reagents and Tools for NER Implementation

Tool / Resource	Type	Function in NER Research
Pre-trained Language Models (MatBERT, SciBERT) [9]	Software Model	Provides foundational, domain-aware word embeddings, drastically reducing the need for training from scratch and improving performance on scientific text.
Annotation Tools (Prodigy, Doccano, BRAT) [11]	Software Platform	Enables efficient manual annotation of text data to create high-quality training sets, often supporting features like collaborative workflow and model-assisted labeling.
Perovskite NER Dataset [9]	Benchmark Data	A publicly available, annotated dataset of 800 abstracts used for training and evaluating the performance of NER models on a specific materials science sub-domain.
Convolutional Neural Network (CNN) [9]	Algorithm	A deep learning component used in the NER model architecture to perform local feature extraction from sequences of word embeddings.
Conditional Random Field (CRF) [9] [6]	Algorithm	A statistical modeling method used as the final layer in an NER model to predict the most logically consistent sequence of labels by considering neighbor dependencies.
Python NLP Libraries (spaCy, NLTK) [6]	Software Library	Open-source libraries that provide robust, pre-built implementations for standard NLP tasks, including tokenization, model training, and entity recognition.

Application Notes

The Role of Named Entity Recognition in Accelerating Research

Named Entity Recognition (NER) has emerged as a critical technology for unlocking the vast knowledge embedded within scientific literature across materials science and drug discovery. With the annual growth of materials science papers at a compounded rate of 6%, manually analyzing this wealth of information to establish chemistry-structure-property relationships has become increasingly impractical [12]. NER systems automatically identify and categorize key entities—from specific polymers and their properties to drug compounds and protein targets—transforming unstructured text into machine-readable data that can power knowledge graphs, predictive models, and intelligent search systems [13] [12]. This capability is particularly valuable in drug discovery, where identifying potential therapeutic entities and their biological targets requires analyzing complex relationships across diverse data sources [14] [15].

The transition from manual literature analysis to automated information extraction represents a paradigm shift in research methodology. Where researchers once spent countless hours poring over journals to extract specific material properties or drug-target interactions, NER pipelines can now process hundreds of thousands of abstracts in days, generating comprehensive databases that capture critical relationships [12]. For instance, one implementation extracted approximately 300,000 material property records from 130,000 polymer abstracts in just 60 hours, demonstrating the profound efficiency gains possible through automated entity extraction [16]. This accelerated knowledge mining directly supports both fields' core objectives: in materials science, the discovery of novel materials with tailored properties, and in drug discovery, the identification of new therapeutic candidates and their mechanisms of action [14] [12].

Key Entity Types and Their Research Applications

In materials science NER, a clearly defined ontology of entity types enables precise information extraction from scientific text. These entities form the foundational building blocks for understanding material systems and their characteristics, with applications spanning from polymer design to energy storage materials [12] [16].

Table 1: Core Entity Types in Materials Science NER

Entity Type	Description	Research Application	Frequency in Annotated Corpora
POLYMER	Material entities that are polymers	Primary subject of study in polymer science	7,364 occurrences
PROPERTY_NAME	Entity type for a material property	Links materials to their characteristics	4,535 occurrences
PROPERTY_VALUE	Numeric value and unit for a property	Quantitative analysis and prediction	5,800 occurrences
MONOMER	Repeat units for a POLYMER entity	Polymer synthesis and design	2,074 occurrences
POLYMER_CLASS	Broad terms for a class of polymers	Categorization and classification	1,476 occurrences
ORGANIC_MATERIAL	Organic materials that are not polymers (e.g., plasticizers)	Formulation development	914 occurrences
INORGANIC_MATERIAL	Inorganic additives in formulations	Composite material design	1,272 occurrences
MATERIAL_AMOUNT	Amount of a particular material in a formulation	Reproducibility and scaling	1,143 occurrences

These entity types enable researchers to automatically construct structured databases from unstructured text, capturing essential relationships between material composition, structure, and performance [12]. For example, extracting tuples containing (POLYMER, PROPERTYNAME, PROPERTYVALUE) allows for large-scale analysis of structure-property relationships that would be prohibitively time-consuming to compile manually. The frequency data included in Table 1, drawn from annotated corpora of polymer literature, reflects the relative importance and occurrence of each entity type in scientific abstracts [16].

In drug discovery, complementary entity types include DRUGCOMPOUND, PROTEINTARGET, BIOLOGICAL_ACTIVITY, and TOXICITY, which facilitate the identification of potential therapeutic candidates and their mechanisms of action [14] [15]. The integration of these entity types across domains enables advanced applications such as drug repurposing, where existing drugs are matched to new disease targets based on their molecular interaction signatures [14].

Advanced NER Frameworks: From Sequence Labeling to Machine Reading Comprehension

Traditional NER approaches treated entity recognition as a sequence labeling problem, using models such as BiLSTM-CRF or BERT with a token classification head to assign labels to individual words in a text [13]. While these methods achieved reasonable performance, they struggled with capturing complex semantic relationships and frequently failed to handle nested entities where one entity contains another within its span [13]. This limitation proved particularly problematic in scientific texts, where complex descriptions often include multiple overlapping entities of different types.

The emerging solution to these challenges is the Machine Reading Comprehension (MRC) framework, which reformulates NER as a question-answering task [13]. Instead of simply labeling each token, the MRC approach generates specific queries for each entity type and identifies text spans that answer these queries. For example, to identify polymer entities, the system might process the query "Which polymers are mentioned in the text?" and extract the relevant spans as answers [13]. This paradigm shift offers several significant advantages:

• Handling of Nested Entities: By processing each entity type through separate queries, the MRC framework can identify overlapping entities that share the same text span, a common occurrence in technical scientific descriptions [13].
• Incorporation of Prior Knowledge: The query mechanism allows domain knowledge to be explicitly encoded through carefully crafted questions, guiding the model to recognize entities based on established scientific understanding [13].
• Improved Utilization of Semantic Context: The question-answering format encourages the model to better understand the contextual relationships between entities and their surrounding text [13].

State-of-the-art implementations of this approach have achieved remarkable performance, with F1-scores of 89.64% on the Matscholar dataset and 94.30% on the BC4CHEMD dataset, outperforming traditional sequence labeling methods across multiple benchmarks [13]. The MRC framework represents a significant advancement in extracting complex scientific information with the precision required for research applications in materials science and drug discovery.

Experimental Protocols

Protocol: Implementing a Machine Reading Comprehension Framework for Materials Science NER

Purpose and Scope

This protocol details the procedure for implementing a Machine Reading Comprehension (MRC) framework for Named Entity Recognition (NER) in materials science literature. The approach enables accurate extraction of key entity types (e.g., POLYMER, PROPERTY_VALUE) from scientific text, significantly accelerating the compilation of structured databases from unstructured literature [13]. The method is particularly effective for handling nested entities and leveraging semantic context, achieving state-of-the-art performance with F1-scores of 89.64% on the Matscholar dataset [13].

Equipment and Software Requirements

• Python 3.7+ programming environment
• Transformer libraries (Hugging Face Transformers)
• PyTorch or TensorFlow deep learning frameworks
• Access to pre-trained language models (MaterialsBERT, MatSciBERT)
• Annotated datasets for model training and evaluation
• Standard NLP preprocessing tools (spaCy, NLTK)

Step-by-Step Procedure

Step 1: Data Preparation and Annotation

• Collect a corpus of materials science abstracts from sources such as the Materials Project database or polymer literature [12] [16].
• Annotate the abstracts using a defined ontology of entity types (see Table 1). For polymer-focused extraction, core entities include POLYMER, PROPERTYNAME, PROPERTYVALUE, MONOMER, and MATERIAL_AMOUNT [16].
• Split the annotated data into training (85%), validation (5%), and test sets (10%) following established practices in the field [16].
• Transform the sequence labeling data into MRC format by creating (Context, Query, Answer) triples for each entity type [13].

Table 2: Example Query Templates for MRC Framework

Entity Type	Query Template	Answer Format
POLYMER	"Which polymers are mentioned in the text?"	Text spans
PROPERTY_NAME	"What material properties are discussed?"	Text spans
PROPERTY_VALUE	"What are the numerical values and units for material properties?"	Text spans
MONOMER	"What monomer units are described?"	Text spans
MATERIAL_AMOUNT	"What are the amounts or concentrations of materials?"	Text spans

Step 2: Model Selection and Configuration

• Select a pre-trained language model as the backbone architecture. Materials-specific models like MaterialsBERT (trained on 2.4 million materials science abstracts) or MatSciBERT typically outperform general-domain models [12] [13].
• Configure the MRC framework with two binary classifiers: one for predicting start indices and another for predicting end indices of entity spans [13].
• Set hyperparameters based on established practices: learning rate of 2e-5, batch size of 16 or 32, and maximum sequence length of 512 tokens [13].

Step 3: Model Training and Fine-tuning

• Input the formatted sequences to the BERT-based model: {[CLS] + Query + [SEP] + Context + [SEP]} [13].
• Train the model using cross-entropy loss for both start and end position prediction [13].
• Implement early stopping based on validation performance to prevent overfitting.
• Fine-tune the model on specific subdomains if needed (e.g., polymer solar cells, fuel cells, supercapacitors) to enhance domain-specific performance [12].

Step 4: Inference and Entity Extraction

• For new text inputs, process each query type sequentially to extract all relevant entities.
• Apply the trained start and end classifiers to identify potential entity spans.
• Implement post-processing rules to handle overlapping spans and validate extracted entities.
• Export the extracted entities in structured formats (JSON, CSV) for downstream applications.

Step 5: Evaluation and Validation

• Evaluate model performance using standard metrics: precision, recall, and F1-score on the held-out test set [13].
• Validate extracted entities against manually annotated gold-standard data to ensure accuracy.
• Perform iterative improvement by analyzing error patterns and refining query formulations [13].

The following diagram illustrates the complete MRC workflow for materials science NER:

Protocol: Natural Language Processing Pipeline for Large-Scale Material Property Extraction

Purpose and Scope

This protocol describes the implementation of a general-purpose natural language processing pipeline for extracting material property data from large corpora of scientific literature. The pipeline enables researchers to automatically identify and structure material property information at scale, generating databases of property records that can be used for materials discovery and prediction tasks [12]. The approach has been successfully applied to extract approximately 300,000 material property records from polymer literature, demonstrating its effectiveness for automating the curation of materials databases [16].

Equipment and Software Requirements

• Corpus of materials science abstracts (e.g., 2.4 million abstracts from materials science journals) [12]
• Pre-trained MaterialsBERT or domain-specific language model
• Annotation tools (e.g., Prodigy for manual annotation)
• Computational resources for model training (GPU recommended)
• Database system for storing extracted property records

Step-by-Step Procedure

Step 1: Corpus Filtering and Preprocessing

• Filter a large abstract corpus to identify domain-relevant texts. For polymer science, filter abstracts containing the string 'poly' and numerical values using regular expressions [16].
• Preprocess text by tokenization, sentence segmentation, and normalization of special characters and symbols.

Step 2: Named Entity Recognition

• Apply the trained NER model to identify entities of interest in the text. The core entity types for material property extraction include:
  • POLYMER and other material entities
  • PROPERTYNAME
  • PROPERTYVALUE (including numerical values and units)
  • MATERIAL_AMOUNT [16]
• Utilize an ensemble of models or the MRC framework described in Protocol 2.1 to improve recognition accuracy [13].

Step 3: Relation Extraction

• Implement heuristic rules or train a relation classification model to associate extracted entities.
• Focus on identifying {material, property, value} tuples that represent complete property records [12].
• Resolve coreferences where materials are referred to by pronouns or abbreviations later in the text.

Step 4: Named Entity Normalization

• Normalize polymer names and material identifiers to account for synonyms and naming variations [12].
• Standardize property values and units to consistent representations for quantitative analysis.

Step 5: Data Export and Application

• Export the extracted material property records to a structured database or knowledge graph.
• Implement a web interface (e.g., polymerscholar.org) to enable researchers to search and explore the extracted data [16].
• Utilize the extracted data for downstream tasks such as training property prediction models or analyzing structure-property relationships [12].

The following workflow diagram illustrates the complete property extraction pipeline:

The Scientist's Toolkit

Research Reagent Solutions for NER Implementation

Table 3: Essential Tools and Resources for Materials Science NER

Tool/Resource	Type	Function	Source/Availability
MaterialsBERT	Language Model	Domain-specific BERT model pre-trained on 2.4M materials science abstracts for contextual understanding of materials terminology	Publicly available [12]
MatSciBERT	Language Model	Alternative BERT model pre-trained on materials science text for NER tasks	Publicly available [13]
PolymerAbstracts Dataset	Annotated Data	750 manually annotated polymer abstracts with 8 entity types for training and evaluation	Research data [16]
Matscholar Dataset	Benchmark Data	Annotated corpus for evaluating materials science NER performance	Publicly available [13]
ChemDataExtractor	Software Tool	Toolkit for automated chemical data extraction from scientific literature	Open source [12]
Prodigy	Annotation Tool	Commercial tool for efficient manual annotation of training data	Commercial license [16]

Practical Implementation Guidance

For researchers implementing NER systems for materials science or drug discovery, several practical considerations can significantly impact success:

• Domain Adaptation: While general-purpose language models like BERT provide a solid foundation, models specifically pre-trained on scientific corpora (MaterialsBERT, MatSciBERT) consistently outperform them on materials science tasks [12] [13]. The domain-specific vocabulary and conceptual understanding embedded in these models is particularly valuable for accurately recognizing technical entities.

• Query Design in MRC Frameworks: The formulation of queries in MRC-based NER significantly influences performance. Queries should be derived from annotation guidelines and incorporate domain knowledge. For example, effective queries for polymer entities might include "Find all polymer materials" or "Identify synthetic macromolecules" [13].

• Handling of Nested Entities: Traditional sequence labeling approaches struggle with nested entities where one entity contains another. The MRC framework's question-answering approach naturally handles this challenge by processing each entity type through separate queries [13].

• Integration with Downstream Applications: The ultimate value of NER extraction is realized when the extracted entities power downstream applications such as property prediction models, knowledge graphs, or materials discovery platforms [12] [17]. Designing the extraction pipeline with these applications in mind ensures the output is structured appropriately for subsequent use.

The continued advancement of NER technologies, particularly through frameworks like MRC and domain-adapted language models, is transforming how researchers access and utilize the knowledge embedded in scientific literature. By implementing the protocols and utilizing the tools described in this document, research teams can significantly accelerate their materials discovery and drug development efforts.

The Critical Role of NER in Accelerating Materials and Drug Discovery Pipelines

Named Entity Recognition (NER) serves as a foundational technology in scientific text mining, enabling the rapid transformation of unstructured text from millions of publications into structured, actionable data. In both materials science and biomedical research, where the volume of literature grows exponentially, NER systems automatically identify and classify critical entities such as material names, properties, synthesis methods, diseases, genes, and chemicals. This capability directly accelerates discovery pipelines by powering knowledge extraction, facilitating inverse design, and enabling large-scale literature-based discovery that would be impossible through manual curation alone [18] [19] [20]. The adaptation of advanced deep learning architectures, particularly transformer-based models, has significantly enhanced the accuracy and scope of these systems, pushing the boundaries of what can be automated in scientific research.

The performance of NER systems is typically evaluated using precision (correctness of extracted entities), recall (completeness of extraction), and F-score (harmonic mean of precision and recall). The following table summarizes reported performance metrics across different scientific domains and datasets:

Table 1: Performance Metrics of NER Systems Across Scientific Domains

Domain	Corpus/Dataset	Key Entity Types	Reported F-Score (%)	Model Architecture
Materials Science	Matscholar [19]	Materials, properties, applications, synthesis methods	87.0	Not Specified
Biomedical (Diseases)	NCBI Disease Corpus [21]	Disease names	85.7	CLSTM (Contextual LSTM)
Biomedical (Genes)	BioCreative II GM [21]	Gene mentions	81.4	CLSTM (Contextual LSTM)
Biomedical (Chemicals/Diseases)	BioCreative V CDR [21]	Chemical and disease names	86.4	CLSTM (Contextual LSTM)

NER in the Materials Science Discovery Pipeline

In materials science, NER directly addresses the critical bottleneck of connecting new research findings with established knowledge by extracting specific entities from unstructured text.

Key Applications and Extracted Entities

• Information Extraction from Abstracts: Applied to 3.27 million materials science abstracts, NER has successfully extracted over 80 million materials-science-related named entities, transforming abstract content into structured database entries [19].
• Complex Query Resolution: The structured data enables researchers to answer complex "meta-questions" about published literature without laborious manual searches, significantly accelerating literature review and hypothesis generation [19].
• Powering Foundation Models: NER-extracted entities form the training data for foundation models in materials discovery, which are applied to downstream tasks including property prediction, synthesis planning, and molecular generation [18].

Experimental Protocol: Materials Science NER Implementation

Objective: Implement a named entity recognition system to extract materials-science-related entities from scientific abstracts and full-text publications.

Materials and Methods:

• Data Collection: Gather a corpus of materials science documents (abstracts and/or full-text articles) in PDF or plain text format [19].
• Annotation Guidelines: Develop comprehensive annotation guidelines defining entity types including:
  • Inorganic material mentions (e.g., "TiO2", "perovskite")
  • Sample descriptors (e.g., "thin film", "nanoparticle")
  • Phase labels (e.g., "cubic phase", "amorphous")
  • Material properties (e.g., "band gap", "conductivity")
  • Applications (e.g., "catalyst", "battery electrode")
  • Synthesis methods (e.g., "sol-gel", "chemical vapor deposition")
  • Characterization methods (e.g., "X-ray diffraction", "SEM") [19]
• Model Training:
  • Pre-processing: Convert PDF documents to text using appropriate tools, handling challenges of multi-modal data (text, tables, images) [18].
  • Annotation: Manually annotate a subset of documents following established guidelines to create gold-standard training data.
  • Algorithm Selection: Implement a suitable deep learning architecture such as BiLSTM-CRF or transformer-based models [21].
  • Training: Train the model on annotated data, using techniques like transfer learning if limited annotated data is available.
  • Evaluation: Assess model performance using standard metrics (precision, recall, F1-score) on a held-out test set [19].

Validation: Apply the trained model to large-scale information extraction from materials science literature (e.g., 3.27 million abstracts) and verify extracted entities against manually annotated samples [19].

NER in the Biomedical and Drug Discovery Pipeline

Biomedical NER (bNER) faces unique challenges including entity ambiguity, proliferation of synonyms and abbreviations, and the constant emergence of newly discovered entities, requiring specialized approaches.

Domain-Specific Challenges and Solutions

• Entity Complexity: Biological entities continually increase with new discoveries, have numerous synonyms, frequent abbreviations, and often consist of mixtures of letters, symbols, and punctuation [21].
• Contextual Understanding: Advanced models like CLSTM (Contextual LSTM) incorporate n-gram features with BiLSTM and CRF to capture important local contexts in biomedical text, achieving state-of-the-art performance [21].
• Multimodal Extraction: Modern systems combine text-based NER with specialized algorithms for processing molecular structures from images and data from plots, enabling comprehensive information extraction [18].

Experimental Protocol: Biomedical NER for Electronic Health Records

Objective: Implement a biomedical NER system to extract clinically relevant entities from electronic health records (EHRs) for treatment prediction and knowledge discovery.

Materials and Methods:

• Data Source: Collect de-identified electronic health records containing clinical notes, physician observations, and patient histories [22].
• Entity Definition: Define target entity types relevant to clinical applications:
  • Diseases and Disorders (e.g., "type 2 diabetes", "myocardial infarction")
  • Drugs and Medications (e.g., "metformin", "aspirin")
  • Dosage Information (e.g., "500mg", "twice daily")
  • Administration Routes (e.g., "oral", "intravenous")
  • Procedures and Treatments (e.g., "coronary artery bypass", "chemotherapy")
  • Anatomical Sites (e.g., "left ventricle", "frontal lobe") [22] [20]
• Model Implementation:
  • Data Preprocessing: Clean and tokenize clinical text, handling domain-specific abbreviations and formatting inconsistencies.
  • Architecture Selection: Employ deep learning architectures such as:
    • BiLSTM-CRF: Bidirectional Long Short-Term Memory networks with Conditional Random Fields for sequence labeling [21]
    • Transformer Models: BERT-based architectures pre-trained on biomedical corpora [21]
    • Contextual LSTM (CLSTM): Incorporates n-gram features for improved contextual understanding [21]
  • Training Strategy: Utilize transfer learning from models pre-trained on biomedical literature (e.g., PubMed abstracts) to address limited annotated clinical data.
  • Evaluation: Assess performance using precision, recall, and F1-score on manually annotated gold-standard EHR datasets.

Validation: Validate extracted entities against expert-annotated corpora and assess utility for downstream tasks including treatment outcome prediction and adverse drug reaction detection [22].

Table 2: Essential Research Reagents for NER Implementation

Reagent/Resource	Type	Function in NER Pipeline	Example Sources
Annotated Corpora	Training Data	Gold-standard data for model training and evaluation	NCBI Disease Corpus [21], BioCreative CDR [21], Matscholar [19]
Pre-trained Word Embeddings	Language Model	Initialize word representations with semantic knowledge	PubMed word embeddings [21], domain-specific embeddings
Deep Learning Frameworks	Software Infrastructure	Implement and train NER architectures	TensorFlow, PyTorch, Transformers
BiLSTM-CRF	Algorithm Architecture	Sequence labeling for entity recognition	Contextual LSTM [21]
Transformer Models	Algorithm Architecture	State-of-the-art contextual representations	BERT [21]
IOB Labeling Scheme	Annotation Format	Standard format for marking entity boundaries	Inside-Outside-Beginning format [21]

Emerging Architectures and Future Directions

The field of scientific NER is rapidly evolving with several emerging trends shaping its future development and application.

Advanced Model Architectures

• Foundation Models: Large language models trained on broad scientific data can be adapted to various downstream NER tasks with minimal fine-tuning, demonstrating significant promise for materials and drug discovery applications [18].
• Multimodal Approaches: Systems that combine text with images, tables, and molecular structures enable more comprehensive information extraction from scientific documents [18].
• Joint Modeling: Methods that simultaneously perform entity recognition and relationship extraction show promising results compared to traditional pipelined approaches [20].

Implementation Challenges and Considerations

• Data Quality and Availability: The performance of NER systems depends heavily on the quality and comprehensiveness of training data, with limited annotated corpora remaining a significant challenge in specialized domains [18].
• Domain Adaptation: Models trained on general scientific text often require significant fine-tuning and adaptation to perform well on specialized subdomains or new entity types.
• Scalability and Efficiency: Processing millions of documents requires efficient algorithms and infrastructure, particularly for complex deep learning models [20].

Named Entity Recognition has evolved from a basic text mining tool to a critical enabling technology for accelerating scientific discovery in both materials science and biomedical research. By automatically transforming unstructured scientific text into structured, queryable knowledge, NER systems directly address the fundamental challenge of information overload that researchers face today. The continued development of specialized NER systems, particularly those leveraging advanced deep learning architectures and multimodal approaches, promises to further accelerate discovery pipelines, enable new forms of literature-based knowledge discovery, and ultimately reduce the time from hypothesis to breakthrough in both materials and drug development.

The COVID-19 pandemic created an unprecedented need for the rapid identification of therapeutic compounds, challenging traditional drug discovery timelines. This application note details a hybrid methodology that combines named entity recognition (NER) from scientific literature with computational chemistry and experimental validation to accelerate the discovery of drug-like molecules. Framed within broader research on NER for materials science, this case study demonstrates how natural language processing (NLP) can efficiently structure unstructured text to identify promising chemical entities. The workflow bridges computational linguistics and experimental bioscience, offering a scalable template for responding to emerging health threats. Researchers applied this integrated approach to the COVID-19 Open Research Dataset (CORD-19), extracting and validating molecules with potential efficacy against SARS-CoV-2, specifically targeting the essential 3C-like protease (3CLpro) [23] [24].

Experimental Protocols and Workflows

Named Entity Recognition for Molecule Extraction

Objective: To automatically identify and extract references to drug-like molecules from the extensive COVID-19 scientific literature.

Materials:

• Text Corpus: COVID-19 Open Research Dataset (CORD-19) containing 198,875 scientific articles [24].
• Computational Tools: SpaCy and Keras long short-term memory (LSTM) models for NER [24].
• Validation: Non-expert human reviewers for model training and verification.

Methodology: A model-in-the-loop methodology was employed to maximize the efficiency of human annotation efforts [24]. The workflow, detailed in Figure 1, proceeded as follows:

• Bootstrap Sampling: An initial NER model was trained on a small, human-verified set of examples to establish baseline capability for identifying drug-like molecules in text.
• Iterative Model Refinement: The model was iteratively applied to the CORD-19 corpus. For each iteration, human reviewers were presented with and verified only the model's most uncertain predictions.
• Model Retraining: The verified predictions were incorporated into the training data, and the model was retrained. This cycle continued until model performance improvements fell below a pre-defined threshold.
• Full Corpus Application: The final, trained model was applied to the entire CORD-19 corpus to extract all putative drug-like molecules.

This targeted labeling approach required only tens of hours of human effort to label 1,778 samples, resulting in an NER model with an F-1 score of 80.5%—performance on par with that of non-expert humans. The process successfully identified 10,912 putative drug-like molecules from the literature, substantially enriching the pool of candidate compounds for further investigation [24].

Computational Validation and Screening

Objective: To computationally assess the bioactivity and drug-like properties of the extracted molecules.

Materials:

• Chemical Databases: ChEMBL database for bioactivity data [23].
• Cheminformatics Tools: RDKit for computing molecular descriptors and PubChem fingerprints [23].
• QSAR Modeling: Six regression algorithms (Extra Tree, Gradient Boosting, XGBoost, Support Vector, Decision Tree, and Random Forest) were compared for building quantitative structure-activity relationship (QSAR) models to predict compound bioactivity based on IC50 values [23].
• ADMET Analysis: Software tools (e.g., SWISSADME) were used to predict absorption, distribution, metabolism, excretion, and toxicity (ADMET) properties [23].

Methodology:

• Data Curation: 133 drug-like bioactive molecules were retrieved from the ChEMBL database specifically targeting SARS coronavirus 3CL Protease. The dataset was divided into active, inactive, and intermediate classes based on standard IC50 values [23].
• Molecular Descriptor Analysis: Exploratory Data Analysis (EDA) was performed using molecular descriptors adhering to Lipinski's rule. Statistical tests, such as the Mann-Whitney U test, identified significant differences between active and inactive molecular classes [23].
• QSAR Model Training and Selection: The dataset was used to train the six QSAR models. The Extra Tree Regressor (ETR) model demonstrated superior predictive performance for compound bioactivity compared to other algorithms and was selected for subsequent screening [23].
• ADMET Screening: The top candidates identified by the QSAR model were subjected to in-silico ADMET analysis to filter for favorable pharmacokinetic and safety profiles. This step identified 13 promising bioactive molecules with suitable drug-like properties [23].

Experimental Validation

Objective: To experimentally verify the efficacy of the shortlisted drug candidates.

Materials:

• Target Protein: SARS-CoV-2 3CL Protease (Mpro).
• Software Tools: AUTODOCK VINA for molecular docking, PyMOL for visualization [23].
• Cellular Assays: Viral infection inhibition assays and phenotypic screening [25].

Methodology:

• Molecular Docking: The binding affinity and interaction mode of the 13 ADMET-filtered compounds with the SARS-CoV-2 3CL Protease were evaluated using molecular docking. This computational simulation predicts how strongly and where a small molecule binds to a target protein [23].
• Hit Confirmation: Docking results were used to rank the compounds based on binding affinity, leading to the shortlisting of six molecules (ChEMBL IDs: 187460, 222769, 225515, 358279, 363535, and 365134) as the most favorable drug candidates [23].
• Phenotypic and Target-Based Screening: In parallel, independent studies performed target-based (3CLpro) and phenotypic screening of in-house compound libraries. This process identified novel hit compounds, including those from a chloroquine-heterocycle chemotype and previously unreported polyhedral‑boron systems (metallacarborane and dicarba-closo-dodecaborane chemotypes), which showed true anti-viral activity in cellular assays [25].

Data Presentation and Analysis

NER Model Performance and Output

Table 1: Performance Metrics of the NER Model for Drug-like Molecule Identification

Metric	Value	Context and Significance
Total Processed Articles	198,875	Size of the CORD-19 corpus [24].
Human-labeled Samples	1,778	Efficient labeling via model-in-the-loop [24].
Final Model F-1 Score	80.5%	Performance on par with non-expert humans [24].
Putative Molecules Identified	10,912	Total molecules extracted from the full corpus [24].
Molecules Enriched for Screening	3,591	New molecules added to screening libraries [24].
High-Performance Docking Hits	18	Molecules ranking in the top 0.1% in docking studies against 3CLPro [24].

Experimental Validation Results

Table 2: Experimentally Validated Hit Compounds Against SARS-CoV-2

Compound / Scaffold	Source / ID	Target / Mechanism	Key Experimental Findings
GC376 / Coronastat	Optimized Lead [26]	SARS-CoV-2 3CLpro inhibitor (covalent, reversible)	Sub-10 nM cellular potency, pan-coronavirus activity, co-crystal structure confirmed binding to C145 [26].
Shortlisted Bioactive Molecules	ChEMBL: 187460, etc. [23]	SARS-CoV-2 3CLpro inhibitor	High binding affinity in molecular docking, favorable ADMET profile [23].
Polyhedral-Boron Derivatives	Novel Hits [25]	3CLpro and PLpro inhibitor (Metallacarborane)	Low µM inhibitory activity against Mpro, true anti-viral activity in phenotypic screening (SI > 2) [25].
Fungal Polysaccharides	Natural Products [27]	Viral entry / RBD-ACE2 binding inhibition	Potent inhibitory effects on SARS-CoV-2 infection and RBD-hACE2 binding in vitro [27].
Rose Bengal, Venetoclax, AKBA	Screening Hits [28]	nsp12 (RNA polymerase) inhibitor	Effectively limited viral replication by blocking nsp12 from initiating replication [28].

The Scientist's Toolkit: Research Reagent Solutions

Table 3: Essential Research Materials and Tools for NER-driven Drug Discovery

Tool or Reagent	Function / Application	Specific Examples / Notes
Named Entity Recognition (NER) Models	Automatically extract drug and molecule names from unstructured text.	SpaCy, Keras-LSTM models; trained on custom corpus [24].
Chemical and Bioactivity Databases	Source of known bioactive molecules for training QSAR models and repurposing.	ChEMBL [23], DrugBank [24], PubChem [29].
Cheminformatics Software	Compute molecular descriptors, fingerprints, and perform structural analysis.	RDKit [23], Osiris DataWarrior [29].
Molecular Docking Tools	Predict binding affinity and mode of interaction between a compound and a protein target.	AUTODOCK VINA [23] [29].
ADMET Prediction Platforms	In-silico assessment of pharmacokinetics and toxicity.	admetSAR server [29], SWISSADME [23].
Structural Biology & Visualization	Determine and analyze 3D protein-ligand complex structures.	Cryo-electron microscopy, X-ray crystallography, PyMOL [23] [30].

Workflow and Pathway Visualizations

Figure 1: The integrated NER and drug discovery workflow. The process begins with text mining a large corpus of scientific literature, from which drug-like molecules are extracted using a trained NER model. These candidates undergo sequential computational filtering via QSAR modeling and ADMET analysis before the most promising hits are validated experimentally through molecular docking and cellular assays [23] [24].

Figure 2: The model-in-the-loop NER training process. This iterative methodology optimizes the use of scarce human labeling resources. The model is first bootstrapped on a small labeled dataset. It then predicts labels on the full corpus, and a human annotator reviews only the predictions with the lowest confidence. These newly labeled samples are added to the training set, and the model is retrained. This loop continues until model performance plateaus, yielding a high-fidelity NER model with minimal human effort [24].

This case study demonstrates a powerful and efficient framework for drug discovery that integrates computational linguistics with bioinformatics and experimental biology. By applying a targeted NER model to the vast COVID-19 literature, researchers rapidly identified thousands of putative drug-like molecules, enriching screening libraries and leading to the experimental validation of several promising candidates against SARS-CoV-2 targets. The success of this workflow, culminating in the identification of specific bioactive molecules with high binding affinity for the 3CL protease, validates the role of named entity recognition as a critical component in modern materials science and drug development research. This approach provides a scalable, rapid-response template for identifying therapeutic agents against future pathogenic threats.

From Theory to Practice: NER Methods, Models, and Real-World Applications

Named Entity Recognition (NER) is a fundamental Natural Language Processing (NLP) technique that identifies and classifies key information in text into predefined categories such as person, location, organization, and more domain-specific entities [31]. In materials science research, NER has become an indispensable tool for automating the extraction of structured information from vast scientific literature, patents, and technical reports [8] [18]. The ability to automatically identify materials, properties, synthesis parameters, and characterization methods from unstructured text enables researchers to build large-scale, structured databases for data-driven materials discovery [8].

The evolution of NER techniques has followed a trajectory from simple rule-based methods to sophisticated deep learning approaches, with each generation offering improved accuracy and adaptability [31] [32]. This progression has been particularly impactful in scientific domains like materials science, where the specialized terminology and contextual dependencies present unique challenges for traditional NLP methods [18]. Modern NER systems now leverage transformer architectures and foundation models specifically fine-tuned for scientific text, enabling unprecedented efficiency in extracting materials science knowledge from the rapidly expanding scientific literature [33] [8].

Historical Progression of NER Methodologies

Dictionary-Based Systems

The earliest NER systems employed dictionary-based approaches, which relied on predefined lists of entities (gazetteers) and string-matching algorithms to identify relevant terms in text [34] [32]. These systems worked by checking whether words in the text appeared in a vocabulary of known entities, making them straightforward to implement and interpret [31].

Key Characteristics:

• Utilized exhaustive lists of entity names (e.g., chemical compounds, material names)
• Employed fuzzy matching to handle minor variations in spelling or formatting
• Required constant manual updates to maintain comprehensive coverage
• Struggled with emerging terminology and domain-specific abbreviations [34]

In materials science, dictionary-based systems faced significant challenges due to the continuous discovery of new materials and the complex nomenclature systems used for chemical compounds [18]. The static nature of dictionaries made it difficult to keep pace with the rapidly evolving terminology in materials research, limiting their long-term utility for comprehensive information extraction [31].

Rule-Based Systems

Rule-based systems represented an advancement by incorporating linguistic patterns and contextual rules alongside lexical resources [34]. These systems used handcrafted rules based on morphological patterns, syntactic structures, and contextual clues to identify and classify entities [32].

Common Rule Types:

• Pattern-based rules (e.g., capitalization patterns for proper nouns)
• Context-based rules (e.g., words following "manufactured by" likely indicate organizations)
• Syntactic rules leveraging part-of-speech tags and dependency parsing [32]

While rule-based systems could capture some linguistic regularities missed by pure dictionary approaches, they remained brittle and required significant domain expertise to develop and maintain [31]. The manual creation of comprehensive rule sets for materials science proved particularly challenging given the field's specialized syntax and terminology [8].

Machine Learning-Based Systems

The introduction of machine learning (ML) approaches marked a significant shift in NER methodology, moving from manually constructed rules to statistically learned models [31]. ML-based NER systems treated entity recognition as a sequence labeling problem, using annotated datasets to train models that could generalize beyond predefined rules and dictionaries [32].

Predominant Algorithms:

• Conditional Random Fields (CRF): Particularly effective for sequence labeling tasks
• Support Vector Machines (SVM): Applied with careful feature engineering
• Decision Trees: Offered interpretability with reasonable performance [31] [32]

These systems relied on extensive feature engineering, incorporating features such as word capitalization, prefix/suffix patterns, part-of-speech tags, and contextual windows [32]. While more robust than previous approaches, ML-based systems still required significant manual effort in feature design and struggled with capturing long-range dependencies in text [31].

Deep Learning Approaches

Deep learning revolutionized NER by enabling end-to-end learning of relevant features directly from data, eliminating the need for manual feature engineering [31]. Initial deep learning architectures for NER included:

Recurrent Neural Networks (RNN) with Long Short-Term Memory (LSTM) or Gated Recurrent Unit (GRU) cells, which could process sequential text while maintaining information about previous context [32]. Bidirectional LSTM (BiLSTM) architectures further enhanced this by processing sequences in both directions, capturing context from preceding and following words [8].

The true transformation came with the transformer architecture, introduced in 2017 [35]. Transformers utilized self-attention mechanisms to weigh the importance of different words in a sequence, enabling parallel processing and capturing long-range dependencies more effectively than recurrent architectures [35] [36].

Modern Foundation Models and LLMs

The current state-of-the-art in NER leverages transformer-based foundation models and Large Language Models (LLMs) pretrained on massive text corpora [33] [8]. Models like BERT (Bidirectional Encoder Representations from Transformers) and GPT (Generative Pre-trained Transformer) create deep contextualized word representations that significantly enhance NER performance [36].

These models excel at disambiguating entity types based on subtle contextual clues – a critical capability in materials science where terms like "crystal" might refer to a specific material structure or a general concept depending on context [8]. Foundation models can be further fine-tuned on domain-specific scientific literature, creating specialized NER systems with exceptional accuracy for materials science terminology [18].

Table 1: Evolution of NER Techniques and Their Characteristics

Era	Primary Approach	Key Technologies	Strengths	Limitations
Early Systems	Dictionary-Based	Gazetteers, String Matching	Simple, Interpretable	Poor recall, Constant updates needed [31] [34]
1990s-2000s	Rule-Based	Pattern Matching, Context Rules	Handles some variability	Labor-intensive, Domain-specific [32]
2000s-2010s	Machine Learning	CRF, SVM, Feature Engineering	Generalizable, Statistical foundation	Requires feature engineering, Data hungry [31]
2010s-2018	Deep Learning	LSTM, BiLSTM, Word Embeddings	Automatic feature learning, Context awareness	Sequential processing, Complex training [8] [32]
2018-Present	Transformer & Foundation Models	BERT, GPT, Attention Mechanisms	State-of-the-art accuracy, Contextual understanding	Computational demands, Large data requirements [33] [36]

Quantitative Comparison of NER Approaches

Table 2: Performance Characteristics Across NER Paradigms

Metric	Dictionary-Based	Rule-Based	ML-Based	Deep Learning	Foundation Models
Typical Precision	High (for known entities)	Medium-High	Medium-High	High	Very High [31]
Typical Recall	Low (limited to dictionary)	Medium	Medium-High	High	Very High [31] [32]
Domain Adaptation Effort	High (manual curation)	High (rule creation)	Medium (retraining)	Medium (retraining)	Low (fine-tuning) [18]
Context Understanding	None	Limited	Moderate	Good	Excellent [8]
Handling Ambiguity	Poor	Moderate	Good	Very Good	Excellent [31] [32]
Training Data Requirements	None	None	Large annotated datasets	Large annotated datasets	Can work with few-shot learning [18]

Recent advances in foundation models have demonstrated particularly strong performance in scientific NER tasks. For instance, specialized models like SciBERT and MatSciBERT have achieved F1 scores exceeding 90% on materials science entity extraction, significantly outperforming previous approaches [8]. The key advantage of these models lies in their ability to understand scientific context and terminology without extensive task-specific architecture modifications [18].

Experimental Protocols for NER Implementation

Protocol 1: Implementing a Rule-Based NER System for Materials Synthesis Parameters

Objective: Extract synthesis conditions (temperature, pressure, time) from materials science literature using rule-based patterns.

Materials and Tools:

• Text preprocessing library (e.g., SpaCy for tokenization and part-of-speech tagging)
• Regular expression engine
• Domain-specific dictionaries for material names and synthesis methods

Procedure:

• Text Preprocessing:
  • Convert PDF documents to plain text using OCR tools if necessary
  • Tokenize text into sentences and words
  • Apply part-of-speech tagging and dependency parsing

• Pattern Development:

  • Create regular expressions for numerical values with units (e.g., \d+\s*°C, \d+\s*MPa)
  • Define contextual patterns for synthesis parameters (e.g., "heated at X for Y hours")
  • Compile lists of synthesis verbs ("sintered", "annealed", "hydrothermal")

• Entity Extraction:

  • Apply pattern matching to identify candidate entities
  • Use contextual rules to validate extractions (e.g., temperature values must be preceded by synthesis verbs)
  • Resolve co-references when parameters are referenced later in text

• Validation:

  • Manually review extractions on a sample of documents
  • Calculate precision and recall against a gold-standard annotated corpus
  • Refine rules based on error analysis

Expected Outcomes: A rule-based system capable of extracting explicit synthesis parameters with high precision but potentially lower recall for implicitly described conditions.

Protocol 2: Fine-Tuning a Foundation Model for Materials Science NER

Objective: Adapt a pre-trained transformer model (e.g., BERT) to recognize materials science-specific entities.

Materials and Tools:

• Pre-trained transformer model (BERT, SciBERT, or materials-specific foundation model)
• Annotated dataset of materials science texts with entity labels
• GPU-enabled computing environment
• Deep learning framework (PyTorch/TensorFlow)

Procedure:

• Dataset Preparation:
  • Collect and annotate materials science texts with entity types (material names, properties, characterization techniques)
  • Split data into training (80%), validation (10%), and test (10%) sets
  • Convert annotations to token-level labels compatible with the model

• Model Configuration:

  • Load pre-trained weights from base model
  • Add a classification head for entity types
  • Set hyperparameters (learning rate: 2e-5, batch size: 16-32, epochs: 3-10)

• Training Process:

  • Employ gradual unfreezing of layers if limited training data available
  • Use AdamW optimizer with linear learning rate decay
  • Monitor loss on validation set to prevent overfitting

• Evaluation:

  • Calculate precision, recall, and F1-score on test set
  • Perform error analysis on false positives/negatives
  • Compare against baseline models (CRF, BiLSTM-CRF)

Expected Outcomes: A specialized NER model achieving F1 scores of 85-95% on materials science entities, with robust performance across diverse text types (abstracts, full papers, patents).

Workflow Visualization of Modern NER Systems

The Scientist's Toolkit: Essential Research Reagents for NER Implementation

Table 3: Essential Tools and Libraries for NER Development

Tool/Library	Type	Primary Function	Materials Science Applicability
SpaCy [31] [32]	Open-source Library	Production-ready NLP pipelines with pre-trained models	Fast processing of large literature corpora, customizable for domain terms
Flair [32]	Open-source Library	Advanced sequence labeling with multiple embeddings	Combines BERT, ELMo for higher accuracy on scientific text
BERT [31] [36]	Foundation Model	Contextual word representations	Base for domain adaptation to materials science
SciBERT [8]	Domain-Specific Model	BERT pretrained on scientific literature	Superior performance on technical texts without extensive fine-tuning
Prodigy [31]	Annotation Tool	Active learning-powered data annotation	Efficient creation of labeled datasets for materials entities
Hugging Face Transformers [36]	Model Library	Access to thousands of pre-trained models	Rapid experimentation with different architectures
Labelbox [31]	Annotation Platform	Collaborative data labeling	Team-based annotation of materials science corpora
Stanford CoreNLP [31] [32]	NLP Toolkit	Comprehensive linguistic analysis	Robust preprocessing and grammatical analysis

Advanced Applications in Materials Science

Modern NER systems have enabled several advanced applications in materials science research:

Automated Knowledge Base Construction: NER is used to automatically populate materials databases from scientific literature, extracting information about material compositions, synthesis conditions, and measured properties [8]. Systems can process thousands of papers to build comprehensive databases that would be infeasible to create manually [18].

Synthesis Route Extraction: Advanced NER models identify and link synthesis parameters with resulting material properties, enabling the mining of synthesis-property relationships from literature [8]. This supports the development of predictive models for materials synthesis and processing optimization.

Multimodal Information Extraction: State-of-the-art systems combine text-based NER with image analysis to extract information from figures, charts, and tables in scientific documents [18]. For example, molecular structures depicted in images can be linked with their textual descriptions in the paper.

Research Trend Analysis: By applying NER at scale across decades of scientific literature, researchers can track the emergence of new materials, characterize the evolution of research focus areas, and identify promising directions for future investigation [8].

The integration of these advanced NER capabilities into materials research workflows represents a significant acceleration in the pace of materials discovery and development. As foundation models continue to improve and incorporate more domain-specific knowledge, the role of NER in materials informatics is expected to grow even more prominent [33] [18].

Named Entity Recognition (NER) is a fundamental task in natural language processing that involves identifying and classifying key information entities in text into predefined categories such as person names, organizations, locations, and, in the context of materials science, material names, properties, and synthesis methods [37]. Within materials informatics, effective NER enables the automated extraction of structured knowledge from scientific literature, experimental data, and research publications, thereby accelerating materials discovery and development cycles [33] [38]. This document provides detailed application notes and experimental protocols for implementing two traditional machine learning models—Conditional Random Fields (CRF) and Naïve Bayes Classifiers—for NER tasks in materials science research.

Model Fundamentals and Performance Analysis

Core Model Characteristics

Table 1: Fundamental characteristics of CRF and Naïve Bayes models for NER.

Feature	Conditional Random Fields (CRF)	Naïve Bayes Classifiers (NBC)
Model Type	Discriminative, probabilistic graphical model	Generative, probabilistic classifier
Theoretical Basis	Models conditional probability (P(Y	X)) of label sequence (Y) given input sequence (X) [39]	Applies Bayes' theorem with strong (naïve) independence assumptions between features [40]
Sequence Handling	Excellently models dependencies and correlations between subsequent labels in a sequence [9]	Typically treats each token/entity independently; can be adapted for sequences
Primary Advantage	Considers contextual information for collective classification of sequences [37]	Computational simplicity, efficiency with small datasets, resistance to overfitting [40]
Key Limitation	Computationally intensive for long sequences and large feature sets	Strong feature independence assumption often violates linguistic realities

Performance Comparison in Scientific Domains

Table 2: Documented performance of CRF and Naïve Bayes in scientific NER tasks.

Model	Application Domain	Reported Performance	Reference
MatBERT-CNN-CRF	Perovskite materials NER	F1-score: 90.8% (1-6% improvement over BERT, SciBERT, MatBERT) [9]	Zhang et al., 2024
Naïve Bayes with Filters	Chemical NER (CHEMDNER corpus)	Precision: 0.74, Recall: 0.95, Balanced Accuracy: 0.92 [40]	unnamed reference, 2022
Diffusion-CRF-BiLSTM	General & biomedical NER	Significant gains in recall, accuracy, and F1 scores on multiple datasets [39]	unnamed reference, 2025
BEM-NER (CRF-based)	Chinese NER (Weibo, Ontonotes)	Significant performance enhancement versus existing models [41]	unnamed reference, 2025

Experimental Protocols

Protocol 1: CRF for Materials Science NER

Application Note: CRF is particularly effective for NER tasks where contextual information and label dependencies are crucial, such as identifying multi-word material names and their properties in scientific literature [9] [39].

Workflow:

Diagram 1: CRF model development workflow.

Materials and Reagents:

Table 3: Research reagents and computational tools for CRF-based NER.

Item	Specification/Function	Example Tools/Libraries
Annotated Dataset	Gold-standard corpus with entity annotations for training and evaluation	CHEMDNER [40], Perovskite dataset [9]
Feature Extraction	Converts text tokens into machine-readable feature vectors	sklearn-crfsuite, CRFSuite
CRF Implementation	Algorithms for training and inference on sequence data	CRF++, sklearn-crfsuite, AllenNLP
Evaluation Framework	Metrics and scripts for assessing model performance	Precision, Recall, F1-score [9]

Step-by-Step Methodology:

• Data Preparation and Annotation

  • Collect a domain-specific corpus of materials science literature (e.g., 800 annotated abstracts for perovskite materials) [9].
  • Annotate entities using a standardized scheme such as IOBES, which provides higher F1-scores compared to other schemes [9]. Common entity types in materials science include:
    • MAT: Material names (e.g., "methylammonium lead iodide")
    • APL: Application contexts (e.g., "photovoltaics")
    • DSC: Material descriptors and properties (e.g., "bandgap", "efficiency")
    • SMT: Synthesis methods and techniques (e.g., "spin-coating")
  • Split the annotated data into training, validation, and test sets (e.g., 70%-15%-15%).

• Feature Engineering

  • Extract a comprehensive set of features for each token in the input sequence:
    • Word-level features: The token itself, its lowercased form, prefixes, and suffixes (typically 2-4 characters).
    • Orthographic features: Word shape (capitalization, punctuation), digit patterns (e.g., contains digits, is a year).
    • Contextual features: Tokens in a window around the current token (e.g., ±2 tokens).
    • Domain-specific features: Incorporation of pre-trained word embeddings from models like MatBERT, which is specifically trained on materials science literature [9].

• Model Training

  • Implement the CRF model using libraries such as sklearn-crfsuite.
  • Define the CRF objective function, which models the conditional probability of the label sequence given the observations: ( P(y|x) = \frac{1}{Z(x)} \exp\left(\sum{k=1}^{K} \thetak fk(y, x)\right) ) where (Z(x)) is the normalization factor, (θk) are the model parameters, and (f_k) are the feature functions [39].
  • Train the model to learn the parameters (θ_k) that maximize the log-likelihood of the training data. Regularization (L1 or L2) is typically applied to prevent overfitting.

• Evaluation and Deployment

  • Evaluate the trained model on the held-out test set using standard metrics: Precision, Recall, and F1-score [9].
  • For the perovskite NER task, the MatBERT-CNN-CRF model achieved an F1-score of 90.8% [9].
  • Deploy the model to process new, unlabeled texts for material entity extraction.

Protocol 2: Naïve Bayes for Chemical NER

Application Note: The Naïve Bayes approach is highly effective for extracting Chemical Named Entities (CNEs) from scientific texts, particularly with imbalanced datasets where CNEs constitute only a small fraction of the text [40]. It is recognized for its easy implementation, accuracy, and speed in cheminformatics and medicinal chemistry applications.

Workflow:

Diagram 2: Naïve Bayes classifier workflow for CNER.

Materials and Reagents:

Table 4: Research reagents and computational tools for Naïve Bayes-based CNER.

Item	Specification/Function	Example Tools/Libraries
Text Corpus	Collection of scientific abstracts/publications for processing	CHEMDNER (10,000 abstracts) [40]
Tokenization Tool	Segments text into processable units/tokens	Python NLTK WordPunktTokenizer [40]
Feature Generator	Creates multi-n-gram representations from text fragments	Custom Python scripts [40]
Naïve Bayes Implementation	Algorithm for calculating posterior probabilities based on n-gram features	PASS software algorithm [40]

Step-by-Step Methodology:

• Text Preprocessing and Tokenization

  • Utilize a tokenizer such as the Python NLTK WordPunkt tokenizer to break input text into tokens [40].
  • For each target token, create Fragments of Text (FoTs) by concatenating the target token with a context window of tokens before and after it (e.g., ±1, ±2, ±3 tokens) [40].

• Multi-n-gram Generation

  • For each FoT, generate a set of "multi-n-grams"—continuous sequences of one to n symbols (typically n=5) from the token sequence [40].
  • This approach allows the recognition of novel CNEs by capturing characteristic symbol patterns.

• Naïve Bayes Classification

  • Implement the Naïve Bayes classifier to calculate the posterior probability that a given FoT belongs to a specific CNE type (e.g., "Systematic", "Trivial", "Formula", "Family", "Abbreviation") [40].
  • The core calculation involves: ( ln\left[\frac{P(C|F)}{1-P(C|F)}\right] \cong ln\left[\frac{P(C)}{1-P(C)}\right] + \sum{i=1}^{m} ln\left[\frac{P(gi|C)}{1-P(gi|C)}\right] ) where (P(C|F)) is the probability that fragment (F) (represented by n-grams (g1, ..., g_m)) belongs to class (C) [40].
  • The coefficients for each n-gram are calculated directly from the training data based on their frequency in different CNE types.

• Filtering and Evaluation

  • Apply specially developed filters to remove false positives and refine the extracted entity list [40].
  • Evaluate performance on a labeled corpus like CHEMDNER, where this method achieved a recall of 0.95, precision of 0.74, and balanced accuracy of 0.92 [40].

The Scientist's Toolkit

Table 5: Essential resources for implementing NER in materials science.

Category	Resource	Description and Application
Datasets	CHEMDNER	Contains 10,000 abstracts with over 80,000 labeled chemical entities for training and evaluation [40].
	Perovskite Dataset	A public dataset of 800 annotated abstracts from perovskite literature, facilitating domain-specific NER [9].
Pre-trained Models	MatBERT	A BERT model pre-trained on materials science literature, generates domain-aware word embeddings to boost NER performance [9].
Software & Libraries	sklearn-crfsuite	Python library for implementing CRF models with scikit-learn compatible interface.
	NLTK	Natural Language Toolkit for tokenization, preprocessing, and n-gram generation [40].
Evaluation Metrics	Precision, Recall, F1-score	Standard metrics for quantifying NER performance and comparing model effectiveness [9] [40].

CRF and Naïve Bayes classifiers represent two distinct yet highly effective traditional machine learning approaches for Named Entity Recognition in materials science. CRF models excel in capturing sequential dependencies and contextual information, making them suitable for accurately identifying entity boundaries and types in complex scientific text [9] [39]. In contrast, Naïve Bayes classifiers offer a computationally efficient and robust solution, particularly valuable for extracting chemical named entities from highly imbalanced datasets where entities constitute a small fraction of the text [40]. The choice between these models depends on specific research requirements, including dataset characteristics, computational resources, and desired performance metrics. Both approaches, when properly implemented using the protocols outlined herein, can significantly accelerate materials discovery by enabling efficient knowledge extraction from the vast and growing scientific literature.

Named Entity Recognition (NER) is a fundamental component of Natural Language Processing (NLP) that involves identifying and classifying key information entities—such as material names, properties, and synthesis methods—within unstructured text. For the materials science domain, efficiently extracting structured data from the rapidly growing body of scientific literature is crucial for accelerating discovery [42]. Deep learning architectures, particularly Bidirectional Long Short-Term Memory (BiLSTM) networks and Convolutional Neural Networks (CNNs), have become cornerstone technologies for this task. Hybrid models that integrate their complementary strengths demonstrate superior capability in handling the complex terminology and contextual dependencies characteristic of materials science literature [43] [39].

This application note provides a detailed technical overview of BiLSTM, CNN, and their hybrid architectures within the context of materials science NER. It presents structured performance comparisons, detailed experimental protocols for implementation, and visualizations of key workflows to equip researchers and scientists with the practical tools needed to deploy these advanced neural networks.

Core Network Components

Convolutional Neural Networks (CNNs) excel at extracting local, position-invariant features from input data. In NER tasks, especially at the character level, CNNs effectively identify morphological patterns—such as prefixes, suffixes, and word roots—that are highly indicative of entity boundaries and categories. This capability is particularly valuable for processing technical compounds and material nomenclature [43].

Bidirectional Long Short-Term Memory Networks (BiLSTMs) process sequential data in both forward and backward directions. This dual processing allows the network to capture long-range contextual dependencies from both past and future words in a sentence, which is essential for resolving entity ambiguities. For instance, determining whether "lead" refers to a metal or the action in a sentence relies heavily on surrounding context [39].

Conditional Random Fields (CRF) are often used as a final output layer in sequence labeling models. While not a deep learning component per se, CRF incorporates grammatical constraints and ensures global consistency in the predicted sequence of labels, which significantly improves the coherence of the final extracted entities [39] [44].

Quantitative Performance of Architectures

The performance of different neural architectures has been systematically evaluated on various NER tasks. The following table summarizes key metrics from recent studies, illustrating the relative strengths of each approach.

Table 1: Performance Comparison of Deep Learning Models on NER Tasks

Model Architecture	Dataset(s)	Key Metric(s)	Reported Performance	Reference
BiLSTM with Domain-Specific Embeddings	Materials Science NER Datasets	F1 Score	Consistently outperformed general BERT	[42]
WP-XAG (WoBERT, XLSTM, Adaptive Attention)	Resume, Weibo, CMeEE, CLUENER2020	F1 Score	96.89%, 74.89%, 72.19%, 80.96%	[43]
Enhanced Diffusion-CRF-BiLSTM (EDCBN)	Multiple NER Datasets	Recall, Accuracy, F1	Significant gains reported	[39]
XLNet-BiLSTM-CRF	Standard NER Benchmarks	F1 Score	State-of-the-art results at time of publication	[44]

The data demonstrates a clear trend: while simpler models like BiLSTM can be highly effective when powered by domain-specific knowledge [42], increasingly sophisticated hybrid architectures are setting new benchmarks for accuracy and robustness on complex NER tasks [43] [39].

Experimental Protocols for Hybrid NER Models

This section provides a detailed, step-by-step methodology for implementing and training a hybrid CNN-BiLSTM-CRF model for Named Entity Recognition, adaptable for domain-specific applications like materials science.

Data Preparation and Preprocessing Protocol

• Text Tokenization: Convert raw text into a sequence of tokens. For languages like Chinese, this involves word or character segmentation. For English, tokenize into words or subwords [43].
• Label Encoding: Annotate tokens with standard NER tags (e.g., B-PER, I-PER, B-MAT, I-MAT, O) using the BIO (Begin, Inside, Outside) scheme. For materials science, define a custom tag set (e.g., B-Material, I-Synthesis, B-Property).
• Embedding Layer Generation:
  • Option A (Pre-trained Embeddings): Load static word vectors from domain-specific pre-trained models (e.g., FastText, Word2Vec). This is highly recommended for leveraging existing materials science corpora [45].
  • Option B (Contextual Embeddings): Use a pre-trained transformer model like BERT or its variants (e.g., SciBERT, MatBERT) to generate dynamic, context-aware embeddings for each token. MatBERT has been shown to provide a 1% to 12% performance improvement on materials science tasks [42].
• Dataset Splitting: Partition the annotated data into training (∼70-80%), validation (∼10-15%), and test (∼10-15%) sets, ensuring a representative distribution of entity classes across splits.

Model Implementation and Training Protocol

• Architecture Assembly:
  • Input Layer: Accepts sequences of token indices or embedding vectors.
  • CNN Feature Extraction: Apply a 1D convolutional layer with multiple filter widths (e.g., 3, 4, 5) to capture local n-gram features. Use ReLU activation and follow with max-pooling [43].
  • Sequence Modeling: Feed the CNN's output features into a BiLSTM layer. The BiLSTM captures long-range contextual dependencies from both directions in the sequence, which is critical for disambiguating entities [39].
  • CRF Output Layer: The output vectors from the BiLSTM are fed into a CRF layer. The CRF learns transition constraints between consecutive labels, preventing invalid sequences (e.g., I-MAT cannot follow O) and improving overall label consistency [44].
• Model Training:
  • Loss Function: Utilize a combined loss, typically the negative log-likelihood from the CRF layer.
  • Optimizer: Use Adam or a similar adaptive optimizer for stable convergence.
  • Regularization: Implement techniques like Dropout (after embedding, CNN, and BiLSTM layers) and Early Stopping based on validation loss to prevent overfitting [45].
  • Hyperparameter Tuning: Experiment with embedding dimensions, CNN filter sizes and counts, BiLSTM hidden unit sizes, and learning rates.

Evaluation and Validation Protocol

• Metric Calculation: Evaluate the model on the held-out test set using standard token-level precision, recall, and F1-score.
• Error Analysis: Manually inspect misclassified examples, particularly false positives and false negatives, to identify model weaknesses or annotation inconsistencies.
• Ablation Studies: Quantify the contribution of each component (CNN, BiLSTM, CRF) by training and evaluating simplified model variants and comparing their performance to the full hybrid model.

The Scientist's Toolkit: Research Reagents & Essential Materials

Table 2: Key Resources for Implementing Deep Learning NER Models

Resource Category	Specific Tool / Dataset	Function / Description	Relevance to Materials Science NER
Pre-trained Language Models	MatBERT, SciBERT, WoBERT	Provide context-aware word representations pre-trained on scientific corpora.	Domain-specific models like MatBERT significantly boost F1 scores by understanding scientific terminology [42] [43].
Annotation Tools	Label Studio, BRAT	Software for manually annotating text documents with entity labels.	Essential for creating gold-standard training data for custom entities (e.g., "perovskite", "MOF").
Deep Learning Frameworks	PyTorch, TensorFlow	Open-source libraries for building, training, and deploying neural networks.	Provide flexible environments for implementing hybrid CNN-BiLSTM-CRF architectures.
Computational Hardware	GPUs (NVIDIA), TPUs	Hardware accelerators for efficient deep learning model training.	Crucial for reducing training time for large models and datasets.
Benchmark Datasets	CLUENER2020, CMeEE, custom materials corpora	Standardized datasets for training and benchmarking model performance.	CMeEE is a Chinese medical dataset; sourcing or creating a similar dataset for materials is critical [43].

Workflow and Architecture Visualization

Diagram 1: CNN-BiLSTM-CRF NER Architecture.

Diagram 2: Materials Science NER Workflow.

The exponential growth of materials science literature has created a significant bottleneck in extracting and utilizing the knowledge contained within published research. Named Entity Recognition (NER)—a fundamental natural language processing (NLP) technique for identifying and classifying key information elements in text—has emerged as a critical solution for transforming unstructured scientific literature into structured, machine-readable data. The transformer revolution, initiated by the development of BERT (Bidirectional Encoder Representations from Transformers), has dramatically advanced the capabilities of NER systems in the materials science domain. These advances have enabled the automated extraction of critical information such as material names, properties, synthesis parameters, and application contexts from vast corpora of scientific text.

Domain-specific adaptations of the original BERT architecture, particularly MatBERT and MaterialsBERT, have demonstrated remarkable improvements in processing materials science literature. These models address the unique challenges posed by domain-specific terminology, notations, and contextual relationships that general-purpose language models often struggle to interpret accurately. The development of these specialized transformers has accelerated materials discovery pipelines, enabled large-scale knowledge graph construction, and facilitated the creation of comprehensive materials property databases that would be impractical to assemble through manual curation alone.

Model Architectures and Technical Specifications

The evolution from BERT to its materials-specific variants represents a paradigm shift in how natural language processing is applied to scientific literature. Each model builds upon its predecessor while introducing domain-specific optimizations that enhance performance on materials science tasks.

BERT (Bidirectional Encoder Representations from Transformers) serves as the foundational architecture upon which domain-specific models are built. Utilizing a multi-layer bidirectional transformer architecture, BERT employs a masked language model objective that randomly masks portions of the input text and trains the model to predict the masked words based solely on their context. This approach enables deep bidirectional representations that fuse left and right contexts across all layers, making it particularly effective for understanding complex linguistic relationships. The model was originally pre-trained on BookCorpus and English Wikipedia, providing broad general language understanding but limited coverage of scientific terminology.

MatSciBERT represents a significant adaptation specifically designed for the materials science domain. Rather than training from scratch, MatSciBERT employs a domain-adaptive pre-training approach that continues pre-training SciBERT on a carefully curated materials science corpus. This corpus encompasses approximately 285 million words drawn from peer-reviewed publications across key materials families including inorganic glasses, metallic glasses, alloys, and cement. The vocabulary overlap between the materials science corpus and SciBERT is approximately 53.64%, making SciBERT a more suitable starting point than the original BERT model. This strategic approach allows MatSciBERT to develop specialized representations of materials science terminology while retaining the general linguistic capabilities of its predecessor.

MaterialsBERT follows a similar domain-adaptive approach but with distinct architectural and training decisions. This model was developed by continuing pre-training from PubMedBERT rather than SciBERT, utilizing a massive corpus of 2.4 million materials science abstracts. The choice of PubMedBERT as a base leverages its strong foundation in scientific language, particularly from the biomedical domain, which shares certain characteristics with materials science literature. MaterialsBERT has demonstrated state-of-the-art performance on multiple materials science NER datasets, outperforming other BERT-based models on three out of five benchmark tasks.

Table 1: Technical Specifications of Materials Science Transformer Models

Model	Base Architecture	Pre-training Corpus	Vocabulary Size	Domain Adaptation Method
BERT	Transformer	BookCorpus, English Wikipedia (3.3B words)	30,522	General language model
MatSciBERT	SciBERT	285M words from materials science publications	~30,000	Continued pre-training on domain corpus
MaterialsBERT	PubMedBERT	2.4M materials science abstracts	~30,000	Continued pre-training on domain corpus

Table 2: Performance Comparison on Materials Science NER Tasks (F1-Scores)

Model	SOFC Dataset	Matscholar Dataset	PolymerAbstracts	Perovskite Dataset
BERT	0.783	0.821	0.734	0.795
SciBERT	0.826	0.857	0.792	0.834
MatSciBERT	0.894	0.901	0.845	0.908
MaterialsBERT	0.882	0.893	0.867	0.896

Experimental Protocols and Implementation

Domain-Specific Pre-training Methodology

The development of effective materials science language models requires careful implementation of domain-adaptive pre-training. The following protocol outlines the key steps for transitioning from a general-purpose language model to a domain-specific expert:

Corpus Curation and Preparation: Assemble a comprehensive collection of materials science texts. The MatSciBERT corpus, for example, contained approximately 285 million words drawn from 150,000 peer-reviewed publications focused on inorganic glasses, metallic glasses, alloys, and cement. Each document undergoes text extraction and cleaning to remove formatting artifacts, tables, and figures while preserving the core scientific content.

Vocabulary Alignment: Analyze the overlap between the domain corpus vocabulary and the base model's tokenizer. For MatSciBERT, the uncased vocabulary showed 53.64% overlap with SciBERT compared to only 38.90% with original BERT, justifying the selection of SciBERT as the foundation. This alignment minimizes the occurrence of unknown tokens and improves the model's ability to represent domain-specific terms.

Continued Pre-training: Initialize the model with base weights (SciBERT or PubMedBERT) and continue pre-training using the masked language modeling objective on the domain corpus. Standard parameters include a batch size of 32-64, learning rate of 5e-5 with linear decay, and training for 2-4 epochs. The training objective randomly masks 15% of tokens, with 80% replaced by [MASK], 10% by random tokens, and 10% left unchanged.

Named Entity Recognition Fine-tuning Protocol

Once pre-training is complete, the model can be fine-tuned for specific NER tasks using annotated datasets:

Annotation Schema Development: Define an ontology of entity types relevant to materials science. A comprehensive schema might include: MATERIAL, PROPERTY, PROPERTYVALUE, SYNTHESISMETHOD, CHARACTERIZATIONTECHNIQUE, and APPLICATION. The PolymerAbstracts dataset, for example, uses 8 entity types including POLYMER, POLYMERCLASS, PROPERTYNAME, and PROPERTYVALUE.

Dataset Preparation: Convert annotated texts into token-label pairs using the model's tokenizer. For BERT-based models, apply WordPiece tokenization and align labels with the resulting tokens. For sequences longer than the model's maximum length (typically 512 tokens), employ sliding window approaches or truncation strategies.

Model Fine-tuning: Add a linear classification layer on top of the pre-trained encoder and train with a cross-entropy loss function. Recommended hyperparameters include: batch size of 16-32, learning rate of 3e-5 to 5e-5, and training for 20-50 epochs with early stopping. Employ gradient accumulation when working with limited GPU memory.

Evaluation Metrics: Assess model performance using standard sequence labeling metrics: precision, recall, and F1-score calculated at the token level. For comprehensive evaluation, also consider span-level F1 which requires exact match of entity boundaries and type.

Advanced Applications and Implementation Frameworks

Large-Scale Materials Property Extraction

Transformer-based NER models have enabled the development of automated pipelines for large-scale materials property extraction from scientific literature. The following protocol outlines a production-scale implementation:

Corpus Filtering: Identify relevant documents using keyword-based filtering. For polymer-focused extraction, this might involve selecting documents containing the string "poly" in titles or abstracts, reducing a corpus of 2.4 million documents to approximately 681,000 relevant papers.

Text Segmentation: Divide documents into logical units (paragraphs or sections) to isolate coherent descriptions of materials and properties. A full-text polymer corpus might yield 23.3 million paragraphs requiring processing.

Hierarchical Filtering: Implement a two-stage filtering approach to identify text segments containing extractable property data. First, apply property-specific heuristic filters to detect paragraphs mentioning target properties, reducing the corpus to approximately 11% of original segments. Second, apply an NER filter to identify paragraphs containing complete material-property-value tuples, further reducing to about 3% of segments with extractable records.

Relation Extraction: Employ rule-based approaches or additional classification layers to associate extracted entities, forming complete material-property records. This step transforms isolated entities into structured tuples: [material, property, value, unit].

Data Normalization: Implement techniques to handle variations in material names (e.g., "PMMA," "poly(methyl methacrylate)," "poly-MMA") and property units, ensuring consistent representation in the final database.

Question Answering for Targeted Information Extraction

Recent research has demonstrated the effectiveness of Question Answering (QA) frameworks for extracting specific material-property relationships, offering a flexible alternative to traditional NER pipelines:

Model Selection: Fine-tune domain-specific transformers (MatSciBERT, MaterialsBERT) on general QA datasets like SQuAD2.0, which includes examples with empty answers to improve the model's ability to recognize when information is not present in the text.

Query Formulation: Design natural language questions tailored to target properties, such as "What is the bandgap of material X?" This approach enables zero-shot extraction without task-specific training data.

Snippet Processing: Divide documents into coherent text segments (snippets) of appropriate length, typically 1-3 paragraphs, to provide sufficient context while maintaining computational efficiency.

Answer Extraction: Apply the QA model to each snippet-question pair, extracting text spans containing the relevant values. Implement confidence thresholds to balance precision and recall, with typical optimal thresholds ranging from 0.1-0.2.

Value Normalization: Convert extracted text spans to standardized units and formats, handling variations in numerical representation and unit notation common in scientific literature.

Table 3: Performance Comparison of Extraction Methods on Perovskite Bandgap Extraction

Method	Precision	Recall	F1-Score	Resource Requirements
Rule-based (CDE2)	0.856	0.317	0.456	Low
QA MatSciBERT	0.812	0.782	0.797	Medium
QA MaterialsBERT	0.795	0.768	0.781	Medium
GPT-4	0.843	0.801	0.822	High
Fine-tuned NER	0.901	0.875	0.888	High (initial setup)

The Scientist's Toolkit: Research Reagent Solutions

Implementing transformer-based NER systems requires both software and data resources. The following table outlines essential components for developing and deploying materials science NER pipelines:

Table 4: Essential Resources for Materials Science NER Implementation

Resource	Type	Description	Application
MatSciBERT	Pre-trained Model	BERT model continued-trained on 285M words from materials science literature	General materials science NER tasks
MaterialsBERT	Pre-trained Model	PubMedBERT further trained on 2.4M materials science abstracts	Polymer-focused extraction and relation classification
PolymerAbstracts	Annotated Dataset	750 abstracts labeled with 8 entity types (POLYMER, PROPERTY_VALUE, etc.)	Training and evaluating polymer NER systems
Matscholar	Annotated Dataset	Comprehensive collection of materials science entity annotations	Benchmarking model performance
ChemDataExtractor	Software Toolkit	Rule-based system for chemical information extraction	Baseline comparison and hybrid approaches
Hugging Face Transformers	Software Library	Python implementation of transformer architectures	Model fine-tuning and inference
Polymerscholar.org	Database	Extracted polymer property data from 2.4M articles	Validation and analysis of extraction results

Future Directions and Challenges

The application of transformer models in materials science NER continues to evolve, with several promising research directions emerging. Parameter-efficient fine-tuning techniques, such as Low-Rank Adaptation (LoRA) and QLoRA, enable resource-effective adaptation of large language models to specialized BioNER and materials NER tasks. These approaches can fine-tune models like Llama3.1 on a single GPU with 16GB memory while maintaining competitive performance, democratizing access to advanced NER capabilities [46].

Multi-task learning approaches represent another frontier, with researchers developing unified models capable of extracting diverse entity types simultaneously. While models like BioBERT and PubMedBERT are typically designed for single-entity-type extraction, recent efforts have demonstrated the feasibility of multi-task BioNER models that can process various entity categories in a single pass, potentially reducing both training and inference costs for comprehensive materials information extraction [46].

The integration of large language models like GPT-4 and Llama 2 with traditional NER pipelines offers opportunities to enhance relation extraction and handle complex entity descriptions, though challenges with hallucination and computational cost remain significant considerations. Future developments will likely focus on hybrid approaches that leverage the strengths of both specialized NER models and general-purpose LLMs, creating systems capable of comprehensive knowledge extraction from the ever-expanding materials science literature [47].

The exponential growth of materials science literature presents a formidable challenge for researchers attempting to manually extract critical information from vast publications. Named Entity Recognition (NER)—the process of automatically identifying and classifying material-specific entities such as compounds, elements, structures, and properties—has become essential for constructing knowledge graphs and accelerating materials discovery [13]. Traditional NER approaches, typically formulated as sequence labeling tasks using models like BiLSTM-CRF, often struggle with capturing complex semantic information and effectively handling nested entities (where one entity contains another) [13] [48].

The Machine Reading Comprehension (MRC) paradigm represents a transformative shift in tackling NER problems. Instead of assigning labels to each word in a sequence, MRC frames entity extraction as a question-answering task. For each entity type (e.g., material, property), a specific query is designed, and the model's objective is to extract the answer span—the named entity—from the context [13] [49]. This innovative approach more naturally leverages the deep contextual understanding capabilities of pre-trained language models, leading to significant performance improvements, particularly for the complex and overlapping entities prevalent in materials science text [13] [48].

Performance of MRC-based NER

Quantitative evaluations demonstrate that the MRC paradigm achieves state-of-the-art performance on key materials science datasets. The following table summarizes the F1 scores reported for a MatSciBERT-MRC model across several benchmarks.

Table 1: Performance of MRC-based NER on materials science datasets.

Dataset	Domain Focus	Reported F1-Score (%)
Matscholar	General materials science entities	89.64 [13]
BC4CHEMD	Chemical entities	94.30 [13]
NLMChem	Chemical entities	85.89 [13]
SOFC	Solid Oxide Fuel Cells	85.95 [13]
SOFC-Slot	Solid Oxide Fuel Cells	71.73 [13]

These results underscore the effectiveness of the MRC approach across diverse sub-domains within materials science. The lower performance on the SOFC-Slot dataset highlights the ongoing challenge of extracting highly specific, slot-filling type information, an area for continued research [13].

Experimental Protocols and Methodologies

Core MRC for NER Protocol

Implementing MRC for NER involves a structured pipeline, from data preparation to model inference. The workflow can be visualized as follows, illustrating the key stages of the process.

Diagram Title: MRC for NER Workflow

Protocol Steps:

• Data Conversion: Transform standard NER annotations (token-sequence labels) into MRC-style triples of (Context, Query, Answer) [13].

  • Example: For a sentence "LiFePO4 has a high capacity." with the entity "LiFePO4" labeled as MATERIAL, the generated triple would be:
    • Context: "LiFePO4 has a high capacity."
    • Query: "Which material is mentioned in the text?"
    • Answer: "LiFePO4"

• Query Generation: Manually design natural language questions for each entity type. These queries incorporate prior knowledge and are critical for guiding the model. Queries should be unambiguous and broadly defined based on dataset annotation guidelines [13].

  • Example Queries from Matscholar Dataset:
    • For MATERIAL: "Which material is mentioned in the text?"
    • For APPLICATION: "Find the application of the material."
    • For PROPERTY: "Which property is mentioned in the text?" [13]

• Model Construction: Build the MRC model architecture.

  • Encoder: Use a pre-trained language model like BERT or its domain-specific variant (e.g., MatSciBERT) as the backbone to encode the concatenated [CLS] query [SEP] context [SEP] sequence [13].
  • MRC Head: Implement two independent binary classifiers on top of the encoded representations to predict the start and end positions of the answer span for a given query [13]. This design allows for the extraction of multiple entities per context.

• Training: Train the model using standard cross-entropy loss for the two classification tasks (start and end positions). The training objective is for the model to correctly identify the text spans that answer the queries [13].

• Inference: For a new text, run forward passes with all predefined entity-type queries. The start and end classifiers will output the probability of each token being a start or end position. Decode these predictions to extract the final entity spans [13].

Advanced Protocol: Multi-Question MRC (MQMRC)

To address computational inefficiency when dealing with many entity types, a Multi-Question MRC approach can be employed.

Diagram Title: Multi-Question MRC Approach

Protocol Steps:

• Input Formulation: Concatenate all queries for different entity types with the context into a single input sequence for the model [50].
• Model Architecture: Utilize a BERT-based architecture with self-attention mechanisms that allow the model to jointly consider all questions and the context in a single forward pass [50].
• Benefits: This method leverages interactions between different entity types and leads to significantly faster training and inference times (reported to be 2.5x faster training and 2.3x faster inference) compared to processing queries one-by-one, without sacrificing accuracy [50].

The Scientist's Toolkit: Research Reagent Solutions

This section details the essential computational "reagents" required to implement MRC-based NER for materials science.

Table 2: Essential Research Reagents for MRC-based NER.

Tool/Resource	Type	Function in the Experiment
MatSciBERT	Pre-trained Language Model	Domain-specific BERT model, pre-trained on a massive corpus of materials science text, providing foundational semantic understanding for the domain [13].
Matscholar Dataset	Annotated Corpus	Benchmark dataset for evaluating NER performance, containing annotated entities like MATERIAL, PROPERTY, and APPLICATION from materials science abstracts [13].
BERT-base Model	Pre-trained Language Model	General-purpose BERT model; can be used as a starting point if a domain-specific model is unavailable, though performance may be lower [13].
R-NET (e.g., MS MRC)	MRC Model Architecture	An example of a neural MRC model using a self-matching attention mechanism, which can be used for analysis and prototyping [51].
Retrieval-Augmented Generation (RAG)	LLM Enhancement Framework	Framework used with Large Language Models (LLMs) to automatically customize and define taxonomies for specific manufacturing processes, reducing reliance on manual definitions [52].

Advanced Application: Integrating Large Language Models

The field is rapidly evolving with the integration of Large Language Models (LLMs). A promising framework uses Retrieval-Augmented Generation (RAG) to automatically customize entity taxonomies for specific manufacturing processes (e.g., fused deposition modeling) by integrating expert knowledge from academic materials and the internal knowledge of LLMs [52]. This approach can be implemented via:

• In-Context Learning (ICL): Providing the LLM with a few annotated examples within the prompt to perform NER without updating the model's weights [52].
• Fine-tuning: Supervised fine-tuning of an LLM on a specific NER dataset, which can achieve high performance (e.g., F1 scores above 0.91) but requires more computational resources [52].

The exponential growth of materials science literature has created a significant bottleneck in research, with the number of published articles increasing at a compounded annual rate of 6% [12]. This deluge of information makes it challenging for researchers to connect new findings with established knowledge and identify quantitative trends through manual analysis alone. Named Entity Recognition (NER) has emerged as a critical natural language processing (NLP) technology that enables the automatic extraction of structured information from unstructured scientific text, thereby accelerating materials discovery [19] [12].

General-purpose material property extraction pipelines represent a paradigm shift in materials informatics. Unlike previous approaches that focused on specific properties using keyword searches or regular expressions, these pipelines aim to extract any material property information at scale [12]. The fundamental challenge lies in transforming published literature—written in natural language that is not machine-readable—into structured database entries that allow for programmatic querying and analysis. This capability is particularly valuable for addressing data scarcity in materials informatics, where training property predictors traditionally requires painstaking manual curation of data from literature [12].

Core NER Models and Performance

Domain-Specific Language Models

The effectiveness of NER in materials science hinges on domain-specific language models that understand materials-specific notations and jargon. Several specialized models have been developed, each with distinct architectures and training corpora:

MatSciBERT is trained on a carefully curated corpus of approximately 150,000 materials science papers containing ~285 million words, focusing on inorganic glasses, metallic glasses, alloys, and cement and concrete [53]. The model was developed using domain-adaptive pre-training, initializing weights from SciBERT due to a 53.64% vocabulary overlap with the materials science corpus [53]. This specialization enables superior performance on materials-specific tasks compared to general-purpose language models.

MaterialsBERT was trained on 2.4 million materials science abstracts, building upon PubMedBERT through continued pre-training on domain-specific text [12]. This model powers a general-purpose property extraction pipeline and has demonstrated outperformance over other baseline models in three out of five named entity recognition datasets [12].

MatBERT represents another approach to domain specialization, pre-trained on a substantial collection of materials science literature [9]. This model has been successfully applied to NER tasks across multiple materials datasets and serves as the foundation for more specialized architectures like MatBERT-CNN-CRF, which combines MatBERT's embedding capabilities with convolutional neural networks for enhanced feature extraction [9].

Performance Comparison of NER Models

Table 1: Performance comparison of NER models on materials science tasks

Model	Training Data	Architecture	Reported Performance (F1)	Key Applications
MatSciBERT	~150K papers, ~285M words [53]	Transformer-based	State-of-the-art on 3 NER datasets [53]	General materials information extraction
MaterialsBERT	2.4M materials science abstracts [12]	BERT-based	Outperforms baselines in 3/5 NER datasets [12]	Polymer property extraction
MatBERT-CNN-CRF	Materials science literature [9]	MatBERT + CNN + CRF	90.8% on perovskite dataset [9]	Perovskite material knowledge extraction
Generic BERT	General corpora (Wikipedia, BookCorpus) [9]	Transformer-based	Lower than domain-specific models [9]	Baseline comparison

The performance advantage of domain-specific models is consistently demonstrated across multiple studies. The MatBERT-CNN-CRF model, for instance, shows performance improvements of 1-6% compared to generic BERT, SciBERT, and even the base MatBERT models on perovskite material datasets [9]. This enhancement is achieved through the incorporation of a convolutional neural network that better captures local contextual features and a conditional random field layer that effectively models label dependencies [9].

Annotation Frameworks and Ontologies

Entity Typing Schemes

The foundation of any effective NER system lies in its annotation framework, which defines the entity types to be extracted. Different research efforts have employed varying ontologies tailored to their specific extraction goals:

The PolymerScholar ontology utilizes eight entity types: POLYMER, POLYMERCLASS, PROPERTYVALUE, PROPERTYNAME, MONOMER, ORGANICMATERIAL, INORGANICMATERIAL, and MATERIALAMOUNT [12]. This ontology is designed to capture key pieces of information commonly found in abstracts, enabling the extraction of material property records for downstream applications. The annotations in this framework avoid the traditional BIO (Beginning-Inside-Outside) tagging scheme, instead opting for a simpler approach where only tokens belonging to the ontology are annotated, and all other tokens are labeled as 'OTHER' [12].

The Matscholar ontology employs a broader set of categories, including inorganic material mentions, sample descriptors, phase labels, material properties and applications, as well as synthesis and characterization methods [19]. This framework has been used to extract more than 80 million materials-science-related named entities from 3.27 million abstracts, achieving an F1 score of 87% [19].

For perovskite-specific applications, researchers have adopted the IOBES (Inside-Outside-Beginning-End-Single) labeling scheme, which has been shown to provide higher F1-scores compared to alternative schemes [9]. This scheme labels individual token entities as S-X (where X represents the entity type), while multi-token entities use B (begin), I (inside), and E (end) tags [9].

Annotation Quality and Consistency

Maintaining annotation quality is crucial for training reliable NER models. The PolymerScholar project reported strong inter-annotator agreement metrics, with a Fleiss Kappa of 0.885 and pairwise Cohen's Kappa scores of (0.906, 0.864, 0.887) across three annotators [12]. These metrics, comparable to those reported elsewhere in the literature, indicate good homogeneity in the annotations and reflect the effectiveness of their annotation guidelines.

The annotation process typically involves multiple rounds with progressive refinement of guidelines. In the PolymerScholar project, annotation was conducted over three rounds using a small sample of abstracts in each round, with previous abstracts re-annotated using refined guidelines [12]. To expedite the process, automatic pre-annotation using dictionaries of entities can be employed for entity types where such resources are available [12].

Experimental Protocols and Implementation

Data Collection and Preprocessing

The first critical step in building a material property extraction pipeline involves assembling a domain-specific corpus. Multiple approaches have been successfully implemented:

The MatSciBERT corpus was created by selecting approximately 150,000 papers from ~1 million papers downloaded from the Elsevier Science Direct Database, focusing on inorganic glasses, metallic glasses, alloys, and cement and concrete [53]. This corpus contained approximately 285 million words, with 40% of words from research papers related to inorganic glasses and ceramics, and 20% each from bulk metallic glasses, alloys, and cement [53].

The MaterialsBERT pipeline utilized a corpus of 2.4 million materials science papers, from which polymer-relevant abstracts were filtered by selecting those containing the string 'poly' and using regular expressions to identify abstracts containing numeric information [12].

For perovskite-specific applications, researchers constructed a dataset of 800 annotated abstracts obtained through web scraping using the Springer-Nature API, selecting literature based on the presence of perovskite materials and the inclusion of multiple entity types in the abstracts [9].

Model Architecture and Training

The typical architecture for NER in materials science follows a multi-layer approach:

Embedding Layer: Domain-specific BERT models (MatSciBERT, MaterialsBERT, or MatBERT) generate contextual token embeddings from the input text [12] [53] [9]. These embeddings capture domain-specific semantic information crucial for accurate entity recognition.

Feature Extraction: Convolutional Neural Networks (CNN) are often employed to extract local contextual features and capture n-gram information [9]. The CNN model comprises convolutional layers and pooling layers, providing outstanding local feature selection capabilities and reducing feature dimensionality [9].

Sequence Labeling: A Conditional Random Field (CRF) layer typically decodes the final sequences, modeling dependencies between output labels [9]. The CRF layer utilizes constraint relationships between labels to predict outputs in the correct order, ensuring the robustness of entity label predictions [9].

The training process employs standard deep learning optimization techniques. Models are typically trained using cross-entropy loss, with dropout regularization (e.g., dropout probability of 0.2) to prevent overfitting [12]. Due to the BERT model's input sequence length limit of 512 tokens, longer sequences are truncated as per standard practice [12].

Diagram 1: End-to-end workflow for material property extraction pipeline showing major stages from data collection to application

Evaluation Metrics and Validation

Model performance is typically evaluated using standard information extraction metrics:

• Precision = TP / (TP + FP)
• Recall = TP / (TP + FN)
• F1-score = 2 × (Precision × Recall) / (Precision + Recall)

where TP represents true positives, FP represents false positives, and FN represents false negatives [9].

Rigorous validation involves holding out test sets from the annotated data, typically 10% of the annotated abstracts, to assess generalizability [12]. For the PolymerScholar project, the dataset was split into 85% for training, 5% for validation, and 10% for testing [12].

The Researcher's Toolkit

Table 2: Key resources for building material property extraction pipelines

Resource Type	Specific Examples	Function/Purpose	Availability
Language Models	MatSciBERT, MaterialsBERT, MatBERT [12] [53]	Generate domain-aware contextual embeddings	Publicly available on Hugging Face and GitHub
Annotation Tools	Prodigy [12]	Manual annotation of training data	Commercial license required
NER Frameworks	BERT + CNN + CRF [9]	Complete architecture for entity recognition	Open-source implementations
Domain Corpora	Polymer abstracts, perovskite papers [12] [9]	Training and evaluation data	Curated from scientific databases
Evaluation Metrics	Precision, Recall, F1-score [9]	Quantitative performance assessment	Standard NLP metrics

Software and Computational Infrastructure

Successful implementation of material property extraction pipelines requires both specialized software and appropriate computational resources. The core natural language processing components typically build upon transformer architectures implemented in PyTorch or TensorFlow. The training process for models like MatSciBERT requires significant computational resources, given the size of the training corpora (hundreds of millions of words) and the complexity of the models [53].

Many research groups have made their pre-trained models publicly available. For instance, MatSciBERT pre-trained weights are hosted at Hugging Face (https://huggingface.co/m3rg-iitd/matscibert), and codes for pre-training and fine-tuning on downstream tasks are available on GitHub (https://github.com/M3RG-IITD/MatSciBERT) [53]. Similarly, the data and functionality from the Matscholar project have been made freely available on GitHub (https://github.com/materialsintelligence/matscholar) and their website (http://matscholar.com) [19].

Applications and Extracted Knowledge

Scale of Extracted Information

The application of general-purpose material property extraction pipelines has enabled the mining of unprecedented amounts of structured information from materials science literature:

The Matscholar project applied its NER model to information extraction from 3.27 million materials science abstracts, extracting more than 80 million materials-science-related named entities [19]. The content of each abstract was represented as a structured database entry, enabling complex "meta-questions" to be answered using simple database queries [19].

The PolymerScholar pipeline obtained approximately 300,000 material property records from about 130,000 abstracts in just 60 hours [12]. This demonstrates the remarkable efficiency of automated extraction compared to manual literature curation.

In the perovskite domain, researchers extracted 24,280 data points from 2,389 literature abstracts, identifying the most frequently appearing entities and trends in the field [9]. This included insights into lead substitution and environmental friendliness discussions within the perovskite community [9].

Domain-Specific Insights

The extracted data enables the recovery of non-trivial insights across diverse applications:

For energy applications, the extracted data has been analyzed for fuel cells, supercapacitors, and polymer solar cells, revealing known trends and phenomena in materials science [12]. This analysis capability provides researchers with quantitative trends that would be difficult to discern through manual literature review.

The extracted data also enables machine learning applications, such as training property predictors using automatically curated data. For example, researchers have trained a machine learning predictor for the glass transition temperature using automatically extracted data [12]. This approach helps address data scarcity in materials informatics where training property predictors traditionally requires manual curation.

Diagram 2: Detailed NER model architecture showing text processing from raw input to structured entities

Best Practices and Implementation Guidelines

Annotation Quality Assurance

Maintaining high annotation quality is paramount for successful NER implementation. The following practices have proven effective:

Progressive Guideline Refinement: Conduct annotation over multiple rounds, refining guidelines with each iteration and re-annotating previous abstracts using the refined guidelines [12]. This iterative approach ensures consistent application of annotation schemas.

Pre-annotation Strategies: Utilize automatic pre-annotation using dictionaries of entities for entity types where such resources are available to speed up the annotation process [12]. This approach improves efficiency while maintaining quality.

Inter-annotator Agreement Measurement: Assess annotation consistency using established metrics such as Cohen's Kappa and Fleiss Kappa [12]. The PolymerScholar project reported excellent agreement metrics, with Fleiss Kappa of 0.885 and pairwise Cohen's Kappa scores of (0.906, 0.864, 0.887) [12].

Model Selection Considerations

Choosing the appropriate model architecture depends on specific use cases:

For general materials extraction, MatSciBERT provides broad coverage across multiple materials families, having been trained on diverse materials science literature [53].

For polymer-focused applications, MaterialsBERT offers specialized capabilities, having been specifically applied to polymer literature and integrated into the PolymerScholar pipeline [12].

For specific material classes like perovskites, the MatBERT-CNN-CRF architecture has demonstrated superior performance, achieving 90.8% F1-score by combining MatBERT's domain awareness with CNN's feature extraction capabilities and CRF's sequence optimization [9].

The performance advantage of domain-specific models is consistent across studies. When tested on specialized NER datasets, these models typically outperform general-purpose language models by significant margins, justifying the investment in domain-specific training [12] [53] [9].

General-purpose material property extraction pipelines represent a transformative approach to knowledge management in materials science. By leveraging domain-specific named entity recognition, these systems can process millions of abstracts to extract structured information at scales impossible through manual curation. The development of specialized language models like MatSciBERT, MaterialsBERT, and MatBERT has been crucial to achieving the high accuracy required for scientific applications.

The implementation of these pipelines follows a systematic process involving corpus collection, annotation schema development, model training with domain-adapted architectures, and rigorous validation. When properly implemented, these systems can extract hundreds of thousands of material property records from scientific literature, enabling complex meta-analyses and machine learning applications that accelerate materials discovery and development.

As these technologies continue to mature, they hold the promise of unlocking the vast knowledge embedded in the materials science literature, transforming how researchers access and utilize published information to advance the field.

The rapid identification of effective therapeutics is a critical component of pandemic response. For SARS-CoV-2, the virus responsible for the COVID-19 pandemic, this process was accelerated through computational approaches, including Named Entity Recognition (NER). This case study explores the application of NER methodologies to extract potential antiviral compounds from scientific literature, framing the approach within the broader context of materials informatics for accelerated discovery.

The challenge was substantial: the volume of COVID-19 literature grew exponentially, creating a corpus far too large for manual review. The CORD-19 collection, for instance, contained nearly 200,000 articles, making traditional curation methods impractical [24]. NER systems provided a scalable solution by automatically identifying and classifying relevant chemical entities within this massive text corpus, thereby enriching candidate molecules for computational and experimental validation.

NER Methodologies for Chemical Compound Recognition

Chemical Named Entity Recognition (CNER) involves identifying text fragments that refer to chemical compounds, such as "remdesivir" or "chloroquine." Various machine learning approaches have been employed for this task, each with distinct advantages.

Table 1: Comparison of NER Approaches for Chemical Compound Identification

Methodology	Underlying Technology	Reported Performance (F1 Score)	Key Advantages
SpaCy Model [24]	Pre-trained NLP library on OntoNotes	~85.85% (general entities)	Fast processing; easy implementation
Naïve Bayes Classifier [40]	Multi-n-gram analysis with NBC	80.5% (Precision: 0.74, Recall: 0.95)	Effective on imbalanced data; recognizes novel entities
BERT-based Models [40]	Transformer architecture, domain-specific pre-training	High performance (domain-specific)	State-of-the-art contextual understanding
OGER Bio-NER [54]	Dependency parsing with ML	N/A (Proof-of-concept)	Links entities to major biological databases

Detailed Workflow: Model-in-the-Loop with Active Learning

A particularly efficient method for generating training data with minimal human effort is the model-in-the-loop active learning approach [24]. This iterative process makes optimal use of scarce human labeling resources by focusing efforts on the most uncertain samples.

• Process Details: This workflow begins with a small, human-verified bootstrap set of examples. An initial NER model is trained and applied to the unlabeled corpus. Rather than verifying all predictions, human reviewers (who can be non-experts, such as administrative staff) label only the samples for which the model is least confident. These newly labeled samples are added to the training set, and the model is retrained. This loop continues until performance improvements fall below a set threshold [24].
• Efficiency: This process required only tens of hours of human labeling time (for 1,778 samples) to produce a model with an F1 score of 80.5%, a performance on par with that of non-expert humans. When applied to the CORD-19 corpus, this model identified 10,912 putative drug-like molecules, enriching the target list for downstream screening by 3,591 molecules [24].

Table 2: Key Research Reagent Solutions for NER in Antiviral Discovery

Item Name	Type/Provider	Function in the Workflow
CORD-19 Corpus [24]	Literature Dataset (Allen Institute for AI)	Primary text source; contains nearly 200,000 COVID-19 related scientific articles for entity mining.
OGER (OntoGene's Biomedical Entity Recogniser) [54]	NERD Tool / API	Performs Named Entity Recognition and Disambiguation (NERD), linking entities to major biological databases (e.g., ChEBI, Gene Ontology, MeSH).
SpaCy NLP Library [24]	Software Library (Open Source)	Provides pre-trained models and frameworks for building custom NER pipelines to process large volumes of text.
CHEMDNER Corpus [40]	Annotated Text Corpus	Benchmark dataset containing 10,000 abstracts with over 80,000 labeled chemical entities for training and evaluating CNER models.
ReFRAME Drug Repurposing Library [55]	Compound Library (Calibr)	A comprehensive library of ~12,000 clinical-stage or FDA-approved molecules used to validate the antiviral activity of NER-derived candidates.

Application to SARS-CoV-2 Antiviral Discovery

Integration with the Drug Discovery Pipeline

The compounds identified through NER become valuable inputs for computational and experimental screening pipelines. A prominent example is the use of the ReFRAME drug repurposing library, a collection of approximately 12,000 clinically tested compounds, which was screened to identify inhibitors of SARS-CoV-2 replication [55]. NER can rapidly populate such screening libraries with candidates mined from the latest literature.

The subsequent validation workflow typically involves a multi-stage process:

Success Stories: From Text to Lead Compound

The practical application of this pipeline is exemplified by several discoveries:

• Literature Mining Outcome: One study screened over 9,000 "safe-in-man" and 249,000 natural compounds in silico against the SARS-CoV-2 RNA-dependent RNA polymerase (RdRp). This computationally intensive process can be efficiently primed by NER, which pre-populates the candidate list with compounds mentioned in relevant virology and pharmacology literature [56].
• Experimental Validation: From the top 119 candidates selected by in silico docking, 42 compounds inhibited the RdRp in a biochemical assay. Four of these showed IC₅₀ and EC₅₀ values in the nanomolar or low micromolar range and also inhibited viral replication in cell-based assays [56].
• Novel Mechanisms: Beyond established targets like the RdRp and main protease (Mpro), NER can also aid in discovering inhibitors for novel viral targets. For instance, the membrane (M) protein was identified as a target for the small-molecule inhibitor JNJ-9676. Cryo-EM revealed that JNJ-9676 stabilizes the M protein dimer in an altered conformation, preventing the release of infectious virus. In Syrian golden hamsters, a pre-exposure dose of 25 mg/kg twice daily reduced viral load in the lung by 3.5 log₁₀ [57].

Detailed Experimental Protocols

Protocol: Model-in-the-Loop Training for Chemical NER

This protocol is adapted from the work of [24] and details the steps for creating a training set and model for recognizing drug-like molecules.

• Bootstrap Set Creation: Manually curate a small set of sentences (e.g., 100-200) from the target domain (e.g., CORD-19 abstracts) where drug-like molecules are clearly labeled.
• Initial Model Training: Train an initial NER model (e.g., a SpaCy model or a Naïve Bayes Classifier) on this bootstrap set. The model should be trained to predict the entity type (e.g., "DRUG") for each word token.
• Uncertainty Sampling: a. Apply the trained model to a large, unlabeled corpus. b. For each prediction, calculate a confidence score (e.g., the probability of the predicted class). c. Rank the predictions by confidence and select the bottom N (e.g., 100-200) least confident samples for human review.
• Human Annotation: Present the selected uncertain samples to human reviewers. The reviewers' task is to correctly label the named entities (e.g., mark the spans of text that are drug-like molecules).
• Model Retraining: Add the newly human-verified samples to the training set and retrain the NER model.
• Iteration: Repeat steps 3-5 until the model's performance on a held-out validation set no longer improves significantly (e.g., F1 score improvement < 1%).

Protocol: Cell-Based Antiviral Assay for SARS-CoV-2 Inhibitors

This protocol summarizes the high-throughput screening method used to validate the antiviral activity of compounds, as described by [55].

• Cell Seeding: Seed Vero E6 cells (kidney epithelial cells from African green monkey, highly permissive to SARS-CoV-2) in a 384-well plate.
• Compound Application: Add compounds from the screening library (e.g., ReFRAME) at a desired concentration (e.g., 5 µM for primary screening).
• Viral Infection: Infect cells with a clinical isolate of SARS-CoV-2 at a low multiplicity of infection (MOI = 0.01) to allow for multiple rounds of viral replication.
• Incubation: Incubate the infected, compound-treated cells for a set period (e.g., 72 hours) to allow the development of virus-induced cytopathic effect (CPE).
• Viability Readout: Measure viral-induced CPE using a cell viability assay (e.g., CellTiter-Glo). Viral replication reduces cell viability.
• Orthogonal Validation: For hits from the primary screen, confirm activity using an orthogonal method. For example, immunostain for the viral nucleoprotein (NP) to directly quantify viral replication in the presence and absence of the compound at varying concentrations (dose-response).

Overcoming Hurdles: Solving Data Scarcity, Ambiguity, and Model Performance Challenges

Named Entity Recognition (NER) plays a crucial role in materials science research by enabling the extraction of structured, machine-interpretable knowledge from unstructured scientific literature. This process facilitates the construction of knowledge graphs and accelerates data-driven materials discovery [58]. However, the field faces significant data scarcity challenges, particularly for specialized subdomains where technical expertise is required for annotation and available datasets are limited [59] [60]. Data augmentation (DA) has emerged as a powerful strategy to mitigate these challenges by expanding and diversifying training datasets, which is especially valuable in few-shot learning scenarios where deep learning techniques might otherwise underperform [61].

In materials science, data scarcity stems from multiple factors: the high cost of expert annotation, the domain-specific terminology that requires specialized knowledge, and the fine-grained entity types that must be distinguished [58] [60]. These challenges are particularly pronounced when entities must conform to pre-existing domain ontologies to ensure semantic alignment and cross-dataset interoperability [58]. This application note outlines practical strategies and protocols for addressing data scarcity in materials science NER, with a focus on experimentally-validated augmentation techniques and their implementation protocols.

Data Augmentation Techniques and Performance Analysis

Data augmentation techniques for NER can be broadly categorized into text-based augmentation and synthetic data generation approaches. The effectiveness of these methods varies based on dataset size, domain specificity, and the underlying NER model architecture.

Table 1: Comparison of Data Augmentation Techniques for NER

Technique	Methodology	Best-SuScenarios	Performance Impact
Contextual Word Replacement (CWR)	Replaces words contextually using BERT models [59]	Low-resource domains; BERT-based NER models	Generally outperforms mention replacement; particularly beneficial for BERT models [59]
Mention Replacement (MR)	Replaces entity mentions with same-label mentions from training set [59]	Smaller datasets; entity-centric recognition	Less effective than CWR; moderate performance gains [59]
Combined Real & Artificial Data	Integrates limited real data with strategically generated artificial data [62]	Scenarios with very limited annotated data; complex material behaviors	Improved prediction robustness; average final thickness error reduction to 5.5% in compaction studies [62]
Domain-Specific Pre-training	Pre-training language models on domain-specific corpora [60]	All materials science NER tasks, especially with limited fine-tuning data	MatBERT outperforms BERT by 1-12%; advantages most pronounced in small data limit [60]

Experimental studies reveal several critical insights about data augmentation effectiveness. First, augmentation provides the most substantial benefits for smaller datasets, while for larger datasets, models trained with augmented data may yield equivalent or even inferior performance compared to those trained without augmentation [59]. Second, there exists a saturation point beyond which additional augmented examples degrade model quality, necessitating careful experimentation with different volumes of augmented data [59]. Third, the choice of NER model architecture influences augmentation effectiveness, with BERT models generally benefiting more from data augmentation than Bi-LSTM+CRF models [59].

Experimental Protocols for Data Augmentation

Protocol: Contextual Word Replacement for NER

Purpose: To generate diverse training examples while preserving semantic meaning and entity context through contextual word replacement.

Materials and Resources:

• Pre-trained BERT model (domain-specific variants preferred)
• Limited annotated NER dataset
• Computational resources with GPU acceleration

Procedure:

• Data Preparation: Prepare annotated NER data in IOB (Inside-Outside-Beginning) format, ensuring consistent annotation standards [59].
• Model Selection: Select an appropriate BERT model for contextual replacement. For materials science domains, consider MatSciBERT or MatBERT for improved domain alignment [58] [60].
• Replacement Strategy:
  • Identify replacement candidates (non-entity words or specific entity types)
  • For each candidate word, use BERT's masked language modeling capability to generate contextually appropriate replacements
  • Replace words while preserving entity tags for adjacent words
• Quality Control:
  • Limit replacement rate to 10-20% of words per sentence
  • Validate semantic coherence of augmented examples
  • Ensure entity boundaries remain correctly annotated
• Dataset Balancing:
  • Generate augmented examples progressively, monitoring performance on validation set
  • Identify saturation point where additional augmentation ceases to improve performance

Applications: Most beneficial for low-resource materials science domains with limited annotated data, particularly when using BERT-based NER models [59].

Protocol: Synthetic Data Generation with DiffRenderGAN

Purpose: To generate realistic synthetic data for nanomaterial segmentation tasks where manual annotation is prohibitively expensive or time-consuming.

Materials and Resources:

• Non-annotated real microscopy images
• Differentiable renderer integrated within GAN framework
• 3D nanoparticle models (meshes)
• Transformation matrix containing positional and scaling information

Procedure:

• Mesh Modeling: Create a collection of particle meshes reflecting shape properties of nanoparticles observed in real images [63].
• Transformation Computation: Calculate transformation tensor Φ containing spatial coordinates and scaling factors for each mesh, encoding different nanoparticle constellations [63].
• Parameter Optimization: Focus generator on optimizing textural parameters (θ_BSDF) that mimic material properties observed in SEM and HIM imaging [63].
• Adversarial Training:
  • Train generator to produce synthetic images from random noise vectors
  • Train discriminator to distinguish between real and synthetic images
  • Iterate until generator produces realistic synthetic images
• Annotation Extraction: Automatically extract annotation masks using unique identifiers assigned to each mesh in the virtual scene [63].
• Validation: Compare synthetic data quality against real annotated data and evaluate segmentation performance on real microscopy images.

Applications: Particularly valuable for nanomaterial segmentation tasks in electron microscopy, including titanium dioxide (TiO₂), silicon dioxide (SiO₂), and silver nanowires (AgNW) [63].

Protocol: Domain-Specific Pre-training and Fine-tuning

Purpose: To leverage domain-specific pre-training for improved NER performance in materials science, especially in low-data regimes.

Materials and Resources:

• Domain-specific pre-trained language model (MatBERT, MatSciBERT)
• Limited annotated NER dataset
• Computational resources for fine-tuning

Procedure:

• Model Selection:
  • For general materials science: MatBERT [60]
  • For materials mechanics and fatigue: MatSciBERT [58]
  • Consider model size and computational constraints
• Parameter Efficient Fine-Tuning (PEFT):
  • For very limited data, employ Low-Rank Adaptation (LoRA) to reduce trainable parameters [58]
  • Freeze base model parameters, adapt with low-rank matrices
• Progressive Fine-tuning:
  • Begin with lower learning rates (1e-5 to 5e-5)
  • Use early stopping based on validation performance
  • Gradually increase sequence length and batch size as resources allow
• Evaluation:
  • Assess on both in-distribution and out-of-distribution datasets
  • Compare against baseline models with general pre-training
  • Evaluate ontology-conformal accuracy for ontology-based applications

Applications: Essential for fine-grained entity recognition in specialized subdomains like materials fatigue, and for ontology-conformal NER requiring semantic alignment with pre-existing domain ontologies [58].

Visualization of Data Augmentation Workflows

Figure 1: Comprehensive workflow for addressing data scarcity in materials science NER, integrating multiple augmentation strategies.

Figure 2: Text-based augmentation techniques showing parallel workflows for CWR and MR approaches.

Table 2: Key Research Reagents and Computational Resources for Materials Science NER

Resource	Type	Function	Domain Specificity
MatBERT	Language Model	Domain-specific pre-trained transformer for materials science NER	High - specifically pre-trained on materials science literature [60]
MatSciBERT	Language Model	Domain-specific pre-trained transformer for materials mechanics and fatigue	High - specialized for materials mechanics [58]
Bi-LSTM+CRF	Model Architecture	Deep neural network with bidirectional LSTM and CRF layer for sequence tagging	Medium - general architecture but can use domain-specific embeddings [59] [60]
DiffRenderGAN	Generative Framework	Integrates differentiable renderer with GAN for synthetic nanomaterial data	High - specifically designed for nanomaterial segmentation [63]
MatWheel Framework	Generative Framework	Conditional generative model for material property prediction synthetic data	Medium - focused on material properties but adaptable [64]
Materials Ontologies	Knowledge Representation	Formal domain conceptualizations ensuring semantic alignment	High - essential for ontology-conformal NER [58]

Implementation Considerations and Best Practices

Successful implementation of data augmentation strategies for materials science NER requires careful consideration of several factors. First, domain specificity should guide technique selection: text-based augmentation methods like CWR and MR work well for general materials science text, while synthetic generation approaches like DiffRenderGAN are invaluable for specialized imaging data [59] [63]. Second, dataset size determines augmentation value: smaller datasets (typically <500 annotated samples) benefit most from augmentation, while performance gains diminish for larger datasets [59]. Third, model architecture influences approach effectiveness: BERT models generally show greater improvement from augmentation compared to Bi-LSTM+CRF architectures [59].

Critical implementation best practices include:

• Progressive Augmentation: Systematically increase augmented data volume while monitoring validation performance to identify saturation points [59].
• Quality Validation: Ensure semantic coherence of augmented examples, particularly for technical materials science terminology [59] [58].
• Domain Alignment: Prioritize domain-specific pre-training (MatBERT, MatSciBERT) over general models, especially in low-data regimes [58] [60].
• Combined Approaches: Integrate multiple strategies, such as using both real and artificial data, for improved robustness [62].
• Ontology Conformance: For knowledge graph applications, ensure augmented data maintains conformity with domain ontologies for semantic interoperability [58].

When properly implemented, these data augmentation strategies can significantly alleviate data scarcity challenges in materials science NER, enabling more effective information extraction from limited annotated resources and accelerating materials discovery through enhanced knowledge graph construction.

In the domain of materials science research, extracting critical information from vast scientific literature—such as material compositions, synthesis conditions, and functional properties—is essential for accelerating discovery. Named Entity Recognition (NER) serves as a foundational natural language processing (NLP) technique to automate this extraction. However, training robust NER models requires large, high-quality, domain-specific annotated datasets, the creation of which is notoriously time-consuming and expensive, requiring scarce domain expertise. This application note explores the integration of human-in-the-loop (HITL) Active Learning (AL) strategies to optimize the training data generation process for NER in materials science. By strategically selecting the most informative data points for human experts to annotate, this approach significantly reduces annotation effort, mitigates computational cold-start problems, and leads to the development of more accurate and generalizable models.

The efficiency gains from employing HITL-AL workflows are substantial across various scientific domains, including materials science and clinical NER. The following table summarizes key performance metrics reported in recent studies.

Table 1: Performance Metrics of Active Learning and Human-in-the-Loop Frameworks

Framework / Study	Domain	Key Metric	Performance Improvement	Reference
LLM-based Active Learning (LLM-AL)	Materials Science	Data required to find optimal candidates	Reduced by over 70% compared to traditional methods [65]
Human-in-the-loop Automated Experiment (hAE)	STEM-EELS	Experiment efficiency	Enabled targeted discovery, avoiding uniform grid sampling [66]
Partially Bayesian Neural Networks (PBNNs)	Molecular & Materials Property Prediction	Computational Cost vs. Accuracy	Achieved accuracy comparable to fully Bayesian networks at lower computational cost [67]
Active LEARNER System (CAUSE algorithm)	Clinical NER	Annotation Efficiency (Simulation)	Outperformed traditional AL and random sampling [68]

Table 2: Comparative Analysis of Surrogate Models for Active Learning

Model Type	Key Advantages	Key Challenges	Suitability for Materials NER
Gaussian Process (GP)	Mathematically grounded uncertainty quantification (UQ) [67]	Struggles with high-dimensional data and non-stationarities [67]	Moderate
Deep Kernel Learning (DKL)	Combines neural networks with GP-based UQ [67]	Scalability issues and potential mode collapse [67]	High
Fully Bayesian Neural Networks (BNNs)	Robust UQ, effective on small/noisy datasets [67]	Prohibitively high computational cost [67]	High
Partially Bayesian NNs (PBNNs)	Comparable UQ to BNNs with lower cost [67]	Requires strategic selection of probabilistic layers [67]	Very High
Large Language Models (LLMs)	Mitigates cold-start, requires no feature engineering [65]	Non-deterministic output, can generate non-physical responses [65]	Very High (for prompt-based NER)

Protocols for HITL-AL in Materials Science NER

Protocol 1: Human-in-the-Loop Active Learning Workflow

Objective: To iteratively and efficiently build a high-performance NER model for materials science text with minimal expert annotation effort.

Materials and Reagents:

• Text Corpus: A large, unlabeled collection of materials science literature (e.g., scientific papers, abstracts, patents).
• Human Expert: A materials scientist or domain expert with knowledge of the entities of interest (e.g., polymers, perovskites, synthesis methods).
• Computing Environment: Standard ML/ NLP computing infrastructure.
• Initial Seed Data: A small, pre-annotated dataset (~50-100 documents) to initialize the model.

Procedure:

• Model Initialization: Train an initial NER model (e.g., a transformer-based model like SciBERT) on the available seed data.
• Model Inference & Uncertainty Scoring: Use the trained model to predict entities on the entire unlabeled pool of documents. For each sentence or document, calculate an uncertainty score (e.g., entropy, least confidence) based on the model's predictive probabilities [68] [67].
• Human Intervention and Querying: Select the top K most uncertain samples from the pool. This prioritizes data points where the model is most confused and expert input would be most informative. Present these selected samples to the human expert for annotation [66] [69].
• Model Retraining: Incorporate the newly annotated, high-value data into the existing training set. Retrain or fine-tune the NER model on this augmented dataset.
• Iteration and Convergence: Repeat steps 2-4 until a predefined stopping criterion is met (e.g., a target performance level is achieved, or an annotation budget is exhausted). Monitor performance on a held-out validation set.

Protocol 2: Fine-Tuning vs. In-Context Learning for NER Model Development

Objective: To establish a protocol for developing the core NER model using either supervised fine-tuning or in-context learning with Large Language Models (LLMs).

Materials and Reagents:

• Base Model: A pre-trained language model (e.g., BioClinicalBERT, RoBERTa-large, GPT-4o).
• Annotation Interface: A tool like BRAT or a custom system for efficient text annotation [68].
• Training Data: The annotated dataset generated from Protocol 1.

Procedure: A. Supervised Fine-Tuning (SFT):

• Data Preparation: Format the annotated data into a sequence labeling format compatible with the base model (e.g., BIO or BILOU tagging scheme).
• Model Setup: Add a token classification head on top of the pre-trained base model.
• Training Loop: Fine-tune the entire model on the annotated dataset using a standard optimizer (e.g., AdamW) and a cross-entropy loss function. Employ early stopping to prevent overfitting.
• Evaluation: Evaluate the fine-tuned model on a held-out test set using standard NER metrics (Precision, Recall, F1-score). SFT has been shown to achieve the strongest overall performance (e.g., F1 of 87.1% on clinical NER tasks) [70].

B. In-Context Learning (ICL) with LLMs:

• Prompt Engineering: Design a prompt that includes the task description, definition of entity types, and a few annotated examples (few-shot learning). Simpler prompts have been shown to outperform longer, instruction-heavy ones [70].
• Model Querying: For each sentence to be analyzed, send the constructed prompt to a large LLM (e.g., GPT-4o) and parse the response to extract the annotated entities.
• Iteration: Refine the prompt based on model errors to improve performance. Note that while flexible, ICL generally underperforms SFT for specialized NER tasks [70].

The Scientist's Toolkit: Research Reagent Solutions

Table 3: Essential Tools and Frameworks for HITL-AL in Materials NER

Tool / Resource	Type	Function in the Workflow
spaCy / Stanza	NLP Library	Provides production-ready pipelines for tokenization, part-of-speech tagging, and baseline NER models.
Hugging Face Transformers	Model Library	Offers access to thousands of pre-trained models (e.g., SciBERT, MatBERT) for fine-tuning [71].
BRAT Annotation Tool	Software	A web-based tool for rapid, structured annotation of text documents, suitable for creating ground truth data [68].
AllenNLP	NLP Research Library	Simplifies experimentation with deep learning models for NLP, offering abstractions for model building and evaluation [71].
LLMs (GPT-4o, Llama)	Large Language Model	Can be used for in-context learning or as a data augmentation tool to generate synthetic training examples [65].
Uncertainty Metrics (Entropy)	Algorithm	Quantifies model uncertainty for each prediction, forming the basis for the AL acquisition function [67].
Partially Bayesian NNs (PBNNs)	Machine Learning Model	Provides reliable uncertainty estimates at a lower computational cost than fully Bayesian methods, ideal for AL [67].

Addressing Semantic Ambiguity and Complex Entity Nesting

Named Entity Recognition (NER) is a fundamental natural language processing (NLP) task that serves as a critical component for information extraction, retrieval, and knowledge graph construction in materials science research [72] [13]. The exponential growth of materials science literature presents both an opportunity and a challenge: while millions of scientific papers contain valuable materials knowledge, extracting structured information from this corpus has become increasingly difficult [72] [9]. The field faces particular challenges with semantic ambiguity, where the same entity may be expressed through multiple textual representations, and complex entity nesting, where entities contain or overlap with other entities in text [13]. This application note examines current methodologies addressing these challenges and provides detailed protocols for implementing advanced NER systems in materials science.

Quantitative Analysis of NER Performance in Materials Science

Recent advances in domain-specific language models and novel NLP frameworks have significantly improved the ability to recognize and disambiguate materials science entities. The performance of various models across multiple datasets demonstrates substantial progress in addressing both semantic ambiguity and entity nesting challenges.

Table 1: Performance Comparison of NER Models on Materials Science Datasets

Model	Dataset	Precision (%)	Recall (%)	F1-Score (%)	Key Capabilities
MatBERT-CNN-CRF [9]	Perovskite (800 abstracts)	-	-	90.8	Handles syntactic variations, captures local semantic relationships
MatSciBERT-MRC [13]	Matscholar	-	-	89.64	Effectively extracts nested entities, utilizes contextual information
MatSciBERT-MRC [13]	BC4CHEMD	-	-	94.30	Resolves semantic ambiguity through query framework
MatSciBERT-MRC [13]	NLMChem	-	-	85.89	Handles complex chemical nomenclature
MatSciBERT-MRC [13]	SOFC	-	-	85.95	Adapts to specialized subdomains
MatSciBERT-MRC [13]	SOFC-Slot	-	-	71.73	Addresses slot filling in structured contexts

Table 2: Entity Distribution in Extracted Materials Science Data

Entity Type	Count in MatKG	Examples	Common Ambiguity Challenges
Materials (CHM)	Not specified	'single crystal LiMnO3', 'lead' [9]	Synonyms, formula variations, syntactic differences
Properties (PRO)	Not specified	'Light-Harvesting Ability' [72]	Semantic variations ('Ability' vs 'Capability')
Applications (APL)	Not specified	'thermoelectric' [72]	Broad contextual dependencies
Synthesis Methods (SMT)	Not specified	'solid state sintering' [72]	Procedural terminology variations
Characterization Methods (CMT)	Not specified	'high-temperature AFM' [72]	Acronym resolution, technique specifications
Total Entities	70,000 [72]	-	-
Total Unique Triples	5.4 million [72]	-	-

Experimental Protocols for Advanced NER in Materials Science

Machine Reading Comprehension Framework for Nested Entity Resolution

The transformation of NER from a sequence labeling task to a Machine Reading Comprehension (MRC) task represents a significant methodological advancement for handling nested entities [13].

Protocol Steps:

• Dataset Transformation: Convert traditional sequence labeling data into (Context, Query, Answer) triples

  • Context: Input sequence X = {x₁, x₂, ..., xₙ}
  • Query: Natural language question Q designed to extract specific entity types
  • Answer: Span of target entity within context

• Query Generation: Develop specific queries for each entity type based on annotation guidelines

  • Example: For material entities, use query "Which material is mentioned in the text?"
  • Example: For property entities, use query "What property is described?" [13]

• Model Architecture:

  • Utilize MatSciBERT as backbone model pre-trained on materials science literature
  • Concatenate query and context with [CLS] and [SEP] tokens
  • Format: {[CLS], q₁, q₂, ..., qₘ, [SEP], x₁, x₂, ..., xₙ}

• Span Prediction:

  • Implement two binary classifiers for start and end indices
  • Start index prediction: Kstart = linear(LQstart) ∈ R^(N×2)
  • End index prediction: Kend = linear(LQend; softmax(K_start)) ∈ R^(N×2)
  • Enable prediction of multiple start and end indices for nested entities [13]

MatBERT-CNN-CRF Architecture for Semantic Disambiguation

The hybrid MatBERT-CNN-CRF model addresses semantic ambiguity through a multi-stage processing pipeline that leverages both global contextual understanding and local feature extraction [9].

Protocol Steps:

• Word Embedding Generation:

  • Utilize MatBERT model pre-trained on 5 million materials science papers
  • Generate contextualized word embeddings incorporating domain-specific knowledge
  • Input: Raw text tokens from materials science abstracts

• Feature Extraction with CNN:

  • Implement 1D Convolutional Neural Network for local feature detection
  • Capture character-level and sub-word patterns indicative of entity boundaries
  • Kernel sizes: Vary between 2-5 characters to detect morphological patterns

• Sequence Labeling with CRF:

  • Apply Conditional Random Field layer for structured prediction
  • Incorporate label transition constraints to enforce valid entity sequences
  • Utilize IOBES labeling scheme (Inside, Outside, Beginning, End, Single) [9]

• Training Configuration:

  • Batch size: 32
  • Learning rate: 2e-5 with linear decay
  • Epochs: 10-20 with early stopping
  • Optimizer: AdamW with weight decay

Data Cleaning and Entity Normalization Protocol

Addressing semantic ambiguity requires extensive post-processing of extracted entities to resolve syntactic and semantic variations [72].

Protocol Steps:

• Non-ASCII Character Filtering:

  • Remove entities containing purely non-ASCII characters (Greek letters, mathematical symbols)
  • Maintain data uniformity and simplify processing pipeline

• Edit Distance Clustering:

  • Calculate Levenshtein edit distance between entity strings
  • Apply Fuzzy Sort algorithm with 90-95% similarity threshold
  • Cluster entities with high syntactic similarity (e.g., 'electrode', 'electrodes')

• Canonical Representation Generation:

  • Employ ChatGPT API with few-shot learning for semantic disambiguation
  • Provide examples of chemically similar and dissimilar terms
  • Generate canonical forms for clustered entities (e.g., 'electrode' for ['electrode', 'electrodes'])

• Iterative Refinement:

  • Execute entire cleaning pipeline for 5 iterations
  • Progressively refine entity clusters and canonical representations
  • Manual validation of ambiguous cases (e.g., 'methanol' vs 'ethanol') [72]

The Scientist's Toolkit: Research Reagent Solutions

Table 3: Essential Research Components for Materials Science NER

Component	Type	Specifications	Function in NER Pipeline
MatBERT [9] [13]	Pre-trained Language Model	110M parameters, trained on 5M materials science papers	Domain-specific contextual embedding generation
MatSciBERT [13]	Pre-trained Language Model	BERT architecture, materials science corpus	Base model for MRC and sequence labeling approaches
Materials Science Corpus [72]	Training Data	5 million scientific papers, abstracts and figure captions	Domain-specific pretraining and fine-tuning
Perovskite Dataset [9]	Annotated Dataset	800 annotated abstracts, IOBES labeling scheme	Model training and evaluation for specialized domains
Matscholar Dataset [13]	Benchmark Dataset	Annotated materials science texts	Performance evaluation and comparative analysis
ChatGPT API [72]	Semantic Disambiguation	Few-shot prompting with chemical examples	Entity normalization and canonical representation
1D-CNN Layer [9]	Feature Extraction	Multiple kernel sizes (2-5)	Local pattern detection in entity texts
CRF Layer [9]	Sequence Labeling	Transition constraint learning	Global sequence optimization and valid label sequences
MRC Framework [13]	NLP Architecture	Query-answer formulation	Handling nested entities through question decomposition

The integration of domain-specific language models like MatBERT and MatSciBERT with innovative frameworks such as MRC and hybrid neural architectures has substantially advanced the state of named entity recognition in materials science. These approaches systematically address the dual challenges of semantic ambiguity and complex entity nesting through specialized data cleaning protocols, contextual understanding, and structured prediction. The detailed protocols and experimental frameworks presented in this application note provide researchers with practical methodologies for implementing robust NER systems capable of extracting structured knowledge from the rapidly expanding materials science literature. As these technologies continue to mature, they promise to accelerate materials discovery and development through enhanced knowledge extraction and organization.

In materials science research, Named Entity Recognition (NER) is a fundamental natural language processing (NLP) technique for automatically extracting structured information—such as material compositions, synthesis methods, and properties—from unstructured scientific text. A significant challenge in deploying NER models is domain shift, where a model trained on one corpus of materials science literature experiences performance degradation when applied to text from a different sub-domain, such as moving from fatigue mechanics to perovskite photovoltaics. This performance drop occurs due to changes in terminology, entity distribution, and writing style [58]. Ensuring model generalizability across these domains is therefore critical for building robust, automated knowledge extraction systems that can accelerate materials discovery.

Quantifying the Domain Shift Problem in Materials NER

The performance gap between in-distribution (ID) and out-of-distribution (OOD) tests concretely illustrates the domain shift problem. Specialized models like MatSciBERT and MatBERT, when tested on materials science text, show a marked performance decrease on OOD data, though they still significantly outperform general foundation models [58]. The following table summarizes the typical performance drop observed in NER tasks due to domain shift.

Table 1: Performance Comparison of NER Models on In-Distribution (ID) vs. Out-of-Distribution (OOD) Data in Materials Science

Model Type	Specific Model	ID F1-Score (%)	OOD F1-Score (%)	Performance Drop (Percentage Points)	Key Characteristics
Fine-tuned Domain-Specific	MatSciBERT [58] [53]	~85 (Est.)	~80 (Est.)	~5	Pre-trained on ~285M word materials science corpus [53]
Fine-tuned Domain-Specific	MatBERT [9]	90.8 (Reported)	N/R	N/R	Pre-trained on materials science literature; used for perovskites [9]
Fine-tuned Domain-Specific	MatBERT-CNN-CRF [9]	90.8 (Reported)	N/R	N/R	Incorporates CNN for local feature extraction & CRF for label decoding [9]
Foundation Model (ICL)	GPT-4 (Few-Shot) [58]	Lower than fine-tuned	Significantly Lower	Larger than fine-tuned	Performance highly dependent on quality of few-shot demonstrations [58]
Foundation Model (ICL)	GPT-3.5-Turbo (Zero-Shot) [73]	Fails to outperform specialized baselines	N/R	N/R	Struggles with complex, domain-specific material entities [73]

Abbreviations: N/R (Not Reported in search results), Est. (Estimated from context), ICL (In-Context Learning)

Experimental Protocols for Evaluating Domain Generalization

A standardized evaluation protocol is essential for diagnosing and mitigating the effects of domain shift. The following workflow and detailed methodology provide a framework for robust benchmarking.

Diagram 1: Domain Generalization Eval Workflow

Detailed Evaluation Methodology

• Dataset Creation and Curation:

  • Source Documents: Select scientific publications from distinct but related sub-domains of materials science (e.g., materials fatigue vs. microstructure-property relationships) [58]. The text should be extracted from full-length articles, not just abstracts, to capture a wider range of entity contexts [58].
  • Annotation Protocol: Annotate the text according to a pre-defined, formal ontology (e.g., the materials mechanics ontology) [58]. This ensures semantic alignment and interoperability across datasets. The IOBES (Inside-Outside-Beginning-End-Single) labeling scheme is recommended, as it has been shown to yield higher F1-scores compared to other schemes [9].
  • Strategic Data Splitting:
    • In-Distribution (ID) Set: Randomly split documents from the primary domain (e.g., fatigue) into training/validation/test sets.
    • Out-of-Distribution (OOD) Set: Use a separate set of documents from a different sub-domain (e.g., perovskites) as the OOD test set [58] [9]. This evaluates performance on a realistic domain shift.

• Model Training and Fine-Tuning:

  • Baseline Models: Fine-tune domain-specific pre-trained models like MatSciBERT [53] or MatBERT [9] on the ID training set. These models have been pre-trained on large corpora of materials science text, providing a strong foundation.
  • Advanced Architectures: For higher performance on a specific ID task, consider augmenting the base model. For example, adding a Convolutional Neural Network (CNN) layer can help extract local contextual features, and a final Conditional Random Field (CRF) layer can improve label sequence consistency [9].
  • Comparison Models: Evaluate foundation Large Language Models (LLMs) like GPT-4 using in-context learning (few-shot demonstrations) on the same test sets [58] [73].

• Performance Metrics and Analysis:

  • Primary Metric: Calculate the F1-score (the harmonic mean of precision and recall) separately on the ID and OOD test sets for all models [9].
  • Generalization Gap: Quantify the domain shift by calculating the difference in F1-score between the ID and OOD results.
  • Error Analysis: Examine the types of entities that are most frequently misclassified in the OOD setting to identify specific terminological or contextual challenges.

Key Strategies for Mitigating Domain Shift

Domain-Specific Pre-training and Fine-Tuning

The most effective strategy, as evidenced by the performance of models like MatSciBERT and MatBERT, is to use a language model that has been pre-trained on a large, diverse corpus of scientific text from the target domain [58] [53]. This process aligns the model's internal representations with the specialized vocabulary and syntax of materials science. Continuing pre-training (domain-adaptive pre-training) on an unlabeled corpus from the specific sub-domain of interest can further enhance OOD performance [53].

Parameter-Efficient Fine-Tuning (PEFT)

For scenarios with limited annotated data, PEFT methods like Low-Rank Adaptation (LoRA) can be highly effective. LoRA freezes the pre-trained model weights and injects trainable rank-decomposition matrices into the transformer layers, significantly reducing the number of parameters that need to be fine-tuned [58]. This approach is particularly valuable in materials science, where large, annotated datasets are often scarce [58].

Improved In-Context Learning for LLMs

When using large foundation models, their ability to handle domain shift is heavily dependent on the quality and relevance of the few-shot examples provided in the prompt [58]. To improve OOD performance:

• Curate Demonstrations: Select few-shot examples that are semantically close to the target OOD domain.
• Leverage Hybrid Pipelines: Implement a two-stage ICL pipeline where the model first identifies challenging entity spans and then performs a focused classification, which can be a cost-effective alternative to full fine-tuning [58].

Multimodal and Domain Adaptation Learning

Emerging techniques from other areas of materials informatics show promise for NER.

• Domain Adaptation (DA): Machine learning models can be adapted to improve OOD prediction performance for material properties by incorporating information from the target domain [74]. While applied to property prediction, similar DA techniques could be explored for NER tasks.
• Multimodal Learning: Frameworks that integrate multiple data modalities (e.g., composition and X-ray diffraction patterns) have shown improved generalization and robustness in materials property prediction [75]. A analogous approach for NER could involve jointly modeling text with other data sources.

The Scientist's Toolkit: Research Reagent Solutions

Table 2: Essential Tools for Materials Science NER Model Development

Tool / Resource	Type	Primary Function in NER
MatSciBERT [53]	Pre-trained Language Model	Provides a foundational model with embedded materials science knowledge, ideal for fine-tuning on specific NER tasks.
Materials Mechanics Ontology [58]	Ontology / Schema	Defines entity types and relationships formally, ensuring consistent annotation and semantic interoperability across datasets.
LoRA (Low-Rank Adaptation) [58]	Fine-tuning Method	Enables efficient adaptation of large models to new domains with limited annotated data, reducing computational cost.
IOBES Labeling Scheme [9]	Annotation Standard	A token-level labeling scheme for marking entity boundaries in text, proven to achieve high F1-scores in NER tasks.
Domain-Adaptation (DA) Models [74]	Machine Learning Technique	Improves prediction accuracy on out-of-distribution target materials by leveraging domain adaptation techniques.

Diagram 2: Model Optimization Strategy

Leveraging Domain-Specific Pre-training for Superior Performance

The exponential growth of materials science literature presents a significant bottleneck for researchers, making the manual extraction of key information an unsustainable and time-consuming task [76] [77]. Named Entity Recognition (NER), a fundamental natural language processing (NLP) technique, offers a solution by automatically identifying and classifying material-specific entities—such as materials names, properties, and synthesis methods—into structured, machine-readable data [78] [77]. However, general-purpose language models often yield suboptimal results on scientific text due to their unfamiliarity with domain-specific notations and jargon [76].

Domain-specific pre-training has emerged as a powerful strategy to overcome this limitation. By continuing the training of a base language model on a large, unlabeled corpus of scientific text, the model learns the statistical representations and contextual relationships unique to the materials science domain [76] [79]. This application note details the quantitative advantages of this approach, provides protocols for its implementation, and outlines the essential toolkit for researchers aiming to leverage NER for accelerated materials discovery.

Quantitative Performance Advantage

Empirical studies consistently demonstrate that models pre-trained on materials science text significantly outperform their general-purpose counterparts on NER tasks. The performance advantage is most pronounced in low-data regimes, a common scenario in scientific research where annotated data is scarce [80] [77].

Table 1: NER Model Performance (F1 Score) Comparison on Materials Science Datasets

Model	Pre-training Corpus	Solid-State Materials Dataset	Doping Dataset	Gold Nanoparticles Dataset
BERT	General Text (BookCorpus, Wikipedia)	Baseline	Baseline	Baseline
SciBERT	Broad Scientific Corpus (Multidisciplinary)	+3% to +12% vs. BERT [77]	+3% to +12% vs. BERT [77]	+3% to +12% vs. BERT [77]
MatBERT	Materials Science Journals	+1% to +12% vs. BERT [80] [77]	+1% to +12% vs. BERT [80] [77]	+1% to +12% vs. BERT [80] [77]
MatSciBERT	Materials Science Publications (Alloys, Glasses, Cement, etc.)	State-of-the-art on Matscholar, SOFC [76]	-	-

Beyond traditional sequence labeling, transforming the NER task into a Machine Reading Comprehension (MRC) framework has set new state-of-the-art benchmarks. In this approach, each entity type is extracted by answering a specific natural language query, which allows for better utilization of semantic context and effectively handles nested entities [13].

Table 2: Performance of the MatSciBERT-MRC Model on Public Benchmarks

Dataset	Primary Domain	F1-Score
Matscholar	General Materials Science	89.64% [13]
BC4CHEMD	Chemicals & Drugs	94.30% [13]
SOFC	Solid Oxide Fuel Cells	85.95% [13]
SOFC-Slot	Solid Oxide Fuel Cells	71.73% [13]

Experimental Protocols

Protocol A: Domain-Adaptive Pre-training of a Language Model

This protocol describes the process of creating a domain-specific language model like MatSciBERT from an existing base model [76].

• Corpus Curation:

  • Source Identification: Identify and gather a large corpus of text from the target domain. For materials science, this typically involves collecting full-text research papers and abstracts from sources like the Elsevier ScienceDirect API [76] [79].
  • Data Scope: The training corpus for MatSciBERT was composed of approximately 150,000 papers from materials science families, including inorganic glasses, metallic glasses, alloys, and cement, totaling about 285 million words [76].

• Model and Initialization:

  • Base Model Selection: Choose a suitable base model for continued pre-training. Due to a higher vocabulary overlap (53.64%), MatSciBERT was initialized using SciBERT weights rather than the original BERT (38.90% overlap) [76].
  • Tokenization: Use the tokenizer from the base model (e.g., SciBERT's WordPiece tokenizer) to process the domain-specific corpus [76].

• Pre-training Execution:

  • Objective: Continue pre-training using the standard Masked Language Modeling (MLM) objective, where the model learns to predict randomly masked tokens in the input sequence.
  • Infrastructure: This process requires significant computational resources, typically involving training on multiple GPUs for several days.

• Model Output:

  • The output is a pre-trained language model (e.g., MatSciBERT) whose weights can be publicly released for downstream tasks like NER [81].

Protocol B: Fine-Tuning for Named Entity Recognition (NER)

This protocol involves adapting a pre-trained model to a specific NER task using a smaller, annotated dataset [79].

• Data Annotation:

  • Ontology Definition: Define an ontology of entity types relevant to the task (e.g., POLYMER, PROPERTY_NAME, PROPERTY_VALUE, SYNTHESIS_METHOD) [82] [79].
  • Annotation Process: Manually annotate a set of texts (e.g., abstracts or paragraphs) with the defined entity labels. Using annotation tools like Prodigy can streamline this process. To ensure quality, measure inter-annotator agreement using metrics like Fleiss Kappa (target >0.8) [79].

• Model Architecture:

  • The standard architecture uses the domain-specific BERT model (e.g., MatSciBERT) as an encoder.
  • The contextualized token embeddings generated by BERT are fed into a linear classification layer with a softmax activation, which predicts the entity label for each token [79].

• Training:

  • The model is trained on the annotated dataset using a cross-entropy loss function.
  • Standard techniques like dropout (e.g., probability of 0.2) are applied to prevent overfitting, which is crucial in low-resource settings [79].

Protocol C: Machine Reading Comprehension for NER

This advanced protocol frames NER as a question-answering task, which is particularly effective for extracting nested entities [13].

• Data Transformation:

  • Convert the labeled NER data into (Context, Query, Answer) triples.
  • The Context is the original text sequence.
  • The Query is a natural language question designed for each entity type (e.g., "What material is mentioned?" for the MATERIAL entity).
  • The Answer is the span of text in the context that corresponds to the entity [13].

• Model Training and Prediction:

  • A model like BERT is trained to take the combined [CLS] Query [SEP] Context [SEP] sequence as input.
  • Instead of a single-token classifier, two binary classifiers are used to predict the start and end positions of the answer span within the context for a given query. This allows for the extraction of multiple entities [13].

The Scientist's Toolkit: Research Reagent Solutions

Table 3: Essential Resources for Materials Science NER

Resource Name	Type	Description & Function
MatSciBERT [76] [81]	Pre-trained Model	A BERT model pre-trained on a corpus of materials science publications. Serves as a powerful base encoder for NER models.
MatBERT [80] [77]	Pre-trained Model	A BERT variant pre-trained exclusively on materials science journal text, showing top performance on various NER tasks.
MaterioMiner Dataset [82]	Annotated Dataset	A fine-granular dataset with 179 distinct entity classes, ideal for training and benchmarking models on detailed information extraction.
MatSci-NLP Benchmark [78]	Evaluation Benchmark	A suite of seven NLP tasks (NER, Relation Classification, etc.) for standardized evaluation of model performance on materials science text.
PolymerScholar NER Dataset [79]	Annotated Dataset	A dataset of 750 polymer abstracts annotated with 8 entity types, facilitating NER work in the polymer sub-domain.
Hugging Face Hub [81]	Model Repository	A platform hosting many pre-trained models like MatSciBERT, allowing for easy download and integration into research pipelines.

Parameter Efficient Fine-Tuning (PEFT) and Low-Rank Adaptation (LoRA)

Parameter-Efficient Fine-Tuning (PEFT) has emerged as a critical paradigm for adapting large pre-trained models to downstream tasks, offering a balance between computational efficiency and model performance. In the context of materials science research, where vast amounts of unstructured data exist in scientific literature, PEFT enables researchers to customize powerful Large Language Models (LLMs) for specialized tasks like Named Entity Recognition (NER) without the prohibitive costs of full parameter optimization. The scarcity of structured data in materials science—evidenced by the minuscule fraction of available research data in usable structured form compared to the volume of published papers—makes efficient information extraction techniques particularly valuable [83]. Low-Rank Adaptation (LoRA) has gained significant popularity within this paradigm by freezing pre-trained weights and decomposing incremental matrices into trainable low-rank matrices, drastically reducing trainable parameters while maintaining competitive performance [84].

The fundamental mathematical principle underlying LoRA is the hypothesis that weight updates during adaptation have a low "intrinsic rank." Instead of fine-tuning all parameters in a weight matrix ( W \in \mathbb{R}^{d \times k} ), LoRA constrains the update by representing it with a low-rank decomposition ( W + \Delta W = W + BA ), where ( B \in \mathbb{R}^{d \times r} ), ( A \in \mathbb{R}^{r \times k} ), and the rank ( r \ll \min(d,k) ) [85] [84]. This approach reduces the number of trainable parameters from ( d \times k ) to ( r \times (d+k) ), typically resulting in a reduction of 7-8× fewer parameters than conventional fine-tuning [86]. For materials science NER applications, this efficiency enables rapid customization of models to recognize specialized entities like material compositions, synthesis parameters, and experimental conditions without requiring massive computational resources.

PEFT Methodologies: Comparative Analysis

Taxonomy of PEFT Approaches

Parameter-efficient methods can be broadly categorized into three main types: addition-based, selection-based, and reparameterization-based methods [84]. Addition-based methods introduce additional parameters or layers and train only these newly introduced components. Examples include prefix tuning and prompt tuning, which introduce supplementary trainable prefix tokens attached either to the input or the hidden layers of the base model. While effective, these methods alter the original model structure and can introduce additional costs during inference [84]. Selection-based methods achieve efficient fine-tuning by selectively choosing specific layers, parameters, or structures in the network. BitFit, for instance, trains only the bias terms in the network, while Diff Pruning learns a task-specific "difference" vector [84]. Both methods significantly reduce trainable parameters while maintaining performance. Reparameterization-based methods, including LoRA, leverage low-rank representations to minimize the number of trainable parameters, offering an optimal balance for many applications [84].

LoRA and Advanced Variants

The core LoRA method has inspired numerous advanced variants that address specific limitations. AdaLoRA represents the incremental matrix in Singular Value Decomposition (SVD) form and performs adaptive rank adjustment by pruning singular values based on their importance [84]. La-LoRA (Layer-wise Adaptive Low-Rank Adaptation) introduces dynamic rank allocation to each layer based on contribution to task performance, addressing the limitation of uniform rank assignment in standard LoRA [84]. NoRA (Nested Low-Rank Adaptation) employs serial structures and activation-aware SVD to optimize initialization and fine-tuning of projection matrices, reducing fine-tuning parameters by 85.5% while enhancing performance by 1.9% on LLaMA-3 8B [87]. QLoRA (Quantized LoRA) further extends efficiency by quantizing the base model to 4-bit precision, making it possible to fine-tune a 65B parameter model on a single 48GB GPU [88]. For materials science NER applications, these advanced methods enable more efficient adaptation to the complex, hierarchical entity relationships characteristic of scientific text.

Table 1: Comparison of Major PEFT Methods for Materials Science NER

Method	Core Principle	Parameter Efficiency	Inference Overhead	Suitability for Materials NER
LoRA	Low-rank decomposition of weight updates	High (7-8× fewer parameters)	Minimal	Excellent for most entity types
AdaLoRA	Adaptive rank allocation via SVD pruning	Very High	Minimal	Optimal for complex entity relations
La-LoRA	Layer-wise dynamic rank allocation	High	None	Excellent for multi-task NER setups
QLoRA	4-bit quantization + LoRA	Extreme	Minimal	Ideal for resource-constrained environments
Prefix Tuning	Trainable prefix tokens	Moderate	Present (increased sequence length)	Moderate for structured outputs
Prompt Tuning	Trainable soft prompts	Moderate	Present (increased sequence length)	Moderate for specialized terminologies

Application to Named Entity Recognition in Materials Science

NER Workflow and LoRA Integration

Named Entity Recognition in materials science involves identifying and classifying specialized entities such as material compositions, synthesis methods, characterization techniques, and property measurements within scientific text. The application of LoRA-fine-tuned models to this task follows a structured workflow that maximizes extraction accuracy while maintaining computational efficiency. As demonstrated in recent studies, LLMs fine-tuned with LoRA can successfully perform joint named entity recognition and relation extraction (NERRE), handling the complex inter-relations characteristic of materials science knowledge [1]. This approach differs from traditional pipeline-based methods where NER and relation extraction are separate steps; instead, a single fine-tuned model can output structured representations of hierarchical entity relationships [1].

The typical workflow begins with data collection and annotation, where domain experts label text passages with the desired entities and relationships. For materials science, this might include annotating sentences with entities like "LiCoO2" (material), "sol-gel" (synthesis method), and "350°C" (synthesis parameter), along with their relationships [1]. The annotation format defines the output structure, which can be simple English sentences or more structured formats like JSON objects. With approximately 100-500 annotated examples, a base LLM can then be fine-tuned using LoRA to perform the extraction task independently [1]. This method has demonstrated strong performance on representative tasks in materials chemistry, including linking dopants and host materials, cataloging metal-organic frameworks, and general composition/phase/morphology/application information extraction [1].

Diagram 1: LoRA Fine-tuning for Materials NER (82 characters)

Hybrid Approaches for Enhanced NER Performance

Recent research has demonstrated that hybrid approaches combining LLMs with supervised Small Language Models (SLMs) can achieve superior NER performance in specialized domains. One study showed that a relatively weaker LLM enhanced with LoRA-based fine-tuning and similarity-based prompting could achieve performance comparable to a SLM baseline [89]. By implementing a fusion strategy that prioritizes the SLM's predictions while using LLM guidance in low-confidence cases, researchers achieved performance surpassing both individual baselines on Chinese NER datasets [89]. This approach leverages the structured prediction capabilities of SLMs while incorporating the semantic understanding and adaptability of LLMs, offering a promising direction for materials science NER where both precision and adaptability to novel entities are crucial.

For materials science applications, this hybrid approach can be particularly valuable when dealing with diverse document types, from historical research papers to contemporary articles with varying reporting formats. The SLM component provides consistent extraction of well-established entities, while the LoRA-enhanced LLM component adapts to novel terminology, complex entity relationships, and contextual variations. This combination addresses the "death by 1000 cuts" problem in chemical data extraction, where the sheer scale of possible variations makes comprehensive rule-based systems intractable [83].

Experimental Protocols and Implementation

Standardized LoRA Fine-tuning Protocol for Materials NER

Objective: Adapt a base language model (e.g., LLaMA-2, Mistral) to extract materials science entities from research text using Low-Rank Adaptation.

Materials and Setup:

• Hardware: NVIDIA GeForce RTX 4090 GPU (or comparable with ≥24GB VRAM) [84]
• Software: Hugging Face Transformers, PEFT library, PyTorch
• Base Model: Pre-trained LLM (7B parameter models recommended for resource efficiency)
• Datasets: Annotated materials science text (100-500 examples minimum) [1]

Table 2: Research Reagent Solutions for LoRA NER Experiments

Component	Specification	Function/Role
Base LLM	LLaMA-2 7B, Mistral 7B	Foundation model providing general language capabilities and knowledge
LoRA Adapters	Rank (r)=8-16, alpha=16-32	Efficient task-specific adaptation with minimal parameters
Annotation Framework	Custom schema for materials entities	Defines entity types and relationships for domain specialization
Optimizer	AdamW, learning rate=3e-4	Controls parameter update process during fine-tuning
Tokenization	SentencePiece, BPE tokenizers	Text preprocessing and model input formatting
Evaluation Metrics	F1-score, precision, recall	Quantifies NER performance and extraction accuracy

Procedure:

• Data Preparation:
  • Collect and annotate 100-500 text passages (abstracts or full paragraphs) from materials science literature [1]
  • Define entity schema covering key materials science concepts (composition, phase, morphology, application, synthesis parameters) [1]
  • Format annotations as JSON objects with consistent structure for model training

• Model Configuration:

  • Load pre-trained base model using Hugging Face Transformers
  • Configure LoRA parameters: rank=8, loraalpha=16, targetmodules="qproj,vproj" (for LLaMA architecture)
  • Set training parameters: batchsize=4, gradientaccumulationsteps=4, learningrate=3e-4, numtrainepochs=10

• Training Cycle:

  • Initialize LoRA matrices with random Gaussian initialization
  • Freeze all base model parameters
  • Train only LoRA adapter layers on annotated dataset
  • Validate on held-out examples every epoch to monitor for overfitting

• Evaluation:

  • Quantitative: Calculate precision, recall, and F1-score for entity extraction
  • Qualitative: Manual inspection of model outputs for complex entity relationships
  • Compare against baseline methods (full fine-tuning, other PEFT approaches)

• Deployment:

  • Merge LoRA adapter with base model for inference efficiency
  • Implement pipeline for processing new research papers
  • Export structured outputs to materials knowledge graph or database

This protocol typically reduces trainable parameters by 85-95% compared to full fine-tuning while maintaining competitive performance for specialized NER tasks in materials science [84] [1].

Advanced Protocol: Layer-wise Adaptive LoRA (La-LoRA)

Objective: Implement dynamic rank allocation across network layers based on their contribution to NER performance.

Rationale: Uniform rank assignment in standard LoRA fails to account for heterogeneous importance of different layers, potentially resulting in suboptimal adaptation [84]. La-LoRA addresses this by treating each layer as an independent unit and progressively adjusting rank allocation during training.

Procedure:

• Initialization:
  • Begin with uniform low rank (r=4) across all layers
  • Establish contribution measurement metric based on gradient norms or activation patterns

• Dynamic Rank Allocation:

  • Implement Truncated Norm Weighted Dynamic Rank Allocation (TNW-DRA) to assess layer contributions [84]
  • Apply Dynamic Contribution-Driven Parameter Budget (DCDPB) to allocate higher ranks to layers with greater contribution potential [84]
  • Gradually increase current allocatable rank (CAR) throughout training rather than setting to maximum immediately [84]

• Progressive Training:

  • Early stages: Fewer parameters allocated to learn foundational features
  • Later stages: Increased budgets to capture more complex features and relationships
  • Monitor layer-specific contributions and adjust rank allocations accordingly

Results: La-LoRA has demonstrated consistent outperformance over standard LoRA benchmarks across multiple tasks, with reduced fine-tuning parameters (85.5%), training time (37.5%), and memory usage (8.9%) while enhancing performance by 1.9% on LLaMA-3 8B [84].

Diagram 2: La-LoRA Layer Architecture (73 characters)

Results and Performance Metrics

Quantitative Performance Analysis

The application of LoRA and its variants to materials science NER has demonstrated compelling results across multiple evaluation metrics. Fine-tuned LLMs using LoRA have achieved strong performance on representative tasks in materials chemistry, including linking dopants and host materials, cataloging metal-organic frameworks, and general composition/phase/morphology/application information extraction [1]. These approaches have proven effective for both sentence-level and document-level materials information extraction, with the capability to output structured representations of complex hierarchical entity relationships [1].

In broader NER applications, studies have shown that LLMs improved with LoRA-based fine-tuning and similarity-based prompting can achieve performance comparable to supervised Small Language Model baselines [89]. Hybrid approaches that prioritize SLM predictions while using LLM guidance in low-confidence cases have demonstrated outperformance over both individual baselines on multiple NER datasets [89]. For materials science applications specifically, this suggests significant potential for accurately extracting complex scientific knowledge with reduced computational requirements.

Table 3: Performance Comparison of PEFT Methods on Model Adaptation

Method	Trainable Parameters	Memory Usage	Training Time	NER F1-Score
Full Fine-tuning	100% (reference)	100% (reference)	100% (reference)	91.5 (reference)
Standard LoRA	12-15%	65-70%	45-50%	90.8
QLoRA	8-10%	45-50%	40-45%	89.2
La-LoRA	7-9%	60-65%	30-35%	92.1
NoRA	5-7%	55-60%	25-30%	91.9
Adapter Tuning	15-18%	70-75%	50-55%	89.5

Qualitative Advantages for Materials Science

Beyond quantitative metrics, LoRA-enhanced NER systems provide qualitative advantages for materials science research. The flexibility to output structured knowledge in customizable formats (e.g., JSON objects, simple English sentences) enables seamless integration with downstream applications such as materials knowledge graphs, automated literature reviews, and data-driven discovery pipelines [1]. This approach successfully handles the complex inter-relations inherent in inorganic materials science, where properties are determined by combinations of elemental composition, atomic geometry, microstructure, morphology, processing history, and environmental factors [1].

The parameter efficiency of LoRA methods also enables more rapid iteration and specialization to sub-domains within materials science. Researchers can maintain a single base model with multiple LoRA adapters specialized for different tasks—extracting battery materials data, catalysis information, or polymer characteristics—without the storage overhead of multiple fully fine-tuned models [88] [84]. This modular approach aligns well with the diverse and specialized nature of materials science research, where extraction requirements may vary significantly across sub-disciplines.

Parameter-Efficient Fine-Tuning, particularly through Low-Rank Adaptation and its advanced variants, represents a transformative approach for adapting large language models to the specialized domain of materials science Named Entity Recognition. The methods detailed in these application notes enable researchers to overcome the historical challenges of extracting structured knowledge from scientific text while maintaining computational efficiency. As the field advances, several promising directions emerge for further enhancing PEFT applications in scientific NER.

Future developments will likely focus on multi-modal extraction capabilities, combining text with molecular structures, spectra, and microscopy images to create comprehensive materials knowledge bases [83]. Cross-document analysis capabilities will enable connecting disjoint data published in separate articles, potentially revealing novel "Swanson links" between disparate research findings [83]. Additionally, continued advancement in PEFT methods—particularly those optimizing layer-wise contributions, dynamic architecture adjustments, and quantization techniques—will further reduce computational barriers while improving extraction accuracy for complex scientific entities and relationships.

The integration of these efficient adaptation methods with domain-specific knowledge validation creates a powerful paradigm for accelerating materials discovery. By enabling accurate, efficient extraction of structured knowledge from the vast corpus of materials science literature, PEFT and LoRA methodologies serve as critical components in the ongoing digital transformation of materials research and development.

Benchmarking NER Models: Accuracy, Performance, and Choosing the Right Tool

In the field of materials science, the exponential growth of scientific publications has created a critical need to automatically extract structured information from vast amounts of unstructured text. Named Entity Recognition (NER)—the computational task of identifying and classifying specific entities like material compositions, properties, and synthesis methods in text—is fundamental to this process. Evaluating the performance of an NER system is not a matter of simple accuracy; it requires a balanced understanding of Precision, Recall, and the F1-Score. These metrics provide a nuanced view of a model's capability, ensuring that the extracted data is reliable enough to build robust materials databases and accelerate discovery.

---

The Core Metrics: Definitions and Calculations

To understand how an NER system performs, we measure its effectiveness in identifying relevant entities and avoiding mistakes. The core concepts are built on the count of True Positives (TP), False Positives (FP), and False Negatives (FN).

• A True Positive (TP) occurs when the model correctly identifies a named entity.
• A False Positive (FP) occurs when the model incorrectly labels a word or phrase as a named entity.
• A False Negative (FN) occurs when the model fails to identify a named entity that is present in the text.

Precision: The Measure of Accuracy

Precision answers the question: "Of all the entities the model labeled, how many were correct?" It is a measure of the model's reliability and accuracy.

Precision = TP / (TP + FP)

A high Precision means the model is trustworthy; when it predicts an entity, it is likely correct. This is crucial in materials science to avoid polluting databases with incorrect data. For example, if a model designed to identify "DOPANT" entities extracts 10 entities, but only 6 are actual dopants, its Precision is 60%.

Recall: The Measure of Completeness

Recall answers the question: "Of all the actual entities present in the text, how many did the model find?" It is a measure of the model's comprehensiveness.

Recall = TP / (TP + FN)

A high Recall means the model misses very few entities. This is vital for ensuring that a literature review is thorough. For instance, if a paragraph contains 20 true "MAT" (material) entities, but the model only finds 6 of them, its Recall is 30%.

F1-Score: The Harmonic Mean

The F1-Score is the harmonic mean of Precision and Recall, providing a single metric that balances both concerns.

F1-Score = (2 × Precision × Recall) / (Precision + Recall)

The F1-Score is the most important metric for getting a holistic view of model performance, especially when you need to find a balance between minimizing false alarms (FP) and minimizing missed entities (FN). A model can have high Precision but low Recall (it doesn't find many entities, but its predictions are correct), or high Recall but low Precision (it finds most entities but makes many mistakes). The F1-Score penalizes extreme values in either, making it the preferred metric for reporting overall NER performance [90].

The following diagram illustrates the logical relationship between these core metrics and the final F1-Score:

Why F1-Score is Critical for Materials Science NER

In materials informatics, the cost of errors in automated data extraction is high. Relying solely on Precision or Recall can lead to suboptimal research outcomes.

• The High Precision Trap: A model with 95% Precision but 40% Recall might seem accurate. However, it fails to capture over half of the relevant data points from the literature (e.g., missing a novel material composition reported in a paper). This incomplete data can lead to biased conclusions and hinder discovery.
• The High Recall Trap: A model with 95% Recall but 40% Precision captures almost everything but introduces significant noise and incorrect entities into the knowledge base (e.g., mislabeling a general term as a specific material). This "dirty data" can corrupt training sets for machine learning models and lead to invalid scientific predictions.

Therefore, the F1-Score provides the essential balance. It ensures that NER systems are both accurate and comprehensive, which is a foundational requirement for building large-scale, trustworthy materials databases from literature [90]. For example, in a recent study, the MatSKRAFT framework for extracting materials knowledge from scientific tables achieved an F1-score of 88.68% for property extraction, demonstrating a strong balance that enables reliable data synthesis [91].

Performance Benchmarks in Materials Science NER

The performance of NER models is highly dependent on their architecture and, critically, on their training data. Domain-specific models that are pre-trained on scientific and materials science text significantly outperform general-purpose models. The following table summarizes quantitative benchmarks from recent studies, highlighting the advantage of domain-specific pre-training.

Table 1: Performance Comparison of NER Models on Materials Science Tasks

Model	Model Description	Dataset / Task	Reported F1-Score	Key Takeaway
MatBERT [60]	BERT model with domain-specific pre-training on materials science text.	Solid-State Materials Dataset	Outperformed BERT by ~12% and SciBERT by ~1%	Domain-specific pre-training provides a measurable advantage.
BiLSTM [60]	A simpler model with domain-specific pre-trained word embeddings.	Solid-State Materials Dataset	Consistently outperformed the general BERT model	Even simpler models with domain knowledge can outperform complex general models.
MatSciBERT [53]	A materials-aware language model trained on ~285M words from peer-reviewed papers.	Matscholar NER Task	Established state-of-the-art results	A materials-specific language model significantly accelerates information extraction.
LLM Fine-tuning [92]	Fine-tuning a Large Language Model (ChatGLM3-6B) with low-quality datasets.	Construction Documents NER	F1 reached 0.756	LLMs can be effectively fine-tuned for NER even with imperfect, domain-specific data.
MatSKRAFT [91]	A specialized framework using constraint-driven Graph Neural Networks (GNNs).	Property Extraction from Scientific Tables	F1 of 88.68% (Properties), 71.35% (Compositions)	Specialized, non-LLM architectures can achieve high performance on structured data.

The experimental workflow for establishing these benchmarks, from data preparation to model evaluation, can be summarized as follows:

Experimental Protocol: Evaluating a Custom NER Model

This protocol provides a step-by-step guide for training and evaluating a transformer-based NER model on a custom materials science dataset, using standard Python libraries.

Research Reagent Solutions

Table 2: Essential Software Tools and Libraries for NER Implementation

Item Name	Function / Application	Specific Use in NER Pipeline
Transformers Library [90]	Provides pre-trained models (e.g., BERT, SciBERT, MatSciBERT).	Serves as the core model architecture for transfer learning and fine-tuning.
PyTorch (torch) [90]	A deep learning framework.	Handles tensor computations, model training, and GPU acceleration.
Datasets Library [90]	Efficient dataset loading and management.	Loads and preprocesses the annotated NER dataset into a suitable format for the model.
Seqeval Library [90]	A specialized evaluation library for sequence labeling tasks.	Critically calculates F1-Score, Precision, and Recall at the entity level, accounting for BIO tags.
pandas [90]	Data handling and processing.	Used for loading, manipulating, and analyzing the dataset from CSV files.

Step-by-Step Procedure

• Data Preparation:

  • Obtain a labeled NER dataset (e.g., the solid-state dataset from Weston et al. or the Entity Annotated Corpus from Kaggle). The data should be in a format where each token (word) is tagged with a label like B-MAT (Beginning of a material), I-MAT (Inside a material), or O (Outside any entity).
  • Split the dataset into training, validation, and test sets (e.g., 80/10/10).
  • Use the datasets library to load and tokenize the text, aligning the labels with the tokenized words.

• Model Selection and Initialization:

  • Select a pre-trained model from the transformers library. For materials science, prefer domain-adapted models like MatSciBERT or SciBERT over the vanilla BERT.
  • Initialize a model for token classification, using the selected model as the base and setting the number of output labels to match your NER tag set.

• Model Fine-Tuning:

  • Define training arguments (batch size, learning rate, number of epochs).
  • Use a Trainer from the transformers library to fine-tune the model on your training dataset. Use the validation set for evaluating performance during training.

• Model Evaluation and Metric Calculation:

  • Use the fine-tuned model to make predictions on the held-out test set.
  • Use the seqeval library to compute the final metrics. This library is essential as it correctly handles the sequence nature of NER, unlike standard accuracy metrics.
  • The typical code snippet for evaluation is:

  • The classification_report will output the Precision, Recall, and F1-Score for each entity class and their overall averages.

For researchers and scientists building the next generation of materials databases, a deep understanding of Precision, Recall, and F1-Score is non-negotiable. These metrics are the gatekeepers of data quality. As evidenced by the superior performance of models like MatSciBERT and MatBERT, investing in domain-specific NER tools and evaluating them with the rigorous F1-Score is a critical step in ensuring that the knowledge extracted from millions of scientific publications is both comprehensive and accurate, thereby truly accelerating the pace of materials discovery and drug development.

Named Entity Recognition (NER) is a fundamental natural language processing (NLP) technique that automatically identifies and classifies key information entities—such as material names, properties, and synthesis methods—within unstructured text [9]. Within materials science research, the exponential growth of scientific publications has created a critical bottleneck in knowledge organization, making NER an essential technology for extracting structured data from literature at scale [93] [9]. The evolution of NER methodologies has progressed from traditional machine learning approaches to deep learning architectures, and most recently to large language models (LLMs), each offering distinct advantages and limitations for materials science applications [8]. This application note provides a comprehensive technical comparison of these three paradigms, offering detailed experimental protocols and implementation resources to guide researchers in selecting and deploying optimal NER solutions for materials science and drug development applications.

Technical Approach Comparison

Table 1: Comparative analysis of NER approaches for materials science

Feature	Traditional Machine Learning	Deep Learning	Large Language Models (LLMs)
Architecture	Conditional Random Fields (CRF), Support Vector Machines (SVM) [41]	BiLSTM-CRF, CNN-CRF, Transformer-based models (SciBERT, MatBERT) [9]	GPT-series, in-context learning, prompt engineering [93]
Data Requirements	Medium-sized labeled datasets with heavy feature engineering [8]	Large labeled datasets (thousands of examples) [9]	Few-shot or zero-shot learning; minimal labeled data [93]
Performance (F1-Score)	~80-85% (highly feature-dependent) [94]	~90.8% (MatBERT-CNN-CRF on perovskite data) [9]	High performance with strategic prompt design, comparable to fine-tuned models [93]
Training Needs	Requires extensive feature engineering and preprocessing [94]	Requires exhaustive fine-tuning with labeled datasets [93]	No fine-tuning needed; uses in-context learning [93]
Domain Adaptation	Requires complete retraining and feature redesign	Requires domain-specific pre-training (e.g., MatBERT) and fine-tuning [9]	Native capability for multiple materials domains via prompt engineering [93]
Key Advantages	Interpretable models; effective with limited data	State-of-the-art accuracy; automatic feature learning [9]	No labeled data requirement; identifies incorrect annotations [93]
Limitations	Labor-intensive feature engineering; limited performance ceiling [8]	Computationally intensive; large labeled datasets required [93]	Cost of API calls; potential hallucinations; context window limits [95]

Experimental Protocols

Deep Learning Approach: MatBERT-CNN-CRF for Perovskite NER

This protocol details the methodology for achieving state-of-the-art NER performance on a perovskite materials dataset, achieving an F1-score of 90.8% [9].

Materials Dataset Preparation

• Source: Collect abstracts from perovskite literature using publisher APIs (e.g., Springer-Nature)
• Annotation: Label entities using the IOBES scheme (Inside, Outside, Beginning, End, Single) which provides higher F1-scores compared to simpler labeling schemes
• Split: Divide annotated dataset into training (80%), validation (10%), and test (10%) sets
• Entities: Define domain-specific entity types including material (MAT), application (APL), property (PRO), and synthesis method (SYN) [9]

Model Implementation Steps

• Word Embedding Generation
  • Utilize MatBERT model, pre-trained on extensive materials science literature, to generate contextualized word embeddings
  • Input tokenized sentences into MatBERT to obtain 768-dimensional embeddings for each token

• Feature Extraction with CNN

  • Pass embeddings through a 1D Convolutional Neural Network (CNN) with multiple filter sizes (2,3,4)
  • Apply ReLU activation function after convolutional layers
  • Use CNN's strength in capturing local semantic relationships and reducing feature dimensionality

• Sequence Labeling with CRF

  • Feed CNN outputs to a Conditional Random Field (CRF) layer
  • CRF models dependencies between subsequent labels, ensuring valid label sequences
  • Calculate training and validation loss using CRF layer outputs [9]

Training Configuration

• Optimizer: Adam with learning rate of 2e-5
• Batch Size: 16 sequences
• Epochs: 50 with early stopping based on validation loss
• Regularization: Dropout (0.1) to prevent overfitting

LLM Approach: In-Context Learning for Materials NER

This protocol leverages GPT-style models for NER without explicit training, using few-shot prompting instead [93].

Prompt Engineering Design

• Task Specification
  • Clearly define the NER task in natural language: "Identify and classify materials science entities from the following text"
  • Specify entity types with examples: MAT (material names), PRO (properties), SYN (synthesis methods), APL (applications)

• Few-Shot Examples

  • Select 3-5 representative examples from the target domain
  • Ensure examples cover all entity types of interest
  • Format examples with input text and desired output entities

• Output Structure

  • Specify JSON output format for structured results
  • Define entity boundaries and confidence measures

Implementation Workflow

• Input Preparation
  • Chunk long documents into paragraphs respecting sentence boundaries
  • Limit input length to model's context window (e.g., 4096 tokens for GPT-3.5)

• API Call Configuration

  • Use ChatCompletion endpoint with GPT-3.5-turbo or GPT-4
  • Set temperature=0 for deterministic outputs
  • Configure max_tokens to accommodate expected output length

• Output Processing

  • Parse JSON response to extract entities and confidence scores
  • Handle potential parsing errors with fallback strategies
  • Merge entities from multiple chunks, resolving boundary conflicts [93]

Validation and Quality Control

• Human Review: Sample and verify model outputs for critical applications
• Consistency Checking: Compare entity extractions across similar documents
• Error Analysis: Identify common failure patterns for prompt refinement

Workflow Visualization

Diagram 1: NER workflow comparison across three approaches

Research Reagent Solutions

Table 2: Essential tools and resources for materials science NER implementation

Resource	Type	Application	Access
MatBERT [9]	Pre-trained language model	Domain-specific word embeddings for materials science	Hugging Face Transformers
GPT-4/ChatGPT [93]	Large language model	Zero-shot/few-shot NER via prompt engineering	API access
BERT/SciBERT [8]	Pre-trained language model	General scientific text processing	Hugging Face Transformers
Perovskite NER Dataset [9]	Labeled dataset	Benchmarking and model training	Available from research papers
spaCy	NLP library	Text preprocessing and pipeline management	Open source
Hugging Face Transformers	Model library	Access to pre-trained models and fine-tuning	Open source
BRAT	Annotation tool	Manual annotation of training data	Open source

The selection of an appropriate NER approach for materials science research depends critically on available resources and project requirements. Traditional ML methods remain viable for limited-data scenarios where interpretability is prioritized. Deep learning approaches, particularly domain-adapted models like MatBERT-CNN-CRF, deliver state-of-the-art accuracy but require significant annotated data and computational resources. LLMs offer a compelling alternative with their minimal data requirements and rapid prototyping capabilities, though concerns regarding cost and hallucination require careful mitigation. As these technologies continue to evolve, hybrid approaches that leverage the strengths of multiple paradigms will likely emerge as the most effective strategy for extracting structured knowledge from the rapidly expanding materials science literature.

In the field of materials science, efficiently extracting structured information from the vast body of scientific literature is a critical challenge. Named Entity Recognition (NER)—a natural language processing (NLP) technique for identifying and classifying key information entities in text—serves as a foundational step for building knowledge graphs and accelerating data-driven research [9] [8]. The emergence of Large Language Models (LLMs) has revolutionized NER, presenting a choice between two primary approaches: using massive, general-purpose LLMs (e.g., GPT-4) or smaller, domain-specific models (e.g., MatBERT) fine-tuned on scientific corpora [18] [58]. This application note examines whether domain specialists genuinely outperform generalists in the context of NER for materials science, providing experimental protocols and quantitative comparisons to guide researcher selection.

Quantitative Performance Comparison

The table below summarizes key performance metrics from published studies, comparing general-purpose and domain-specific LLMs on materials science NER tasks.

Table 1: Performance Comparison of LLMs on Materials Science NER Tasks

Model Type	Example Model	Key Performance Metrics	Domain / Dataset	Comparative Result
Domain-Specific	MatBERT-CNN-CRF [9]	F1 Score: 90.8%	Perovskite Material Abstracts	Outperformed BERT, SciBERT, and MatBERT by 1-6% [9].
Domain-Specific	MatBERT [42]	F1 Score	General Materials Science	Improved over BERT and SciBERT by ~1-12% across three materials datasets [42].
Domain-Specific	Fine-tuned Task-Specific (e.g., MatSciBERT) [58]	F1 Score	Materials Fatigue & Microstructure	Significantly outperformed GPT-4 in ontology-conformal NER, especially on fine-grained entities [58].
General-Purpose	GPT-4 [58]	F1 Score	Materials Fatigue & Microstructure	Underperformed fine-tuned domain-specific models on ontology-conformal NER [58].
General-Purpose	General-purpose LLMs (e.g., GPT-4o, Qwen2.5) [96]	Accuracy: ~80%	Materials Simulation Tool QA (pymatgen)	Significantly outperformed domain-specific materials LLMs (which scored <32%) on tool knowledge [96].
Domain-Specific	Materials Chemistry LLMs [96]	Accuracy: <32%	Materials Simulation Tool QA (pymatgen)	Fell far behind general-purpose models in understanding tool usage [96].

Experimental Protocols for Materials Science NER

Protocol A: Fine-Tuning a Domain-Specific Model (MatBERT-CNN-CRF)

This protocol details the methodology for achieving state-of-the-art performance on a perovskite NER task, as documented by Zhang et al. [9].

1. Objective: To construct a named entity recognition model for extracting material-related entities from perovskite scientific abstracts.

2. Research Reagent Solutions & Computational Tools: Table 2: Essential Tools for Protocol A

Item / Tool	Function in the Protocol
MatBERT Model	Provides domain-adapted, contextualized word embeddings from materials science text [9].
1D Convolutional Neural Network (CNN)	Acts as a downstream model to extract local contextual features and character-level patterns from the word embeddings [9].
Conditional Random Field (CRF) Layer	Decodes the final sequence of entity labels by modeling dependencies between adjacent labels, ensuring global consistency [9].
IOBES Annotation Scheme	A labeling scheme that outperforms simpler ones (e.g., IOB) by specifying if a token is the Beginning, Inside, or End of an entity, a Single-token entity, or Outside an entity [9].
Perovskite Dataset (800 annotated abstracts)	A specialized, human-annotated dataset used for training and evaluating the model, containing entities like material names (MAT) and applications (APL) [9].

3. Workflow:

The following diagram illustrates the end-to-end workflow for the MatBERT-CNN-CRF model.

4. Procedure:

• Data Preparation & Annotation:
  • Collect text corpus from scientific abstracts (e.g., via Springer-Nature API).
  • Annotate the text with relevant entity labels (e.g., MAT, APL) following the IOBES scheme to ensure high-fidelity sequence labeling [9].
• Model Architecture & Training:
  • Embedding Generation: Pass tokenized input text through MatBERT to generate contextualized word embeddings. MatBERT is preferred because it is pre-trained on a massive corpus of materials science literature, providing a significant domain advantage over general BERT [9] [42].
  • Feature Extraction: Feed the embeddings into a 1D-CNN layer. The CNN is effective at capturing local semantic relationships and character-level features between words in a sentence [9].
  • Sequence Labeling: The features from the CNN are passed to a CRF layer. The CRF layer uses a transition matrix to learn the legal sequences of entity tags (e.g., an "I-APL" cannot follow a "B-MAT"), thereby decoding the most probable global sequence of labels [9].
  • Train the combined model (MatBERT-CNN-CRF) on the annotated dataset, using cross-entropy loss.
• Validation & Evaluation:
  • Evaluate model performance on a held-out test set using standard metrics: Precision, Recall, and F1-Score [9].

Protocol B: In-Context Learning with a General-Purpose LLM

This protocol outlines the use of a foundation LLM like GPT-4 for NER without fine-tuning, relying on its inherent reasoning capabilities and instructions provided in the prompt [58].

1. Objective: To perform ontology-conformal NER on materials science text using a general-purpose LLM with few-shot in-context learning.

2. Research Reagent Solutions & Computational Tools: Table 3: Essential Tools for Protocol B

Item / Tool	Function in the Protocol
Foundation LLM (e.g., GPT-4)	The core model that performs the reasoning and entity recognition based on the provided prompt and demonstrations [58].
Domain Ontology	A formal, machine-interpretable specification of the domain concepts (e.g., the Materials Mechanics Ontology). It defines the entity types and ensures semantic alignment and interoperability of extracted data [58].
Few-Shot Demonstrations	A small number of carefully curated, correctly annotated text examples included in the prompt to illustrate the NER task to the LLM [58].

3. Workflow:

The following diagram illustrates the two-stage in-context learning pipeline.

4. Procedure:

• Task Formulation & Prompt Design:
  • Define the entity types based on a pre-existing domain ontology (e.g., for materials fatigue) [58].
  • Construct the Prompt: Create a detailed instruction that explains the NER task, lists the target entity types, and specifies the desired output format (e.g., JSON).
  • Few-Shot Selection: Select 3-5 representative text snippets from the training data, along with their correct entity annotations, to serve as in-context examples within the prompt. The quality and relevance of these examples are critical for performance [58].
• Model Inference:
  • Submit the constructed prompt, containing the task instruction, few-shot examples, and the target text, to a general-purpose LLM like GPT-4 via its API.
• Output Processing:
  • The LLM will generate a completion based on the prompt. This output must be programmatically parsed and validated to extract the structured entities, as LLM outputs can be unstructured or inconsistent [58].

The evidence indicates that the superiority of domain-specific versus general-purpose LLMs is not absolute but contingent on the task's nature and constraints.

• For specialized, fine-grained NER: Domain-specific models like MatBERT consistently demonstrate superior performance [9] [42] [58]. Their specialized pre-training on scientific text allows for a deeper understanding of domain-specific terminology and context. This advantage is most pronounced in tasks requiring fine-grained entity distinction and strict conformity to a domain ontology [58]. Furthermore, they are more resource-efficient to run after fine-tuning.

• For broader knowledge and tool usage: Recent benchmarks reveal that general-purpose LLMs (e.g., GPT-4o, Qwen2.5) can significantly outperform smaller domain-specific models on tasks requiring broader reasoning, such as answering questions about materials simulation tools or generating functional code [96]. Their extensive pre-training provides a wider base of general knowledge, including programming.

Conclusion: For the core task of information extraction via NER from materials science literature, domain specialists do outperform generalists. Models like MatBERT, fine-tuned on specialized corpora, provide higher accuracy and reliability for extracting structured, ontology-conformal data. However, the optimal strategy for a materials research pipeline may be hybrid, leveraging domain-specific models for precise NER while employing general-purpose LLMs for broader tasks like literature summarization or code generation. Researchers should prioritize domain-specific models for high-fidelity NER but remain aware of the complementary strengths of generalists in the broader AI for science landscape.

The overwhelming volume of materials science literature presents a significant bottleneck for research and discovery. Named Entity Recognition (NER)—a fundamental Natural Language Processing (NLP) task that identifies and classifies specific entities like material compositions, synthesis methods, and properties in unstructured text—is crucial for automating the construction of structured materials databases [8]. The advent of Large Language Models (LLMs) has introduced two dominant paradigms for adapting these powerful tools to specialized domains like materials science: in-context learning (ICL) and supervised fine-tuning (SFT). This article examines the rise of these approaches, providing a comparative analysis and detailed protocols for their application in materials science NER to guide researchers and scientists in deploying these technologies effectively.

Key Concepts and Definitions

• Large Language Models (LLMs): Advanced AI systems, such as GPT-4o and Claude 3.5 Sonnet, based on the Transformer architecture. They are trained on massive text datasets to understand and generate human-like text, serving as the foundation for both ICL and SFT [97] [98].
• In-Context Learning (ICL): A method where a pre-trained, general-purpose LLM performs a specific task by following instructions and examples provided directly within its input prompt, without any updates to the model's internal parameters [99].
• Supervised Fine-Tuning (SFT): A process that further trains a pre-trained LLM on a specialized, task-specific dataset. This updates the model's weights, tailoring its knowledge and behavior to a particular domain, such as materials science NER [99] [100].
• Named Entity Recognition (NER): An NLP task that involves identifying and classifying named entities (e.g., material names, properties, synthesis conditions) in unstructured text into pre-defined categories [101].

Comparative Analysis: Performance and Resource Requirements

The choice between ICL and SFT involves a critical trade-off between performance, data availability, and computational resources. The following table summarizes quantitative findings from recent evaluations.

Table 1: Comparative Performance of LLM Approaches on NER and Related Tasks

Model / Approach	Task / Domain	Key Metric (F1 Score)	Data & Resource Requirements
GPT-4o (SFT) [99]	Clinical NER (CADEC)	87.1%	High cost; requires task-specific annotated data
GPT-4o (Few-Shot ICL) [99]	Clinical NER (CADEC)	Lower than SFT	Lower cost; requires few examples in prompt
Fine-tuned BERT/BART Models [102]	Various BioNLP Tasks	~0.65 (Macro-average)	Domain-specific annotated data
Zero/Shot LLMs (e.g., GPT-4) [102]	Various BioNLP Tasks	~0.51 (Macro-average)	No task-specific data; relies on model's pre-existing knowledge
Fine-tuned BERT-based models [73]	Materials Science NER	Outperformed zero-shot LLMs	Specialized, annotated materials science datasets

Table 2: Qualitative Comparison of ICL vs. SFT for Materials Science NER

Feature	In-Context Learning (ICL)	Supervised Fine-Tuning (SFT)
Performance	Competitive on reasoning tasks [102]; struggles with complex, domain-specific entities [73]	State-of-the-art in most extraction tasks; superior for complex, specialized entities [102] [73]
Data Needs	No annotated training data; only few examples in prompt	Requires high-quality, task-specific annotated data [100]
Cost & Speed	Lower initial cost; faster prototyping [100]	Higher upfront training cost; more efficient inference for high-volume tasks [100]
Flexibility	Highly flexible; tasks changed via prompt	Less flexible; model retraining needed for task changes
Hallucination & Consistency	Higher risk of hallucination and missing information [102]	Improved consistency and adherence to output format [100]

Experimental Protocols for Materials Science NER

Protocol A: Implementing In-Context Learning

Objective: To extract material names and properties from scientific literature using ICL.

Workflow:

Materials & Reagents:

• Base LLM API Key: Provides access to a powerful model like GPT-4o or Claude 3.5 Sonnet for text generation [97] [98].
• Prompt Template: A pre-defined text structure containing instructions, entity definitions, and examples [99].
• Few-Shot Examples: A small set (e.g., 3-5) of annotated text snippets demonstrating correct entity extraction for materials and properties [99].
• Input Text Corpus: The collection of target materials science literature (e.g., PDFs, raw text) for processing [73].

Procedure:

• Task Definition: Explicitly define the entity types to be extracted (e.g., MATERIAL, PROPERTY, VALUE, SYNTHESIS_METHOD).
• Prompt Engineering: Construct a clear, instructional prompt. Use a simple prompt structure over a complex, instruction-heavy one for better performance [99]. Example Prompt:

• Model Selection & Execution: Choose a capable LLM (e.g., GPT-4o via API) and submit the prompt along with the target text for processing.
• Output Parsing: Extract and validate the structured output (e.g., JSON) generated by the LLM.

Protocol B: Implementing Supervised Fine-Tuning

Objective: To create a high-performance, specialized model for materials science NER via SFT.

Workflow:

Materials & Reagents:

• Annotated Training Dataset: A high-quality dataset of materials science text annotated with entity labels. This is the most critical reagent [100].
• Base Foundation Model: An open-weight model like Llama 3.3 70B or Mistral 3 as the starting point for fine-tuning [97] [100].
• Fine-Tuning Framework: Software such as axolotl or the QLoRA library, which reduces memory requirements by using low-rank adaptation [100].
• GPU Cluster: Access to high-performance GPUs (e.g., A100/H100 with 80GB VRAM) via cloud services like RunPod or vast.ai [100].

Procedure:

• Data Curation: Collect and annotate a dataset of materials science text. This can be done manually by experts or by generating a silver-standard dataset using a powerful LLM like GPT-4o with few-shot prompting [100].
• Model & Framework Setup: Select an appropriate base model and configure the fine-tuning framework (e.g., QLoRA) with hyperparameters.
• Training Execution: Run the fine-tuning job on the GPU cluster. Monitor the training loss to ensure convergence.
• Evaluation: Systematically evaluate the fine-tuned model's performance on a held-out test set not seen during training, using metrics like F1-score [100].

The Scientist's Toolkit: Research Reagent Solutions

Table 3: Essential Resources for LLM-Based NER in Materials Science

Reagent / Resource	Function / Role	Examples & Notes
Pre-trained LLMs	Foundation for ICL or starting point for SFT	Closed-source: GPT-4o, Claude 3.5 [97] [98]. Open-source: LLaMA series, Mistral Medium 3 [97].
Annotation Tools	Create labeled data for fine-tuning	Tools like Label Studio; crucial for generating high-quality training sets [100].
Fine-Tuning Software	Enables efficient model adaptation	QLoRA, axolotl; reduces computational cost of SFT [100].
Computing Hardware	Provides compute for model training/inference	Cloud GPUs (A100, H100); essential for running SFT protocols [100].
Evaluation Framework	Measures model performance systematically	Tools like promptfoo; critical for comparing different models and approaches [100].

The choice between ICL and SFT is not one-size-fits-all but should be guided by project-specific constraints and goals. The following decision pathway can assist researchers in selecting the appropriate strategy.

In conclusion, both ICL and SFT are powerful techniques for adapting LLMs to the critical task of NER in materials science. In-context learning offers a rapid, accessible entry point for prototyping and tasks that align well with an LLM's pre-existing knowledge. In contrast, supervised fine-tuning delivers superior accuracy and efficiency for high-stakes, high-volume, or highly specialized applications, justifying the investment in data and computation. For researchers aiming to automate the extraction of knowledge from the vast materials science literature, a hybrid strategy—beginning with ICL for exploration and transitioning to SFT for production-scale systems—will likely yield the most robust and effective outcomes.

Retrieval-Augmented Generation (RAG) and Agentic Systems for NER

The application of Named Entity Recognition (NER) in materials science research has been transformed by the integration of Retrieval-Augmented Generation (RAG) and agentic AI systems. These technologies address the fundamental challenge of keeping pace with the rapidly expanding and highly specialized scientific literature. RAG frameworks enhance the accuracy and reliability of NER by grounding the entity recognition process in dynamically retrieved, authoritative knowledge sources, rather than relying solely on a model's static parametric knowledge [103]. This is particularly crucial in materials science, where new compounds, synthesis methods, and characterization techniques emerge constantly. Agentic systems further amplify this capability by orchestrating complex, multi-step reasoning processes—decomposing intricate NER tasks into manageable subtasks, dynamically deciding which specialized tools or knowledge sources to consult, and validating intermediate results to ensure final output quality [104]. The synergy between RAG's knowledge retrieval strengths and agentic systems' procedural intelligence creates a powerful paradigm for extracting structured knowledge from unstructured scientific text, enabling researchers to build comprehensive knowledge graphs and accelerate materials discovery with unprecedented efficiency.

Core Architectural Frameworks

RAG Architectures for Enhanced NER

Retrieval-Augmented Generation models for NER have evolved from simple retrieval pipelines to sophisticated architectures designed for high-precision domains like materials science. Three primary architectural patterns have emerged:

• Naive RAG implements a basic pipeline of indexing, retrieval, and generation. While straightforward, this approach often struggles with the nuanced terminology and contextual dependencies of scientific text. [103]
• Advanced RAG incorporates pre-retrieval and post-retrieval optimizations. Key enhancements include sliding window techniques to process long documents, automatic taxonomy generation to adapt to domain-specific vocabularies, and re-ranking mechanisms to prioritize the most relevant retrieved passages. [52]
• Modular RAG represents the most flexible approach, integrating specialized components for query routing, multiple retrieval strategies (vector, keyword, graph-based), and memory management. This architecture is particularly suited for the complex entity relationships found in materials science literature. [103]

A particularly impactful innovation is the Context-Aware RAG (CARE-RAG) framework, which dynamically adjusts retrieval parameters—such as search depth, traversal strategies, and scoring weights—based on classified query intent. This query-type-aware retrieval prevents contextual dilution and enables more precise handling of diverse entity types, from material compositions to synthesis parameters. [105]

Agentic System Designs for Scientific NER

Agentic systems for scientific NER move beyond single-model inference to coordinated multi-agent workflows. These systems typically employ specialized agents with distinct roles:

• Decomposition Agents break down complex extraction tasks—such as identifying all entities in a full research paper—into focused subtasks (e.g., extract materials from methods section, find characterization techniques from results). [104]
• Tool-Use Agents interface with external resources and computational tools, such as querying materials databases, accessing application programming interfaces of specialized NER models, or performing semantic similarity searches across knowledge graphs. [104]
• Validation Agents cross-reference extracted entities against authoritative sources, check for consistency with domain knowledge, and flag potentially spurious extractions for human review. [105]

The BiomedKAI architecture demonstrates the power of this approach with six specialized agents (Diagnostic, Treatment, Drug Interaction, General Medical, Preventive Care, and Research) operating on a comprehensive biomedical knowledge graph. While designed for biomedical applications, this multi-agent framework provides a template for materials science adaptation, where specialized agents could focus on distinct subdomains such as synthesis protocols, structural characterization, or functional properties. [105]

Quantitative Performance Analysis

Table 1: Performance Benchmarks of NER and RAG Systems Across Scientific Domains

Model/System	Domain	Dataset/Metric	Performance Score	Key Innovation
MRC-for-NER [106]	Materials Science	Matscholar	89.64% F1	Converts sequence labeling to Machine Reading Comprehension
MRC-for-NER [106]	Chemistry	BC4CHEMD	94.30% F1	Handles nested entities via question-answering format
MRC-for-NER [106]	Chemistry	NLMChem	85.89% F1	Effectively utilizes semantic information in datasets
LLM-based NER Framework [52]	Manufacturing	Fused Deposition Modeling	91.92% F1	Uses RAG for automatic taxonomy customization
BiomedKAI (CARE-RAG) [105]	Biomedical	MedQA Accuracy	84.4%	Query-type-aware retrieval with multiple specialized agents
BiomedKAI (CARE-RAG) [105]	Biomedical	NEJM Diagnostic Precision	85.7%	Dynamic knowledge graph integration
Hybrid AI-NER Pipeline [107]	Drug Discovery	CORD-19 Corpus	80.5% F1	Model-in-the-loop with iterative human labeling

Table 2: Efficiency Metrics of Advanced RAG Systems

System	Token Efficiency	Hallucination Reduction	Multi-hop Reasoning Accuracy	Computational Requirements
BiomedKAI (CARE-RAG) [105]	66.5% (vs conventional RAG)	99.22% effectiveness	95.02%	General-purpose LLMs with standard hardware
Traditional RAG Systems [103]	Baseline	Limited	~70-80%	Often requires specialized infrastructure
Agentic Systems [104]	Varies by design	High with validation agents	>90% (target)	Multi-component with tool integration

Application Notes for Materials Science NER

Specialized Workflow for Materials Entity Extraction

Implementing RAG and agentic systems for materials science NER requires a specialized workflow tuned to the domain's unique characteristics:

• Taxonomy Development: Leverage RAG to automatically generate and refine domain-specific entity taxonomies by retrieving and synthesizing definitions from materials science textbooks, review articles, and ontologies. The LLM-based NER framework for additive manufacturing demonstrates this approach, using RAG to create precise taxonomies for manufacturing processes without manual definition. [52]

• Nested Entity Handling: Address the challenge of nested entities (e.g., "Nd₂Fe₁₄B" where "Nd", "Fe", and "B" are also entities) using Machine Reading Comprehension approaches. This method transforms sequence labeling into a question-answering task, effectively resolving overlapping entities through multiple independent queries. [106]

• Multi-modal Integration: Develop agentic systems that can correlate textual entities with experimental data, such as linking synthesis conditions mentioned in text with corresponding characterization results (XRD patterns, SEM images) from supplementary materials. [104]

• Knowledge Graph Population: Use extracted entities to build and continuously update materials knowledge graphs, creating rich networks connecting compositions, synthesis methods, properties, and applications. These knowledge graphs then serve as enhanced retrieval sources for future NER tasks, creating a virtuous cycle of improvement. [105]

The Scientist's Toolkit: Research Reagent Solutions

Table 3: Essential Components for Implementing RAG and Agentic NER Systems

Component/Tool	Function	Example Implementations
Knowledge Graphs	Provide structured, interconnected knowledge for retrieval	BiomedKAI's graph with 43K genes, 230K proteins, 17K drugs [105]
Specialized Agents	Decompose complex NER tasks into manageable subtasks	Diagnostic, Treatment, Research agents in BiomedKAI [105]
Machine Reading Comprehension (MRC) Models	Handle nested and overlapping entity recognition	MRC-for-NER achieving 89.64% F1 on Matscholar [106]
Iterative Human-in-the-Labeling	Efficiently generate training data with minimal expert input	Model-in-the-loop approach requiring only tens of hours [107]
Retrieval-Augmented NER (RA-NER)	Augment recognition with relevant external knowledge	Amazon's RA-NER for e-commerce, adaptable to materials science [108]
LLM Fine-tuning Frameworks	Adapt base models to domain-specific terminology	Framework for additive manufacturing achieving 91.92% F1 [52]
Query Intent Classification	Dynamically adjust retrieval strategies based on query type	CARE-RAG's context-aware retrieval parameters [105]

Experimental Protocols

Protocol 1: Implementing MRC for Nested Entity Recognition

Purpose: To extract nested and overlapping material entities from scientific text using a Machine Reading Comprehension approach.

Workflow:

• Dataset Preparation:
  • Select and annotate materials science texts with nested entity spans (e.g., "TiO₂ nanotube arrays" where "TiO₂" and "nanotube" are nested).
  • Convert traditional sequence labels into question-answer pairs. For example, create questions like "What is the base material composition?" and "What is the material morphology?"

• Model Training:

  • Implement a model architecture that encodes the context passage and question simultaneously.
  • Train the model to predict answer spans from the original text using datasets like Matscholar, BC4CHEMD, or NLMChem.

• Inference and Extraction:

  • For each input text, generate multiple questions targeting different entity types.
  • Aggregate the answer spans from all questions to reconstruct the complete set of entities, including nested structures.
  • Validate performance using standard F1 metrics, targeting >89% on materials science benchmarks. [106]

Protocol 2: Deploying Multi-Agent RAG System

Purpose: To implement a collaborative multi-agent system for comprehensive materials knowledge extraction.

Workflow:

• System Architecture:
  • Deploy specialized agents for key materials science subdomains: Synthesis Agent, Characterization Agent, and Application Agent.
  • Implement a query router that analyzes input text and directs specific entity recognition tasks to relevant agents.

• Knowledge Integration:

  • Construct or integrate with existing materials knowledge graphs containing entities such as compositions, crystal structures, synthesis methods, and properties.
  • Enable each agent to retrieve relevant context from this knowledge graph using vector semantic search combined with graph traversal.

• Collaborative Extraction:

  • Each agent processes the input text using its specialized retrieval strategy and recognition model.
  • Implement a validation agent that cross-references extracted entities across multiple sources and checks for consistency.
  • Synthesize final entity extractions with confidence scores and source attributions. [105] [104]

Future Directions and Challenges

The integration of RAG and agentic systems for materials science NER presents several promising research directions alongside significant challenges. A primary frontier is the development of continuously learning systems that can autonomously update their knowledge bases and extraction models as new research is published, addressing the current limitation of static knowledge. [104] Additionally, enhancing multi-modal reasoning capabilities would enable systems to correlate entities mentioned in text with data from figures, tables, and experimental datasets, creating more comprehensive knowledge representations. [103] Significant challenges remain in validating and ensuring the reproducibility of AI-extracted entities, particularly for novel materials where ground truth may be limited. [104] Furthermore, computational efficiency remains a concern, though approaches like BiomedKAI's context compression that achieves 66.5% token efficiency compared to conventional RAG demonstrate promising pathways forward. [105] As these systems become more sophisticated, establishing robust evaluation frameworks and benchmarks specific to materials science NER will be crucial for measuring progress and guiding future development.

Analysis of State-of-the-Art Performance on Key Materials Science Datasets

This application note provides a detailed analysis of state-of-the-art performance on key Named Entity Recognition (NER) datasets within the materials science domain. The exponential growth of materials science publications has created a critical need for automated information extraction tools to unlock knowledge buried in unstructured text [60]. This analysis documents benchmark results, methodologies, and experimental protocols that enable researchers to select appropriate models and datasets for accelerating materials discovery through automated knowledge extraction. The findings are contextualized within the broader thesis that domain-specific adaptation is crucial for achieving high-performance NER in the technically complex field of materials science, where specialized nomenclature and complex entity relationships present unique challenges for natural language processing systems.

State-of-the-Art Performance Analysis

Quantitative Performance Metrics on Benchmark Datasets

Domain-specific language models consistently outperform general-purpose models across diverse materials science NER tasks. The following tables summarize state-of-the-art performance on key benchmark datasets.

Table 1: Performance Comparison of BERT-based Models on Materials Science NER Tasks (F1 Scores)

Model	Pre-training Corpus	Solid-State Dataset	Doping Dataset	Gold Nanoparticle Dataset
MatBERT	Materials Science Literature	92.5%	89.7%	86.2%
SciBERT	Scientific Multidisciplinary	91.4%	87.2%	83.5%
BERT~BASE~	General (Wikipedia/BookCorpus)	88.3%	82.1%	79.8%

Data adapted from Trewartha et al. showing domain-specific pre-training advantages [60].

Table 2: LLM Performance on Graduate-Level Materials Science Reasoning (MSQA Benchmark)

Model Type	Exemplary Models	Accuracy (Binary QA)	Accuracy (Long-Form QA)
Proprietary API-based	GPT-4o, Gemini-2.0-Pro	Up to 84.5%	72.8%
Open-Source	Deepseek-v3, Llama-series	Up to 60.5%	51.3%
Domain-Fine-Tuned	Various	Often <50%	~40%

Data from Cheung et al. demonstrating performance gaps in complex reasoning tasks [109].

Table 3: Traditional NER Model Performance on Scientific Literature (F1 Scores)

Model	Architecture	Polymer NER	Materials Property Extraction
SciNER	Bi-LSTM + CRF + External Lexicon	81.3%	78.9%
BiLSTM	BiLSTM + Domain Embeddings	79.8%	79.5%
Rule-based	Dictionary Lookup + Patterns	62.4%	58.7%

Performance on specialized scientific NER tasks shows advantages of hybrid approaches [110].

Critical Performance Insights

The quantitative results reveal several critical patterns in materials science NER performance:

• Domain-specific pre-training provides measurable advantages: MatBERT improves over BERT~BASE~ by 1-12% across datasets, with the most significant gains observed in highly specialized subdomains like doping and nanoparticle synthesis [60].

• The performance gap widens in low-data regimes: SciBERT and MatBERT outperform original BERT to a greater extent when training data is limited, highlighting the value of domain-specific pre-training for practical applications with scarce annotations [60].

• Architectural simplicity can outperform general complexity: Despite relative architectural simplicity, BiLSTM models with domain-specific pre-trained word embeddings consistently outperform general BERT, demonstrating that domain knowledge can trump model complexity [60].

• Retrieval augmentation enhances LLM performance: Incorporating retrieved contextual data notably enhances model performance on the MSQA benchmark, showing retrieval augmentation as a crucial adaptation strategy for complex reasoning tasks [109].

Experimental Protocols

Domain-Specific Language Model Pre-training Protocol

Objective: Create a materials science domain-specific language model through continued pre-training of base transformer models.

Materials and Setup:

• Hardware: 4-8 NVIDIA A100 or V100 GPUs with at least 40GB memory each
• Software: Python 3.8+, PyTorch 1.9+, Transformers library, HuggingFace ecosystem
• Base Model: SciBERT or PubMedBERT as starting checkpoint
• Training Corpus: 2.4 million materials science abstracts [12] or 150,000 full-text articles [53]

Procedure:

• Corpus Preparation: Collect and preprocess materials science text corpus, removing non-text elements, tables, and formatting artifacts. For full-text processing, use specialized parsers like ChemistryHTMLPaperParser to preserve mathematical formulas and chemical representations [109].

• Tokenization: Apply domain-appropriate tokenization using the SciBERT or BERT tokenizer. For materials science text, vocabulary overlap is approximately 53.64% with SciBERT and 38.90% with BERT, favoring SciBERT as the starting point [53].

• Pre-training Configuration:

  • Batch Size: 32-64 per GPU (adjust based on GPU memory)
  • Learning Rate: 1e-4 to 5e-5 with linear decay
  • Sequence Length: 512 tokens with truncation for longer sequences
  • Training Steps: 50,000-100,000 steps (approximately 2-5 epochs)

• Pre-training Objectives: Implement masked language modeling (MLM) with 15% masking probability, using whole word masking for multi-token materials science terms.

• Validation: Monitor perplexity on held-out validation set (5-10% of corpus).

Troubleshooting Notes:

• Gradient clipping at 1.0 to prevent explosion
• Mixed precision (FP16) training to reduce memory usage
• Checkpoint saving every 5,000 steps for recovery

NER Model Fine-tuning Protocol

Objective: Adapt pre-trained domain-specific language models for named entity recognition on materials science text.

Materials and Setup:

• Annotated NER Datasets: Solid-state (800 abstracts), Doping (455 abstracts), or Gold Nanoparticle datasets [60]
• Model Architecture: Pre-trained MatSciBERT, MaterialsBERT, or SciBERT as encoder
• Computational Resources: 1-2 NVIDIA V100 or A100 GPUs

Procedure:

• Data Preparation:
  • Apply BIO (Beginning-Inside-Outside) or IO labeling scheme to annotated datasets
  • For materials science entities, common labels include: MAT (inorganic materials), SPL (symmetry/phase labels), PRO (material properties), APL (applications), SMT (synthesis methods), CMT (characterization methods) [60]
  • Split data into training (70-80%), validation (10-15%), and test (10-15%) sets

• Model Architecture:

  • Use pre-trained transformer as encoder with frozen or lightly fine-tuned parameters
  • Add linear classification layer on top of encoder outputs ([CLS] token or token-level embeddings)
  • Optional: Add Conditional Random Field (CRF) layer for structured prediction

• Training Configuration:

  • Batch Size: 16-32
  • Learning Rate: 2e-5 to 5e-5 with linear warmup for first 10% of steps
  • Weight Decay: 0.01
  • Dropout: 0.1-0.3 on classification layer
  • Maximum Epochs: 10-20 with early stopping based on validation F1

• Evaluation Metrics:

  • Primary: Entity-level F1 score (strict boundary matching)
  • Secondary: Precision, Recall, and Accuracy

Validation Approach:

• Cross-validation on smaller datasets (<1000 samples)
• Hold-out test set evaluation for larger datasets
• Statistical significance testing via bootstrap sampling

LLM Evaluation Protocol for Materials Science Reasoning

Objective: Evaluate large language models on complex materials science reasoning tasks using the MSQA benchmark.

Materials and Setup:

• MSQA Benchmark Dataset: 1,757 graduate-level materials science questions in two formats (detailed explanatory responses and binary True/False assessments) [109]
• LLMs: Various proprietary (GPT-4o, Gemini-2.0-Pro) and open-source (Deepseek-v3) models
• Evaluation Framework: Custom evaluation scripts for accuracy measurement

Procedure:

• Prompt Construction:
  • For binary questions: Use direct question-answering format with "True" or "False" as expected outputs
  • For long-answer questions: Use instruction-following prompts requesting detailed explanations
  • Implement few-shot learning with 3-5 exemplars for complex reasoning questions

• Inference Configuration:

  • Temperature: 0.0 for deterministic outputs in evaluation
  • Maximum Token Length: 512 for binary, 1024 for long-form answers
  • No beam search for efficiency

• Evaluation Methodology:

  • Binary Questions: Exact match for True/False assessment
  • Long-Form Answers: GPT-4 based evaluation for reasoning quality and factual accuracy
  • Expert human evaluation on 100-question subset for validation

• Retrieval Augmentation:

  • Implement dense passage retrieval from materials science literature
  • Concatenate retrieved context with original question
  • Evaluate performance improvement with and without retrieval

Workflow Visualization

NER Model Training and Evaluation Workflow

Diagram 1: End-to-end workflow for materials science NER model development

Dataset Quality Evaluation Framework

Diagram 2: Statistical evaluation framework for NER dataset quality

Research Reagent Solutions

Table 4: Essential Research Reagents for Materials Science NER Experiments

Reagent / Tool	Specifications	Function / Application
MatSciBERT	Pre-trained on 285M words from 150K materials science papers [53]	Domain-specific language model encoder for materials science NER tasks
MaterialsBERT	Trained on 2.4 million materials science abstracts, based on PubMedBERT [12]	Polymer-focused NER and relation extraction, powers material property data extraction pipeline
MSQA Benchmark	1,757 graduate-level questions across 7 subfields with binary and long-answer formats [109]	Evaluation of reasoning capabilities and factual knowledge of LLMs in materials science
Solid-State Dataset	800 annotated abstracts with 8 entity types (MAT, SPL, DSC, PRO, APL, SMT, CMT) [60]	Benchmark for general materials science NER performance evaluation
PolymerAbstracts Dataset	750 annotated abstracts with 8 entity types (POLYMER, PROPERTY_VALUE, etc.) [12]	Training and evaluation for polymer-focused information extraction
SciNER Model	Bi-LSTM + CRF + DBpedia lexicon features [110]	Traditional neural approach with external knowledge integration for scientific NER
ChemistryHTMLPaperParser	XML-based parser for materials science publications [109]	Preserves mathematical formulas and chemical representations during text extraction
Domain-Adaptive Pre-training Corpus	2.4 million materials science abstracts or 150K full-text articles [53] [12]	Continued pre-training of base language models for domain adaptation

Technical Discussion

Performance Pattern Analysis

The consistent outperformance of domain-specific models like MatSciBERT and MaterialsBERT establishes a clear hierarchy in materials science NER capabilities. MatBERT's 1-12% improvement over BERT~BASE~ demonstrates that domain-specific pre-training effectively captures materials science nomenclature and conceptual relationships [60]. This performance advantage stems from several factors: specialized vocabulary coverage (53.64% overlap with SciBERT vs. 38.90% with BERT), exposure to domain-specific syntactic patterns, and conceptual understanding of materials science relationships [53].

The unexpected success of BiLSTM models over general BERT, despite architectural simplicity, reveals that domain-specific word embeddings can compensate for architectural limitations [60]. This suggests a hybrid approach combining domain embeddings with transformer architectures may yield optimal results. The performance gap expansion in low-data regimes further emphasizes that domain knowledge becomes increasingly critical when annotated examples are scarce.

Dataset Quality Implications

Statistical dataset evaluation reveals significant quality variations across materials science NER benchmarks. Wang et al.'s framework identifies critical dimensions for dataset assessment: reliability (redundancy, accuracy, leakage), difficulty (unseen entity ratio, ambiguity, density), and validity (imbalance, entity-null rate) [111]. These metrics explain performance variations across datasets and guide improvement efforts.

The annotation quality significantly impacts model performance, with studies identifying 5.38% label error rates in widely used benchmarks like CoNLL03 [111]. For materials science specifically, domain expertise requirements compound annotation challenges, necessitating expert involvement and rigorous quality control processes. The PolymerAbstracts dataset achieved high inter-annotator agreement (Fleiss Kappa: 0.885) through multiple annotation rounds with refined guidelines [12], establishing a protocol for high-quality materials science corpus creation.

Emerging Trends and Future Directions

The integration of retrieval-augmented generation (RAG) with LLMs for materials science QA represents a promising direction for overcoming knowledge cutoff limitations [109]. The MSQA benchmark demonstrates that retrieval augmentation significantly enhances performance on complex reasoning tasks, suggesting hybrid approaches that combine parametric knowledge with external scientific databases.

Future advancements will likely focus on multi-modal extraction, combining textual information with chemical structures, phase diagrams, and experimental data. The development of unified frameworks that integrate symbolic AI for interpretability with deep learning for pattern recognition will address current "black box" limitations while maintaining high performance [112]. As dataset quality evaluation becomes more sophisticated, targeted dataset improvement will emerge as a more efficient path to performance gains than architectural modifications alone [111].

Conclusion

Named Entity Recognition has matured into an indispensable tool for unlocking the vast knowledge contained within materials science and biomedical literature. This guide has demonstrated that while foundational machine learning methods provide a strong baseline, transformer-based models like MatBERT and innovative frameworks like MRC consistently deliver state-of-the-art performance. The key to success lies in addressing domain-specific challenges through targeted strategies such as domain-adaptive pre-training and human-in-the-loop annotation. Looking forward, the integration of large language models and structured ontologies promises to further enhance the accuracy and interoperability of extracted knowledge. For biomedical research, the implications are profound: NER pipelines can systematically identify candidate drug molecules, repurpose existing therapeutics, and build comprehensive knowledge graphs that map complex disease mechanisms, ultimately accelerating the pace of drug discovery and clinical innovation.

References

• 1. Structured information extraction from scientific text with ... [https://www.nature.com/articles/s41467-024-45563-x]
• 2. Recent Advances in Named Entity Recognition [https://arxiv.org/html/2401.10825v3]
• 3. Evolution and emerging trends of named entity recognition [https://pmc.ncbi.nlm.nih.gov/articles/PMC11066397/]
• 4. A Systematic Review for Classification and Selection of ... [https://www.sciencedirect.com/science/article/pii/S2772662224000936]
• 5. Recent Advances in Named Entity Recognition [https://arxiv.org/html/2401.10825v2]
• 6. What Is Named Entity Recognition? [https://www.ibm.com/think/topics/named-entity-recognition]
• 7. What Is Named Entity Recognition (NER) [https://www.tonic.ai/guides/named-entity-recognition-models]
• 8. Applications of natural language processing and large ... [https://www.nature.com/articles/s41524-025-01554-0]
• 9. Named entity recognition in the perovskite field based on ... [https://www.sciencedirect.com/science/article/abs/pii/S0927025624002350]
• 10. Text Annotation: Techniques to Label Data for NLP Projects [https://labelyourdata.com/articles/data-annotation/text-annotation]
• 11. Exploring the Different Types of Text Annotation and Use ... [https://medium.com/@takouasaadani/exploring-the-different-types-of-text-annotation-and-use-cases-a75ed0a7580c]
• 12. A general-purpose material property data extraction ... [https://www.nature.com/articles/s41524-023-01003-w]
• 13. Application of machine reading comprehension techniques for ... [https://jcheminf.biomedcentral.com/articles/10.1186/s13321-024-00874-5]
• 14. Drug Design and Discovery: Principles and Applications [https://pmc.ncbi.nlm.nih.gov/articles/PMC6155886/]
• 15. A review of transformer models in drug discovery and beyond [https://www.sciencedirect.com/science/article/pii/S2095177924001783]
• 16. A general-purpose material property data extraction ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC10073792/]
• 17. Accurate, interpretable predictions of materials properties ... [https://www.sciencedirect.com/science/article/pii/S2666389923001587]
• 18. Foundation models for materials discovery – current state ... [https://www.nature.com/articles/s41524-025-01538-0]
• 19. Named Entity Recognition and Normalization Applied to Large ... [https://eta.lbl.gov/publications/named-entity-recognition-and]
• 20. Named entity recognition and relationship extraction for ... [https://www.sciencedirect.com/science/article/abs/pii/S0925231224019428]
• 21. Biomedical named entity recognition using deep neural ... [https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-019-3321-4]
• 22. A Review on Electronic Health Record Text-Mining for ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC10178605/]
• 23. Hybrid Approach to Identifying Druglikeness Leading ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC9695639/]
• 24. Models and Processes to Extract Drug-like Molecules From ... [https://www.frontiersin.org/journals/molecular-biosciences/articles/10.3389/fmolb.2021.636077/full]
• 25. Unveiling novel pharmacophores and hit compounds for ... [https://pubmed.ncbi.nlm.nih.gov/40639033/]
• 26. Development of optimized drug-like small molecule ... [https://www.nature.com/articles/s41467-022-29413-2]
• 27. Screening and identifying natural products with SARS-CoV ... [https://www.sciencedirect.com/science/article/pii/S2590053623001568]
• 28. Researchers identify 4 antiviral drug candidates for COVID-19 [https://cen.acs.org/pharmaceuticals/drug-discovery/Researchers-identify-4-antiviral-drug/103/web/2025/10]
• 29. Structure-based screening of natural product libraries in ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC8938336/]
• 30. A materials-science perspective on tackling COVID-19 [https://www.nature.com/articles/s41578-020-00247-y]
• 31. Named Entity Recognition: A Practical 2025 Guide [https://labelyourdata.com/articles/data-annotation/named-entity-recognition]
• 32. The Ultimate Guide to Named Entity Recognition (NER) [https://www.koncile.ai/en/ressources/named-entity-recognition-ner-guide]
• 33. Foundation Models, LLM Agents, Datasets, and Tools [https://arxiv.org/html/2506.20743v1]
• 34. Named Entity Recognition (NER) — The Concept, Types, and ... [https://weareshaip.medium.com/named-entity-recognition-ner-the-concept-types-and-applications-d5876a87ef6d]
• 35. Transformer (deep learning) [https://en.wikipedia.org/wiki/Transformer_(deep_learning)]
• 36. The Complete Guide to AI Architectures: From Neural ... [https://huggingface.co/blog/ProCreations/the-mega-article]
• 37. A survey on Named Entity Recognition — datasets, tools, ... [https://www.sciencedirect.com/science/article/pii/S2949719123000146]
• 38. Materials Expert-Artificial Intelligence for materials discovery [https://www.nature.com/articles/s43246-025-00928-7]
• 39. A diffusion enhanced CRF and BiLSTM framework for ... [https://www.nature.com/articles/s41598-025-04036-x]
• 40. Chemical named entity recognition in the texts of scientific ... [https://jcheminf.biomedcentral.com/articles/10.1186/s13321-022-00633-4]
• 41. A boundary enhanced multi-task neural attention approach ... [https://www.nature.com/articles/s41598-025-25317-5]
• 42. Quantifying the advantage of domain-specific pre-training ... [https://www.sciencedirect.com/science/article/pii/S2666389922000733]
• 43. Chinese Named Entity Recognition Based on Multi-Level ... [https://www.mdpi.com/2076-3417/14/19/9083]
• 44. Named Entity Recognition by Using XLNet-BiLSTM-CRF [https://www.semanticscholar.org/paper/Named-Entity-Recognition-by-Using-XLNet-BiLSTM-CRF-Yan-Jiang/55238da106f07f6b1d724d9e52f2923076b1cb41]
• 45. Development of a Hybrid CNN-BiLSTM Architecture to ... [https://dm.ageditor.ar/index.php/dm/article/view/726]
• 46. Resource-efficient instruction tuning of large language ... [https://www.sciencedirect.com/science/article/abs/pii/S153204642500125X]
• 47. Data extraction from polymer literature using large ... [https://www.nature.com/articles/s43246-024-00708-9]
• 48. Geo-MRC: Dynamic Boundary Inference in Machine ... [https://www.mdpi.com/2220-9964/14/11/431]
• 49. Named Entity Recognition via Machine Reading ... [https://arxiv.org/abs/2309.11027]
• 50. NER-MQMRC: Formulating named entity recognition as ... [https://www.amazon.science/publications/ner-mqmrc-formulating-named-entity-recognition-as-multi-question-machine-reading-comprehension]
• 51. An exploratory study on the potential of machine reading ... [https://www.sciencedirect.com/science/article/pii/S0346251X22001440#!]
• 52. Knowledge extraction for additive manufacturing process ... [https://www.sciencedirect.com/science/article/pii/S073658452400187X]
• 53. MatSciBERT: A materials domain language model for text ... [https://www.nature.com/articles/s41524-022-00784-w]
• 54. Improving classification of low-resource COVID-19 ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC8510872/]
• 55. Discovery of SARS-CoV-2 Antivirals through Large-scale ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC7603405/]
• 56. Dual-Site Inhibition of SARS-CoV‑2 RNA-Dependent RNA ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12519464/]
• 57. A small-molecule SARS-CoV-2 inhibitor targeting the ... [https://www.nature.com/articles/s41586-025-08651-6]
• 58. Ontology-conformal recognition of materials entities using ... [https://www.nature.com/articles/s41598-025-03619-y]
• 59. An Experimental Study on Data Augmentation Techniques ... [https://arxiv.org/html/2411.14551v1]
• 60. Quantifying the advantage of domain-specific pre-training ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC9024010/]
• 61. A survey of data augmentation in named entity recognition [https://www.sciencedirect.com/science/article/abs/pii/S0925231225015280]
• 62. Addressing data scarcity in deep learning: Leveraging real ... [https://www.sciencedirect.com/science/article/pii/S0264127525009566]
• 63. Addressing data scarcity in nanomaterial segmentation ... [https://www.nature.com/articles/s41524-025-01702-6]
• 64. MatWheel: Addressing Data Scarcity in Materials Science ... [https://arxiv.org/abs/2504.09152]
• 65. Training-Free Active Learning Framework in Materials ... [https://arxiv.org/html/2511.19730v1]
• 66. Building workflows for an interactive human-in-the-loop ... [https://pubs.rsc.org/en/content/articlehtml/2025/dd/d5dd00033e]
• 67. and transfer Active with partially Bayesian neural networks... learning [https://pubs.rsc.org/en/content/articlehtml/2025/dd/d5dd00027k]
• 68. An active learning-enabled annotation system for clinical ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC5506567/]
• 69. Human-in-the-loop active learning for goal-oriented molecule ... [https://jcheminf.biomedcentral.com/articles/10.1186/s13321-024-00924-y]
• 70. Supervised Fine-Tuning or In-Context Learning ... [https://arxiv.org/html/2510.22285v1]
• 71. A FIT4NER Generic Approach for Framework-Independent ... [https://www.mdpi.com/2078-2489/16/7/554]
• 72. MatKG: An autonomously generated knowledge graph in ... [https://www.nature.com/articles/s41597-024-03039-z]
• 73. Mining experimental data from Materials Science literature ... [https://arxiv.org/html/2401.11052v1]
• 74. Realistic material property prediction using domain ... [https://www.sciencedirect.com/org/science/article/pii/S2635098X2400010X]
• 75. Structure Agnostic Multimodal Learning for Materials Science [https://arxiv.org/html/2507.01054v1]
• 76. MatSciBERT: A materials domain language model for text ... mining [https://www.nature.com/articles/s41524-022-00784-w?error=cookies_not_supported&code=001cfb13-debc-4eac-aa32-406a98106793]
• 77. The Impact of Domain-Specific Pre-Training on Named ... [https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3950755]
• 78. Advancing Language Models in Materials - Simple Science Science [https://scisimple.com/en/articles/2025-11-08-advancing-language-models-in-materials-science--a3d6xle]
• 79. A general-purpose material property data extraction pipeline from large... [https://www.nature.com/articles/s41524-023-01003-w?error=cookies_not_supported&code=0a9504d9-8e24-4111-a123-50543863a498]
• 80. Quantifying the advantage of domain-specific pre-training ... [https://pubmed.ncbi.nlm.nih.gov/35465225/]
• 81. m3rg-iitd/matscibert · Hugging Face [https://huggingface.co/m3rg-iitd/matscibert]
• 82. MaterioMiner — An ontology-based text dataset for extraction... mining [https://arxiv.org/html/2408.04661v1]
• 83. From text to insight: large language models for chemical data ... [https://pubs.rsc.org/en/content/articlehtml/2025/cs/d4cs00913d]
• 84. La-LoRA: Parameter-Efficient Fine-Tuning with Layer-wise ... [https://www.sciencedirect.com/science/article/abs/pii/S089360802500975X]
• 85. Parameter-efficient fine-tuning in large language models [https://link.springer.com/article/10.1007/s10462-025-11236-4]
• 86. Low-Rank Adaptation of Neural Fields [https://arxiv.org/html/2504.15933v2]
• 87. NoRA: Nested Low-Rank Adaptation for Efficient Fine- ... [https://openreview.net/forum?id=6nZwOYDcQx]
• 88. The Fine-Tuning Landscape in 2025: A Comprehensive ... [https://medium.com/@pradeepdas/the-fine-tuning-landscape-in-2025-a-comprehensive-analysis-d650d24bed97]
• 89. LLM's Weakness in NER Doesn't Stop It from Enhancing a ... [https://aclanthology.org/2025.alp-1.21/]
• 90. F1 Score for NER: A Metric To Evaluate Precision And Recall [https://thatware.co/f1-score-for-ner/]
• 91. MatSKRAFT: A framework for large-scale materials ... [https://arxiv.org/html/2509.10448v1]
• 92. Named entity recognition for construction documents ... [https://www.sciencedirect.com/science/article/abs/pii/S0926580525001918]
• 93. Accelerating materials language processing with large language models | Communications Materials [https://www.nature.com/articles/s43246-024-00449-9]
• 94. Recent advances and applications of deep learning methods in materials science | npj Computational Materials [https://www.nature.com/articles/s41524-022-00734-6]
• 95. Materials science in the era of large language models: a perspective - Digital Discovery (RSC Publishing) DOI:10.1039/D4DD00074A [https://pubs.rsc.org/en/content/articlehtml/2024/dd/d4dd00074a]
• 96. MatTools: Benchmarking Large Language Models for ... [https://arxiv.org/html/2505.10852v1]
• 97. Top LLMs in 2025: Performance, Features & Use Cases [https://kanerika.com/blogs/comparison-of-llms/]
• 98. The Best LLMs in 2025: Top 5 Models Compared for Real- ... [https://www.knacklabs.ai/blogs/the-best-llms-in-2025-top-5-models-compared-for-real-world-use]
• 99. Supervised Fine-Tuning or In-Context Learning ... [https://arxiv.org/abs/2510.22285]
• 100. Is fine-tuning LLMs still worth it in 2025? [https://www.kadoa.com/blog/is-fine-tuning-still-worth-it]
• 101. Large-scale application of named entity recognition to ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC9931203/]
• 102. Benchmarking large language models for biomedical ... [https://www.nature.com/articles/s41467-025-56989-2]
• 103. A survey on retrieval-augmentation generation (RAG) ... [https://link.springer.com/article/10.1007/s00521-025-11666-9]
• 104. Agentic AI for Scientific Discovery [https://iclragenticai.github.io/]
• 105. A Novel Context-Aware Retrieval Framework for ... [https://www.sciencedirect.com/science/article/abs/pii/S1566253525009649]
• 106. Application of machine reading comprehension techniques ... [https://pubmed.ncbi.nlm.nih.gov/38956728/]
• 107. Models and Processes to Extract Drug-like Molecules From ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC8435623/]
• 108. Retrieval augmented NER for knowledge intensive named ... [https://www.amazon.science/publications/ra-ner-retrieval-augmented-ner-for-knowledge-intensive-named-entity-recognition]
• 109. MSQA: Benchmarking LLMs on Graduate-Level Materials Science Reasoning and Knowledge [https://arxiv.org/html/2505.23982]
• 110. SciNER: Extracting Named Entities from Scientific Literature [https://pmc.ncbi.nlm.nih.gov/articles/PMC7302801/]
• 111. Statistical dataset evaluation: A case study on named entity ... [https://www.cambridge.org/core/journals/natural-language-processing/article/statistical-dataset-evaluation-a-case-study-on-named-entity-recognition/BF7FA3EF95004830F233CF5D743D98B2]
• 112. Frontiers | Digitized material design and performance prediction driven by high-throughput computing [https://www.frontiersin.org/journals/materials/articles/10.3389/fmats.2025.1599439/full]