Evaluatology-based AI-Ready materials dataset quality evaluation framework

ZUO Wei; YANG Zhengwei; SHI Siqi; LIU Yue

doi:10.11959/j.issn.2096-0271.BDR26219

Chinese

您当前的位置：

首页 >

文章列表页 >

Evaluatology-based AI-Ready materials dataset quality evaluation framework

更新时间：2026-08-24

- Evaluatology-based AI-Ready materials dataset quality evaluation framework
- Big Data Research (2026)
- 作者机构：
  
  1.上海大学计算机工程与科学学院&核电关键材料全国重点实验室，上海200444
  2.上海大学核电关键材料全国重点实验室&材料科学与工程学院，上海200444
  3.上海大学材料基因组工程研究院，上海 200444
- 作者简介：
- 基金信息：
  
  The National Natural Science Foundation of China(92270124;52073169;61936001)
- DOI：10.11959/j.issn.2096-0271.BDR26219
  CLC： TB3;TP181;
- Received：31 May 2026，
  
  Revised：2026-07-15，
  
  Accepted：24 August 2026，
- 稿件说明：
移动端阅览
ZUO Wei, YANG Zhengwei, SHI Siqi, et al. Evaluatology-based AI-Ready materials dataset quality evaluation framework[J/OL]. Big Data Research, 2026.
DOI：

ZUO Wei, YANG Zhengwei, SHI Siqi, et al. Evaluatology-based AI-Ready materials dataset quality evaluation framework[J/OL]. Big Data Research, 2026. DOI： 10.11959/j.issn.2096-0271.BDR26219.

摘要

数据质量的科学评价是数据有效治理的核心前提，直接关系到高质量AI-Ready数据集在下游任务建模的效果。当前研究聚焦于单一数据模态或特定质量维度，导致跨模态、跨维度的评价结果缺乏可比性，难以形成系统的数据质量画像。因此，提出基于评价学的AI-Ready多模态材料数据集质量评价框架，明确评价对象、评价条件与评价方法以构建统一评价体系，并籍此通过“继承‑发展”六大规则与五步构建流程，实现从统一框架向分模态评价体系的系统性延伸。进一步，对AI-Ready材料时序数据集完整性进行案例实验，验证了基于该评价框架设计评价体系的有效性，为构建高质量AI-Ready材料数据集、开发高可靠数据驱动模型提供坚实的理论支撑与科学依据。

Abstract

Scientific evaluation of data quality is a core prerequisite for effective data governance

as it directly affects the utility of high-quality AI-Ready dataset for downstream tasks. However

current research mainly focuses on specific modalities or dimensions

leading to incomparable evaluation results across different modalities and dimensions

and making it difficult to establish a systematic data quality profile. To address this issue

this work proposes an Evaluatology-based multimodal data quality evaluation framework for AI-Ready materials dataset

by defining the evaluation subject

evaluation conditions

and evaluation method to construct a unified evaluation system. Then

the six rules and five-step construction process of "Inheritance–Development" are employed to extend the unified framework into modality-specific evaluation systems. Furthermore

a case study on the completeness of AI-Ready materials time-series data was conducted

validating the effectiveness of the evaluation system designed based on the proposed framework. This work provides a theoretical and scientific foundation for constructing high-quality AI-ready dataset and developing highly reliable data-driven models.

关键词

Keywords

references

Polat C , Kurban H , Serpedin E , et al . Beyond atomic geometry representations in materials science: a human-in-the-loop multimodal framework [C ] // Proceedings of the 13th International Conference on Learning Representations (ICLR) . Singapore , 2025 .

Dreger M , Malek K , Eikerling M . Large language models for knowledge graph extraction from tables in materials science [J ] . Digital Discovery , 2025 , 4 ( 5 ): 1221 - 1231 .

Shahzad K , Mardare A I , Hassel A W . Accelerating materials discovery: combinatorial synthesis, high-throughput characterization, and computational advances [J ] . Science and Technology of Advanced Materials: Methods , 2024 , 4 ( 1 ): 2314561 .

Liu Y , Yang Z W , Zou X X , et al . A general framework to govern machine learning oriented materials data quality [J ] . Materials Science and Engineering: R: Reports , 2025 , 162 : 100821 .

Liu Y , Yang Z W , Zou X X , et al . Data quantity governance for machine learning in materials science [J ] . National Science Review , 2023 , 10 ( 7 ): nwad125 .

Liu Y , Ma S C , Yang Z W , et al . Domain knowledge-assisted materials data anomaly detection towards constructing high-performance machine learning models [J ] . Journal of Materiomics , 2025 , 11 ( 6 ): 101066 .

Liu Y , Zou X X , Ma S C , et al . Feature selection method reducing correlations among features by embedding domain knowledge [J ] . Acta Materialia , 2022 , 238 : 118195 .

Liu , Y , Wang S Y , Yang Z W , et al . Auto-MatRegressor: liberating machine learning alchemists [J ] . Science bulletin 2023

Liu Y , Liu D H , Ge X Y , et al . A high-quality dataset construction method for text mining in materials science [J ] . Acta Physica Sinica , 2023 , 72 ( 7 ): 070701 .

Liu Y , Liu , Zuo W , et al . A closed-loop framework for material text annotation correction and validation based on large language models [C ] // Proceedings of the International Joint Conference on Neural Networks (IJCNN) . 2026 .

Liu Y , Yu Z Y , Liu Z T , et al . Improving disentanglement in variational auto-encoders via feature imbalance informed dimension weighting [J ] . Knowledge-Based Systems , 2024 , 263 : 110261 .

Yu Z Y , Yang Z W , Zhang W , et al . Focus on latent: toward a meaningful and interpretable latent representation for diffusion models [J ] . Information Processing & Management , 2026 , 63 ( 7 ), 104854 .

Zhan J F , Wang L , Gao W L , et al . Evaluatology: The science and engineering of evaluation [J ] . BenchCouncil Transactions on Benchmarks, Standards and Evaluations , 2024 , 4 ( 1 ): 100162 .

Bernardo M C , João V S , Marcelo C S , et al . Optimizing deep neural networks for nuclear power plant temperature estimation: A study on feature importance and outlier detection [J ] . Progress in Nuclear Energy , 2025 , 191 : 106039 .

Li K M , Persaud D , C houdhary K , et al . Exploiting redundancy in large materials datasets for efficient machine learning with less data [J ] . Nature Communications , 2023 , 14 ( 1 ): 7845 .

Xu P , Ji X , Li M , et al . Small data machine learning in materials science [J ] . npj Computational Materials , 2023 , 9 ( 1 ): 85 .

Khan A , et al . Leveraging generative adversarial networks to create realistic scanning transmission electron microscopy images [J ] . npj Computational Materials , 2023 , 9 ( 1 ): 85 .

Liu N , Jafarzadeh S , Lattimer B Y , et al . Harnessing large language models for data-scarce learning of polymer properties [J ] . Nature Computational Science , 2025 , 5 : 245 - 254 .

Wang R Y , Strong D M . Beyond accuracy: what data quality means to data consumers [J ] . Journal of Management Information Systems , 1996 , 12 ( 4 ): 5 - 33 .

L uo C X , X iong H X , Ye Y Z , et al . TDQE: a quality evaluation method for text data in deep learning [J ] . Big Data Research , 2025 , 11 ( 6 ): 95 - 107 .

崔丁洁 . 面向语义的文本质量评价研究 [D ] . 哈尔滨 : 哈尔滨工业大学 , 2022 .

Cui D J . Research on Semantic-Oriented Text Quality Evaluation [D ] . Harbin : Harbin Institute of Technology , 2022 .

谢沛文 . 面向知识图谱的质量评估技术研究 [D ] . 南京 : 东南大学 , 2022 .

Xie P W . Research on Quality Assessment Technology for Knowledge Graphs [D ] . Nanjing : Southeast University , 2022 .

曹玉东 , 刘海燕 , 贾旭 , 等 . 基于深度学习的图像质量评价方法综述 [J ] . 计算机工程与应用 , 2021 , 57 ( 23 ): 27 - 36 .

CAO Y D , LIU H Y , JIA X , et al . Review of Image Quality Assessment Methods Based on Deep Learning [J ] . Computer Engineering and Applications , 2021 , 57 ( 23 ): 27 - 36 .

宋洪涛 , 于江生 , 韩启龙 . 工业多元时序数据质量评估方法 [J ] . 计算机应用 , 2024 , 44 ( 6 ): 1743 - 1750 .

Song H T , Yu J S , Han Q L . Quality Assessment Method for Industrial Multivariate Time Series Data [J ] . Journal of Computer Applications , 2024 , 44 ( 6 ): 1743 - 1750 .

刘小戈 . 多维传感器时序数据的质量评估及分类研究 [D ] . 哈尔滨 : 哈尔滨工程大学 , 2023 .

Liu X G . Research on Quality Assessment and Classification of Multidimensional Sensor Time Series Data [D ] . Harbin : Harbin Engineering University , 2023 .

尤祺 , 袁堂晓 , 汪惠芬 . 基于神经网络的工业时序数据质量管理方法 [J ] . 机械制造与自动化 , 2022 , 51 ( 3 ): 96 - 99, 112 .

You Q , Yuan T X , Wang H F . Industrial Time Series Data Quality Management Method Based on Neural Network [J ] . Machine Building & Automation , 2022 , 51 ( 3 ): 96 - 99, 112 .

Alampara N , Schilling-Wilhelmi M , Jablonka K . Lessons from the trenches on evaluating machine-learning systems in materials science [J ] . Computational Materials Science , 2025 , 248 : 113452 .

忤丹丹 , 梅健 , 李明 , 等 . 工程建设环境时序数据质量评价模型研究 [J ] . 环境科学与管理 , 2024 , 49 ( 12 ): 168 - 173 .

Wu D D , Mei J , Li M , et al . Research on quality evaluation model of time-series data for engineering construction environment [J ] . Environmental Science and Management , 2024 , 49 ( 12 ): 168 - 173 .

刘晋瑞 , 刘雪鹏 , 刘速 , 等 . 面向炼化生产运行的高质量多源时序数据集构建方法研究 [J ] . 化工学报 , 2026 , 1 - 14 .

Liu J R , Liu X P , Liu S , et al . Research on construction method of high-quality multi-source time-series dataset for refining and chemical production operation [J ] . CIESC Journal , 2026 , 1 - 14 .

Zhang Y , Tang Q , Zhang Y , et al . Identifying degradation patterns of lithium ion batteries from impedance spectroscopy using machine learning [J ] . Nature Communications , 2020 , 11 ( 1 ): 1706 .

Battery Archive . Battery Archive [EB/OL ] . https://www.batteryarchive.org/ https://www.batteryarchive.org/ , 2021

NASA Ames Research Center . NASA Prognostics Center of Excellence Data Set Repository [EB/OL ] . https://ti.arc.nasa.gov/tech/dash/groups/pcoe/prognostic-data-repository/ https://ti.arc.nasa.gov/tech/dash/groups/pcoe/prognostic-data-repository/ , 2007

Deng Z , Xu L , Liu H , et al . Prognostics of battery capacity based on charging data and data-driven methods for on-road vehicles [J ] . Applied Energy , 2023 , 339 : 120954 .

Zhu J , Wang Y , Huang Y , et al . Data-driven capacity estimation of commercial lithium-ion batteries from voltage relaxation [J ] . Nature Communications , 2022 , 13 ( 1 ): 2261 .

Center for Advanced Life Cycle Engineering (CALCE) . CALCE Battery Research Group [EB/OL ] . https://web.calce.umd.edu/batteries/ https://web.calce.umd.edu/batteries/ , 2011

Lu J , Xiong R , Tian J , et al . Battery Degradation Datasets (Two Types of Lithium-ion Batteries) [EB/OL ] . DataMendeley, V1, 2023 . doi: 10.17632/v8k6bsr6tf.1 http://dx.doi.org/10.17632/v8k6bsr6tf.1 .

Birkl C , Howey D . Oxford Battery Degradation Dataset 1 [EB/OL ] . University of Oxford , 2017 . doi: 10.5287/ora-pb591qk7p http://dx.doi.org/10.5287/ora-pb591qk7p .

Ma G , Xu S , Jiang B , et al . Real-time personalized health status prediction of lithium-ion batteries using deep transfer learning [J ] . Energy & Environmental Science , 2022 , 15 ( 10 ): 4083 - 4094 .

Severson K A , Attia P M , Jin N , et al . Data-driven prediction of battery cycle life before capacity degradation [J ] . Nature Energy , 2019 , 4 ( 5 ): 383 - 391 .

Xu-Ancha . Datasets [EB/OL ] . https://www.xu-ancha.com/datasets https://www.xu-ancha.com/datasets .

Materials Cloud Archive . Materials Cloud Archive: Record ed 9q1 - h0 m 33 [EB/OL ] . https://archive.materialscloud.org/records/ed9q1-h0m33 https://archive.materialscloud.org/records/ed9q1-h0m33 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Research on the construction path of government trusted data space

Time series data multi-scale missing value imputation

Research on speaker anonymization method via latent space splicing and contrastive learning

Research on legal risks and regulatory paths of enterprise data crawling — from the perspective of balancing rights and interests

A cross-data-space search system for scenario-based data usage

Related Author

No data

Related Institution

No data

AI问答

Postal code：100079
Tel：（010）53879206 Email：tmw@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09082226号-64 京公网安备11010602201714号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰