1. 天津大学 电气自动化与信息工程学院,天津,300072
2. 国网天津市电力公司,天津,300010
3. 天津市能源大数据仿真企业重点实验室,天津,300010
金志刚(1972—),男,教授,博士生导师
纸质出版:2023
移动端阅览
金志刚, 何晓勇, 岳顺民, 等. 融合知识图谱的医疗领域命名实体识别[J]. 哈尔滨工业大学学报, 2023,55(5):50-58.
JIN Zhigang, HE Xiaoyong, YUE Shunmin, et al. Named entity recognition in medical domain combined with knowledge graph[J]. 2023, 55(5): 50-58.
金志刚, 何晓勇, 岳顺民, 等. 融合知识图谱的医疗领域命名实体识别[J]. 哈尔滨工业大学学报, 2023,55(5):50-58. DOI: 10.11918/202201126.
JIN Zhigang, HE Xiaoyong, YUE Shunmin, et al. Named entity recognition in medical domain combined with knowledge graph[J]. 2023, 55(5): 50-58. DOI: 10.11918/202201126.
为了改善通用预训练模型不适应医疗领域的命名实体识别任务这一不足
提出了一种融合医疗领域知识图谱的神经网络架构
该架构利用弹性位置和遮盖矩阵使预训练模型计算自注意力时避免语义混淆和语义干扰
在微调时使用多任务学习的思想
利用回忆学习的优化算法使预训练模型均衡通用语义表达和目标任务的学习
最终得到更为高效的向量表示并进行标签预测。实验结果表明:本文提出的命名实体识别架构在医疗领域上取得了优于主流预训练模型的效果
在通用领域也有较为良好的效果。该架构避免了重新训练针对某个领域的预训练模型和引入额外的编码结构从而精简了计算代价和模型大小。此外
通过消融实验对比
医疗领域对于知识图谱的依赖程度较通用领域依赖程度更大
这说明在医疗领域中融合知识图谱方法的有效性。通过参数分析
证明本文使用回忆学习的优化算法可以有效控制模型参数的更新
使模型可以保留更多的通用语义信息并得到更符合语义的向量表达。本文也通过实验分析说明了所提方法在实体数量少的种类上具有更优的表现。
In view of the problem that general pre-trained models are not suitable for named entity recognition tasks in the medical domain
a neural network architecture that integrates knowledge graph in the medical domain was proposed. The elastic position and masking matrix were used to avoid semantic confusion and semantic interference in self-attention calculation of pre-trained model. The idea of multi-task learning in fine-tuning was adopted
and the optimization algorithm of recall learning was employed for pre-trained model to balance between general semantic expression and learning of the target task. Finally
a more efficient vector representation was obtained and label prediction was conducted. Experimental results showed that the proposed architecture achieved better results than the mainstream pre-trained models in the medical domain
and had relatively good results in the general domain. The architecture avoided retraining pre-trained models in particular domain and additional coding structures
which greatly reduced computational cost and model size. In addition
according to the ablation experiments
the medical domain was more dependent on the knowledge graph than the general domain
indicating the effectiveness of integrating the knowledge graph method in the medical domain. Parameter analysis proved that the optimization algorithm which used recall learning could effectively control the update of model parameters
so that the model retained more general semantic information and obtained more semantic vector representation. Besides
the experimental analysis showed that the proposed method had better performance in the category with a small number of entities.
0
浏览量
15
下载量
0
CSCD
关联资源
相关文章
相关作者
相关机构
京公网安备11010602201714号