Design and Realization of Distributed Web Crawler

JIN Yue-fu; FAN Jian-ying; FENG Yang

Chinese

您当前的位置：

首页 >

文章列表页 >

Design and Realization of Distributed Web Crawler

更新时间：2026-07-10

- Design and Realization of Distributed Web Crawler
- Vol. 15, Issue 1, Pages: 116-119(2010)
- 作者机构：
  
  哈尔滨理工大学测控技术与通信工程学院
- 作者简介：
- 基金信息：
- DOI：
  CLC： TP274.2
- Published：2010
- 稿件说明：
移动端阅览
JIN Yue-fu, FAN Jian-ying, FENG Yang. Design and Realization of Distributed Web Crawler[J]. 2010, 15(1): 116-119.
DOI：

JIN Yue-fu, FAN Jian-ying, FENG Yang. Design and Realization of Distributed Web Crawler[J]. 2010, 15(1): 116-119. DOI：

摘要

Web信息的急剧增长

给信息的有效使用带来了巨大挑战.作为搜索引擎的重要组成部分

Web信息采集扮演着越来越重要的角色.文中从并行性、负载均衡、体系结构以及可扩展性这4个方面详细讨论了分布式Web信息采集系统的设计与实现.经验证

该分布式Web信息采集系统能够实现负载均衡

并具有良好的可扩展性.

Abstract

Great enhancement on the web information brings an enormous challenge to human on how to use it effectively. As a key part of search engine

web crawler plays a more and more important role. This paper introduces the design and realization of a distributed web crawler

covers the following four aspects: parallel

load balancing

architecture and expansibility. Practice shows that the distributed web crawler can achieve load balancing and have a good expansibility.

关键词

Keywords

references

Views

2395

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

(θ,φ) - derivations and Jordan(θ,φ) - derivations of Hom-Lie Supertriple Systems

Online Courseresource Recommendation MethodFusing Multi-modal Features

Collaborative Navigation and Observability Analysis of Cluster UAVs Based on Magnetometer Fusion

Fault Diagnosis Method of Miniature Circuit Breaker Based on RTH-BP Neural Network

Hölder Continuity of Solutions for a Class of Elliptic Equations

Related Author

No data

Related Institution

No data

AI问答

Postal code：100079
Tel：（010）53879206 Email：tmw@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09082226号-64 京公网安备11010602201714号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰