An Improved T-Spider Distributed Crawler

JIN Fan; GU Jin-guang

Chinese

您当前的位置：

首页 >

文章列表页 >

An Improved T-Spider Distributed Crawler

更新时间：2026-09-04

- An Improved T-Spider Distributed Crawler
- Vol. 28, Issue 8, Pages: 102-104(2011)
- 作者机构：
  
  1. 武汉科技大学机构计算机科学与技术学院,湖北,武汉,430065
  2. 东南大学计算机科学与工程学院,江苏,南京,210096
- 作者简介：
- 基金信息：
- DOI：
  CLC：
- Published：2011
- 稿件说明：
移动端阅览
金凡, 顾进广. 一种改进的T-Spider分布式爬虫[J]. 微电子学与计算机, 2011, 28(8): 102-104.

JIN Fan, GU Jin-guang. An Improved T-Spider Distributed Crawler[J]. 2011, 28(8): 102-104.
金凡, 顾进广. 一种改进的T-Spider分布式爬虫[J]. 微电子学与计算机, 2011, 28(8): 102-104. DOI：

JIN Fan, GU Jin-guang. An Improved T-Spider Distributed Crawler[J]. 2011, 28(8): 102-104. DOI：

摘要

为了提高互联网网页的抓取速度

提出了一个改进的T-Spider分布式爬虫模型.该爬虫在解析URL阶段将页面进行切割以并行解析

在页面调度阶段使用改进的链接优先权计算方法

提高爬虫的抓取速度和稳定性.通过实验结果分析

验证了该方法的有效性.

Abstract

To increase the speed of the crawler

this paper proposes a model that is based on the T-Spider.During the time of extracting links from the page content

the crawler takes use of the page cutting algorithm

and then uses a new algorithm of link priority computing to enhance the stability and increase the speed of the crawler.The experiment shows that it is availability.

关键词

Keywords

references

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Simulation on the DC characteristics of double heterojunction p-GaN HEMT with AlN/GaN hybrid buffer layer

Aerial vehicle detection algorithm based on multi-scale fusion and small-object enhancement

Design of ultra-low power real-time clock circuits

Implementation and verification of a dual-chain power gating strategy for low power design

A feature-oriented context understanding and slice analysis network for tuberculosis detection

Related Author

No data

Related Institution

No data

AI问答

Postal code：100079
Tel：（010）53879206 Email：tmw@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09082226号-64 京公网安备11010602201714号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰