tornado 异步检测网址目标是否存在

通过 tornado 异步检测目标网址文件是否存在。

tornado 异步检测网址目标是否存在

代码实现

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
#!/usr/bin/env python
# -*- coding: utf-8 -*-

import logging

import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web


from tornado import gen
from tornado.httpclient import AsyncHTTPClient, HTTPRequest


from tornado.options import define, options

define("port", default=10076, help="run on the given port", type=int)


class MainHandler(tornado.web.RequestHandler):
    @gen.coroutine
    def get(self):
        _url = 'http://www.test.com/logo.jpg'

        http_client = AsyncHTTPClient()
        req = HTTPRequest(url=_url, method='HEAD')
        http_client.fetch(req, callback=(yield gen.Callback("check_url_by_head")))
        _response = yield gen.Wait("check_url_by_head")
        logging.error(_response)
        if _response.code == 200:
            print 'file exist'
        else:
            print _response.code


def main():
    tornado.options.parse_command_line()
    application = tornado.web.Application([
        (r"/", MainHandler),
    ])
    http_server = tornado.httpserver.HTTPServer(application)
    http_server.listen(options.port)
    tornado.ioloop.IOLoop.instance().start()


if __name__ == "__main__":
    main()

注意

需要目标文件服务器支持 HEAD 请求。

本文网址: https://pylist.com/topic/107.html 转摘请注明来源

Suggested Topics

向各搜索引擎主动提交网址的经验

当网站添加了新内容后,都想第一时间让搜索引擎知道并且来抓取,省得“原创”都给了采集站。当采取主动提交时,Google 响应很快,60秒左右收录!百度响应也不赖。...

Leave a Comment