IT博客汇
  • 首页
  • 精华
  • 技术
  • 设计
  • 资讯
  • 扯淡
  • 权利声明
  • 登录 注册

    ubuntu14.04安装Scrapy框架

    admin发表于 2016-04-02 13:20:52
    love 0

    ubuntu14.04安装Scrapy框架,首先,安装scrapy 需要以下程序或者库:

    • python 2.7
    • lxml
    • OpenSSL
    • pip or easy_install

    一、pip是作为python包索引器easy_install的替代品,用于安装和管理Python包。

    sudo apt-get install python-pip

    二、我们必须要用下面的命令安装python开发库。如果包没有安装那么就会在安装scrapy框架的时候报关于python.h头文件的错误。

    sudo apt-get install python-dev

    三、验证是否安装Python 2.7 和 lxml 和 OpenSSL 和 pip

    1、验证Python 2.7 : python -V

    2、验证lxml和OpenSSL:

    root@hoster:~# python
    Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
    [GCC 4.8.2] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import lxml
    >>> import OpenSSL

    3、验证pip:

    root@hoster:~# pip
    
    Usage:   
      pip <command> [options]
    
    Commands:
    ......

    注:如何安装lxml?:

    1、sudo apt-get update

    2、sudo apt-get install libxml2-dev libxslt-dev python-dev

    3、sudo apt-get build-dep python-lxml(依赖包)

    4、pip install lxml

    四、scrapy框架既可从deb包安装也可以从源码安装。我们用pip(Python 包管理器)安装了deb包了。

    sudo pip install scrapy

    五、验证是否安装scrapy成功:

    root@hoster:~/testing# scrapy
    Scrapy 1.0.5 - project: testing
    
    Usage:
      scrapy <command> [options] [args]
    
    Available commands:
      bench         Run quick benchmark test
      check         Check spider contracts
      commands      
      crawl         Run a spider
      edit          Edit spider
      fetch         Fetch a URL using the Scrapy downloader
      genspider     Generate new spider using pre-defined templates
      list          List available spiders
      parse         Parse URL (using its spider) and print the results
      runspider     Run a self-contained spider (without creating a project)
      settings      Get settings values
      shell         Interactive scraping console
      startproject  Create new project
      version       Print Scrapy version
      view          Open URL in browser, as seen by Scrapy
    
    Use "scrapy <command> -h" to see more info about a command
    

    Copyright:www.cplusplus.me Share、Open- C/C++程序员之家



沪ICP备19023445号-2号
友情链接