08 2019 档案
摘要:COOKIES_ENABLED 默认: True 是否启用cookiesmiddleware。如果关闭,cookies将不会发送给web server。 COOKIES_DEBUG 默认: False 如果启用,Scrapy将记录所有在request(cookie 请求头)发送的cookies及re
阅读全文
摘要:C:\Windows\System32\drivers\etc 修改host
阅读全文
摘要:# -*- coding: utf-8 -*-import sysreload(sys)sys.setdefaultencoding('utf-8')
阅读全文
摘要:sudo apt-get update sudo apt-get -y install apt-transport-https ca-certificates curl software-properties-common # step 2: 安装GPG证书 curl -fsSL http://mi
阅读全文
摘要:import execjs # eval 和 complie 是要构建一个JS的环境 e = execjs.eval('a = new Array(1,2,3)') # 可以直接执行JS代码 print(e) x = execjs.compile(''' function add(x,y){ ret
阅读全文
摘要:from scrapy.crawler import CrawlerProcess from scrapy.utils.project import get_project_settings # 注意与scrapy.cfg在同一级目录 if __name__ == '__main__': process = CrawlerProcess(get_project_settings()) ...
阅读全文
摘要:import json import pymongo if __name__ == '__main__': client = pymongo.MongoClient(host="127.0.0.1", port=27017) # 第一种 # db = client['toy'] # # collection = db["stu"] # 第二...
阅读全文
摘要:#基础数据添加db.stu.drop() db.stu.insert({name:'郭靖',hometown:'蒙古',age:20,gender:true}) db.stu.insert({name:'黄蓉',hometown:'桃花岛',age:18,gender:false}) db.stu.insert({name:'华筝',hometown:'蒙古',age:18,gende...
阅读全文
摘要:import js2py # 实例化一个执行js的环境对象 context_js_obj = js2py.EvalJs() js_str = """ function A(a,b){ return a+b } """ # 传递js_str,执行js context_js_obj.execute(js_str) result = context_js_obj.A...
阅读全文
摘要:import threading import time from queue import Queue from multiprocessing.dummy import Pool import requests from lxml import etree class QiuBaiSpider(object): # 1.爬取的的网站,和请求头 def __init__(s...
阅读全文