python是一个好东西，爬虫是真的很强
这个爬虫代码爬取的对象是360图片 https://image.so.com
需要的配件
Python requests threading
代码-注意在目录下创建一个img文件夹
import requests
import threading
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36'
}
def get_response(html_url):
response = requests.get(url=html_url, headers=headers)
return response
def save(img_url, title):
path = 'img\' + title + '.jpg'
img_content = get_response(img_url).content
with open(path, mode='wb') as f:
f.write(img_content)
print('正在保存：', title)
def main(url):
html_data = get_response(url).json()
lis = html_data['list']
for li in lis:
img_url = li['qhimg_downurl']
title = li['title']
save(img_url, title)
if __name__ == '__main__':
for page in range(0, 301, 30):
url = 'https://image.so.com/zjl?ch=beauty&sn={}&listtype=new&temp=1'.format(page)
main_thread = threading.Thread(target=main, args=(url,))
main_thread.start()
直接下载
文章来源于互联网:python爬虫爬取360图片代码多线程1分钟几百+图片到手
