前言
蔬菜是指可以做菜、烹饪成为食品的一类植物或菌类,蔬菜是人们日常饮食中必不可少的食物之一。
近期,蔬菜价格有所上涨,引起广泛关注。“双节”将至,物价走势如何?
9月16日,国家发改委召开9月份例行新闻发布会。针对蔬菜价格上涨相关情况,国家发改委新闻发言人孟玮表示,蔬菜的生长周期比较短,后期随着极端天气减少,秋季蔬菜陆续上市,市场供应有望在较短时间内恢复,鲜菜价格将随之回落。
项目目标
爬取食品商务网蔬菜价格数据
受害网址
https://price.21food.cn/
www#gaodaima.com来源gaodaimacom搞#^代%!码&网搞代码
爬虫代码
导入工具
<span style="color: #0000ff"><a href="https://www.gaodaima.com/tag/import" title="查看更多关于import的文章" target="_blank">import</a></span><span style="color: #000000"> requests </span><span style="color: #0000ff">import</span><span style="color: #000000"> parsel </span><span style="color: #0000ff">import</span><span style="color: #000000"> csv </span><span style="color: #0000ff">import</span> time
解析网站,爬取数据
<span style="color: #0000ff">for</span> page <span style="color: #0000ff">in</span> range(1, 19<span style="color: #000000">): time.sleep(</span>1<span style="color: #000000">) url </span>= <span style="color: #800000">"</span><span style="color: #800000">https://price.21food.cn/guoshu-p{}.html</span><span style="color: #800000">"</span><span style="color: #000000">.format(page) headers </span>=<span style="color: #000000"> { </span><span style="color: #800000">"</span><span style="color: #800000">user-agent</span><span style="color: #800000">"</span>: <span style="color: #800000">"</span><span style="color: #800000">Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36</span><span style="color: #800000">"</span><span style="color: #000000"> } response </span>= requests.get(url=url, headers=<span style="color: #000000">headers) selector </span>=<span style="color: #000000"> parsel.Selector(response.text) lis </span>= selector.css(<span style="color: #800000">"</span><span style="color: #800000">.gs_top_t2_left div:nth-child(1) .sjs_top_cent_erv ul li</span><span style="color: #800000">"</span><span style="color: #000000">) dit </span>=<span style="color: #000000"> {} </span><span style="color: #0000ff">for</span> li <span style="color: #0000ff">in</span><span style="color: #000000"> lis: name </span>= li.css(<span style="color: #800000">"</span><span style="color: #800000">a::text</span><span style="color: #800000">"</span><span style="color: #000000">).get() dit[</span><span style="color: #800000">"</span><span style="color: #800000">品种</span><span style="color: #800000">"</span>] =<span style="color: #000000"> name price </span>= li.css(<span style="color: #800000">"</span><span style="color: #800000">td span::text</span><span style="color: #800000">"</span><span style="color: #000000">).get() dit[</span><span style="color: #800000">"</span><span style="color: #800000">平均价格</span><span style="color: #800000">"</span>] =<span style="color: #000000"> price qushi </span>= li.css(<span style="color: #800000">"</span><span style="color: #800000">td .sc_up::text</span><span style="color: #800000">"</span><span style="color: #000000">).get() </span><span style="color: #0000ff">if</span> qushi ==<span style="color: #000000"> None : dit[</span><span style="color: #800000">"</span><span style="color: #800000">趋势</span><span style="color: #800000">"</span>] = <span style="color: #800000">"</span><span style="color: #800000">下降</span><span style="color: #800000">"</span> <span style="color: #0000ff">else</span><span style="color: #000000">: dit[</span><span style="color: #800000">"</span><span style="color: #800000">趋势</span><span style="color: #800000">"</span>] =<span style="color: #000000"> qushi csv_write.writerow(dit) </span><span style="color: #0000ff">print</span>(dit)
保存数据
f = open(<span style="color: #800000">"</span><span style="color: #800000">data.csv</span><span style="color: #800000">"</span>, mode=<span style="color: #800000">"</span><span style="color: #800000">a</span><span style="color: #800000">"</span>, encoding=<span style="color: #800000">"</span><span style="color: #800000">utf-8-sig</span><span style="color: #800000">"</span>, newline=<span style="color: #800000">""</span><span style="color: #000000">) csv_write </span>= csv.DictWriter(f, fieldnames=[<span style="color: #800000">"</span><span style="color: #800000">品种</span><span style="color: #800000">"</span>, <span style="color: #800000">"</span><span style="color: #800000">平均价格</span><span style="color: #800000">"</span>, <span style="color: #800000">"</span><span style="color: #800000">趋势</span><span style="color: #800000">"</span><span style="color: #000000">]) csv_write.writeheader()</span>