前言
本文的文字及图片来源于网络,仅供学习、交流使用,不具有任何商业用途,版权归原作者所有,如有问题请及时联系我们以作处理
以下文章来源于菜鸟学Python数据分析，作者J哥
1.网页分析
本文以爬取《乐队的夏天第2季》第13期上视频弹幕为例，首先通过以下步骤找到存放弹幕的真实url。
分析弹幕真实url，我们发现，参数5981449914376200是视频tvid，参数62是tvid倒数4为的前两位，参数00是tvid的最后两位，.z前的参数1为视频总时长除以300秒向上取整。观察相邻两个弹幕文件包，可以看出爱奇艺每5分钟更新一次弹幕文件。
由于直接爬取出来的弹幕文件存在乱码，需要进行二进制编码，方可得到最终的弹幕数据。
2.爬虫实战
<span>import</span><span> zlib
</span><span>import</span><span> requests
</span><span>#</span><span> 1.爬取xml文件</span>
<span>def</span><span> download_xml(url):
bulletold </span>= requests.get(url).content <span>#</span><span> 二进制内容</span>
<span>return</span><span> zipdecode(bulletold)
</span><span>def</span><span> zipdecode(bulletold):
</span><span>"</span><span>对zip压缩的二进制内容解码成文本</span><span>"</span><span>
decode </span>= zlib.decompress(bytearray(bulletold), 15 + 32).decode(<span>"</span><span>utf-8</span><span>"</span><span>)
</span><span>return</span><span> decode
</span><span>for</span> x <span>in</span> range(1,12<span>):
</span><span>#</span><span> x是从1到12，12怎么来的，这一集总共57分钟，爱奇艺每5分钟会加载新的弹幕,57除以5向上取整</span>
url = <span>"</span><span>https://cmts.iqiyi.com/bullet/62/00/5981449914376200_300_</span><span>"</span> + str(x) + <span>"</span><span>.z</span><span>"</span><span>
xml </span>=<span> download_xml(url)
</span><span>#</span><span> 把编码好的文件分别写入17个xml文件中（类似于txt文件），方便后边取数据</span>
with open(<span>"</span><span>./aiqiyi/iqiyi</span><span>"</span> + str(x) + <span>"</span><span>.xml</span><span>"</span>, <span>"</span><span>a+</span><span>"</span>, encoding=<span>"</span><span>utf-8</span><span>"</span><span>) as f:
f.write(xml)
</span><span>#</span><span> 2.读取xml文件中的弹幕数据数据</span>
<span>from</span> xml.dom.minidom <span>import</span><span> parse
</span><span>import</span><span> xml.dom.minidom
</span><span>def</span><span> xml_parse(file_name):
DOMTree </span>=<span> xml.dom.minidom.parse(file_name)
collection </span>=<span> DOMTree.documentElement
</span><span>#</span><span> 在集合中获取所有entry数据</span>
entrys = collection.getElementsByTagName(<span>"</span><span>entry</span><span>"</span><span>)
</span><span>print</span><span>(entrys)
result </span>=<span> []
</span><span>for</span> entry <span>in</span><span> entrys:
content </span>= entry.getElementsByTagName(<span>"</span><span>content</span><span>"</span><span>)[0]
</span><span>print</span><span>(content.childNodes[0].data)
i </span>=<span> content.childNodes[0].data
result.append(i)
</span><span>return</span><span> result
with open(</span><span>"</span><span>aiyiqi_danmu.txt</span><span>"</span>, mode=<span>"</span><span>w</span><span>"</span>, encoding=<span>"</span><span>utf-8</span><span>"</span><span>) as f:
</span><span>for</span> x <span>in</span> range(1,12<span>):
l </span>= xml_parse(<span>"</span><span>./aiqiyi/iqiyi</span><span>"</span> + str(x) + <span>"</span><span>.xml</span><span>"</span><span>)
</span><span>for</span> line <span>in</span><span> l:
f.write(line)
f.write(</span><span>"</span><span>
</span><span>"</span>
www#gaodaima.com来源gaodai#ma#com搞@@代~&码*网搞代码
3.数据预览
搞代码网（gaodaima.com）提供的所有资源部分来自互联网，如果有侵犯您的版权或其他权益，请说明详细缘由并提供版权或权益证明然后发送到邮箱[email protected]，我们会在看到邮件的第一时间内为您处理，或直接联系QQ：872152909。本网站采用BY-NC-SA协议进行授权
转载请注明原文链接：Python爬虫练习：爬取爱奇艺视频弹幕
转载请注明原文链接：Python爬虫练习：爬取爱奇艺视频弹幕
