python入门——爬取整个网页源码

<span style="color: #008080"> 1</span> <span style="color: #0000ff">import</span><span style="color: #000000"> requests
</span><span style="color: #008080"> 2</span> <span style="color: #008000">#</span><span style="color: #008000"> encoding:utf-8  #默认格式utf-8</span>
<span style="color: #008080"> 3</span> 
<span style="color: #008080"> 4</span> <span style="color: #0000ff">def</span> get_html(url): <span style="color: #008000">#</span><span style="color: #008000">爬取源码函数</span>
<span style="color: #008080"> 5</span>     <a href="https://www.gaodaima.com/tag/headers" title="查看更多关于headers的文章" target="_blank">headers</a> =<span style="color: #000000"> {
</span><span style="color: #008080"> 6</span>         <span style="color: #800000">"</span><span style="color: #800000">User-Agent</span><span style="color: #800000">"</span>: <span style="color: #800000">"</span><span style="color: #800000">Mozilla/5.0(Macintosh; Intel Mac OS X 10_11_4)
</span><span style="color: #008080"> 7</span> <span style="color: #800000">        AppleWebKit/537.36(KHTML, like Gecko) Chrome/52 .0.2743. 116 Safari/537.36</span><span style="color: #800000">"</span>
<span style="color: #008080"> 8</span> 
<span style="color: #008080"> 9</span>     }  <span style="color: #008000">#</span><span style="color: #008000"> 模拟浏览器访问</span>
<span style="color: #008080">10</span>     response = requests.get(url, headers=headers)  <span style="color: #008000">#</span><span style="color: #008000"> 请求访问网站</span>
<span style="color: #008080">11</span>     response.encoding = response.apparent_encoding <span style="color: #008000">#</span><span style="color: #008000">设置字符编码格式</span>
<span style="color: #008080">12</span>     html = response.text  <span style="color: #008000">#</span><span style="color: #008000"> 获取网页源码</span>
<span style="color: #008080">13</span>     <span style="color: #0000ff">return</span> html  <span style="color: #008000">#</span><span style="color: #008000"> 返回网页源码</span>
<span style="color: #008080">14</span> 
<span style="color: #008080">15</span> r = get_html(<span style="color: #800000">"</span><span style="color: #800000">https://www.baidu.com/</span><span style="color: #800000">"</span><span style="color: #000000">)
</span><span style="color: #008080">16</span> <span style="color: #0000ff">print</span>(r) <span style="color: #008000">#</span><span style="color: #008000">打印网页源码</span>

www#gaodaima.com来源gaodai#ma#com搞@@代~&码*网搞代码