一次LVS+MySQL的主主负载均衡实战

这是去年做的一个项目的记录，如果大家有更好的解决方案，欢迎指出。 先说说项目需求，用户需要在两个地市部署两套应用系统和两套数据库，在一个地市主用，在另一个热备；数据要互备；而且如果主用地市流量很大，可以分流到备用地市。 再说说用户环境，用户

这是去年做的一个项目的记录，如果大家有更好的解决方案，欢迎指出。

先说说项目需求，用户需要在两个地市部署两套应用系统和两套数据库，在一个地市主用，在另一个热备；数据要互备；而且如果主用地市流量很大，可以分流到备用地市。

再说说用户环境，用户所有地市之间的通信使用MPLS VPN，两个地市之间使用不同的网段（甲方的内网）。

应用系统的主备和负载均衡下次再说，这儿先主要讲讲数据库内容。

1.选择数据库

　　现在有很多数据库可以满足主备和负载均衡的需求，有关系型数据库，也有NoSQL数据库。

　　我们选择了MySQL，做项目嘛，首先考虑的是项目成本，在关系型数据库中，能满足需求而且成本低的也就是MySQL了。对NoSQL数据库，我们考虑过mongodb和Hadoop，而且对这两个数据库都做了前期的研究评估，但因为我们的应用系统改造的成本太大，就放弃了。

　　采用了MySQL，那么就看使用它的什么技术了，我们需要两个地市的数据库都能访问，那么主备技术不能采用；两个地市的数据独立完整，而且在大多数情况下使用主用地市系统，备用地市只做数据同步，不参与应用计算，不消耗网络流量，MySQL Cluster也排除了。

　　最终采用了MySQL的主主备份技术，来达到数据的备份和两个数据库系统都可以访问的目的。

2.选择负载均衡技术

　　对MySQL的负载均衡技术，主要有HAPROXY和LVS等，由于我们的数据库要部署在两个不同地市的不同网段上，只有LVS的隧道模式能满足我们的需求。

3.架构图

4.安装LVS

　　1).安装必须文件

<span>yum</span> <span>install</span> -y <span>gcc</span> <span>gcc</span>-c++ makepcre pcre-devel kernel-devel openssl-devel libnl-devel popt-devel

　　2).建立内核软连接

<span>ln</span> -s /usr/src/kernels/<span>2.6</span>.<span>18</span>-<span>164</span>.el5-x86_64/ /usr/src/linux

　　3).安装lvs程序

<span>　　　　modprobe</span> -l|<span>grep</span><span> ipvs　　　　</span><span>tar</span> xvf  ipvsadm-<span>1.24</span>.<span>tar</span><span>.gz       cd ipvsadm</span>-<span>1.24</span>　　　　<span>make</span>       <span>make</span> <span>install</span>       /etc/init.d/<span>ipvsadm save       </span>/etc/init.d/ipvsadm restart

　　在真实机器上的配置脚本：

#!/bin/<span>bash# description: Config realserver lo and apply noarp SNS_VIP</span>=<span>192.168</span>.<span>1.1</span> /etc/rc.d/init.d/<span>functions </span><span>case</span> <span>"</span><span>$1</span><span>"</span> <span>in</span><span>start)       </span><span>ifconfig</span> tunl0 $SNS_VIP netmask <span>255.255</span>.<span>255.255</span> broadcast <span>192.168</span>.<span>2.255</span>       /sbin/route add -<span>host $SNS_VIP dev tunl0       </span><span>echo</span> <span>"</span><span>1</span><span>"</span> >/proc/sys/net/ipv4/conf/tunl0/<span>arp_ignore      #主要是实现禁止响应对VIP的ARP请求       </span><span>echo</span> <span>"</span><span>2</span><span>"</span> >/proc/sys/net/ipv4/conf/tunl0/<span>arp_announce       </span><span>echo</span> <span>"</span><span>1</span><span>"</span> >/proc/sys/net/ipv4/conf/all/<span>arp_ignore       </span><span>echo</span> <span>"</span><span>2</span><span>"</span> >/proc/sys/net/ipv4/conf/all/<span>arp_announce       </span><span>echo</span> <span>0</span> > /proc/sys/net/ipv4/conf/tunl0/<span>rp_filter       </span><span>echo</span> <span>1</span> > /proc/sys/net/ipv4<span>本文来源gaodai#ma#com搞*!代#%^码$网*</span>/conf/tunl0/<span>forwarding       </span><span>echo</span> <span>1</span> > /proc/sys/net/ipv4/<span>ip_forwarding       sysctl </span>-p >/dev/<span>null</span> <span>2</span>>&<span>1</span>       <span>echo</span> <span>"</span><span>RealServer Start OK</span><span>"</span><span>        ;;stop)       </span><span>ifconfig</span><span> tunl0 down       route del $SNS_VIP </span>>/dev/<span>null</span> <span>2</span>>&<span>1</span>       <span>echo</span> <span>"</span><span>0</span><span>"</span> >/proc/sys/net/ipv4/conf/tunl0/<span>arp_ignore       </span><span>echo</span> <span>"</span><span>0</span><span>"</span> >/proc/sys/net/ipv4/conf/tunl0/<span>arp_announce       </span><span>echo</span> <span>"</span><span>0</span><span>"</span> >/proc/sys/net/ipv4/conf/all/<span>arp_ignore       </span><span>echo</span> <span>"</span><span>0</span><span>"</span> >/proc/sys/net/ipv4/conf/all/<span>arp_announce       </span><span>echo</span> <span>"</span><span>RealServer Stoped</span><span>"</span><span>       ;;</span>*<span>)       </span><span>echo</span> <span>"</span><span>Usage: $0 {start|stop}</span><span>"</span><span>       exit </span><span>1</span><span>esac</span><span>exit </span><span>0</span>

5.安装Keepalived

<span>　　tar</span> xvf keepalived-<span>1.1</span>.<span>19</span>.<span>tar</span><span>.gz　　cd keepalived</span>-<span>1.1</span>.<span>19</span><span>　　.</span>/configure --prefix=/usr/local/keepalived --with-kernel-<span>dir</span>=/usr/src/kernels/<span>2.6</span>.<span>18</span>-<span>164</span>.el5-x86_64 --enable-<span>lvs　　</span><span>make</span>　　<span>make</span> <span>install</span>　　<span>cp</span> /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/　　<span>cp</span> /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/　　<span>mkdir</span> -p /etc/<span>keepalived　　</span>/usr/local/keepalived/etc/keepalived/keepalived.conf /etc/<span>keepalived　　</span>/usr/local/keepalived/sbin/keepalived /usr/sbin/<span>   chkconfig keepalived on</span>

　 keepalived.conf文件内容

! Configuration File <span>for</span><span> keepalivedglobal_defs {   notification_email {    　　**********   }   notification_email_from *****   smtp_server </span><span>192.168</span>.<span>200.1</span><span>   smtp_connect_timeout </span><span>30</span><span>   router_id LVS_MASTER}vrrp_instance VI_1 {    state MASTER    interface eth0    virtual_router_id </span><span>51</span><span>    priority </span><span>100</span><span>    advert_int </span><span>1</span><span>    authentication {        auth_type PASS        auth_pass </span><span>1111</span><span>    }    virtual_ipaddress {        </span><span>192.168</span>.<span>2.1</span> <span>            }</span><span>}virtual_server </span><span>192.168</span>.<span>2.1</span> <span>3306</span><span> {    delay_loop </span><span>6</span><span>    lb_algo wrr    lb_kind TUN    nat_mask </span><span>255.255</span>.<span>255.0</span><span>    persistence_timeout </span><span>50</span><span>    protocol TCP real_server </span><span>192.168</span>.<span>1.1</span> <span>3306</span><span> {         weight </span><span>10</span><span>                TCP_CHECK {        connect_timeout </span><span>3</span><span>        nb_get_retry </span><span>3</span><span>        delay_before_retry </span><span>3</span><span>        connect_port </span><span>3306</span><span>}     } real_server </span><span>192.168</span>.<span>3.1</span> <span>3306</span><span> {         weight </span><span>1</span><span>                    TCP_CHECK {        connect_timeout </span><span>3</span><span>        nb_get_retry </span><span>3</span><span>        delay_before_retry </span><span>3</span><span>        connect_port </span><span>3306</span><span>}}}</span>

6.安装MySQL

　　MySQL数据库的安装我就不记录了，主要记录一下主主备份的配置。

　　1).在两台数据库上创建同步账户

<span>grant</span> <span>replication</span> slave <span>on</span> <span>*</span>.<span>*</span> <span>to</span> mysqldb1@<span>'</span><span>192.168.3.1</span><span>'</span> identified <span>by</span> <span>'</span><span>*****</span><span>'</span>;

　　2).修改数据库配置文件

<span>[client]default</span>-character-set=<span>utf8[mysqld]character</span>-set-server=<span>utf8server</span>-<span>id</span>=<span>1</span><span>         #mysql5.5默认有log</span>-bin=mysql-<span>bin   #mysql5.5默认有binlog</span>-ignore-db=<span>mysql #mysql库将不同步auto_increment_offset</span>=<span>1</span><span> #是基数，两边不同server1设置1，server2设置2auto_increment_increment</span>=<span>2</span> #大于等于复制集群中服务器的数量

<span>[client]default</span>-character-set=<span> utf8[mysqld]character</span>-set-server=<span> utf8server</span>-<span>id</span>=<span>2</span><span>   #将默认的1改为2，该值为两边不相同的任意数字log</span>-bin=mysql-<span>bin  #mysql5.5默认有binlog</span>-ignore-db=<span>mysql #mysql库将不同步auto_increment_offset</span>=<span>2</span><span>  #是基数，两边不同server1设置1，server2设置2auto_increment_increment</span>=<span>2</span> #大于等于复制集群中服务器的数量

　　3).执行数据同步

<span lang="EN-US"> 　</span>

<span>　show master status\G　　　结果：　　　　　　　　</span><span>File</span>: binlog.<span>000007</span><span>　　　　　　Position: </span><span>107</span><span>　　　　　　Binlog_Do_DB: test　　　　　　Binlog_Ignore_DB: mysql　　　　　　</span><span>1</span> row <span>in</span> <span>set</span> (<span>0.00</span> sec)

　　 依据结果，执行下面的语句

<span lang="EN-US">　　</span>

change master <span>to</span> master_host<span>=</span><span>'</span><span>192.168.1.2</span><span>'</span>, master_user<span>=</span><span>'</span><span>xxx</span><span>'</span>, master_password<span>=</span><span>'</span><span>*******</span><span>'</span>, master_log_file<span>=</span><span>'</span><span>binlog.000007</span><span>'</span>, master_log_pos<span>=</span><span>107</span>;

7.注意事项

　　1).数据库链接数要依据项目调整

　　2).数据库链接的生命周期和LVS会话管理周期的调整 ipvsadm –set 120 20 100，如果时间过短，每次外部请求都会是一个新的链接，超过数据库最大链接数

　　3).在真实机器上要配置隧道接口，并且要关闭VIP的ARP请求

　　4).在lvs虚拟机上不手动配置隧道接口，用keepalived去自动管理，但在真实机上要手动创建（可以写开机脚本）

　　5).在项目中lvs所在区域如果断网，则负载均衡也失效了，只能手动切换至备用系统

　　搭建lvs，是个要十分细心的工作，很多细节要注意，就不一一说了。

喜欢 (0)赏

[搞代码]

分享 (0)