MySQL+Amoeba实现数据库读写分离
参考:https://www.cnblogs.com/liuyisai/p/6009379.html
一,Amoeba是什么
Amoeba(变形虫)项目,专注 分布式数据库 proxy 开发。座落与Client、DB Server(s)之间。对客户端透明。具有负载均衡、高可用性、sql过滤、读写分离、可路由相关的query到目标数据库、可并发请求多台数据库合并结果。
主要解决:
• 降低 数据切分带来的复杂多数据库结构
• 提供切分规则并降低 数据切分规则 给应用带来的影响
• 降低db 与客户端的连接数
• 读写分离
二,为什么要用Amoeba
目前要实现mysql的主从读写分离,主要有以下几种方案:
1、 通过程序实现,网上很多现成的代码,比较复杂,如果添加从服务器要更改多台服务器的代码。
2、 通过mysql-proxy来实现,由于mysql-proxy的主从读写分离是通过lua脚本来实现,目前lua的脚本的开发跟不上节奏,而写没有完美的现成的脚本,因此导致用于生产环境的话风险比较大,据网上很多人说mysql-proxy的性能不高。
3、 自己开发接口实现,这种方案门槛高,开发成本高,不是一般的小公司能承担得起。
4、 利用阿里巴巴的开源项目Amoeba来实现,具有负载均衡、高可用性、sql过滤、读写分离、可路由相关的query到目标数据库,并且安装配置非常简单。
三,Amoeba的安装
测试环境
amoeba:172.16.20.31
masterDB:172.16.20.180
slaveDB:172.16.20.188
主从配置已经配置好
安装jave版本1.8(安装过程不详述)
参考版本
安装amoeba
可以从https://sourceforge.net/projects/amoeba/下载amoeba-mysql-3.0.5-RC-distribution.zip
无需安装解压即可使用解压目录为/usr/local/amoeba
1 2 3 4 5 6 7 8 9 10 11 | [root@test - cobbler amoeba] # pwd / usr / local / amoeba [root@test - cobbler amoeba] # ll 总用量 12 - rw - r - - r - - 1 root root 5 1 月 24 10 : 05 Amoeba - MySQL.pid drwxrwxrwx 2 root root 63 7 月 5 2013 benchmark drwxrwxrwx 2 root root 166 7 月 5 2013 bin drwxrwxrwx 2 root root 243 1 月 24 10 : 04 conf - rwxrwxrwx 1 root root 729 1 月 24 09 : 13 jvm.properties drwxrwxrwx 2 root root 4096 7 月 5 2013 lib drwxr - xr - x 2 root root 152 1 月 24 09 : 13 logs |
配置文档位于目录/usr/local/amoeba/conf下配置读写分离仅需配置两个文件dbServers.xml和amoeba.xml
dbServers.xml
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 | <?xml version = "1.0" encoding = "gbk" ?> <!DOCTYPE amoeba:dbServers SYSTEM "dbserver.dtd" > <amoeba:dbServers xmlns:amoeba = "http://amoeba.meidusa.com/" > <! - - Each dbServer needs to be configured into a Pool, If you need to configure multiple dbServer with load balancing that can be simplified by the following configuration: add attribute with name virtual = "true" in dbServer, but the configuration does not allow the element with name factoryConfig such as 'multiPool' dbServer - - > <dbServer name = "abstractServer" abstractive = "true" > <factoryConfig class = "com.meidusa.amoeba.mysql.net.MysqlServerConnectionFactory" > < property name = "connectionManager" >${defaultManager}< / property > < property name = "sendBufferSize" > 64 < / property > < property name = "receiveBufferSize" > 128 < / property > <! - - mysql port - - > < property name = "port" > 3306 < / property > #设置Amoeba要连接的mysql数据库的端口,默认是3306 <! - - mysql schema - - > < property name = "schema" >testdb< / property > #设置缺省的数据库,当连接amoeba时,操作表必须显式的指定数据库名,即采用dbname.tablename的方式,不支持 use dbname指定缺省库,因为操作会调度到各个后端dbserver <! - - mysql user - - > < property name = "user" >test1< / property > #设置amoeba连接后端数据库服务器的账号和密码,因此需要在所有后端数据库上创建该用户,并授权amoeba服务器可连接 < property name = "password" > 111111 < / property > < / factoryConfig> <poolConfig class = "com.meidusa.toolkit.common.poolable.PoolableObjectPool" > < property name = "maxActive" > 500 < / property > #最大连接数,默认500 < property name = "maxIdle" > 500 < / property > #最大空闲连接数 < property name = "minIdle" > 1 < / property > #最新空闲连接数 < property name = "minEvictableIdleTimeMillis" > 600000 < / property > < property name = "timeBetweenEvictionRunsMillis" > 600000 < / property > < property name = "testOnBorrow" >true< / property > < property name = "testOnReturn" >true< / property > < property name = "testWhileIdle" >true< / property > < / poolConfig> < / dbServer> <dbServer name = "writedb" parent = "abstractServer" > #设置一个后端可写的dbServer,这里定义为writedb,这个名字可以任意命名,后面还会用到 <factoryConfig> <! - - mysql ip - - > < property name = "ipAddress" > 172.16 . 20.180 < / property > < / factoryConfig> < / dbServer> <dbServer name = "slave" parent = "abstractServer" > #设置后端可写dbserver <factoryConfig> <! - - mysql ip - - > < property name = "ipAddress" > 172.16 . 20.188 < / property > < / factoryConfig> < / dbServer> <dbServer name = "myslave" virtual = "true" > #设置定义一个虚拟的dbserver,实际上相当于一个dbserver组,这里将可读的数据库ip统一放到一个组中,将这个组的名字命名为myslave <poolConfig class = "com.meidusa.amoeba.server.MultipleServerPool" > <! - - Load balancing strategy: 1 = ROUNDROBIN , 2 = WEIGHTBASED , 3 = HA - - > < property name = "loadbalance" > 1 < / property > #选择调度算法,1表示负载均衡,2表示权重,3表示HA, 这里选择1 <! - - Separated by commas,such as: server1,server2,server1 - - > < property name = "poolNames" >slave< / property > #myslave组成员 < / poolConfig> < / dbServer> < / amoeba:dbServers> |
另一个配置文件amoeba.xml
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 | <?xml version = "1.0" encoding = "gbk" ?> <!DOCTYPE amoeba:configuration SYSTEM "amoeba.dtd" > <amoeba:configuration xmlns:amoeba = "http://amoeba.meidusa.com/" > <proxy> <! - - service class must implements com.meidusa.amoeba.service.Service - - > <service name = "Amoeba for Mysql" class = "com.meidusa.amoeba.mysql.server.MySQLService" > <! - - port - - > < property name = "port" > 8066 < / property > #默认监听端口8066 <! - - bind ipAddress - - > #下面配置监听的接口,如果不设置,默认监听所有的IP <! - - < property name = "ipAddress" > 127.0 . 0.1 < / property > - - > < property name = "connectionFactory" > <bean class = "com.meidusa.amoeba.mysql.net.MysqlClientConnectionFactory" > < property name = "sendBufferSize" > 128 < / property > < property name = "receiveBufferSize" > 64 < / property > < / bean> < / property > < property name = "authenticateProvider" > <bean class = "com.meidusa.amoeba.mysql.server.MysqlClientAuthenticator" > # 提供客户端连接amoeba时需要使用这里设定的账号 (这里的账号密码和amoeba连接后端数据库服务器的密码无关) < property name = "user" >root< / property > < property name = "password" > 123456 < / property > < property name = "filter" > <bean class = "com.meidusa.toolkit.net.authenticate.server.IPAccessController" > < property name = "ipFile" >${amoeba.home} / conf / access_list.conf< / property > < / bean> < / property > < / bean> < / property > < / service> <runtime class = "com.meidusa.amoeba.mysql.context.MysqlRuntimeContext" > <! - - proxy server client process thread size - - > < property name = "executeThreadSize" > 128 < / property > <! - - per connection cache prepared statement size - - > < property name = "statementCacheSize" > 500 < / property > <! - - default charset - - > < property name = "serverCharset" >utf8< / property > <! - - query timeout( default: 60 second , TimeUnit:second) - - > < property name = "queryTimeout" > 60 < / property > < / runtime> < / proxy> <! - - Each ConnectionManager will start as thread manager responsible for the Connection IO read , Death Detection - - > <connectionManagerList> <connectionManager name = "defaultManager" class = "com.meidusa.toolkit.net.MultiConnectionManagerWrapper" > < property name = "subManagerClassName" >com.meidusa.toolkit.net.AuthingableConnectionManager< / property > < / connectionManager> < / connectionManagerList> <! - - default using file loader - - > <dbServerLoader class = "com.meidusa.amoeba.context.DBServerConfigFileLoader" > < property name = "configFile" >${amoeba.home} / conf / dbServers.xml< / property > < / dbServerLoader> <queryRouter class = "com.meidusa.amoeba.mysql.parser.MysqlQueryRouter" > < property name = "ruleLoader" > <bean class = "com.meidusa.amoeba.route.TableRuleFileLoader" > < property name = "ruleFile" >${amoeba.home} / conf / rule.xml< / property > < property name = "functionFile" >${amoeba.home} / conf / ruleFunctionMap.xml< / property > < / bean> < / property > < property name = "sqlFunctionFile" >${amoeba.home} / conf / functionMap.xml< / property > < property name = "LRUMapSize" > 1500 < / property > < property name = "defaultPool" >writedb< / property > #设置amoeba默认的池,这里设置为wrutedb < property name = "writePool" >writedb< / property > #这两个选项默认是注释的,需要取消注释,这里用来指定前面两个定义好的读写池 < property name = "readPool" >myslave< / property > < property name = "needParse" >true< / property > < / queryRouter> < / amoeba:configuration> |
在masterdb上创建数据库testdb
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | mysql> create database testdb; Query OK, 1 row affected ( 0.00 sec) mysql> show databases; + - - - - - - - - - - - - - - - - - - - - + | Database | + - - - - - - - - - - - - - - - - - - - - + | information_schema | | mysql | | performance_schema | | sys | | test | | testdb | + - - - - - - - - - - - - - - - - - - - - + 6 rows in set ( 0.00 sec) |
查看slavedb是否复制成功
1 2 3 4 5 6 7 8 9 10 11 12 | mysql> show databases; + - - - - - - - - - - - - - - - - - - - - + | Database | + - - - - - - - - - - - - - - - - - - - - + | information_schema | | mysql | | performance_schema | | sys | | test | | testdb | + - - - - - - - - - - - - - - - - - - - - + 6 rows in set ( 0.00 sec) |
在masterdb上为访问amoedb授权,从会复制用户
1 | GRANT ALL ON testdb. * TO 'test1' @ '172.16.20.31' IDENTIFIED BY '111111' ; |
启动amoeba
1 | / usr / local / amoeba / bin / launcher |
启动报错需要设置好java环境变量及jvm参数
1 2 3 | vim / usr / local / amoeba / jvm.properties 改成:JVM_OPTIONS = "-server -Xms1024m -Xmx1024m -Xss256k -XX:PermSize=16m -XX:MaxPermSize=96m" 原为:JVM_OPTIONS = "-server -Xms256m -Xmx1024m -Xss196k -XX:PermSize=16m -XX:MaxPermSize=96m" |
测试
远程登陆mysql客户端通过指定amoeba配置文件中指定的用户名、密码、和端口以及amoeba服务器ip地址链接mysql数据库
1 | mysql - uroot - p123456 - h172. 16.20 . 31 - P8066 |
在testdb中创建表test并插入数据
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | mysql> use testdb Database changed mysql> create table test_table( id int ); Query OK, 0 rows affected ( 0.02 sec) mysql> show tables; + - - - - - - - - - - - - - - - - - - + | Tables_in_testdb | + - - - - - - - - - - - - - - - - - - + | test_table | + - - - - - - - - - - - - - - - - - - + 1 row in set ( 0.01 sec) mysql> insert into test_table values( 1 ); Query OK, 1 row affected ( 0.00 sec) mysql> select * from test_table; + - - - - - - + | id | + - - - - - - + | 1 | + - - - - - - + 1 row in set ( 0.00 sec) |
分别登录masterdb和salvaedb查看数据
masterdb
1 2 3 4 5 6 7 | mysql> select * from testdb.test_table; + - - - - - - + | id | + - - - - - - + | 1 | + - - - - - - + 1 row in set ( 0.00 sec) |
slavedb
1 2 3 4 5 6 7 | mysql> select * from testdb.test_table; + - - - - - - + | id | + - - - - - - + | 1 | + - - - - - - + 1 row in set ( 0.00 sec) |
停掉masterdb分别在客户端插入和查询
1 2 | mysql> insert into test_table values( 2 ); ERROR 1053 ( 08S01 ): Server shutdown in progress |
1 2 3 4 5 6 7 | mysql> select * from test_table; + - - - - - - + | id | + - - - - - - + | 1 | + - - - - - - + 1 row in set ( 0.01 sec) |
关掉master以后写报错读取正常
开启masterdb关闭slavedb再测试
1 2 3 4 5 | mysql> insert into test_table values( 2 ); Query OK, 1 row affected ( 0.01 sec) mysql> select * from test_table; ERROR 1044 ( 42000 ): poolName = myslave, no valid pools |
关掉slavedb以后写正常读取失败
启动slavedb再查询读取也正常,可以直接登录slave查看数据是否同步
1 2 3 4 5 6 7 | mysql> select * from testdb.test_table; + - - - - - - + | id | + - - - - - - + | 1 | | 2 | + - - - - - - + |
MySQL数据库读写分离配置完毕,应用只需要连接只amobe的8066端口即可实现读写分离
配置文件说明
1 2 3 4 5 6 7 8 9 10 11 12 13 | Amoeba主配置文件($AMOEBA_HOME / conf / amoeba.xml),用来配置Amoeba服务的基本参数,如Amoeba主机地址、端口、认证方式、用于连接的用户名、密码、线程数、超时时间、其他配置文件的位置等。 数据库服务器配置文件($AMOEBA_HOME / conf / dbServers.xml),用来存储和配置Amoeba所代理的数据库服务器的信息,如:主机IP、端口、用户名、密码等。 切分规则配置文件($AMOEBA_HOME / conf / rule.xml),用来配置切分规则。 数据库函数配置文件($AMOEBA_HOME / conf / functionMap.xml),用来配置数据库函数的处理方法,Amoeba将使用该配置文件中的方法解析数据库函数。 切分规则函数配置文件($AMOEBA_HOME / conf / ruleFunctionMap.xml),用来配置切分规则中使用的用户自定义函数的处理方法。 访问规则配置文件($AMOEBA_HOME / conf / access_list.conf),用来授权或禁止某些服务器IP访问Amoeba。 日志规格配置文件($AMOEBA_HOME / conf / log4j.xml),用来配置Amoeba输出日志的级别和方式。 |
【推荐】国内首个AI IDE,深度理解中文开发场景,立即下载体验Trae
【推荐】编程新体验,更懂你的AI,立即体验豆包MarsCode编程助手
【推荐】抖音旗下AI助手豆包,你的智能百科全书,全免费不限次数
【推荐】轻量又高性能的 SSH 工具 IShell:AI 加持,快人一步
· SQL Server 2025 AI相关能力初探
· Linux系列:如何用 C#调用 C方法造成内存泄露
· AI与.NET技术实操系列(二):开始使用ML.NET
· 记一次.NET内存居高不下排查解决与启示
· 探究高空视频全景AR技术的实现原理
· 阿里最新开源QwQ-32B,效果媲美deepseek-r1满血版,部署成本又又又降低了!
· AI编程工具终极对决:字节Trae VS Cursor,谁才是开发者新宠?
· 开源Multi-agent AI智能体框架aevatar.ai,欢迎大家贡献代码
· Manus重磅发布:全球首款通用AI代理技术深度解析与实战指南
· 被坑几百块钱后,我竟然真的恢复了删除的微信聊天记录!