背景
Redis多数据源常见的场景:
- 分区数据处理:当数据量增长时,单个Redis实例可能无法处理所有的数据。通过使用多个Redis数据源,可以将数据分区存储在不同的实例中,使得数据处理更加高效。
- 多租户应用程序:对于多租户应用程序,每个租户可以拥有自己的Redis数据源,以确保数据隔离和安全性。
现在在网上的一些办理方案无法完全满足特定需求,比方不支持动态添加或更新数据源、缺乏数据源的负载均衡功能,大概不能开箱即用必要自行封装一些常用方法。
为相识决这些题目,封装了一个轻量级的Redis多数据源库,已在生产环境中长时间稳固运行。现分享出来,盼望这个库能对你的项目有所帮助。
https://github.com/codebaorg/redis-keeper
如果这篇文章帮助到了你,欢迎star一下,感谢你的支持。
特性
- 基于redisson封装,保存redisson的所有强盛功能。
- 支持redis多数据源设置的及时更新。
- 支持redis多数据源的负载均衡。
- 支持redis数据源的“只读”、“只写”、“读写”、“跳过”的状态切换。
- 提供 130+ 常用的redis操纵方法,包括但不限string、list、hash、set、zset、geo、bitmap、hyberloglog、分布式锁、布隆过滤器、限流等。
快速开始
- <dependency>
- <groupId>org.redisson</groupId>
- <artifactId>redisson</artifactId>
- <version>3.15.5</version>
- <scope>provided</scope>
- </dependency>
- <dependency>
- <groupId>org.codeba</groupId>
- <artifactId>redis-keeper-spring-boot-starter</artifactId>
- <version>2024.0.0</version>
- </dependency>
复制代码
- springboot的设置文件,添加redis多数据源的设置
- redis-keeper:
- redis:
- datasource:
- ds1: // 此处的命名可以随便取,但是要保证唯一不重复
- host: localhost
- port: 6379
- database: 0
- password: yourPass
-
- ds2: // 此处的命名可以随便取,但是要保证唯一不重复
- host: localhost
- port: 6379
- database: 0
- password: yourPass
-
复制代码
- 按自定义的数据源别名获取相应的cacheTemplate,执行redis下令。
方式一:通过CacheTemplateProvider获取cacheTemplate,示比方下:- public class AppTest {
- @Autowired
- private CacheTemplateProvider<CacheTemplate> provider;
- public void test() {
- final Optional<CacheTemplate> templateOptional = provider.getTemplate("ds1");
- if (templateOptional.isPresent()) {
- final CacheTemplate cacheTemplate = templateOptional.get();
- String key = "foo";
- String value = "bar";
- cacheTemplate.set(key, value);
- }
- }
-
- }
复制代码 方式二:通过代理类CacheTemplateProxy获取cacheTemplate,示比方下:- public class AppTest {
- private final CacheTemplate cacheTemplate = CacheTemplateProxy.asTemplate("ds1");
- public void test() {
- String key = "foo";
- String value = "bar";
- cacheTemplate.set(key, value);
- }
-
- }
复制代码 更多示例
多数据源及时更新
必要:spring cloud + nacos
现有的数据源设置示比方下:- redis-keeper:
- redis:
- datasource:
- ds1:
- host: localhost
- port: 6379
- database: 10
- password: xxx
- invoke-params-print: true
复制代码 测试方法:- @RestController
- @RequiredArgsConstructor
- public class TestController {
- private final CacheTemplateProvider<CacheTemplate> provider;
- @RequestMapping("/test")
- public void testRefresh() {
- final Optional<CacheTemplate> templateOptional = provider.getTemplate("ds1");
- templateOptional.ifPresent(cacheTemplate -> {
- cacheTemplate.set("foo", "bar");
- cacheTemplate.del("foo");
- });
- }
- }
复制代码 访问上面的方法,日记打印如下(可以看到写入的redis库是10库):- 2024-03-12T11:33:37.850+08:00 INFO 62914 --- [nio-8000-exec-1] o.c.r.k.support.DefaultRedissonTemplate : cmd:set, params:[foo, bar], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=10}]
- 2024-03-12T11:33:37.858+08:00 INFO 62914 --- [nio-8000-exec-1] o.c.r.k.support.DefaultRedissonTemplate : cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=10}]
复制代码 修改数据源设置,比如修改库为0库,此时可以看到nacos设置及时更新日记:- 2024-03-12T11:43:51.683+08:00 INFO 62914 --- [r-localhost-216] com.alibaba.nacos.common.remote.client : [6253dca3-136e-4ac1-b894-fb4c11483703_config-0] Receive server push request, request = ConfigChangeNotifyRequest, requestId = 1
- 2024-03-12T11:43:51.683+08:00 INFO 62914 --- [r-localhost-216] c.a.n.client.config.impl.ClientWorker : [6253dca3-136e-4ac1-b894-fb4c11483703_config-0] [server-push] config changed. dataId=redis-keeper-demo.yaml, group=DEFAULT_GROUP,tenant=null
- 2024-03-12T11:43:51.684+08:00 INFO 62914 --- [r-localhost-216] com.alibaba.nacos.common.remote.client : [6253dca3-136e-4ac1-b894-fb4c11483703_config-0] Ack server push request, request = ConfigChangeNotifyRequest, requestId = 1
- 2024-03-12T11:43:51.700+08:00 INFO 62914 --- [s.client.Worker] c.a.n.client.config.impl.ClientWorker : [fixed-localhost_8848] [data-received] dataId=redis-keeper-demo.yaml, group=DEFAULT_GROUP, tenant=, md5=dddd513021564c6d6322e18783ac4f2b, content=server:
- redis-keeper:
- redis:
- datasource:
- ds1:
- host: localhost
- ..., type=yaml
- 2024-03-12T11:43:51.701+08:00 INFO 62914 --- [s.client.Worker] c.a.nacos.client.config.impl.CacheData : [fixed-localhost_8848] [notify-listener] time cost=0ms in ClientWorker, dataId=redis-keeper-demo.yaml, group=DEFAULT_GROUP, md5=dddd513021564c6d6322e18783ac4f2b, listener=com.alibaba.cloud.nacos.refresh.NacosContextRefresher$1@178412d1
- 2024-03-12T11:43:51.701+08:00 INFO 62914 --- [ternal.notifier] c.a.nacos.client.config.impl.CacheData : [fixed-localhost_8848] [notify-context] dataId=redis-keeper-demo.yaml, group=DEFAULT_GROUP, md5=dddd513021564c6d6322e18783ac4f2b
- 2024-03-12T11:43:51.915+08:00 INFO 62914 --- [ternal.notifier] c.a.c.n.c.NacosConfigDataLoader : [Nacos Config] Load config[dataId=redis-keeper-demo.yaml, group=DEFAULT_GROUP] success
- 2024-03-12T11:43:51.950+08:00 INFO 62914 --- [ternal.notifier] o.s.c.e.event.RefreshEventListener : Refresh keys changed: [redis-keeper.redis.datasource.ds1.database]
- 2024-03-12T11:43:51.950+08:00 INFO 62914 --- [ternal.notifier] c.a.nacos.client.config.impl.CacheData : [fixed-localhost_8848] [notify-ok] dataId=redis-keeper-demo.yaml, group=DEFAULT_GROUP, md5=dddd513021564c6d6322e18783ac4f2b, listener=com.alibaba.cloud.nacos.refresh.NacosContextRefresher$1@178412d1 ,cost=249 millis.
复制代码 再次访问上面的方法,日记打印如下(可以看到写入的redis库是0库):- 2024-03-12T11:46:11.849+08:00 INFO 62914 --- [nio-8000-exec-4] o.c.r.k.support.DefaultRedissonTemplate : cmd:set, params:[foo, bar], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=0}]
- 2024-03-12T11:46:11.861+08:00 INFO 62914 --- [nio-8000-exec-4] o.c.r.k.support.DefaultRedissonTemplate : cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=0}]
复制代码 多数据源负载均衡
有的业务场景必要多套redis集群,来承接同一业务的流量,起到分流的作用,这时就必要从多个数据源中获取其中的某一个进行redis的读写。
负载均衡的算法现在支持两种:轮询、随机。
redis多数据源设置,示比方下:
注意:只有 redis-keeper.redis.datasources 大概 redis-keeper.redisson.datasources 设置下的多数据源支持负载均衡。- redis-keeper:
- redis:
- datasources:
- ds2:
- - host: localhost
- port: 6379
- database: 1
- password: xxx
- invoke-params-print: true
- - host: localhost
- port: 6379
- database: 2
- password: xxx
- invoke-params-print: true
- - host: localhost
- port: 6379
- database: 3
- password: xxx
- invoke-params-print: true
复制代码 轮询示比方下:
- public class AppTest {
- @Autowired
- private CacheTemplateProvider<CacheTemplate> provider;
- public void testPollTemplate() {
- String key = "foo";
- for (int i = 1; i <= 10; i++) {
- final Optional<CacheTemplate> templateOptional = provider.pollTemplate("ds2");
- if (templateOptional.isPresent()) {
- final CacheTemplate template = templateOptional.get();
- final String str = String.valueOf(i);
- template.set(key, str);
- template.get(key).ifPresent(el -> {
- assert str.equals(el);
- });
- template.del(key);
- }
- }
- }
-
- }
复制代码 日记打印结果,可以看到设置的redis数据源的出现的次数是相近的:- 2024-03-12 12:15:15.614 INFO 67575 --- [main] : cmd:set, params:[foo, 1], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=1}]
- 2024-03-12 12:15:15.621 INFO 67575 --- [main] : cmd:get, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=1}]
- 2024-03-12 12:15:15.624 INFO 67575 --- [main] : cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=1}]
- 2024-03-12 12:15:15.627 INFO 67575 --- [main] : cmd:set, params:[foo, 2], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=2}]
- 2024-03-12 12:15:15.629 INFO 67575 --- [main] : cmd:get, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=2}]
- 2024-03-12 12:15:15.635 INFO 67575 --- [main] : cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=2}]
- 2024-03-12 12:15:15.635 INFO 67575 --- [main] : cmd:set, params:[foo, 3], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:15:15.636 INFO 67575 --- [main] : cmd:get, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:15:15.638 INFO 67575 --- [main] : cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:15:15.639 INFO 67575 --- [main] : cmd:set, params:[foo, 4], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=1}]
- 2024-03-12 12:15:15.652 INFO 67575 --- [main] : cmd:get, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=1}]
- 2024-03-12 12:15:15.653 INFO 67575 --- [main] : cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=1}]
- 2024-03-12 12:15:15.654 INFO 67575 --- [main] : cmd:set, params:[foo, 5], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=2}]
- 2024-03-12 12:15:15.655 INFO 67575 --- [main] : cmd:get, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=2}]
- 2024-03-12 12:15:15.656 INFO 67575 --- [main] : cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=2}]
- 2024-03-12 12:15:15.656 INFO 67575 --- [main] : cmd:set, params:[foo, 6], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:15:15.658 INFO 67575 --- [main] : cmd:get, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:15:15.658 INFO 67575 --- [main] : cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:15:15.659 INFO 67575 --- [main] : cmd:set, params:[foo, 7], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=1}]
- 2024-03-12 12:15:15.660 INFO 67575 --- [main] : cmd:get, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=1}]
- 2024-03-12 12:15:15.661 INFO 67575 --- [main] : cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=1}]
- 2024-03-12 12:15:15.667 INFO 67575 --- [main] : cmd:set, params:[foo, 8], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=2}]
- 2024-03-12 12:15:15.669 INFO 67575 --- [main] : cmd:get, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=2}]
- 2024-03-12 12:15:15.669 INFO 67575 --- [main] : cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=2}]
- 2024-03-12 12:15:15.670 INFO 67575 --- [main] : cmd:set, params:[foo, 9], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:15:15.672 INFO 67575 --- [main] : cmd:get, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:15:15.673 INFO 67575 --- [main] : cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:15:15.674 INFO 67575 --- [main] : cmd:set, params:[foo, 10], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=1}]
- 2024-03-12 12:15:15.675 INFO 67575 --- [main] : cmd:get, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=1}]
- 2024-03-12 12:15:15.676 INFO 67575 --- [main] : cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=1}]
复制代码 随机示比方下:
- public class AppTest {
- @Autowired
- private CacheTemplateProvider<CacheTemplate> provider;
- public void testRandomTemplate() {
- String key = "foo";
- for (int i = 1; i <= 10; i++) {
- final Optional<CacheTemplate> templateOptional = provider.randomTemplate("ds2");
- if (templateOptional.isPresent()) {
- final CacheTemplate template = templateOptional.get();
- final String str = String.valueOf(i);
- template.set(key, str);
- template.get(key).ifPresent(el -> {
- assert str.equals(el);
- });
- template.del(key);
- }
- }
- }
-
- }
复制代码 日记打印结果,可以看到设置的redis数据源的出现的次数是随机的:- 2024-03-12 12:05:06.450 INFO 66199 --- [main]: cmd:set, params:[foo, 1], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:05:06.457 INFO 66199 --- [main]: cmd:get, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:05:06.461 INFO 66199 --- [main]: cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:05:06.464 INFO 66199 --- [main]: cmd:set, params:[foo, 2], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:05:06.465 INFO 66199 --- [main]: cmd:get, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:05:06.467 INFO 66199 --- [main]: cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:05:06.468 INFO 66199 --- [main]: cmd:set, params:[foo, 3], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:05:06.469 INFO 66199 --- [main]: cmd:get, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:05:06.470 INFO 66199 --- [main]: cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:05:06.472 INFO 66199 --- [main]: cmd:set, params:[foo, 4], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=2}]
- 2024-03-12 12:05:06.473 INFO 66199 --- [main]: cmd:get, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=2}]
- 2024-03-12 12:05:06.475 INFO 66199 --- [main]: cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=2}]
- 2024-03-12 12:05:06.475 INFO 66199 --- [main]: cmd:set, params:[foo, 5], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=1}]
- 2024-03-12 12:05:06.476 INFO 66199 --- [main]: cmd:get, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=1}]
- 2024-03-12 12:05:06.477 INFO 66199 --- [main]: cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=1}]
- 2024-03-12 12:05:06.478 INFO 66199 --- [main]: cmd:set, params:[foo, 6], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:05:06.479 INFO 66199 --- [main]: cmd:get, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:05:06.481 INFO 66199 --- [main]: cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:05:06.482 INFO 66199 --- [main]: cmd:set, params:[foo, 7], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:05:06.484 INFO 66199 --- [main]: cmd:get, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:05:06.485 INFO 66199 --- [main]: cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:05:06.486 INFO 66199 --- [main]: cmd:set, params:[foo, 8], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:05:06.488 INFO 66199 --- [main]: cmd:get, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:05:06.489 INFO 66199 --- [main]: cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:05:06.491 INFO 66199 --- [main]: cmd:set, params:[foo, 9], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=2}]
- 2024-03-12 12:05:06.492 INFO 66199 --- [main]: cmd:get, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=2}]
- 2024-03-12 12:05:06.493 INFO 66199 --- [main]: cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=2}]
- 2024-03-12 12:05:06.494 INFO 66199 --- [main]: cmd:set, params:[foo, 10], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:05:06.495 INFO 66199 --- [main]: cmd:get, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
- 2024-03-12 12:05:06.496 INFO 66199 --- [main]: cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=3}]
复制代码 获取指定状态标识的数据源
默认环境下数据源的状态标识是读写,还有三个可配状态标识:只读、只写、跳过。
注意:此处的状态只是一种标识,用于筛选获取指定的数据源,并不用于控制redis的读写下令。
数据源设置如下:- redis-keeper:
- redis:
- datasource:
- ds1:
- host: localhost
- port: 6379
- database: 10
- password: xxx
- status: RW
- invoke-params-print: true
复制代码 测试方法:- public void testTemplateWithStatus() {
- final Optional<CacheTemplate> templateOptional = provider.getTemplate("ds1", CacheDatasourceStatus.RW);
- if (templateOptional.isPresent()) {
- final CacheTemplate cacheTemplate = templateOptional.get();
- String key = "foo";
- String value = "bar";
- cacheTemplate.set(key, value);
- final Optional<Object> optional = cacheTemplate.get(key);
- optional.ifPresent(el -> {
- assert value.equals(el);
- });
- cacheTemplate.del(key);
- }
- assert !provider.getTemplate("ds1", CacheDatasourceStatus.SKIP).isPresent();
- }
复制代码 访问上面的方法,日记打印如下:- 2024-03-12 14:28:34.183 INFO 77230 --- [main]: cmd:set, params:[foo, bar], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=0}]
- 2024-03-12 14:28:34.190 INFO 77230 --- [main]: cmd:get, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=0}]
- 2024-03-12 14:28:34.193 INFO 77230 --- [main]: cmd:del, params:[foo], connectionInfo:[SingleServer:{Address=redis://localhost:6379,Database=0}]
复制代码 更多详细示例,请查看:https://github.com/codebaorg/redis-keeper/blob/main/README_zh.md
对比其他方案
如果这篇文章帮助到了你,欢迎评论、点赞、转发。
本文由博客一文多发平台 OpenWrite 发布!
免责声明:如果侵犯了您的权益,请联系站长,我们会及时删除侵权内容,谢谢合作!更多信息从访问主页:qidao123.com:ToB企服之家,中国第一个企服评测及商务社交产业平台。 |