spring data redis master slave config

Question

Following is my jedis config

@Bean
public JedisConnectionFactory getJedisConnectionFactory() {
    JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory();
    jedisConnectionFactory.setUsePool(true);
    return jedisConnectionFactory;
}

@Bean
public RedisTemplate<String, Object> getRedisTemplate() {
    RedisTemplate<String, Object> redisTemplate = new RedisTemplate<String, Object>();
    redisTemplate.setConnectionFactory(getJedisConnectionFactory());
    return redisTemplate;
}

This config works well when I have single server. What I want to do is have 1 redis master and multiple redis slaves. Per redis documentation, read should happen from slaves, and write should happen from master. How do I change above configuration to use master for writing and slave for reading?

Lets say that my master is at 192.168.10.10 and slave is at localhost.

Thanks!

Solution 1

At this time there's no configuration option in Spring Data Redis that would enable the desired behaviour. Nor does Jedis iteself offer support for this kind of scenario (see jedis #458). RedisConnection requests a connection from the factory when executing operations. At this point the usage purpose of the resource requested it is not clear as the command could be r, w or rw.

One potential solution would be a custom RedisConnectionFactory capable of providing the connection - to one of the slaves you have - in case of a readonly command being executed.

SlaveAwareJedisConnectionFactory factory = new SlaveAwareJedisConnectionFactory();
factory.afterPropertiesSet();

RedisConnection connection = factory.getConnection();

// writes to master
connection.set("foo".getBytes(), "bar".getBytes());

// reads from slave
connection.get("foo".getBytes());

/**
 * SlaveAwareJedisConnectionFactory wraps JedisConnection with a proy that delegates readonly commands to slaves.
 */
class SlaveAwareJedisConnectionFactory extends JedisConnectionFactory {

  /**
    * Get a proxied connection to Redis capable of sending
    * readonly commands to a slave node
    */
  public JedisConnection getConnection() {

    JedisConnection c = super.getConnection();

    ProxyFactory proxyFactory = new ProxyFactory(c);
    proxyFactory.addAdvice(new ConnectionSplittingInterceptor(this));
    proxyFactory.setProxyTargetClass(true);

    return JedisConnection.class.cast(proxyFactory.getProxy());
  };

  /**
   * This one will get the connection to one of the slaves to read from there
   * 
   * @return
   */
  public RedisConnection getSlaveConnection() {

    //TODO: find the an available slave serving the data
    return new JedisConnection(new Jedis("your slave host lookup here"));
  }

  static class ConnectionSplittingInterceptor implements MethodInterceptor,
      org.springframework.cglib.proxy.MethodInterceptor {

    private final SlaveAwareJedisConnectionFactory factory;

    public ConnectionSplittingInterceptor(SlaveAwareJedisConnectionFactory factory) {
      this.factory = factory;
    }

    @Override
    public Object intercept(Object obj, Method method, Object[] args, MethodProxy proxy) throws Throwable {

      RedisCommand commandToExecute = RedisCommand.failsafeCommandLookup(method.getName());

      if (!commandToExecute.isReadonly()) {
        return invoke(method, obj, args);
      }

      RedisConnection connection = factory.getSlaveConnection();

      try {
        return invoke(method, connection, args);
      } finally {
        // properly close the connection after executing command
        if (!connection.isClosed()) {
          connection.close();
        }
      }
    }

    private Object invoke(Method method, Object target, Object[] args) throws Throwable {

      try {
        return method.invoke(target, args);
      } catch (InvocationTargetException e) {
        throw e.getCause();
      }
    }

    @Override
    public Object invoke(MethodInvocation invocation) throws Throwable {
      return intercept(invocation.getThis(), invocation.getMethod(), invocation.getArguments(), null);
    }
  }
}

There solution above holds serveral issues. Eg. MULTI EXEC blocks in you application might no longer work as expected since commands now potentially get piped somewhere you do not want them to be. So maybe it would also make sense to have multiple RedisTemplates for dedicated read, write purpose.

Solution 2

You should use Redis Sentinel to keep master/slave configuration of redis...you can use below to connect to sentinel pool-

@Bean
public RedisConnectionFactory jedisConnectionFactory() {
RedisSentinelConfiguration sentinelConfig = new RedisSentinelConfiguration() .master("mymaster")
    .sentinel("127.0.0.1", 26379) .sentinel("127.0.0.1", 26380);
return new JedisConnectionFactory(sentinelConfig);
}

reference: Spring Sentinel support

Solution 3

You want write-to-master-read-from-slave

This is how to configure it with a sentinal cluster and spring-boot

spring:
  data:
    redis:
      sentinel: 
        master: mymaster
        nodes: my.sentinel.hostname1:26379,my.sentinel.hostname2:26379
      port: 6379

And spring config

@Configuration
public class RedisDatasourceConfig {

  @Bean
  public LettuceClientConfigurationBuilderCustomizer lettuceClientConfigurationBuilderCustomizer() {
    return p -> p.readFrom(SLAVE_PREFERRED);
  }
}

Comments

[hrishikeshp19]

how about two two jedis connection factories and two redis templates, one for read and the other for write? Will that work (in theory) ?

[dan carter]

using Redis Sentinel support alone, you get fail-over when sentinal promotes a new node to master, but you don't get load-balancing, everything still goes to master.