Skip to content

Option to reduce connection churn #261

Description

@fydelity

Hi,

We're using Rqueue 3.2.0 with Redis Cloud (should have available 1000+ connections). Even with Lettuce pooling (20 max-active) we still hit Redis Cloud's hard connection-limit because each poll opens a dedicated connection via openPipeline(). When that happens Lettuce times-out after 10 s and the listener logs. This happen when I have high volume of enqueues (some queues can break down data and enqueue 100x new messages).

I tried to control the number of pool connections with, but no luck. I wonder if there is a way to reuse an open channel?

rqueue:
  redis:
    database: 0
    timeout: 30s
    lettuce:
      pool:
        max-active: 20
        max-idle: 10
        min-idle: 5
        max-wait: 10s

Stack trace

org.springframework.data.redis.RedisConnectionFailureException: Unable to connect to Redis
	at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$ExceptionTranslatingConnectionProvider.translateException(LettuceConnectionFactory.java:1847) ~[spring-data-redis-3.3.1.jar!/:3.3.1]
	at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$ExceptionTranslatingConnectionProvider.getConnection(LettuceConnectionFactory.java:1778) ~[spring-data-redis-3.3.1.jar!/:3.3.1]
	at org.springframework.data.redis.connection.lettuce.LettuceConnection.doGetAsyncDedicatedConnection(LettuceConnection.java:968) ~[spring-data-redis-3.3.1.jar!/:3.3.1]
	at org.springframework.data.redis.connection.lettuce.LettuceConnection.getOrCreateDedicatedConnection(LettuceConnection.java:1029) ~[spring-data-redis-3.3.1.jar!/:3.3.1]
	at org.springframework.data.redis.connection.lettuce.LettuceConnection.openPipeline(LettuceConnection.java:568) ~[spring-data-redis-3.3.1.jar!/:3.3.1]
	at jdk.internal.reflect.GeneratedMethodAccessor174.invoke(Unknown Source) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
	at org.springframework.data.redis.core.CloseSuppressingInvocationHandler.invoke(CloseSuppressingInvocationHandler.java:61) ~[spring-data-redis-3.3.1.jar!/:3.3.1]
	at jdk.proxy3/jdk.proxy3.$Proxy311.openPipeline(Unknown Source) ~[na:na]
	at org.springframework.data.redis.core.RedisTemplate.lambda$executePipelined$1(RedisTemplate.java:472) ~[spring-data-redis-3.3.1.jar!/:3.3.1]

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions