What if a health check fails?

[0xWJ]

I encountered a health check fail. My assumption is when a health check fails, it should abandon the current connection and make a new connection to the redis server.

But turns out it will just throw the failure? Am I missing something or this is by design?

[petyaslavova]

Hi @0xWJ , do you have retries enabled for your Redis client or are you using the default config? One more important details - what client are you using - sync, async, standalone, cluster?

[0xWJ]

Thanks.
I think both sync/async have the same problem. Currently I am using sync.

It is very troublesome to make it work in a not-very-reliable network.

At first, I just use

Redis.from_url(
            redis_uri,
        )

And it hangs sometimes (redis op never returns).

Then I update it to

Redis.from_url(
            redis_uri,
            socket_keepalive_options={socket.TCP_KEEPIDLE: 2, socket.TCP_KEEPINTVL: 1, socket.TCP_KEEPCNT: 2}
        )

But it still hangs sometimes with less chance and raise connection error (timeout or broken pipeline). So I then change it to

Redis.from_url(
            redis_uri,
            socket_connect_timeout=120,
            socket_timeout=socket_timeout,
            socket_keepalive=True,
            socket_keepalive_options={socket.TCP_KEEPIDLE: 2, socket.TCP_KEEPINTVL: 1, socket.TCP_KEEPCNT: 2}
        )

This time no hang happens (at less until now), but still raise error sometimes.

So I try health check

Redis.from_url(
            redis_uri,
            socket_connect_timeout=120,
            socket_timeout=socket_timeout,
            socket_keepalive=True,
            health_check_interval=30,
            socket_keepalive_options={socket.TCP_KEEPIDLE: 2, socket.TCP_KEEPINTVL: 1, socket.TCP_KEEPCNT: 2}
        )

Looks it can't mitigate the problem.

You mean health check only work with retry? I am wondering do you guys have recommended options that work best in a bad network?

[petyaslavova]

@0xWJ in case any of the calls during the communication with Redis fails - an error will be raised, and if you don't have a retry configuration applied - nothing will be retried - you will just see the raised error.
Can you please try to provide a retry configuration when you initialize the client and see if this helps?

[0xWJ]

No problem. I will do it. Hope you can point out how the health check should be used. in your document.

[petyaslavova]

Hi @0xWJ, did you have some time to test with the retry config enabled? One more thing you can try now is to run with the latest release - it has retries enabled by default..

[0xWJ]

Thank you for you quick response. I will try it.