Skip to content

Run pipeline in current thread if all the keys on same node #4149

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
May 7, 2025

Conversation

xrayw
Copy link
Contributor

@xrayw xrayw commented Apr 23, 2025

fix #4148

@xrayw xrayw changed the title Perf pipeline sync threadpool and fix the connection leak Perf pipeline sync threadpool if all the keys on same node Apr 25, 2025
Copy link
Collaborator

@ggivo ggivo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
I think we should also add dedicated tests to verify MultiNodePipeline with single command to a node and MultiNodePipeline multiple commands to the same node are processed successfully

@xrayw
Copy link
Contributor Author

xrayw commented Apr 28, 2025

@ggivo May i check should i add the test to ClusterPipeliningTest class?

@ggivo
Copy link
Collaborator

ggivo commented Apr 29, 2025

@ggivo May i check should i add the test to ClusterPipeliningTest class?

Yes, I would suggest adding them there.

@xrayw xrayw changed the title Perf pipeline sync threadpool if all the keys on same node Run pipeline in current thread if all the keys on same node Apr 29, 2025
@xrayw
Copy link
Contributor Author

xrayw commented Apr 29, 2025

@ggivo I add a test, can you help me to check it. cause i don't have the cluster environment. thank you very much

@ggivo
Copy link
Collaborator

ggivo commented Apr 30, 2025

@ggivo I add a test, can you help me to check it. cause i don't have the cluster environment. thank you very much

@xrayw
Hey, thanks for working on it! If you have Docker, you should be able to bootstrap a test environment locally using
make start-test-env
This will also include the Redis Cluster used by other tests.

details can be found in contribution guideline

@xrayw
Copy link
Contributor Author

xrayw commented Apr 30, 2025

@ggivo test passed now, if the test code is correct.
i can't run make start-test-env on Colima ( MacOS ), So i test it by make test.

@ggivo
Copy link
Collaborator

ggivo commented Apr 30, 2025

@xrayw
Thanks again, will take a final look and provide an update.

@ggivo test passed now, if the test code is correct. i can't run make start-test-env on Colima ( MacOS ), So i test it by make test.

make test will do in this case but we are planning to replace it with the Dockerised environment at some point and rely only on containers. I am on Sonoma ( MacOS ), and it works for me. Curious what is the error you are getting so we can try to address it.

@ggivo ggivo added this to the 6.1.0 milestone Apr 30, 2025
@xrayw
Copy link
Contributor Author

xrayw commented Apr 30, 2025

image

Error response from daemon: invalid reference format
Started test environment with Redis version 8.0.

Although it says it is started, but, actually there is no redis-server started.

Copy link
Collaborator

@ggivo ggivo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ggivo ggivo merged commit eb34e05 into redis:master May 7, 2025
11 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[MultiNodePipelineBase] perf cluster pipeline if all the keys on same node
2 participants