does it support Tensor Parallelism and RDMA #96
Charlie-Louis
started this conversation in
General
Replies: 1 comment
-
|
So it is on the roadmap? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi, tittle says it all, I've seen some examples where even MoE models see a benefit of prompt pre processing and TTFT times go down when using multiple Macs hooked over TB5 and macOS 26.2 / RDMA also the overall inference speed up is real.
So I'm wondering if this project support MLX distributed.
Beta Was this translation helpful? Give feedback.
All reactions