Proposal: Integrating a compression-based acceleration method into xDiT

[Cobalt-27]

Hi👋

We’ve been working on a small project called CompactFusion (repo), built directly on top of xDiT.

Our motivation came from a simple observation: diffusion models show strong temporal redundancy. During parallel inference, we repeatedly transmit large amounts of near-duplicate activations between GPUs.

Our insight is that if we remove this redundant information, we can dramatically reduce communication volume without hurting generation quality. CompactFusion achieves this by transmitting only the compressed residuals (the actual changes between steps), together with a lightweight error-feedback mechanism.

This simple approach cuts communication by 8–16× and delivers significant speedups on PCIe or Ethernet setups, all while maintaining fidelity. In many settings, this compression-based approach yields better generation quality than previous overlap-based methods, since it avoids using stale activations during inference.

The work has just been accepted to NeurIPS 2025 (paper). If the team is open to it, we’d love to have this work integrated into xDiT as an optional module.

Thanks again for making xDiT such an outstanding open-source foundation.

[jcaraban]

Hi @Cobalt-27 , impressive work! as a (fresh) xDiT maintainer I'd be happy to review and support the integration, if @feifeibear or others don't oppose it. I'd say the main requirement is for CompactFusion to be orthogonal and optional.

Would you be able to rebase your changes on latest xDiT and submit a PR for testing and review? We've been trying to resync with Diffusers and keep xDiT from going stale, so your work might have diverted quite a bit from TIP

[Cobalt-27]

Thanks @jcaraban ! Really appreciate the feedback and your interest in supporting the integration.

We’ll wait to hear what the other maintainers think. Personally, I believe this approach fits well with xDiT’s massive-parallel, especially given its quality and performance benefits.

If it’s okay to move forward, we’ll keep it optional and orthogonal. Since xDiT is a large and evolving framework, it might take us some time to adapt and test everything properly; As we want to make sure it’s robust and cleanly integrated.

[feifeibear]

Sorry for the late reply — I had a quick discussion with the CompactFusion author on WeChat.
First of all, I’m very happy to see CompactFusion being integrated into xDiT.

That said, I also think it could work well as a standalone project that simply imports xfuser.

It really depends on whether you have the human bandwidth to make CompactFusion sufficiently general within xfuser without affecting other parts of xDiT.

[Cobalt-27]

Thanks for the reply and clarification @feifeibear :)

I understand the concern. Making CompactFusion stable inside xDiT would take quite some time and effort, and it’s important to do it carefully.

I’ll take some time to carefully evaluate whether there’s enough capacity to make it part of xDiT, or if it would be better to keep it as a standalone project for now. I’ll keep this in mind and think it through. If there’s any further progress, I’ll update here.

(Just to confirm, does vefuser mean xfuser?)

[feifeibear]

sorry, vefuser is typo. It should be xfuser