Implementing GaussianBlendshapes in SparkJS

[yosun]

Summary

I’m exploring whether Gaussian Splat blendshapes can be implemented efficiently in SparkJS.
The goal is to support real-time deformation of splat fields (similar to mesh-based morph targets), but applied to 3D Gaussian Splat representations.

Use case

We’re prototyping interactive web apps with expressive Gaussian Splatting content (character faces, poses, object morphing).
SparkJS is attractive because of its browser runtime, reactive data model, and renderer extensibility, and we want to understand how best to map Gaussian Splat attributes into Spark’s scene graph.

Prior work

• GaussianBlendshapes (zjumsj)
  https://github.com/zjumsj/GaussianBlendshapes
  Uses linear combination of splat basis parameters (mean, covariance, opacity).
• 3D Gaussian Splatting for Real-Time Radiance Field Rendering (Kerbl et al., 2023)
  Base splat representation and tile-based rasterization pipeline.

Core question

What is the recommended representation + update pathway for splat attributes in SparkJS if the splats are to be morphed per-frame?

---

Implementation idea (tentative)

1. Data model

We’re considering a per-splat structure (N = splat count), e.g.:

Float32Array[N] = [
  x, y, z,                // mean
  qx, qy, qz, qw,         // orientation quaternion
  sx, sy, sz,             // scale / sigma
  opacity
]

Blendshapes = delta arrays with identical layout, weighted via linear combination:

S_base + Σ (w_i * S_delta_i)

2. Pipeline candidates

A. GPU-side
Upload base + blend deltas to a buffer and compute blended attributes in the shader.
Pros: no large CPU->GPU copies.
Cons: might require custom render node or compute step.

B. CPU-side
Blend arrays in JS, then upload full updated buffer each frame.
Pros: simplest integration.
Cons: likely too slow for >30–50k splats.

---

Questions

1. Representation

• Does SparkJS prefer splat attributes as GPU buffers (SSBOs / textures) or CPU-side typed arrays?
• Is there a recommended layout for custom geometry / point-draw pipelines?

2. Interpolation

• Does SparkJS have an existing morph target / attribute blending abstraction, or should it be implemented in a custom renderer?
• For attributes like rotation + covariance, is there guidance on non-linear interpolation (quaternion blend, log-space covariance) vs. plain linear blends?

3. Performance + limits

• Practical upper bound for GPU-side splat morphing?
• Performance implications of buffer updates vs per-instance uniforms vs texture-backed attribute tables?

4. Integration

• Are there examples of procedural geometry or custom render nodes we can extend?
• Are WebGPU APIs exposed in a way that supports compute-pass blending before render?

---

Desired outcome

We’d like to:

1. Import a base Gaussian Splat representation.
2. Load N blend targets (array deltas).
3. Apply continuous blend weights in real time (0–1) to morph the splat field.
4. Keep performance viable for tens of thousands of splats in the browser.

Any insight into best practices, preferred APIs, or SparkJS architectural direction would be greatly appreciated.

Thanks!