Disable static cache on certain MoE models#39108
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
run-slow: deepseek_v3 |
|
This comment contains run-slow, running the specified jobs: models: ['models/deepseek_v3'] |
|
|
||
| @auto_docstring | ||
| class DeepseekV3Model(DeepseekV3PreTrainedModel): | ||
| _supports_static_cache = False |
There was a problem hiding this comment.
we only need the one in class DeepseekV3PreTrainedModel(PreTrainedModel) as DeepseekV3Model here is just a subclass of it.
| _tied_weights_keys = ["lm_head.weight"] | ||
| _tp_plan = {"lm_head": "colwise_rep"} | ||
| _pp_plan = {"lm_head": (["hidden_states"], ["logits"])} | ||
| _supports_static_cache = False |
|
|
||
|
|
||
| class DeepseekV3Model(LlamaModel): | ||
| _supports_static_cache = False |
There was a problem hiding this comment.
Wait this isn't the same is it?
There was a problem hiding this comment.
hm, but do we still need this one, if the one in modeling_deepseek_v3 is removed?
There was a problem hiding this comment.
i mean, adding this here means we want it to appear for
class DeepseekV3Model(DeepseekV3PreTrainedModel):
but we don't need it there, so we can remove this one here?
There was a problem hiding this comment.
I'm sure you're right hehe.
Haven't yet had the time to fully understand exactly how modular works 😅
There was a problem hiding this comment.
(me neither just my raw thoughts 😅 )
ydshieh
left a comment
There was a problem hiding this comment.
OK, but a nit that is good to change 🙏
Thanks a lot!
|
@ydshieh could you take a look at the results from the model tests I ran? |
|
we should skip the integration test
that one doesn't use the flag to control the test. It's integration test and usually not control by the flag. |
|
[For maintainers] Suggested jobs to run (before merge) run-slow: deepseek_v3, dots1 |
662d1e0 to
ee85f48
Compare
|
run-slow: deepseek_v3, dots1 |
|
This comment contains run-slow, running the specified jobs: models: ['models/deepseek_v3', 'models/dots1'] |
|
run-slow: deepseek_v3, dots1 |
|
This comment contains run-slow, running the specified jobs: models: ['models/deepseek_v3', 'models/dots1'] |
No description provided.