llama : bump max layers from 256 to 512

[ggerganov]

fix #8528

• I have read the contributing guidelines
• Self-reported review complexity:
  • Low
  • Medium
  • High

[slaren]

I would suggest moving the assert to the case where the key is actually an array only. That way it wouldn't affect other models. The assert should also be replaced with an exception instead to avoid crashing on invalid models. Ideally it should be a dynamic container instead of a fixed array, but that requires more changes.

[ggerganov]

I would suggest moving the assert to the case where the key is actually an array only. That way it wouldn't affect other models.

Wouldn't this loop potentially overflow if we check just array keys?

https://github.com/ggerganov/llama.cpp/blob/b268edf87c3bb996feb88e2d8b2c85b38bff6f36/src/llama.cpp#L4061-L4063

[slaren]

Yes, I assumed that it would only set the first element in that case. From what I can tell, only OpenELM ever uses more than the first element.