Skip to content

bug: when loading models in huggingface, we should device_map= instead of to(device) #581

@nrfulton

Description

@nrfulton

Marked as priority because we get OOMs on some hardware (including colab)

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions