Search results

I don't know if I should input attention_mask in the SFT process

Status: Open.

#40 In syncdoth/RetNet;

· wac81 opened on Mar 22, 2024

Parallel inference is thought to be faster than recurrent inference, but it turns out that it is not in play file

Status: Open.

#39 In syncdoth/RetNet;

· wac81 opened on Mar 22, 2024

contains the question when inference

Status: Open.

#38 In syncdoth/RetNet;

· wac81 opened on Mar 22, 2024

Integration with transformers library

Status: Open.

#37 In syncdoth/RetNet;

· kiucho opened on Mar 12, 2024

HuggingFace checkpoint

Status: Open.

#36 In syncdoth/RetNet;

· xtwigs opened on Jan 2, 2024

how to load model with device_map="auto"

Status: Open.

#35 In syncdoth/RetNet;

· wac81 opened on Dec 27, 2023

The number of parameters does not match the setting in paper

Status: Open.

#34 In syncdoth/RetNet;

· ziHoHe opened on Dec 27, 2023

Can you support streaming when generating?

Status: Open.

#32 In syncdoth/RetNet;

· wac81 opened on Dec 27, 2023

Info/Documentation on chunkwise training

Status: Open.

#30 In syncdoth/RetNet;

· pkpro opened on Nov 27, 2023

Added description for torch.compile

Status: Open.

#29 In syncdoth/RetNet;

· ce-lery opened on Nov 9, 2023

Add Hidden Size for DeepSpeed integration

Status: Open.

#23 In syncdoth/RetNet;

· infosechoudini opened on Oct 5, 2023

How to use multiple GPUs for model parallel training

Status: Open.

#16 In syncdoth/RetNet;

· zhihui-shao opened on Sep 4, 2023