German DistilBertModel raises an issue

[Svito-zar]

Environment info

• transformers version: 4.2.2
• Platform: Linux-5.4.0-65-generic-x86_64-with-debian-buster-sid
• Python version: 3.7.7
• PyTorch version (GPU?): 1.8.0.dev20201202 (False)
• Tensorflow version (GPU?): not installed (NA)
• Using GPU in script?: No
• Using distributed or parallel set-up in script?: No

Who can help

@julien-c @stefan-it @LysandreJik

Information

Model I am using: DistilBert

The problem arises when using:

• the official example scripts: (give details below)

from transformers import DistilBertTokenizer, DistilBertModel
import torch

tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-german-cased')
model = DistilBertModel.from_pretrained('distilbert-base-german-cased')

The tasks I am working on is:

• my own task or dataset:
  Word2Vec encoding

To reproduce

Steps to reproduce the behavior:

1. Simply run the code above
2. See the error message:

Traceback (most recent call last):
  File "/home/tarask/Desktop/Work/Code/Git/probabilistic-gesticulator/my_code/data_processing/annotations/encode_text.py", line 5, in <module>
    model = DistilBertModel.from_pretrained('distilbert-base-german-cased')
  File "/home/tarask/anaconda3/envs/gesture_flow/lib/python3.7/site-packages/transformers/modeling_utils.py", line 1034, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
  File "/home/tarask/anaconda3/envs/gesture_flow/lib/python3.7/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 419, in __init__
    self.embeddings = Embeddings(config)  # Embeddings
  File "/home/tarask/anaconda3/envs/gesture_flow/lib/python3.7/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 89, in __init__
    n_pos=config.max_position_embeddings, dim=config.dim, out=self.position_embeddings.weight
  File "/home/tarask/anaconda3/envs/gesture_flow/lib/python3.7/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 76, in create_sinusoidal_embeddings
    out[:, 0::2] = torch.FloatTensor(np.sin(position_enc[:, 0::2]))
RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place operation.

Expected behavior

No errors

[LysandreJik]

Hello, running the code does not raise an error:

>>> from transformers import DistilBertTokenizer, DistilBertModel
... import torch
... 
... tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-german-cased')
... model = DistilBertModel.from_pretrained('distilbert-base-german-cased')
Downloading: 100%|██████████| 240k/240k [00:00<00:00, 690kB/s]
Downloading: 100%|██████████| 464/464 [00:00<00:00, 199kB/s]
Downloading: 100%|██████████| 270M/270M [00:07<00:00, 36.6MB/s]

Please put the error in your issue, otherwise it's impossible to help you.

[Svito-zar]

Sorry, @LysandreJik , it was a copy paste error. Added it to the issue now:

Traceback (most recent call last):
  File "/home/tarask/Desktop/Work/Code/Git/probabilistic-gesticulator/my_code/data_processing/annotations/encode_text.py", line 5, in <module>
    model = DistilBertModel.from_pretrained('distilbert-base-german-cased')
  File "/home/tarask/anaconda3/envs/gesture_flow/lib/python3.7/site-packages/transformers/modeling_utils.py", line 1034, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
  File "/home/tarask/anaconda3/envs/gesture_flow/lib/python3.7/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 419, in __init__
    self.embeddings = Embeddings(config)  # Embeddings
  File "/home/tarask/anaconda3/envs/gesture_flow/lib/python3.7/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 89, in __init__
    n_pos=config.max_position_embeddings, dim=config.dim, out=self.position_embeddings.weight
  File "/home/tarask/anaconda3/envs/gesture_flow/lib/python3.7/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 76, in create_sinusoidal_embeddings
    out[:, 0::2] = torch.FloatTensor(np.sin(position_enc[:, 0::2]))
RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place operation.

[LysandreJik]

Ah, I see, thanks! This is because you're using the latest PyTorch 1.8+. We patched this issue yesterday in #9917, if you install from source you shouldn't see this error anymore.

[Svito-zar]

Ah, I see, thanks! So I should install transformers from the source, right?

[Svito-zar]

Reverting Pytorch to 1.7 also fixed this error.
Thank you so much for such prompt help @LysandreJik !

[LysandreJik]

Glad you could solve it!