[Failing Test]: pytorch_inference_it_test for MaskedLM is failing to load state dict

[riteshghorse]

What happened?

Looks like there is some problem with loading the state dict for BertForMaskedLM in this example with integration test.

      if len(error_msgs) > 0:
>           raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
                               self.__class__.__name__, "\n\t".join(error_msgs)))
�[1m�[31mE           RuntimeError: Error(s) in loading state_dict for BertForMaskedLM:�[0m
�[1m�[31mE           	Unexpected key(s) in state_dict: "bert.embeddings.position_ids".  [while running 'PyTorchRunInference/BeamML_RunInference']�

https://ci-beam.apache.org/job/beam_PostCommit_Python38_PR/755/consoleText

Issue Failure

Failure: Test is continually failing

Issue Priority

Priority: 2 (backlog / disabled test but we think the product is healthy)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[riteshghorse]

I tried locally with the current state_dict_path apache-beam-ml/models/huggingface.BertForMaskedLM.bert-base-uncased.pth and it shows the same problem. Saved a new instance for the same model and it works fine.

I'm going to replace this file on GCS

[riteshghorse]

tests are passing now https://ci-beam.apache.org/job/beam_PostCommit_Python38_PR/756/consoleText

[riteshghorse]

huggingface/transformers#24505 explains why the old saved file didn't work