Provide clearer instructions on how to specify target language.#38786
Open
khof312 wants to merge 2 commits into
Open
Provide clearer instructions on how to specify target language.#38786khof312 wants to merge 2 commits into
khof312 wants to merge 2 commits into
Conversation
In the current documentation, I believe there is an error in the sample code. While the language token of the input is specified as `<2pt>` (Portuguese), the returned translation is in Italian, so actually the language should be `<2it>`. I didn't initially understand that this was a token for the target language, so I have added an explanation of this to the documentation as well as instructions on how to get the available languages based on the [code](https://huggingface.co/spaces/jbochi/madlad400-3b-mt/blob/main/app.py) in Juarez Bochi's demo space. It would be good to check that I have retrieved the correct range of tokens (4-459) and not omitted any (there were some additional tokens that looked to refer to scripts instead of languages, but I was not sure). If these changes are approved, then they should also be perpetuated to the documentation of the different model checkpoints from google and jbochi.
Member
|
Hi @khof312, "Eu amo pizza!" is Portuguese, not Italian. The reference to pizza is deceptive, I realize, but in Italian this would be "Amo la pizza!", or maybe "Io amo la pizza!". |
Fix the mistake on misidentifying the language.
Author
|
Oops, you are right about the language, my mistake. In this case the <2pt> language code (line 56) need not be changed, but is it still worth adding documentation of how to find the respective language codes? (I have changed the associated file accordingly to fix my error). |
Member
|
cc @stevhliu for the docs question! |
stevhliu
approved these changes
Jul 21, 2025
stevhliu
left a comment
Member
There was a problem hiding this comment.
Yeah I think adding docs for how to find the language code is still useful!
| ['Eu amo pizza!'] | ||
| ``` | ||
|
|
||
| Note that in the input sentence, "`<2pt>`" is a token determining the target language of the translation. To see all supported language codes and their corresponding tokens: |
Member
There was a problem hiding this comment.
I would add this sentence before the code example so users know what it is beforehand
Suggested change
| Note that in the input sentence, "`<2pt>`" is a token determining the target language of the translation. To see all supported language codes and their corresponding tokens: | |
| Include the target language token in the input sentence. `<2pt>` is the token for Portuguese. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In the current documentation, I believe there is an error in the sample code. While the language token of the input is specified as
<2pt>(Portuguese), the returned translation is in Italian, so actually the language should be<2it>.I didn't initially understand that this was a token for the target language, so I have added an explanation of this to the documentation as well as instructions on how to get the available languages based on the code in Juarez Bochi's demo space.
It would be good to check that I have retrieved the correct range of tokens (4-459) and not omitted any (there were some additional tokens that looked to refer to scripts instead of languages, but I was not sure).
If these changes are approved, then they should also be perpetuated to the documentation of the different model checkpoints from google and jbochi.