BF luis:convert errors in multilingual LU files

[bobokids]

Run "bf luis:convert -i calendar.lu --culture=de-de", got following error:
line 1211:20 token recognition error at: 'ü'
line 1231:19 token recognition error at: 'ö'
line 1251:21 token recognition error at: 'ß'
line 1255:21 token recognition error at: 'ß'
line 1257:21 token recognition error at: 'ß'
line 1287:22 token recognition error at: 'ä'
[ERROR] line 1174:0 - line 1174:20: PREBUILT entity 'personName' is not available for the requested locale 'de-de'

Two issues when handling de-de lu file:

1. cannot recognize chars like 'ü' in list entity
2. don't support personName for locales other than en-us and zh-cn

The LU file is here https://github.com/microsoft/botframework-solutions/blob/master/skills/csharp/calendarskill/Deployment/Resources/LU/de-de/Calendar.lu

[bobokids]

Also repro in ludown v1.4. Not repro in ludown v1.3.2.

[vishwacsena]

@feich-ms appears to be related to parser issues with ANTLR. Can you help take a look and PR a fix?

Once you have the core parser issue addressed, assign the issue to me for the prebuilt issue.

[feich-ms]

Will take a look and prepare a fix.

[ssss141414]

Token recognition error has been fixed in bf cli 4.7. But
[ERROR] line 1174:0 - line 1174:20: PREBUILT entity 'personName' is not available for the requested locale 'de-de'
still remains.

[feich-ms]

@ssss141414 Yes, the first token issue should be fixed by #358 which was merged one month ago. Updating package to 4.7 resolved this issue. Assign to @vishwacsena for the prebuilt issue

[vishwacsena]

Fixed via #358. Verified that this .lu file parses E2E without any issues with personName prebuilt entity. https://github.com/microsoft/botframework-solutions/blob/master/skills/csharp/calendarskill/Deployment/Resources/LU/de-de/Calendar.lu

[GitDruid]

Hi @vishwacsena
I still get these errors when trying to deploy for IT and DE:

Converting it-it Calendar LU file ...[ERROR] line 1858:0 - line 1858:20: PREBUILT entity 'personName' is not available for the requested locale 'it-it'
Converting de-de Calendar LU file ...[ERROR] line 1880:0 - line 1880:20: PREBUILT entity 'personName' is not available for the requested locale 'de-de'

Using:

• latest version of the botframework-skills code downloaded on 3/7/2020
• botframework-cli version 4.9.1

Should I file a new issue or will you reopen this one?

Ciao,
Alessandro.

[GitDruid]

Fixed via #358. Verified that this .lu file parses E2E without any issues with personName prebuilt entity. https://github.com/microsoft/botframework-solutions/blob/master/skills/csharp/calendarskill/Deployment/Resources/LU/de-de/Calendar.lu

Please note that this link is no more valid.