Human prompts for dataset generation.

[winglian]

We instruct the user model with carefully designed prompts to mimic human user behavior

@ningding97 could you provide these prompts you used? Thanks!

[ningding97]

We instruct the user model with carefully designed prompts to mimic human user behavior

@ningding97 could you provide these prompts you used? Thanks!

Hi, thanks for your interest in our work. Below are prompt examples we use in the three sectors respectively to instruct the model to generate a new turn of response as a human user. In our experiences, the key point is to avoid "role exchange", thus the prompt is appended to the dialogue history. We provide three example prompts to generate a new turn of response in three sectors, and we also sample different prompts to mimic different kinds of users (for example, the user can be very professional or casual). Hope it helps!

• Sector 1

Above is a conversation between a user and an intelligent assistant. Now suppose you are the user, say something to continue the conversation based on given context. Make the response short and the language casual.

• Sector 2

Above is a conversation between a user and an intelligent assistant. Now suppose you are the user, generate response according to the generated material to continue the conversation. Bear in mind your major request is to ask the assistant to generate some material. So you can ask the assistant either to make it more detailed, add more related information, or any other request to improve the generated material. Be creative and diverse in your request. Make the response short and the language casual.

• Sector 3

Above is a conversation between a user and an intelligent assistant. Now suppose you are the user, say something to continue the conversation based on given context. Bear in mind your major request is to ask the assistant to do something based on a given text material. So it is better you ask question or give instruction that is accomplishable with only the given text available. Make the response short and the language casual.

To provide more details about the data generation process, we will take Sector 1 as an example and provide all the codes, prompts and meta-information.

[renatz]

hi, I attempted to reproduce its procedure but the human often does not raise questions but evaluates the assistants' answers.
what the prompt is appended to the dialogue history mean?
do you append the system prompt to every request's content?

[yulinchen99]

Yes, we put everything under the role "user" for openai chatgpt api. So for example, if you already have the first round of conversation as below

User: user utterance 1
Assistant: assistant response 1

Now you want to generate a new user utterance, then your input message is

{
   "role": "user", 
   "content": """User: user utterance 1
Assistant: assistant response 1
Above is a conversation between a user and an intelligent assistant. Now suppose you are the user, say something to continue the conversation based on given context. Make the response short and the language casual.
"""
}