-
Notifications
You must be signed in to change notification settings - Fork 74
[Docs] Clarify get-events API usage #320
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
9f8c965
c162784
ea4afd5
066f724
405e1b7
304ec12
f0f6b8c
10cd149
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -141,8 +141,8 @@ from llmengine import FineTune | |
|
|
||
| response = FineTune.create( | ||
| model="llama-2-7b", | ||
| training_file="file-7DLVeLdN2Ty4M2m", | ||
| validation_file="file-ezSRtpgKQyItI26", | ||
| training_file="file-AbCDeLdN2Ty4M2m", | ||
| validation_file="file-ezSRpgtKQyItI26", | ||
| ) | ||
|
|
||
| print(response.json()) | ||
|
|
@@ -152,7 +152,35 @@ See the [Model Zoo](../../model_zoo) to see which models have fine-tuning suppor | |
|
|
||
| See [Integrations](../integrations.md) to see how to track fine-tuning metrics. | ||
|
|
||
| Once the fine-tune is launched, you can also [get the status of your fine-tune](../../api/python_client/#llmengine.fine_tuning.FineTune.get). You can also [list events that your fine-tune produces](../../api/python_client/#llmengine.fine_tuning.FineTune.get_events). | ||
| ## Monitoring the fine-tune | ||
|
|
||
| Once the fine-tune is launched, you can also [get the status of your fine-tune](../../api/python_client/#llmengine.fine_tuning.FineTune.get). | ||
| You can also [list events that your fine-tune produces](../../api/python_client/#llmengine.fine_tuning.FineTune.get_events). | ||
| ```python | ||
| from llmengine import FineTune | ||
|
|
||
| fine_tune_id = "ft-cabcdefghi1234567890" | ||
| fine_tune = FineTune.get(fine_tune_id) | ||
| print(fine_tune.status) # BatchJobStatus.RUNNING | ||
| print(fine_tune.fine_tuned_model) # "llama-2-7b.700101-000000 | ||
|
|
||
| fine_tune_events = FineTune.get_events(fine_tune_id) | ||
| for event in fine_tune_events.events: | ||
| print(event) | ||
| # Prints something like: | ||
| # timestamp=1697590000.0 message="{'loss': 12.345, 'learning_rate': 0.0, 'epoch': 0.97}" level='info' | ||
| # timestamp=1697590000.0 message="{'eval_loss': 23.456, 'eval_runtime': 19.876, 'eval_samples_per_second': 4.9, 'eval_steps_per_second': 4.9, 'epoch': 0.97}" level='info' | ||
| # timestamp=1697590020.0 message="{'train_runtime': 421.234, 'train_samples_per_second': 2.042, 'train_steps_per_second': 0.042, 'total_flos': 123.45, 'train_loss': 34.567, 'epoch': 0.97}" level='info' | ||
|
|
||
|
|
||
| ``` | ||
|
|
||
| The status of your fine-tune will give a high-level overview of the fine-tune's progress. | ||
| The events of your fine-tune will give more detail, such as the training loss and validation loss at each epoch, | ||
| as well as any errors that may have occurred. If you encounter any errors with your fine-tune, | ||
| the events are a good place to start debugging. For example, if you see `Unable to read training or validation dataset`, | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we have a separate error for malformed files, e.g. files that don't have Would it make sense to put these in a table?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Discussed offline: |
||
| you may need to make your files accessible to LLM Engine. If you see `Invalid value received for lora parameter 'lora_alpha'!`, | ||
| you should [check that your hyperparameters are valid](../../api/python_client/#llmengine.fine_tuning.FineTune.create). | ||
|
|
||
| ## Making inference calls to your fine-tune | ||
|
|
||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.