Checkpoints tab "Run Inference" button to run inference pipeline from

[bghira]

This pull request introduces a comprehensive set of enhancements focused on expanding checkpoint inference capabilities, improving validation and inference settings handling, and increasing extensibility for both backend and UI state management. The main themes are: adding a robust checkpoint inference API, improving validation logic and flexibility, and introducing mechanisms for handling custom process commands and UI state persistence.

Checkpoint Inference API and UI Enhancements:

• Adds new REST API endpoints for checkpoint inference, including session management, prompt sources, media retrieval, and history operations in checkpoints.py. New request/response models and error handling for inference are introduced. [1] [2] [3] [4]
• Introduces UI state endpoints and persistence for checkpoint inference settings, allowing users to save and retrieve their preferred inference configuration in webui_state.py and webui_state.py service. [1] [2]

Validation and Inference Logic Improvements:

• Refactors validation resolution parsing to support multiple resolutions and model flavors, improving error handling and flexibility in validation.py.
• Updates the validation preview callback logic to support both webhook and direct callback invocation, and ensures proper disabling if neither is configured. [1] [2] [3]
• Adds a method to prepare models for inference mode, ensuring proper registration and evaluation state in trainer.py.
• Allows preservation of text encoders during validation initialization, providing finer control over validation state in trainer.py. [1] [2]

Process Command Extensibility:

• Introduces a custom command queue and handler to allow the process keeper to receive and expose commands not handled by the default training bridge, enhancing extensibility for future features in process_keeper.py. [1] [2] [3] [4] [5]

Resource Management and Safety:

• Prevents starting a new training job if a checkpoint inference session is active, ensuring exclusive access to accelerator resources in training_service.py.

These changes collectively provide a more robust, flexible, and user-friendly experience for checkpoint inference, validation, and UI state management, while laying the groundwork for future extensibility and operational safety.

---

References:

• [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19]

[bghira]