Fix #291: Add constraint checking to initial designs by apaleyes-bot · Pull Request #488 · EmuKit/emukit

apaleyes-bot · 2026-07-03T20:55:04Z

Summary

Implement rejection sampling in initial designs to ensure all generated points respect constraints.

Problem

Initial designs (Random, Sobol, Latin hypercube) did not check whether generated points satisfy defined constraints, leading to invalid samples being returned to users.

Solution

Add constraint checking to InitialDesignBase.get_samples()
Implement rejection sampling: if any point violates a constraint, regenerate entire batch
Preserve space-filling properties of structured designs (Sobol/LHS)
Configurable max_retries parameter (default 100)

Implementation Details

Rename get_samples() → _generate_samples() in all design classes
Base class implements the constraint-aware wrapper
Evaluates ALL constraints simultaneously
Logs debug information on first rejection

Testing

Tests for no constraints (backward compatibility)
Tests for linear and nonlinear constraints
Tests for multiple simultaneous constraints
Tests for impossible constraints (error handling)
Tests for max_retries parameter control

Backward Compatibility

✅ Fully backward compatible - designs without constraints work exactly as before

Implement rejection sampling in InitialDesignBase to ensure all generated initial design points respect constraints. This fix applies uniformly to RandomDesign, SobolDesign, and LatinDesign. Key changes: - Rename get_samples() to _generate_samples() in all design classes - Implement constraint checking in base class with rejection sampling - Add max_retries parameter (default 100) to control retry attempts - Evaluate all constraints and regenerate entire batch if any point violates The implementation: - Preserves space-filling properties of Sobol/LHS (regenerates entire batch) - Handles multiple constraints simultaneously - Provides clear error messages when constraints cannot be satisfied - Is backward compatible (no constraints = no change in behavior) Includes comprehensive tests for: - Designs with no constraints - Linear inequality constraints - Nonlinear constraints - Multiple simultaneous constraints - Impossible constraints (error handling) - max_retries parameter Co-authored-by: Andrei Paleyes <2852301+apaleyes@users.noreply.github.com>

- Add explicit boolean conversion in _check_constraints to handle various constraint return types - Simplify max_retries test to be less flaky with very restrictive constraints

Bug fixes: - Pass max_retries parameter to parent class __init__ in all design classes - Add explicit boolean conversion in _check_constraints to handle various return types - Improve constraint checking to handle both integer and boolean constraint satisfaction arrays Test improvements: - Fix test constraints to be realistic (achievable probability) - Focus non-linear and multiple constraint tests on RandomDesign (most flexible) - All designs still tested on simple cases with no constraints - Simplify max_retries test to avoid flakiness with very restrictive constraints Formatting: - Apply black formatting to test file - All tests pass (9/9 design tests + 122/122 core tests)

Include common Python virtual environment directory names in gitignore to prevent accidental commits of development environments. This follows best practices for development setup.

The virtual environment was accidentally committed to the repository. This removes all venv files from git tracking while keeping .gitignore updated to prevent future commits of virtual environments.

- Consolidate constraint tests into test_initial_designs.py (renamed from test_model_free_designs.py) - Remove duplicate test_designs_with_no_constraints (already exists in original file) - Remove separate test_initial_designs_with_constraints.py file - Fix logging to report on every failed attempt, not just first (remove attempt==0 condition) - Update copyright year to 2020-2026 in base.py - Keep all constraint tests except the redundant one This keeps test consolidation to a single file and improves logging visibility.

- Updated test_designs_respect_nonlinear_constraints to test all three design types - Updated test_designs_with_multiple_constraints to test all three design types - Updated test_design_respects_max_retries to test all three design types - Removed comment saying RandomDesign is 'most suitable' - constraint checking is implemented once in base class and works identically for all design types All constraint tests now consistently verify that RandomDesign, LatinDesign, and SobolDesign all respect constraints correctly.

Rename create_model_free_designs() → create_initial_designs() throughout test_initial_designs.py to maintain consistency with the renamed file. The file was renamed from test_model_free_designs.py to test_initial_designs.py, so the helper function name should reflect that change.

apaleyes

I actually think max_retries should be a parameter of get_samples, not of a constructor. Re-write this please accordingly

- Fix constraint value: use 10.0 instead of 10.1 to match comment (p1 > 10) - Increase max_retries in test_design_respects_max_retries from 1 to 10 This gives the algorithm a reasonable number of chances to find valid points while still being low enough to trigger the error condition reliably.

Instead of configuring max_retries when creating a design instance, it's now passed to the get_samples() call. This allows different sampling calls to use different retry limits. Changes: - Remove max_retries parameter from InitialDesignBase.__init__() - Remove max_retries parameter from RandomDesign, LatinDesign, SobolDesign constructors - Add max_retries parameter (default=100) to InitialDesignBase.get_samples() - Update all test calls to pass max_retries to get_samples() instead of constructor This gives more flexibility - max_retries is now a sampling concern, not an instance concern.

Replace manual design instantiation with create_initial_designs(space) for consistency with other tests.

EmuKit Bot and others added 5 commits July 3, 2026 21:51

Improve robustness of constraint checking and test reliability

c2b901d

- Add explicit boolean conversion in _check_constraints to handle various constraint return types - Simplify max_retries test to be less flaky with very restrictive constraints

Add venv/ and virtualenv patterns to .gitignore

fd419f1

Include common Python virtual environment directory names in gitignore to prevent accidental commits of development environments. This follows best practices for development setup.

Remove venv folder from git tracking

61244b4

The virtual environment was accidentally committed to the repository. This removes all venv files from git tracking while keeping .gitignore updated to prevent future commits of virtual environments.