The Schema Configuration tests in test_bulk_create.py (and related schema config test files) currently rely heavily on mocks to simulate the internal workings of migration functions. Using the database models and ORM should be heavily preferred over mocks. This is primarily due to the complexity of the involved operations and models, as well as the customizability of the Schema Configuration as a whole.
Mocks require imposing control over the environment and "migration internals" that is not present in non-test situations, such as production. Similar to frontend tests running isolated from the backend, responses must be mocked in good faith to the corresponding requests: the response data is entirely controlled by the developer within an isolated sandbox, which does not reflect production or development environments.
A more black-box, result-focused approach, rather than one requiring expectations of internal function implementation ensures that future changes to function internals that yield equivalent results do not cause test failures.
Therefore, these Schema Configuration tests should be updated to leverage the database and non-mocked functionality as much as possible (though mocking side-effects like stdout in run_key_migration_functions remains an appropriate use case for a mock).
Originally posted by @melton-jason in #8225 (comment)
The Schema Configuration tests in test_bulk_create.py (and related schema config test files) currently rely heavily on mocks to simulate the internal workings of migration functions. Using the database models and ORM should be heavily preferred over mocks. This is primarily due to the complexity of the involved operations and models, as well as the customizability of the Schema Configuration as a whole.
Mocks require imposing control over the environment and "migration internals" that is not present in non-test situations, such as production. Similar to frontend tests running isolated from the backend, responses must be mocked in good faith to the corresponding requests: the response data is entirely controlled by the developer within an isolated sandbox, which does not reflect production or development environments.
A more black-box, result-focused approach, rather than one requiring expectations of internal function implementation ensures that future changes to function internals that yield equivalent results do not cause test failures.
Therefore, these Schema Configuration tests should be updated to leverage the database and non-mocked functionality as much as possible (though mocking side-effects like stdout in run_key_migration_functions remains an appropriate use case for a mock).
Originally posted by @melton-jason in #8225 (comment)