Skip to content

additional options for controlling simID and the main output file#24

Merged
wzf-cn merged 2 commits intoV2Xgithub:dev-parallel-and-tracefilefrom
SUTDFCCLab:print-main-file-and-sim-id
Oct 6, 2023
Merged

additional options for controlling simID and the main output file#24
wzf-cn merged 2 commits intoV2Xgithub:dev-parallel-and-tracefilefrom
SUTDFCCLab:print-main-file-and-sim-id

Conversation

@caspycat
Copy link
Contributor

Hi V2Xsim maintainers,

The purpose of this pull request is to add additional options to control whether the application writes the summary of results to MainOut.xls, in addition to being able to control the name. There is also the option to manually specify the simID - if not specified, it will use the old logic of auto incrementing (if printing to main out is disabled, defaults to simID 1)

The motivation behind this is to enable parallelism such as running many simulations at once using parfeval or parfor. If you want to call WiLabV2Xsim.m in a parallel environment, the original behavior of trying to auto increment the ID will cause a race condition between the processes. In such cases, it is much more conducive to prepopulate a list of sim IDs depending on the research question being investigated, and have each simulation not write over each others' files because they might complete at different times.

caspycat and others added 2 commits September 27, 2023 20:53
1. When writing the "mainout" file, using a "marker file" to indicate that the "mainout" file is writing now. In case of multiple processes write the file simultaneously.

2. If the outputs of different simulations are in one folder, the auto-set simIDs could not support "parfor". We may need another effective way to fix this promble.
@caspycat
Copy link
Contributor Author

caspycat commented Oct 4, 2023

hi wzf,

Thank you for your changes, i think the lock file is a good idea.
I think it is reasonable to assume that if an advanced user is using parfor to do parameter sweeps, he should be responsible to set unique simulationids. One would usually write helper code, the experiment runner, to use parfor, the helper code will have a better idea of what id to use. It is very difficult for an individual experiment proceess to decide on auto-increment id unless we change to randomly generated uuids.

I don't have permissions to merge, but I am okay with the current fix.

@wzf-cn wzf-cn changed the base branch from main to dev-parallel-and-tracefile October 6, 2023 00:55
@wzf-cn wzf-cn merged commit f2d63c8 into V2Xgithub:dev-parallel-and-tracefile Oct 6, 2023
@caspycat caspycat deleted the print-main-file-and-sim-id branch November 20, 2023 08:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants