Allow Browser Config Options from Command Line

[DeathBeforeDecaf]

🚀 Feature Proposal

Allow browser-specific configuration options to be passed as a json object and initialized via command line. This eliminates the need for the node json configuration file on the host machine.

Motivation

When node capabilities are initialized via the command line, there's no way to properly initialize browser-specific configuration options, since the capabilities list is being split on ',' character, then key/value pairs are split on '=' character.

Example

Here is the command line initialization of a node that instantiates headless versions of chrome and firefox, please see goog:chromeOptions and moz:firefoxOptions options:

java -Dwebdriver.chrome.driver=/opt/openqa/selenium/browsers/chromedriver-78.0.3904.70 -Dwebdriver.chrome.logfile=/opt/openqa/selenium/logs/node/chrome78_wdc-tst-se3grid-001.log -Dwebdriver.gecko.driver=/opt/openqa/selenium/browsers/geckodriver-v0.26.0 -Dwebdriver.firefox.logfile=/opt/openqa/selenium/logs/node/firefox60_wdc-tst-se3grid-001.log -Dselenium.LOGGER=/opt/openqa/selenium/logs/node/node_wdc-tst-se3grid-001.log -Dselenium.LOGGER.level=INFO -cp /opt/openqa/selenium/bin/selenium-server-standalone-3.141.59.jar:/opt/openqa/selenium/lib/* org.openqa.grid.selenium.GridLauncherV3 -role node -hub http://wdc-tst-se3grid-001.openqa.com:4444/grid/register -port 5555 -maxSession 5 -id wdc-tst-se3grid-001:5555 -capabilities browserName=chrome,version=78.0,maxInstances=5,platform=LINUX,seleniumVersion=3.141.59,acceptSslCerts=true,acceptInsecureCerts=true,takesVideo=false,instance=wdc-tst-se3grid-001,goog:chromeOptions={"args":["--headless","--disable-gpu"]} -capabilities browserName=firefox,version=60.8.0esr,maxInstances=5,platform=LINUX,seleniumVersion=3.141.59,acceptSslCerts=true,acceptInsecureCerts=true,takesVideo=false,instance=wdc-tst-se3grid-001,moz:firefoxOptions={"args":["-headless"],"binary":"/opt/openqa/selenium/browsers/firefox-60.8.0esr/firefox","log":{"level":"info"}}

Note:

The change has been made from the tag of selenium-3.141.59 and placed it on a branch called selenium-3.141.592_json, the specific file update is located here:

https://github.com/DeathBeforeDecaf/selenium/blob/639707ce9967efffd607c58a45e33dd696887267/java/server/src/org/openqa/grid/internal/cli/BrowserDesiredCapabilityConverter.java

Unfortunately I'm not familiar with using tags to interact with git (we generally use branches for releases) - so please forgive any stupid mistakes on my part.

[DeathBeforeDecaf]

Won't anybody take a look at this? It's basically a matched acceptor, allowing json to pass through along with the normal key=value pairs. The work has already been done (the proof-of-concept is at the previously mentioned link).

[diemol]

@DeathBeforeDecaf sorry, I am not getting the idea. Are you suggesting to get rid of the JSON config file for nodes?

[DeathBeforeDecaf]

Hi Diego,

I'm not suggesting getting rid of config files for nodes, I'm just suggesting allowing valid json command line parameter values. With Selenium 2 I was able to initialize everything via the command line but with Selenium 3 some webdriver instances are requiring json-specific settings in their initialization. I want to give people the option to continue to initialize everything with the command line.

Thanks and please let me know if you have additional questions!
-Anton

[diemol]

Grid 4 is allowing the use of TOML for configuration (docs coming soon).

On the other hand, what is the benefit of passing JSON blobs as command line parameters, the command to start a Grid will look very complex and prone to error Why not using JSON (or from now on, TOML files)?.

[DeathBeforeDecaf]

Hi Diego,

Thanks for asking why I use JSON command line parameters instead of JSON configuration files! The short answer is by using JSON command line parameters there is no chance that a selenium client will be started with stale configuration data loaded from an old configuration file.

The long answer is that in the company where I work, some fool in management left me in charge of maintaining several dozen selenium instances that are required for testing. The company has also rolled their own network-based configuration service which centrally performs configuration of all our internal services. To start the selenium 3 client, I have to know how to parse the selenium 3 configuration files to write out key values into a readable JSON file in the file system that the client uses to pass to the w3c webdrivers. More confusingly, we have multiple configurations per selenium 3 client instance. For example, if we want to exercise Chrome on Linux and record the video of the executing test for troubleshooting the selenium test case - we have a configuration for that. If we want to run that same selenium 3 client Chrome on Linux client in a headless mode - we also have a configuration for that.

In an attempt to make things a bit easier to maintain long-term I've created a simple update to selenium 3 client that allows JSON configuration data to pass through the command line processing routines. I thought this functionality might be useful for others so I provided an example implementation to share. Please check out the link above in the initial post for the changes I made to enable this feature. This implementation should be 100% backward compatible with the existing implementation, it merely allows a user to pass the required data in command line values. The data passed via the command line is no more complex than the data used in the configuration files.

While the use of TOML files does sound interesting for version 4, it doesn't solve the fundamental problem of storing data in files rather than allowing for key values to be passed via the command line.

Please let me know if you have more questions!

Thanks,
-AntonW

[barancev]

You can run a series of two commands: one stores data to a temp file and the next one starts selenium with this temp file as a config.

[DeathBeforeDecaf]

Hi @barancev,

That is a solution. However that solution requires write permissions to the file system and knowledge of the file format being written. If the file format changes (like for selenium version 4) then it will require more adjustments. The alternate implementation that I've provided allows the data to pass through on the command line without those requirements and should have no impact on existing users.

Thanks for your consideration,
-AntonW

[barancev]

The arguments don't sound strong enough.

1. Selenium requires write permissions to the file system anyway because it creates temp browser profiles.
2. Using command line options requires knowledge of the options format, especially if they have complex format like a JSON-string.
3. And command line format (JSON or not) can be changed with the same probability as config file format.

[barancev]

Also:

1. It's much easier to write complex configuration in a multiline form with indentation (JSON) or section headers (TOML) that visualize structure, in a text editor.
2. Dealing with spaces and quotes in command line is depressing...
3. On Windows the maximum command-line length is 32767 characters.

[DeathBeforeDecaf]

Sorry, I didn't know I was here for an argument. :^}

I just thought that since Selenium 2 allowed instantiation of selenium clients with the command line, that some people might find that same functionality beneficial if provided in selenium 3.

Thanks,
-AntonW

[barancev]

Yes, "Motivation" section was incuded in the feature proposal template because we want arguments :)

Yes, "it used to work this way in version 2" is an argument. And exactly the experience of using version 2 with overcomplicated error prone CLI options led us to config files.

[diemol]

@DeathBeforeDecaf thank you for taking the time to explain the use case, but I believe it is not ideal to include something like this in the code base for the reasons stated in different comments.

I'll close this issue.