(Discussion) Easier Installation Process for DSpace

[tdonohue]

Describe the bug

This is a discussion ticket related to finding ways to simplify the Installation process for DSpace 7/8/9. It's a place to gather public notes & recommendations while developers work on solutions that can be brought back into DSpace code and documentation.

Brainstorms

Some (unproven) approaches to simplifying the installation process may include:

• Using Docker more heavily to containerize everything and automate installation. We should keep in mind this may not align with the needs of all institutions, as not everyone uses Docker or is comfortable with Docker.
• Removing Apache Ant & only using Maven (as suggested by @pnbecker in a comment below) for installation of the backend.
• Building Tools to help automate/manage configuration and/or validate configuration between frontend & backend. Such tools might minimize common installation issues.
• Is there any way to potentially embed (or download) the User Interface using the Maven build process of the backend?
  • Some examples (one and two) exist for embedding Angular apps into Spring Boot. But these examples are all are specific to only using Client-Side Rendering (CSR) and not using SSR via Node.js. This approach may not be possible/easy, since SSR requires running the frontend via Node.js, rather than deploying as a single page application (SPA).

How can you help?

DSpace Committers & Developers are striving to improve the installation process of DSpace 7+, to make it easier to setup (less prone to configuration errors), and also easier to upgrade. We welcome brainstorms, solutions, shared code, improved docs that aligns with these goals. DSpace is community built/maintained software (with no centralized development team). The more details community members can share, the more quickly we can locate/design solutions to benefit everyone.

[pnbecker]

I think it would already be a benefit, if we could get rid of Ant. Having to use Maven and Ant is something people seem to find hard to understand.
Having production ready Docker images would also be a huge win.

[pnbecker]

To reduce complexity in this area: With ant we currently filter the following files: default.context.xml, the log4j*.properties, log4j2*.xml and some RDF-Configuration. I think we have a good chance to get rid of this:

• The RDF-Configuration is just being filtered to set a directory where the example configuration for a fuseki triple store is storing its files. We could remove this and let people set it manually if we documented that.
• In default.context.xml we set the place where dspace finds its configuration. Since DSpace 8 the embedded tomcat is using a command-line parameter to get the same information. Is that already good enough? How does other spring application solve this?
• For the logging we do set the log.level and the log.dir. I think it is fair to configure the log-level just in the xml files and not filter them. I have no idea yet how to solve the log.dir, but with going more and more to docker, we might use by default stdout and document how to change this to storing log files.

What would be a concrete use case? It would make it easier to start cli jobs out of IDEs. (You can do that already now, if you know what and how to do it, but this would make it easier).

[mwoodiupui]

• I agree that we do not need to filter the RDF configuration (or any other) to customize examples.
• "using a command-line parameter to get the same information." If that means setting a Java system property, this will break at sites which run multiple DSpace instances in a single Tomcat (as we do) because each instance needs a different value. The ServletContext is the proper place for the container to communicate with a single webapp.
• Logging has a chicken/egg problem: DSpace could configure this programmatically, but then we could not log early startup issues because logging is not yet configured. I think this is another place where we should simply document that one must edit a single line in this file.

[mwoodiupui]

We may be trying to do too much because we cannot do enough. Filtering may not be powerful enough.

Perhaps we should view installation and configuration as two steps, and concentrate on promising that DSpace will start when installed according to directions but that one must then configure it to get a working repository. Logs will be written in /tmp or %TMPDIR% and you should change this. DSpace will run without a configuration path (or with a minimal configuration in WAR resources) but will not be usable until one is provided. There should be enough built in that a new instance can run well enough to tell you what is missing.

Configuration that is not necessary to get to a running state with a working administrative interface might be moved into the database and managed through administrative pages. This would create opportunity to validate nonessential configuration ("this property must be a valid {link ISO 8601 date}").

We should still try to ensure that the configuration step is not needlessly onerous.

[mwoodiupui]

We should consider moving some configuration out of dspace.cfg/local.cfg. There are places where we do ugly things with property names to simulate tables, and even trees of linked structures. Those would be more comprehensible as XML: one could see the relationships in the nesting. Spring DI could do the parsing and inject ready-made structures into the classes that manage them.

[mwoodiupui]

We should think about what must be configured to build, what must be configured to install, and what must be configured to run. Build time probably depends only on some Maven properties. Installation needs to know "where to copy the code" and "how to contact the database". I expect that everything else is runtime.

[mwoodiupui]

I'm not so sure about doing away with Ant. Ant is procedural and we need support for this during installation. Ant is also cross-platform and we need that.

Maven works by example using its own procedures, and may not be what we need for installation. It is billed as a project comprehension and build tool, not installation support. Maven creates build artifacts.

[pnbecker]

Great thoughts @mwoodiupui. Reading that, I thought DSpace is very unique with its installer. Most other software provides tarballs, rpms and/or deb packages. Should we overthink our installation approach? Most other comparable software would expect configuration at /etc/dspace and log to /var/log without asking. We could use these defaults and document how to change this.

[mwoodiupui]

Yes. Here I invented a new configuration property dspace.var and use it to locate the volatile directories separately from the code.