DS-3763: Scripts and processes endpoint

processes-endpoint.md

@@ -0,0 +1,185 @@
+# Admin Processes Endpoints
+[Back to the list of all defined endpoints](endpoints.md)
+
+This endpoint allows users to list and manipulate script processes created by the [Scripts endpoint](scripts-endpoint.md).
+
+## Execution List
+**GET /api/system/processes**
+
+This endpoint will return a list of all created processes. The JSON response document is as follows:
+
+```json
+{
+  "page": {
+      	"size": 5,
+      	"totalElements": 14,
+      	"totalPages": 3,
+      	"number": 0
+  },
+  "_embedded" : {
+    "processes" : [
+      {
+        "processId" : "1",
+        "userId" : "aa0263e2-b90a-4528-89fa-116ea4859de1",
+        "startTime" : "2017-11-22T10:29:11Z",
+        "endTime" : null,
+        "scriptName": "metadata-import",
+        "processStatus" : "RUNNING",
+        "parameters" : [
+          {
+            "name" : "-c",
+            "value" : "954e5cfa-6990-4c85-ae42-f30d8c7888e2"
+          },
+          {
+            "name" : "-n",
+            "value" : "true"
+          }
+        ],
+        "_links" : {
+          "self" : {
+            "href" : "/api/system/processes/1"
+          },
+          "script" : {
+            "href" : "/api/system/scripts/import"
+          },
+          "output" : {
+            "href" : "/api/system/processes/1/output"
+          },
+          "files" : {
+            "href" : "/api/system/processes/1/files"
+          }
+        }
+      },
+      {
+        "processId" : "2",
+        "userId" : "c7d85e7f-63e5-4bc0-96cb-5d80be48d62e",
+        "startTime" : "2017-11-20T10:29:11Z",
+        "endTime" : "2017-11-20T10:30:11Z",
+        "scriptName": "metadata-import",
+        "processStatus" : "FAILED",
+        "parameters" : [
+          {
+            "name" : "-i",
+            "value" : "c70893a6-ac55-48c7-9447-61e026b62929"
+          }
+        ],
+        "_links" : {
+          "self" : {
+            "href" : "/api/system/processes/2"
+          },
+          "script" : {
+            "href" : "/api/system/scripts/metadata-export"
+          },
+          "output" : {
+            "href" : "/api/system/processes/2/output"
+          },
+          "files" : {
+            "href" : "/api/system/processes/2/files"
+          }
+        }
+      }
+    ]
+  }
+}
+```
+
+Optional parameters to query the processes:
+* `script`: The name of the script (e.g. import)
+* `userId`: The UUID of the person who created the process (only admins can retrieve processes they didn't start)
+* `status`: The status of the script
+* `parameter.xyz`: Which parameters have been used, and their value
+
+## Execution Details
+**GET /api/system/processes/<:process-id>**
+
+This endpoint will return details on the requested process.
+
+```json
+{
+  "processId" : "3",
+  "userId" : "aa0263e2-b90a-4528-89fa-116ea4859de1",
+  "startTime" : "2017-11-22T10:29:11Z",
+  "endTime" : null,              
+  "scriptName": "metadata-import",
+  "processStatus" : "RUNNING",
+  "parameters" : [
+    {
+    "name" : "-c",
+    "value" : "954e5cfa-6990-4c85-ae42-f30d8c7888e2"
+    },
+    {
+    "name" : "-n",
+    "value" : "true"
+    }
+  ],
+  "_links" : {
+    "self" : {
+      "href" : "/api/system/processes/3"
+    },
+    "script" : {
+      "href" : "/api/system/scripts/import"
+    },
+    "output" : {
+      "href" : "/api/system/processes/3/output"
+    },
+    "files" : {
+      "href" : "/api/system/processes/3/files"
+    }
+  }
+}
+```
+
+The possible `status` values are `RUNNING`, `COMPLETED` and `FAILED`.
+
+## Execution Console Output
+**GET /api/system/processes/<:process-id>/output**
+
+This endpoint will return the console output of the script. To keep the scope of this work limited, it will print the full console output without datestamps.
+In a future version, a solution supporting a live-feed can also be included.
+
+## Execution File Output List
+**GET /api/system/processes/<:process-id>/files**
+
+This endpoint will let an administrator download an output file created by a process. If the file is found, it will be presented as a download. This endpoint will support "Range" HTTP headers so that downloads can be paused and resumed.
+
+```json
+{
+  "processId" : "4",
+  "_links" : {
+    "self" : {
+      "href" : "/api/system/processes/4/files"
+    },
+    "mapfile" : {
+      "href" : "/api/system/processes/4/files/mapfile"
+    },
+    "zipfile" : {
+      "href" : "/api/system/processes/4/files/zipfile"
+    }
+  }
+}
+```
+
+## Execution File Output
+**GET /api/system/processes/<:process-id>/files/<:file-name>**
+
+This endpoint will let an administrator download an output file created by a process. If the file is found, it will be presented as a download. This endpoint will support "Range" HTTP headers so that downloads can be paused and resumed.
+
+Required parameters:
+* `file-name`: The name of the output file to retrieve
+
+## Execution Deletion
+**DELETE /api/system/processes/<:process-id>**
+
+The will delete the process and associated files and output. Only processes with states of `SCHEDULED`, `COMPLETED` and `FAILED` can be deleted. 
+
+Support for stopping a process can be included in a future version.
+
+Status codes:
+* 201 Created - if the operation succeed
+* 404 Not found - if the processes doesn't exist
+* 422 Unprocessable Entity - If the process is running
+
+
+## Script Invocation
+
+See the [scripts endpoint](scripts-endpoint.md#script-invocation) for details on how to start a script

scripts-endpoint.md

@@ -0,0 +1,189 @@
+# Admin Scripts Endpoints
+[Back to the list of all defined endpoints](endpoints.md)
+
+DSpace has functionality to import and export items in CSV and ZIP format, to start a collection harvest run, to run or schedule curation tasks, …. Each of these functionalities also map on a DSpace CLI script. While we could implement each of these operations as a separate endpoint, this contract tries to describe a generic endpoint that allows administrators to explore, run or schedule DSpace CLI scripts from the REST API.
+
+## Scripts Endpoint
+**GET /api/system/scripts**
+
+This endpoint will list all (REST supported) scripts defined in `dspace/config/spring/rest/scripts.xml`. The script entries are embedded with a name, description and a self link. By "REST supported" we mean all scripts that have been updated to allow invocations from the REST API.
+
+The JSON response document is as follows
+```json
+{
+  "page": {
+      	"size": 5,
+      	"totalElements": 14,
+      	"totalPages": 3,
+      	"number": 0
+  },
+  "_embedded" : {
+    "scripts" : [
+      {
+        "id" : "import",
+        "name" : "import",
+        "type" : "script",
+        "description" : "Import items into DSpace",
+        "_links" : {
+          "self" : {
+            "href" : "/api/system/scripts/import"
+          }
+        }
+      },
+      {
+        "name" : "metadata-import",
+        "type" : "script",
+        "description" : "Import metadata after batch editing",
+        "_links" : {
+          "self" : {
+            "href" : "/api/system/scripts/metadata-import"
+          }
+        }
+      }
+    ]
+  }
+}
+```
+
+## Script Name Endpoint
+
+### Script Details
+**GET /api/system/scripts/<:script-name>**
+
+This endpoint will return information on all the parameters that are required to invoke the script. This depends on the parameters defined in the implementation of the script.
+
+The JSON response document is as follows
+```json
+{
+   "id" : "import",
+   "name" : "import",
+   "description" : "Import items into DSpace",
+   "type" : "script",
+   "parameters" : [
+    {
+      "name" : "-a",
+      "description" : "add items to DSpace",
+      "type" : "boolean"
+    },
+    {
+      "name" : "-b",
+      "description" : "add items to DSpace via Biblio-Transformation-Engine (BTE)",
+      "type" : "boolean"
+    },
+    {
+      "name" : "-c",
+      "description" : "destination collection(s) database ID",
+      "type" : "id"
+    },
+    {
+      "name" : "-d",
+      "description" : "delete items listed in mapfile",
+      "type" : "file"
+    },
+    {
+      "name" : "-e",
+      "description" : "email of eperson doing importing",
+      "type" : "string"
+    },
+    {
+      "name" : "-i",
+      "description" : "input type in case of BTE import",
+      "type" : "string"
+    },
+    {
+      "name" : "-n",
+      "description" : "if sending submissions through the workflow, send notification emails",
+      "type" : "boolean"
+    },
+    {
+      "name" : "-p",
+      "description" : "apply template",
+      "type" : "string"
+    },
+    {
+      "name" : "-q",
+      "description" : "don't display metadata",
+      "type" : "boolean"
+    },
+    {
+      "name" : "-m",
+      "description" : "mapfile items in mapfile",
+      "type" : "file"
+    },
+    {
+      "name" : "-r",
+      "description" : "replace items in mapfile",
+      "type" : "boolean"
+    },
+    {
+      "name" : "-R",
+      "description" : "resume a failed import (add only)",
+      "type" : "boolean"
+    },
+    {
+      "name" : "-t",
+      "description" : "test run - do not actually import items",
+      "type" : "boolean"
+    },
+    {
+      "name" : "-w",
+      "description" : "send submission through collection's workflow",
+      "type" : "boolean"
+    },
+    {
+      "name" : "-z",
+      "description" : "zip file containing the import",
+      "type" : "file"
+    }
+   ]
+}
+```
+
+Following parameter types are available:
+* `string`: A regular string value
+* `date`: A string that has a valid date pattern
+* `boolean`: true or false
+* `file`: For parameters with this type, the user has to provide a file. This would be a multipart POST request using the same filename
+* `output`: Parameters with this type define the name of the output file. This name can be used later to download the the output file (e.g. when running `export` or `metadata-export`).
+
+## Script Invocation
+**POST /api/system/scripts/<:script-name>/processes**
+
+POST requests to this endpoint will start the corresponding script with the provided parameters. All parameter values should be provided in the body that has to use the `multipart/form-data` content type. Once the upload is complete and the script was started successfully, this endpoint will return details on the scripts execution
+
+Delayed scripts using a  `startTime` can be supported in a future version.
+
+```json
+{
+  "processId" : "5",
+  "userId" : "aa0263e2-b90a-4528-89fa-116ea4859de1",
+  "processStatus" : "RUNNING",
+  "parameters" : [
+    {
+    "name" : "-c",
+    "value" : "954e5cfa-6990-4c85-ae42-f30d8c7888e2"
+    },
+    {
+    "name" : "-n",
+    "value" : "true"
+    }
+  ],
+  "_links" : {
+    "self" : {
+      "href" : "/api/system/processes/5"
+    },
+    "script" : {
+      "href" : "/api/system/scripts/import"
+    },
+    "output" : {
+      "href" : "/api/system/processes/5/output"
+    }
+  }
+}
+```
+
+The possible `status` values are `SCHEDULED`, `RUNNING`, `COMPLETED` and `FAILED`.
+
+Status codes:
+* 202 Accepted - if the task is accepted for processing
+* 404 Not found - if the script doesn't exist