Add new engine command named "continue"

CHANGELOG.rst

@@ -4,6 +4,13 @@ Changelog
 In development
 --------------
 
+Changed
+~~~~~~~
+
+* Replace "noop" with "continue" when "do" is not specified. The new "continue" command
+  will not alter the previous task state and will continue to conduct the workflow
+  execution. StackStorm/st2#4740 (improvement)
+
 Fixed
 -----
 

docs/source/languages/orquesta.rst

@@ -457,7 +457,11 @@ of ``do``. The following is a revision of the previous example:
       - result: <% ctx().abcd %>
 
 The following example illustrates separate task transitions with different publishes
-on different condition:
+on different condition. After different message is published, both transition to the
+same task to log the message. In the task transition for failure, an explicit
+``fail`` command is specified to tell the workflow execution to fail. If the ``fail``
+command is not specified, ``task2`` is considered a remediation task and the workflow
+execution will succeed:
 
 .. code-block:: yaml
 
@@ -475,10 +479,13 @@ on different condition:
         next:
           - when: <% succeeded() %>
             publish: msg="Successfully posted message."
-            do: task2
+            do:
+              - task2
           - when: <% failed() %>
             publish: msg="Unable to post message due to error: <% result() %>"
-            do: task2
+            do:
+              - task2
+              - fail
       task2:
         action: core.log message=<% ctx(msg) %>
 
@@ -490,43 +497,150 @@ The following is a list of engine commands with special meaning to the workflow
 When specified under ``do`` in the task transition, the engine will act accordingly. These
 commands are also reserved words that cannot be used for task name.
 
-+-------------+-------------------------------------------------------------------+
-| Command     | Description                                                       |
-+=============+===================================================================+
-| noop        | No operation or do not execute anything else.                     |
-+-------------+-------------------------------------------------------------------+
-| fail        | Fails the workflow execution.                                     |
-+-------------+-------------------------------------------------------------------+
++-------------+------------------------------------------------------------------------------------+
+| Command     | Description                                                                        |
++=============+====================================================================================+
+| continue    | Default value when ``do`` is not specified. The workflow engine will not alter the |
+|             | previous task state and will continue to conduct the workflow execution. If the    |
+|             | previous task state is one of the failure states, the conductor will continue and  |
+|             | fail the workflow execution.                                                       |
++-------------+------------------------------------------------------------------------------------+
+| fail        | The workflow engine will fail the workflow execution.                              |
++-------------+------------------------------------------------------------------------------------+
+| noop        | The workflow engine will perform no operation given previous task state. If the    |
+|             | previous task state is one of the failure states, the conductcor will ignore the   |
+|             | task failure and assume a remediation has occurred.                                |
++-------------+------------------------------------------------------------------------------------+
+
+The following example illustrates the use of the default ``continue`` command to let the workflow
+continue processing the task failure (or any other state) as normal. If ``task1`` fails, the second
+task transition will publish the ``stderr`` and the conductor will continue with ``failed`` as the
+final state of the workflow execution:
+
+.. code-block:: yaml
+
+    version: 1.0
+
+    description: >
+        A workflow example that illustrates error handling. By default if no task
+        is specified under "do", the "continue" command is assumed. In this case
+        where there is a task failure, the "continue" command will process the
+        publish and then cascade the task failure to the workflow and the workflow
+        execution will fail as a result.
+
+    input:
+      - cmd
+
+    vars:
+      - stdout: null
+      - stderr: null
+
+    tasks:
+      task1:
+        action: core.local cmd=<% ctx(cmd) %>
+        next:
+          - when: <% succeeded() %>
+            publish: stdout=<% result().stdout %>
+          - when: <% failed() %>
+            publish: stderr=<% result().stderr %>
+
+    output:
+      - stdout: <% ctx(stdout) %>
+      - stderr: <% ctx(stderr) %>
 
-The following example illustrates the use of the ``fail`` command:
+The following example is the same as the example above except the ``continue`` command is
+explicit:
 
 .. code-block:: yaml
 
     version: 1.0
 
     description: >
-        A workflow example that illustrates error handling. By default
-        when any task fails, the notify_on_error task will be executed
-        and the workflow will transition to the failed state.
+        A workflow example that illustrates error handling. In this case, the "continue"
+        command is explicit. When there is a task failure, the "continue" command will
+        process the publish and then cascade the task failure to the workflow and the
+        workflow execution will fail as a result.
 
     input:
       - cmd
 
+    vars:
+      - stdout: null
+      - stderr: null
+
     tasks:
       task1:
         action: core.local cmd=<% ctx(cmd) %>
         next:
           - when: <% succeeded() %>
             publish: stdout=<% result().stdout %>
+            do: continue
           - when: <% failed() %>
             publish: stderr=<% result().stderr %>
-            do: notify_on_error
-      notify_on_error:
-        action: core.echo message=<% ctx(stderr) %>
+            do: continue
+
+    output:
+      - stdout: <% ctx(stdout) %>
+      - stderr: <% ctx(stderr) %>
+
+The following example illustrates the use of the ``noop`` command to let the workflow
+complete successfully even when there is a failure:
+
+.. code-block:: yaml
+
+    version: 1.0
+
+    description: >
+        A workflow example that illustrates error handling. When there is a task
+        failure, the "noop" command specified will be treated as a remediation task
+        and the conductor will succeed the workflow execution as normal.
+
+    input:
+      - cmd
+
+    vars:
+      - stdout: null
+      - stderr: null
+
+    tasks:
+      task1:
+        action: core.local cmd=<% ctx(cmd) %>
+        next:
+          - when: <% succeeded() %>
+            publish: stdout=<% result().stdout %>
+          - when: <% failed() %>
+            publish: stderr=<% result().stderr %>
+            do: noop
+
+    output:
+      - stdout: <% ctx(stdout) %>
+      - stderr: <% ctx(stderr) %>
+
+
+The following example is similar to the the one in previous section where it illustrates the use of
+the ``fail`` command to explicitly fail the workflow. In this case where the failure of the http
+call is communicated with a status number, a task transition is used to catch error when the
+status code is not 200. An explicit ``fail`` command is used to signal the workflow execution
+to fail:
+
+.. code-block:: yaml
+
+    version: 1.0
+
+    description: A sample workflow to fetch data from a REST API.
+
+    vars:
+      - body: null
+
+    tasks:
+      task1:
+        action: core.http url="https://api.xyz.com/objects"
         next:
-          # The fail specified here tells the workflow to go into
-          # failed state on completion of the notify_on_error task.
-          - do: fail
+          - when: <% succeeded() and result().status_code = 200 %>
+            publish: body=<% result().body %>
+          - when: <% succeeded() and result().status_code != 200 %>
+            publish: body=<% result().body %>
+            do: fail
 
     output:
-      - result: <% ctx(stdout) %>
+      - body: <% ctx(body) %>

orquesta/conducting.py

@@ -534,6 +534,10 @@ def has_next_tasks(self, task_id=None, route=None):
                     (next_task_id, str(seq_key))
                 )
 
+                # Ignore if the next task is the engine command to "continue".
+                if next_task_id == 'continue':
+                    continue
+
                 # Evaluate if outbound criteria is satisfied.
                 if not task_state_entry['next'].get(task_transition_id):
                     continue

orquesta/events.py

@@ -297,10 +297,12 @@
 
 
 # Events for special workflow engine operations.
+TASK_CONTINUE_REQUESTED = 'task_continue_requested'
 TASK_NOOP_REQUESTED = 'task_noop_requested'
 TASK_FAIL_REQUESTED = 'task_fail_requested'
 
 ENGINE_OPERATION_EVENTS = [
+    TASK_CONTINUE_REQUESTED,
     TASK_NOOP_REQUESTED,
     TASK_FAIL_REQUESTED
 ]
@@ -347,6 +349,15 @@ class EngineOperationEvent(ExecutionEvent):
     pass
 
 
+class TaskContinueEvent(EngineOperationEvent):
+
+    def __init__(self):
+        self.name = TASK_CONTINUE_REQUESTED
+        self.status = statuses.SUCCEEDED
+        self.result = None
+        self.context = None
+
+
 class TaskNoopEvent(EngineOperationEvent):
 
     def __init__(self):
@@ -366,6 +377,7 @@ def __init__(self):
 
 
 ENGINE_EVENT_MAP = {
+    'continue': TaskContinueEvent,
     'noop': TaskNoopEvent,
     'fail': TaskFailEvent
 }

orquesta/machines.py

@@ -230,6 +230,7 @@
         events.ACTION_DELAYED: statuses.DELAYED,
         events.ACTION_RUNNING: statuses.RUNNING,
         events.ACTION_PENDING: statuses.PENDING,
+        events.TASK_CONTINUE_REQUESTED: statuses.SUCCEEDED,
         events.TASK_NOOP_REQUESTED: statuses.SUCCEEDED,
         events.TASK_FAIL_REQUESTED: statuses.FAILED
     },

orquesta/specs/native/v1/models.py

@@ -83,7 +83,7 @@ def __init__(self, *args, **kwargs):
         do_spec = getattr(self, 'do', None)
 
         if not do_spec:
-            self.do = 'noop'
+            self.do = 'continue'
 
 
 class TaskTransitionSequenceSpec(native_v1_specs.SequenceSpec):

orquesta/tests/fixtures/workflows/native/error-handling-continue.yaml

@@ -0,0 +1,27 @@
+version: 1.0
+
+description: A basic workflow that demonstrates error handler with continue.
+
+vars:
+  - message: null
+
+tasks:
+  task1:
+    action: core.noop
+    next:
+      - when: <% succeeded() %>
+        do: task2
+      - when: <% failed() %>
+        publish:
+          - message: "$%#&@#$!!!"
+        do: continue
+
+  task2:
+    action: core.noop
+    next:
+      - when: <% succeeded() %>
+        publish:
+          - message: "hooray!!!"
+
+output:
+  - message: <% ctx().message %>

orquesta/tests/fixtures/workflows/native/error-handling-fail.yaml

@@ -0,0 +1,33 @@
+version: 1.0
+
+description: A basic workflow that demonstrates error handler with remediation and explicit fail.
+
+vars:
+  - message: null
+
+tasks:
+  task1:
+    action: core.noop
+    next:
+      - when: <% succeeded() %>
+        do: task2
+      - when: <% failed() %>
+        do: task3
+
+  task2:
+    action: core.noop
+    next:
+      - when: <% succeeded() %>
+        publish:
+          - message: "hooray!!!"
+
+  task3:
+    action: core.noop
+    next:
+      - when: <% succeeded() %>
+        publish:
+          - message: "$%#&@#$!!!"
+        do: fail
+
+output:
+  - message: <% ctx().message %>

orquesta/tests/fixtures/workflows/native/error-handling-noop.yaml

@@ -0,0 +1,27 @@
+version: 1.0
+
+description: A basic workflow that demonstrates error handler with noop to ignore error.
+
+vars:
+  - message: null
+
+tasks:
+  task1:
+    action: core.noop
+    next:
+      - when: <% succeeded() %>
+        do: task2
+      - when: <% failed() %>
+        publish:
+          - message: "$%#&@#$!!!"
+        do: noop
+
+  task2:
+    action: core.noop
+    next:
+      - when: <% succeeded() %>
+        publish:
+          - message: "hooray!!!"
+
+output:
+  - message: <% ctx().message %>