E2E test workflow not executing on commenting "/run e2e" or "/test e2e"

[jescalada]

#1165 added a GitHub CI workflow for running E2E tests, however, these tests seem to only execute on main after merging. This means that the CI might pass on the PR, but fail after merging on main, which is the case on the latest commit.

There's a section that enables running the tests if there's a "/run e2e" or "/test e2e" comment present:

if: |
    github.event_name != 'issue_comment' || (
      github.event.issue.pull_request &&
      (contains(github.event.comment.body, '/test e2e') || contains(github.event.comment.body, '/run e2e')) &&
      (github.event.comment.author_association == 'OWNER' || 
       github.event.comment.author_association == 'MEMBER' || 
       github.event.comment.author_association == 'COLLABORATOR')
    )

In #1341, I tried triggering the e2e tests via commenting, but it didn't work.

[coopernetes]

GitHub Actions workflow triggers are not flexible enough for our needs in this regard. As such, after #1359 I just enabled e2e tests to run on every PR. They're still fairly quick (on the order of a few minutes) and run in parallel with normal CI jobs. I think this is a fair trade-off for the time being. There are also runtime features that do not have good integration test coverage so the e2e tests serve a useful purpose as a PR check.

I'm happy to refactor these tests to run later on (pre-release seems appropriate), do we wish to re-add the comment-based triggers? workflow_dispatch is also included and maintainers (those with write permissions on the repo) can trigger it directly via github.

[jescalada]

If test execution is quick and compulsory, then it's totally fine not to have comment triggers 🙂

Will close this issue!