Skip to content

[Merged by Bors] - Spark history server#187

Closed
razvan wants to merge 61 commits intomainfrom
124-implement-adr-22-spark-history-server
Closed

[Merged by Bors] - Spark history server#187
razvan wants to merge 61 commits intomainfrom
124-implement-adr-22-spark-history-server

Conversation

@razvan
Copy link
Copy Markdown
Member

@razvan razvan commented Jan 3, 2023

@razvan razvan linked an issue Jan 3, 2023 that may be closed by this pull request
4 tasks
Copy link
Copy Markdown
Member

@sbernauer sbernauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review finished. I would like to try it out in the meantime

Comment thread docs/modules/ROOT/pages/usage.adoc Outdated
Comment thread rust/operator-binary/src/pod_driver_controller.rs
Comment thread rust/crd/src/history.rs Outdated
Comment thread rust/crd/src/history.rs
Comment thread rust/crd/src/lib.rs Outdated
Comment thread rust/operator-binary/src/history_controller.rs
Comment thread rust/operator-binary/src/history_controller.rs Outdated
Comment thread rust/operator-binary/src/history_controller.rs
Comment thread rust/operator-binary/src/history_controller.rs Outdated
Comment thread rust/operator-binary/src/spark_k8s_controller.rs Outdated
razvan and others added 7 commits January 19, 2023 12:54
Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de>
Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de>
Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de>
Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de>
Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de>
Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de>
Copy link
Copy Markdown
Member

@sbernauer sbernauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
During review we noticed that we currently can't verify S3 connections using a SecretClass. Ideally this would be fixed in this PR, but I'm fine with releasing without it and working on the Issue #194 directly afterwards

bors Bot pushed a commit to stackabletech/stackablectl that referenced this pull request Jan 19, 2023
## Description

I suggest reviewing commits separately

Currently with stackabletech/spark-k8s-operator#187 it produces.
We should probably adopt the exposed services

```
┌──────────────────────┬───────────────┬──────────────────────────────┬───────────────────────────────────────────┬──────┐
│ Product              ┆ Name          ┆ Namespace                    ┆ Endpoints                                 ┆ Info │
╞══════════════════════╪═══════════════╪══════════════════════════════╪═══════════════════════════════════════════╪══════╡
│ spark-history-server ┆ spark-history ┆ kuttl-test-credible-crayfish ┆ node-http         http://172.18.0.4:32525 ┆      │
│                      ┆               ┆                              ┆ node-cleaner-http http://172.18.0.4:32589 ┆      │
└──────────────────────┴───────────────┴──────────────────────────────┴───────────────────────────────────────────┴──────┘
```

## Review Checklist
- [ ] Code contains useful comments
- [ ] (Integration-)Test cases added (or not applicable)
- [ ] Documentation added (or not applicable)
- [ ] Changelog updated (or not applicable)
- [ ] Cargo.toml only contains references to git tags (not specific commits or branches)

Once the review is done, comment `bors r+` (or `bors merge`) to merge. [Further information](https://bors.tech/documentation/getting-started/#reviewing-pull-requests)
@razvan
Copy link
Copy Markdown
Member Author

razvan commented Jan 20, 2023

bors merge

bors Bot pushed a commit that referenced this pull request Jan 20, 2023
Fixes #124 #162
Jenkins: [Azure:heavy_check_mark:](https://ci.stackable.tech/view/02%20Operator%20Tests%20(custom)/job/spark-k8s-operator-it-custom/60/)

Co-authored-by: Andrew Kenworthy <andrew.kenworthy@stackable.de>
@bors
Copy link
Copy Markdown

bors Bot commented Jan 20, 2023

Pull request successfully merged into main.

Build succeeded:

@bors bors Bot changed the title Spark history server [Merged by Bors] - Spark history server Jan 20, 2023
@bors bors Bot closed this Jan 20, 2023
@bors bors Bot deleted the 124-implement-adr-22-spark-history-server branch January 20, 2023 06:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement ADR 22 (Spark history server)

3 participants