From b95caf8da667ff3e360c26d33a4949d839051356 Mon Sep 17 00:00:00 2001 From: Robert Metzger Date: Thu, 6 May 2021 10:44:15 +0200 Subject: [PATCH 1/3] [hotfix] Make reactive warning less strong, clarifications --- docs/content/docs/deployment/elastic_scaling.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/content/docs/deployment/elastic_scaling.md b/docs/content/docs/deployment/elastic_scaling.md index 8a06749c500e9..e4046ce38d752 100644 --- a/docs/content/docs/deployment/elastic_scaling.md +++ b/docs/content/docs/deployment/elastic_scaling.md @@ -31,7 +31,7 @@ This page describes options where Flink automatically adjusts the parallelism in ## Reactive Mode -{{< hint danger >}} +{{< hint info >}} Reactive mode is an MVP ("minimum viable product") feature. The Flink community is actively looking for feedback by users through our mailing lists. Please check the limitations listed on this page. {{< /hint >}} @@ -122,8 +122,8 @@ The [limitations of Adaptive Scheduler](#limitations-1) also apply to Reactive M ## Adaptive Scheduler -{{< hint danger >}} -Using Adaptive Scheduler directly (not through Reactive Mode) is only advised for advanced users. +{{< hint warning >}} +Using Adaptive Scheduler directly (not through Reactive Mode) is only advised for advanced users because slot allocation on a session cluster with multiple jobs is not defined. {{< /hint >}} The Adaptive Scheduler can adjust the parallelism of a job based on available slots. It will automatically reduce the parallelism if not enough slots are available to run the job with the originally configured parallelism; be it due to not enough resources being available at the time of submission, or TaskManager outages during the job execution. If new slots become available the job will be scaled up again, up to the configured parallelism. From 147808d63058d8b4216d7dff18c51593387ce3a4 Mon Sep 17 00:00:00 2001 From: Robert Metzger Date: Thu, 6 May 2021 10:51:21 +0200 Subject: [PATCH 2/3] [hotfix][docs] Re-introduce note about FLINK_CONF_DIR --- docs/content/docs/deployment/config.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/content/docs/deployment/config.md b/docs/content/docs/deployment/config.md index bb9f1f1a34a43..6711ce55dc58b 100644 --- a/docs/content/docs/deployment/config.md +++ b/docs/content/docs/deployment/config.md @@ -34,6 +34,8 @@ The configuration is parsed and evaluated when the Flink processes are started. The out of the box configuration will use your default Java installation. You can manually set the environment variable `JAVA_HOME` or the configuration key `env.java.home` in `conf/flink-conf.yaml` if you want to manually override the Java runtime to use. +You can specifiy a different configuration directory location by defining the `FLINK_CONF_DIR` environment variable. For resource-providers which provide non-session deployments, you can specifiy per-job configurations this way. Make a copy of the `conf` directory from the Flink distribution and modify the settings on a per-job basis. Note that this is not supported in Docker or standalone Kubernetes deployments. + # Basic Setup The default configuration supports starting a single-node Flink session cluster without any changes. From 1b449c5732f03f99a95fffa3e6592907222a3a0c Mon Sep 17 00:00:00 2001 From: Robert Metzger Date: Thu, 6 May 2021 11:11:47 +0200 Subject: [PATCH 3/3] address comments --- docs/content/docs/deployment/config.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/content/docs/deployment/config.md b/docs/content/docs/deployment/config.md index 6711ce55dc58b..907af78a812b7 100644 --- a/docs/content/docs/deployment/config.md +++ b/docs/content/docs/deployment/config.md @@ -34,7 +34,9 @@ The configuration is parsed and evaluated when the Flink processes are started. The out of the box configuration will use your default Java installation. You can manually set the environment variable `JAVA_HOME` or the configuration key `env.java.home` in `conf/flink-conf.yaml` if you want to manually override the Java runtime to use. -You can specifiy a different configuration directory location by defining the `FLINK_CONF_DIR` environment variable. For resource-providers which provide non-session deployments, you can specifiy per-job configurations this way. Make a copy of the `conf` directory from the Flink distribution and modify the settings on a per-job basis. Note that this is not supported in Docker or standalone Kubernetes deployments. +You can specify a different configuration directory location by defining the `FLINK_CONF_DIR` environment variable. For resource providers which provide non-session deployments, you can specify per-job configurations this way. Make a copy of the `conf` directory from the Flink distribution and modify the settings on a per-job basis. Note that this is not supported in Docker or standalone Kubernetes deployments. On Docker-based deployments, you can use the `FLINK_PROPERTIES` environment variable for passing configuration values. + +On session clusters, the provided configuration will only be used for configuring [execution](#execution) parameters, e.g. configuration parameters affecting the job, not the underlying cluster. # Basic Setup