Skip to content

Commit cda5121

Browse files
committed
[SPARK-54469][DSTREAM][TESTS] Fix StreamingContextSuite.stop slow receiver gracefully test to clean up SprakContext
### What changes were proposed in this pull request? This PR aims to fix `StreamingContextSuite.stop slow receiver gracefully` test case to clean up `SprakContext`. ### Why are the changes needed? According to the log, `20000s` setting is propagated to the other test case and causes flaky failures. - https://github.com/apache/spark/actions/runs/19608517338/job/56150848587 ``` - SPARK-22955 graceful shutdown shouldn't lead to job generation error *** FAILED *** The code passed to failAfter did not complete within 20 minutes. (StreamingContextSuite.scala:861) ``` `20000s` setting is originated from `StreamingContextSuite.stop slow receiver gracefully`. We should close the SparkContext correctly. https://github.com/apache/spark/blob/997525cae9b00e47626ce09f00e779b279551ace/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala#L342-L344 ### Does this PR introduce _any_ user-facing change? No behavior change. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #53182 from dongjoon-hyun/SPARK-54469. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent 997525c commit cda5121

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -365,6 +365,8 @@ class StreamingContextSuite
365365
assert(runningCount > 0)
366366
assert(runningCount == totalNumRecords)
367367
Thread.sleep(100)
368+
sc.stop()
369+
LocalStreamingContext.ensureNoActiveSparkContext()
368370
}
369371

370372
test ("registering and de-registering of streamingSource") {

0 commit comments

Comments
 (0)