[Bug]: Portable runner queryMetrics returns duplicate result

[Abacn]

What happened?

For portable runner pipeline result, it is found running PipelineResult.metrics().queryMetrics(...).getCounter/getDistribution/etc returns each metric twice, found in #31999 (comment)

The reson is in

beam/runners/portability/java/src/main/java/org/apache/beam/runners/portability/PortableMetrics.java

Lines 93 to 94 in 4479281

	monitoringInfoList.addAll(jobMetrics.getAttemptedList());
	monitoringInfoList.addAll(jobMetrics.getCommittedList());

it adds both attempted list and committed list, which essentially has the same info, then in convert...MonitoringInfoTo... always creates attempted metrics, e.g. here

beam/runners/portability/java/src/main/java/org/apache/beam/runners/portability/PortableMetrics.java

Line 191 in 4479281

return MetricResult.create(key, false, decodeInt64Counter(counterMonInfo.getPayload()));

(second parameter isCommitSupported=False)

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[github-actions]

This issue has been marked as stale due to 150 days of inactivity. It will be closed in 30 days if no further activity occurs. If you think that’s incorrect or this issue still needs to be addressed, please simply write any comment. If closed, you can reopen the issue at any time. Thank you for your contributions.