SAMZA-2276: Add Metdata store putAll API#1112
SAMZA-2276: Add Metdata store putAll API#1112dnishimura wants to merge 2 commits intoapache:masterfrom
Conversation
mynameborat
left a comment
There was a problem hiding this comment.
A few questions to understand the motivation of PutAll
- Does
Puthave to be synchronous? Sinceflush()is exposed as part of API, what do you think about callers handling it in two steps? Have we evaluated this? - What is the use of
flush()? Looking at the implementation ofPutandPutAll, both are synchronous. Do we intend to have different behavior for future MetadataStore implementation? If so, I suppose we should follow up this PR with another PR that details the guarantees and expectations of these APIs.
I believe the original intent of the metadata store is to have consistent behavior regardless of the underlying store. Since it's simpler to make async calls synchronous than vice-versa, the expected behavior is synchronous. The intent of the |
|
@mynameborat if you don't have any more comments, please approve and merge this commit. This is required to address an internal performance regression with job startup time. Thanks! |
mynameborat
left a comment
There was a problem hiding this comment.
Thanks for the clarification. Minor documentation comment.
Let me know when its ready to be merged.
samza-api/src/main/java/org/apache/samza/metadatastore/MetadataStore.java
Show resolved
Hide resolved
|
@mynameborat thanks for the review. I addressed your comment. This is ready to be merged. |
Add a putsAll API to the metadata store API. This allows for a delayed flush() call for underlying stores that require a flush() after write such as Kafka.