Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Atomic operation when writing to Icebergtables via flink #11944

Open
adityakumarsinha opened this issue Jan 10, 2025 · 3 comments
Open

Atomic operation when writing to Icebergtables via flink #11944

adityakumarsinha opened this issue Jan 10, 2025 · 3 comments
Labels
question Further information is requested

Comments

@adityakumarsinha
Copy link

Query engine

Flink

Question

I have a requirement to write to multiple Iceberg tables based on single source event. I like to know whether multiple iceberg sinks / commiter can write both the table atomically within one checkpoint.

@adityakumarsinha adityakumarsinha added the question Further information is requested label Jan 10, 2025
@pvary
Copy link
Contributor

pvary commented Jan 10, 2025

There are currently no multitable commits available in Iceberg.

Flink will make sure that the data is written exactly once into the tables, but there is no consistency guarantees between the 2 tables

@adityakumarsinha
Copy link
Author

In Iceberg transactions it is mentioned that Nessie supports multi-table transactions. How that works?

@pvary
Copy link
Contributor

pvary commented Jan 13, 2025

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants