New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to avoid mid-write of a BLOB in GCS? #790
Comments
I investigated the hadoop-connectors project code and opted to use reflection in order to get access to
I have made an own version of the
Would it be possible to expose |
Let's take the following code snippet as context:
When the
FSDataOutputStream
output stream is closed the filelines.txt
will appear on GCS containing only the content:I'm exploring how to reach the functionality of seeing on GCS a file either when the file has been fully written or not see it all in the situation when it has been mid-written.
One possibility of doing this would be to close the stream only and only if ALL the content of the file has been successfully written.
In such a case however, when an exception occurs while writing the file, the stream would remain unclosed resulting in "leaking" of resources within the application.
Is there a recipe of achieving the functionality of writing a BLOB on GCS with the following constraints:
The text was updated successfully, but these errors were encountered: