Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: First pass downloadMany #1945

Merged
merged 2 commits into from
Mar 23, 2023
Merged

Conversation

sydney-munro
Copy link
Collaborator

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> ☕️

If you write sample code, please follow the samples format.

@sydney-munro sydney-munro requested a review from a team as a code owner March 22, 2023 20:54
@product-auto-label product-auto-label bot added the size: m Pull request size is medium. label Mar 22, 2023
@product-auto-label product-auto-label bot added the api: storage Issues related to the googleapis/java-storage API. label Mar 22, 2023
Copy link
Contributor

@danielduhh danielduhh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks solid just had a couple questions & request for method docs

public void downloadBlobs() {}
public void downloadBlobs() throws IOException {
TransferManagerConfig config =
TransferManagerConfig.newBuilder().setAllowChunking(false).setMaxWorkers(1).build();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't setAllowChunking false by default?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes it is, can remove that.


@Override
public DownloadResult call() throws Exception {
// TODO: Check for chunking
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the plan here? To check setAllowChunking, and decide whether to download byte stream ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, we would check allowChunking from the ParallelDownloadConfig first to ensure that the customer is allowing us to chunk downloads on their behalf, then we would check the file size to see if we want to do direct or chunked download.

@@ -56,6 +56,14 @@ public TransferManagerImpl(TransferManagerConfig transferManagerConfig) {

@Override
public @NonNull DownloadJob downloadBlobs(List<BlobInfo> blobs, ParallelDownloadConfig opts) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docs will be a followup, just getting first passes out first.

@ddelgrosso1
Copy link

Overall the implementation looks good to me. I will leave it to the Java experts to look more closely at the specifics.

@sydney-munro sydney-munro merged commit fedbd9c into feat/transfer-manager Mar 23, 2023
@sydney-munro sydney-munro deleted the transfer-download branch March 23, 2023 22:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the googleapis/java-storage API. size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants