Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Java VerifyError occurs when using com.google.cloud libraries-bom > 26.31.0 #10698

Open
rnaval opened this issue Apr 15, 2024 · 9 comments
Open

Java VerifyError occurs when using com.google.cloud libraries-bom > 26.31.0 #10698

rnaval opened this issue Apr 15, 2024 · 9 comments
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. type: question Request for information or clarification. Not an issue.

Comments

@rnaval
Copy link

rnaval commented Apr 15, 2024 •
edited

Environment details

  1. API tested: CloudFilestoreManagerClient.create(), SubscriptionAdminClient.create()
  2. OS type and version: CentOS 7 docker
  3. Java version: Java 17
  4. Version(s):
  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>com.google.cloud</groupId>
        <artifactId>libraries-bom</artifactId>
        <version>26.37.0</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>
    <dependency>
      <groupId>com.google.cloud</groupId>
      <artifactId>google-cloud-monitoring</artifactId>
    </dependency>
    <dependency>
      <groupId>com.google.cloud</groupId>
      <artifactId>google-cloud-compute</artifactId>
    </dependency>
    <dependency>
      <groupId>com.google.cloud</groupId>
      <artifactId>google-cloud-storage</artifactId>
    </dependency>
    <dependency>
      <groupId>com.google.cloud</groupId>
      <artifactId>google-cloud-filestore</artifactId>
    </dependency>
    <dependency>
      <groupId>com.google.protobuf</groupId>
      <artifactId>protobuf-java-util</artifactId>
      <version>4.26.1</version>
    </dependency>
  </dependencies>

Code example

  private CloudFilestoreManagerClient createClient() {
    try {
      return CloudFilestoreManagerClient.create();
    } catch (Exception e) {
      logger.get().error("{} Error when creating client for CloudFilestoreManagerClient : {}",
          this.getClass().getSimpleName(), e.getMessage());
    }
    return null;
  }

Stack trace

java.lang.VerifyError: Bad return type
Exception Details:
  Location:
    com/google/cloud/filestore/v1/Instance.internalGetMapFieldReflection(I)Lcom/google/protobuf/MapFieldReflectionAccessor; @24: areturn
  Reason:
    Type 'com/google/protobuf/MapField' (current frame, stack[0]) is not assignable to 'com/google/protobuf/MapFieldReflectionAccessor' (from method signature)
  Current Frame:
    bci: @24
    flags: { }
    locals: { 'com/google/cloud/filestore/v1/Instance', integer }
    stack: { 'com/google/protobuf/MapField' }
  Bytecode:
    0000000: 1bab 0000 0000 0018 0000 0001 0000 0009
    0000010: 0000 0013 2ab7 0004 b0bb 001d 59bb 001e
    0000020: 59b7 001f 1220 b600 211b b600 22b6 0023
    0000030: b700 24bf                              
  Stackmap Table:
    same_frame(@20)
    same_frame(@25)

	at com.google.cloud.filestore.v1.stub.GrpcCloudFilestoreManagerStub.<clinit>(GrpcCloudFilestoreManagerStub.java:88)
	at com.google.cloud.filestore.v1.stub.CloudFilestoreManagerStubSettings.createStub(CloudFilestoreManagerStubSettings.java:494)
	at com.google.cloud.filestore.v1.CloudFilestoreManagerClient.<init>(CloudFilestoreManagerClient.java:517)
	at com.google.cloud.filestore.v1.CloudFilestoreManagerClient.create(CloudFilestoreManagerClient.java:499)
	at com.google.cloud.filestore.v1.CloudFilestoreManagerClient.create(CloudFilestoreManagerClient.java:490)

Another one:

java.lang.VerifyError: Bad return type
Exception Details:
Location:
com/google/pubsub/v1/Subscription.internalGetMapFieldReflection(I)Lcom/google/protobuf/MapFieldReflectionAccessor; @24: areturn
Reason:
Type 'com/google/protobuf/MapField' (current frame, stack[0]) is not assignable to 'com/google/protobuf/MapFieldReflectionAccessor' (from method signature)
Current Frame:
bci: @24
flags: { }
locals: { 'com/google/pubsub/v1/Subscription', integer }
stack: { 'com/google/protobuf/MapField' }
Bytecode:
0000000: 1bab 0000 0000 0018 0000 0001 0000 0009
0000010: 0000 0013 2ab7 0007 b0bb 0069 59bb 006b
0000020: 59b7 006d 126e b600 701b b600 74b6 0077
0000030: b700 7bbf
Stackmap Table:
same_frame(@20)
same_frame(@25)

    at com.google.cloud.pubsub.v1.stub.GrpcSubscriberStub.<clinit>(GrpcSubscriberStub.java:78)
    at com.google.cloud.pubsub.v1.stub.SubscriberStubSettings.createStub(SubscriberStubSettings.java:376)
    at com.google.cloud.pubsub.v1.SubscriptionAdminClient.<init>(SubscriptionAdminClient.java:544)
    at com.google.cloud.pubsub.v1.SubscriptionAdminClient.create(SubscriptionAdminClient.java:526)
    at com.google.cloud.pubsub.v1.SubscriptionAdminClient.create(SubscriptionAdminClient.java:517)

Any additional information below

Issue only occurs when we use 26.32.0 onwards. 26.31.0 works fine.

Unit tests seem to be working fine for all versions, but not sure what is happening in the docker image.
@diegomarquezp diegomarquezp added type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. priority: p2 Moderately-important priority. Fix may not be included in next release. labels Apr 16, 2024
@diegomarquezp
Copy link
Contributor

diegomarquezp commented Apr 16, 2024 •
edited

Hi @rnaval, thanks for reporting this. Would you mind sharing the tag of the docker image you are using? Has this occurred when running your java application on your host machine? Thanks!

@diegomarquezp diegomarquezp added type: question Request for information or clarification. Not an issue. and removed type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Apr 16, 2024
@diegomarquezp
Copy link
Contributor

It looks like protobuf-java-util may be involved (thanks @suztomo).

@diegomarquezp
Copy link
Contributor

diegomarquezp commented Apr 16, 2024 •
edited

@rnaval our libraries are not using protobuf 4.26 just yet. Would you mind trying to modify your dependency on java-protobuf-util to 3.25.3 3.25.2 (the one our libraries use)?

From https://github.com/googleapis/java-cloud-bom/releases/tag/v26.37.0
image

@rnaval
Copy link
Author

rnaval commented Apr 17, 2024

Thanks for the response.

Tried it out with the recommended protobuf library, issue still occurs.

java.lang.VerifyError: Bad return type
Exception Details:
  Location:
    com/google/cloud/filestore/v1/Instance.internalGetMapFieldReflection(I)Lcom/google/protobuf/MapFieldReflectionAccessor; @24: areturn
  Reason:
    Type 'com/google/protobuf/MapField' (current frame, stack[0]) is not assignable to 'com/google/protobuf/MapFieldReflectionAccessor' (from method signature)
  Current Frame:
    bci: @24
    flags: { }
    locals: { 'com/google/cloud/filestore/v1/Instance', integer }
    stack: { 'com/google/protobuf/MapField' }
  Bytecode:
    0000000: 1bab 0000 0000 0018 0000 0001 0000 0009
    0000010: 0000 0013 2ab7 0004 b0bb 001d 59bb 001e
    0000020: 59b7 001f 1220 b600 211b b600 22b6 0023
    0000030: b700 24bf                              
  Stackmap Table:
    same_frame(@20)
    same_frame(@25)

	at com.google.cloud.filestore.v1.stub.GrpcCloudFilestoreManagerStub.<clinit>(GrpcCloudFilestoreManagerStub.java:88)
	at com.google.cloud.filestore.v1.stub.CloudFilestoreManagerStubSettings.createStub(CloudFilestoreManagerStubSettings.java:494)
	at com.google.cloud.filestore.v1.CloudFilestoreManagerClient.<init>(CloudFilestoreManagerClient.java:517)
	at com.google.cloud.filestore.v1.CloudFilestoreManagerClient.create(CloudFilestoreManagerClient.java:499)
	at com.google.cloud.filestore.v1.CloudFilestoreManagerClient.create(CloudFilestoreManagerClient.java:490)

Docker image is based on CentOS 7:

FROM centos:7
...

Here's a look at the dependency tree:

INFO] +- com.google.cloud:google-cloud-monitoring:jar:3.42.0:compile
[INFO] |  +- io.grpc:grpc-api:jar:1.62.2:compile
[INFO] |  +- com.google.code.findbugs:jsr305:jar:3.0.2:compile
[INFO] |  +- com.google.errorprone:error_prone_annotations:jar:2.26.1:compile
[INFO] |  +- io.grpc:grpc-stub:jar:1.62.2:compile
[INFO] |  +- io.grpc:grpc-protobuf:jar:1.62.2:compile
[INFO] |  +- io.grpc:grpc-protobuf-lite:jar:1.62.2:runtime
[INFO] |  +- com.google.api:api-common:jar:2.29.1:compile
[INFO] |  +- com.google.auto.value:auto-value-annotations:jar:1.10.4:compile
[INFO] |  +- javax.annotation:javax.annotation-api:jar:1.3.2:compile
[INFO] |  +- com.google.j2objc:j2objc-annotations:jar:3.0.0:compile
[INFO] |  +- com.google.protobuf:protobuf-java:jar:3.25.2:compile
[INFO] |  +- com.google.api.grpc:proto-google-common-protos:jar:2.37.1:compile
[INFO] |  +- com.google.api.grpc:proto-google-cloud-monitoring-v3:jar:3.42.0:compile
[INFO] |  +- com.google.guava:guava:jar:33.1.0-jre:provided
[INFO] |  +- com.google.guava:failureaccess:jar:1.0.2:compile
[INFO] |  +- com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava:compile
[INFO] |  +- org.checkerframework:checker-qual:jar:3.42.0:compile
[INFO] |  +- com.google.api:gax:jar:2.46.1:compile
[INFO] |  +- com.google.auth:google-auth-library-credentials:jar:1.23.0:compile
[INFO] |  +- io.opencensus:opencensus-api:jar:0.31.1:compile
[INFO] |  +- io.grpc:grpc-context:jar:1.62.2:compile
[INFO] |  +- io.opentelemetry:opentelemetry-api:jar:1.36.0:compile
[INFO] |  +- io.opentelemetry:opentelemetry-context:jar:1.36.0:compile
[INFO] |  +- com.google.api:gax-grpc:jar:2.46.1:compile
[INFO] |  +- io.grpc:grpc-inprocess:jar:1.62.2:compile
[INFO] |  +- io.grpc:grpc-core:jar:1.62.2:compile
[INFO] |  +- com.google.android:annotations:jar:4.1.1.4:runtime
[INFO] |  +- org.codehaus.mojo:animal-sniffer-annotations:jar:1.23:runtime
[INFO] |  +- io.grpc:grpc-alts:jar:1.62.2:compile
[INFO] |  +- io.grpc:grpc-grpclb:jar:1.62.2:compile
[INFO] |  +- org.conscrypt:conscrypt-openjdk-uber:jar:2.5.2:compile
[INFO] |  +- io.grpc:grpc-auth:jar:1.62.2:compile
[INFO] |  +- io.grpc:grpc-netty-shaded:jar:1.62.2:compile
[INFO] |  +- io.grpc:grpc-util:jar:1.62.2:runtime
[INFO] |  +- io.perfmark:perfmark-api:jar:0.27.0:runtime
[INFO] |  +- io.grpc:grpc-googleapis:jar:1.62.2:runtime
[INFO] |  +- io.grpc:grpc-xds:jar:1.62.2:runtime
[INFO] |  +- io.opencensus:opencensus-proto:jar:0.2.0:runtime
[INFO] |  +- io.grpc:grpc-services:jar:1.62.2:runtime
[INFO] |  +- com.google.re2j:re2j:jar:1.7:runtime
[INFO] |  +- org.threeten:threetenbp:jar:1.6.8:compile
[INFO] |  +- com.google.auth:google-auth-library-oauth2-http:jar:1.23.0:compile
[INFO] |  +- com.google.http-client:google-http-client:jar:1.44.1:compile
[INFO] |  +- org.apache.httpcomponents:httpclient:jar:4.5.14:compile
[INFO] |  +- commons-codec:commons-codec:jar:1.16.1:compile
[INFO] |  +- org.apache.httpcomponents:httpcore:jar:4.4.16:compile
[INFO] |  +- io.opencensus:opencensus-contrib-http-util:jar:0.31.1:compile
[INFO] |  +- com.google.http-client:google-http-client-gson:jar:1.44.1:compile
[INFO] |  \- com.google.code.gson:gson:jar:2.10.1:compile
[INFO] +- com.google.cloud:google-cloud-compute:jar:1.51.0:compile
[INFO] |  +- com.google.api.grpc:proto-google-cloud-compute-v1:jar:1.51.0:compile
[INFO] |  \- com.google.api:gax-httpjson:jar:2.46.1:compile
[INFO] +- com.google.cloud:google-cloud-storage:jar:2.36.1:compile
[INFO] |  +- com.google.http-client:google-http-client-jackson2:jar:1.44.1:compile
[INFO] |  +- com.google.api-client:google-api-client:jar:2.4.0:compile
[INFO] |  +- com.google.oauth-client:google-oauth-client:jar:1.35.0:compile
[INFO] |  +- com.google.http-client:google-http-client-apache-v2:jar:1.44.1:compile
[INFO] |  +- com.google.apis:google-api-services-storage:jar:v1-rev20240311-2.0.0:compile
[INFO] |  +- com.google.cloud:google-cloud-core:jar:2.36.1:compile
[INFO] |  +- com.google.cloud:google-cloud-core-http:jar:2.36.1:compile
[INFO] |  +- com.google.http-client:google-http-client-appengine:jar:1.44.1:compile
[INFO] |  +- com.google.cloud:google-cloud-core-grpc:jar:2.36.1:compile
[INFO] |  +- com.google.api.grpc:proto-google-iam-v1:jar:1.32.1:compile
[INFO] |  +- com.google.api.grpc:proto-google-cloud-storage-v2:jar:2.36.1-alpha:compile
[INFO] |  +- com.google.api.grpc:grpc-google-cloud-storage-v2:jar:2.36.1-alpha:compile
[INFO] |  +- com.google.api.grpc:gapic-google-cloud-storage-v2:jar:2.36.1-alpha:compile
[INFO] |  +- com.fasterxml.jackson.core:jackson-core:jar:2.16.1:compile
[INFO] |  \- io.grpc:grpc-rls:jar:1.62.2:runtime
[INFO] +- com.google.cloud:google-cloud-filestore:jar:1.42.0:compile
[INFO] |  +- com.google.api.grpc:proto-google-cloud-filestore-v1:jar:1.42.0:compile
[INFO] |  \- com.google.api.grpc:proto-google-cloud-filestore-v1beta1:jar:0.44.0:compile
[INFO] +- com.google.protobuf:protobuf-java-util:jar:3.25.2:compile

@suztomo
Copy link
Member

suztomo commented Apr 17, 2024

@rnaval Do you observe the problem only happens in the CentOS docker container?

@diegomarquezp
Copy link
Contributor

cc: @suztomo
Hi @rnaval, I created a reproducer based on our comments here.

I could run the main function without any issues in IntelliJ. +1 to @suztomo asking if the problem occurs in your host machine as well.

@rnaval
Copy link
Author

rnaval commented Apr 29, 2024

Sorry for the late response. Based on testing results, we have verified that it works on:

  • MacOS host machine
  • Windows host machine

We also tried it on a Centos 7 VM - same error was encountered.

@rnaval
Copy link
Author

rnaval commented May 17, 2024

@diegomarquezp @suztomo Also tried it out with Ubuntu 22.04 docker, seems to work.

@diegomarquezp
Copy link
Contributor

What's the image tag of this CentOS image? Are you using a dockerfile with this image as base or are you running your code directly on it?
Maybe a small reproducer such as a bash script showing how you setup your code can help us find out what's missing in this CentOS image.

If no dockerfile is involved, then we should look at the jdk versions in each environment you tested, or more generally the differences in each execution environment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. type: question Request for information or clarification. Not an issue.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@suztomo @rnaval @diegomarquezp