Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Proxy support in Flank #527

Closed
leinardi opened this issue Mar 20, 2019 · 9 comments
Closed

Proxy support in Flank #527

leinardi opened this issue Mar 20, 2019 · 9 comments
Assignees
@bootstraponline
Labels
Blocked Feature

Comments

@leinardi
Copy link

When I try to run java flank.jar firebase test android run I get this error:

$ java -jar flank.jar firebase test android run
Request failed, retrying 1x java.net.ConnectException: Connection refused (Connection refused)
Request failed, retrying 2x java.net.ConnectException: Connection refused (Connection refused)
Request failed, retrying 3x java.net.ConnectException: Connection refused (Connection refused)
Request failed, retrying 4x java.net.ConnectException: Connection refused (Connection refused)
Request failed, retrying 5x java.net.ConnectException: Connection refused (Connection refused)
Request failed, retrying 6x java.net.ConnectException: Connection refused (Connection refused)
Request failed, retrying 7x java.net.ConnectException: Connection refused (Connection refused)
Request failed, retrying 8x java.net.ConnectException: Connection refused (Connection refused)
Request failed, retrying 9x java.net.ConnectException: Connection refused (Connection refused)
Request failed, retrying 10x java.net.ConnectException: Connection refused (Connection refused)
Exception in thread "main" picocli.CommandLine$ExecutionException: Error while running command (ftl.cli.firebase.test.android.AndroidRunCommand@67304a40): java.io.IOException: Request failed
        at picocli.CommandLine.execute(CommandLine.java:1168)
        at picocli.CommandLine.access$800(CommandLine.java:141)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:1367)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:1335)
        at picocli.CommandLine$AbstractParseResultHandler.handleParseResult(CommandLine.java:1243)
        at picocli.CommandLine.parseWithHandlers(CommandLine.java:1526)
        at picocli.CommandLine.run(CommandLine.java:1974)
        at picocli.CommandLine.run(CommandLine.java:1920)
        at ftl.Main$Companion.main(Main.kt:41)
        at ftl.Main.main(Main.kt)
Caused by: java.io.IOException: Request failed
        at ftl.http.ExecuteWithRetryKt.executeWithRetry(ExecuteWithRetry.kt:20)
        at ftl.gc.GcToolResults.getDefaultBucket(GcToolResults.kt:79)
        at ftl.args.ArgsHelper.createGcsBucket(ArgsHelper.kt:110)
        at ftl.args.AndroidArgs.<init>(AndroidArgs.kt:95)
        at ftl.args.AndroidArgs$Companion.load(AndroidArgs.kt:197)
        at ftl.args.AndroidArgs$Companion.load(AndroidArgs.kt:188)
        at ftl.cli.firebase.test.android.AndroidRunCommand.run(AndroidRunCommand.kt:33)
        at picocli.CommandLine.execute(CommandLine.java:1160)
        ... 9 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:666)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
        at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
        at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
        at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1334)
        at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1309)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:259)
        at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:113)
        at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:84)
        at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1040)
        at com.google.auth.oauth2.UserCredentials.refreshAccessToken(UserCredentials.java:227)
        at com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:181)
        at com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:167)
        at com.google.auth.http.HttpCredentialsAdapter.initialize(HttpCredentialsAdapter.java:96)
        at ftl.http.HttpTimeoutIncrease.initialize(HttpTimeoutIncrease.kt:10)
        at com.google.api.client.http.HttpRequestFactory.buildRequest(HttpRequestFactory.java:93)
        at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.buildHttpRequest(AbstractGoogleClientRequest.java:397)
        at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:515)
        at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:448)
        at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:565)
        at ftl.http.ExecuteWithRetryKt.executeWithRetry(ExecuteWithRetry.kt:13)
        ... 16 more
16:20:18.575 [pool-3-thread-1] WARN com.bugsnag.delivery.SyncHttpDelivery - Error not reported to Bugsnag - exception when making request
java.net.ConnectException: Connection refused (Connection refused)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:666)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
        at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
        at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
        at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1334)
        at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1309)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:259)
        at com.bugsnag.delivery.SyncHttpDelivery.deliver(SyncHttpDelivery.java:82)
        at com.bugsnag.delivery.AsyncHttpDelivery$2.run(AsyncHttpDelivery.java:83)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

My company requires to use a proxy to access internet so I also tried with
java -Dhttp.proxyHost=http://localhost -Dhttp.proxyPort=3128 -Dhttps.proxyHost=http://localhost:3128 -Dhttps.proxyPort=3128 -jar flank.jar firebase test android run
but the result is the same.

If I use a direct connection to internet (no proxy in between) it works fine.

Any idea of how can I see if is a real proxy issue of if is some other network problem?
Is proxy supported at all by flank?
@bootstraponline
Copy link
Contributor

Does gcloud CLI work? I've never tried to use Flank with a proxy before.

Maybe if gcloud works I can figure out how they support a proxy.

@leinardi
Copy link
Author

Now I can just try something quick, like this:

$ gcloud firebase test android models list
┌───────────────────┬────────────────────┬─────────────────────────────────────┬──────────┬─────────────┬───────────────────┬───────────────┐
│      MODEL_ID     │        MAKE        │              MODEL_NAME             │   FORM   │  RESOLUTION │   OS_VERSION_IDS  │      TAGS     │
├───────────────────┼────────────────────┼─────────────────────────────────────┼──────────┼─────────────┼───────────────────┼───────────────┤
│ A0001             │ OnePlus            │ OnePlus One                         │ PHYSICAL │ 1920 x 1080 │ 22                │               │
│ FRT               │ HMD Global         │ Nokia 1                             │ PHYSICAL │  854 x 480  │ 27                │               │
│ G8142             │ Sony               │ Sony XPERIA XZ Premium              │ PHYSICAL │ 1920 x 1080 │ 25                │               │
│ G8441             │ Sony               │ Xperia XZ1 Compact                  │ PHYSICAL │ 1280 x 720  │ 26                │               │
│ HWMHA             │ Huawei             │ Huawei Mate 9                       │ PHYSICAL │ 1920 x 1080 │ 24                │               │
│ Nexus10           │ Samsung            │ Nexus 10                            │ VIRTUAL  │ 2560 x 1600 │ 19,21,22          │               │
│ Nexus4            │ LG                 │ Nexus 4                             │ VIRTUAL  │ 1280 x 768  │ 19,21,22          │               │
│ Nexus5            │ LG                 │ Nexus 5                             │ VIRTUAL  │ 1920 x 1080 │ 19,21,22,23       │               │
│ Nexus5X           │ LG                 │ Nexus 5X                            │ VIRTUAL  │ 1920 x 1080 │ 23,24,25,26       │               │
│ Nexus6            │ Motorola           │ Nexus 6                             │ VIRTUAL  │ 2560 x 1440 │ 21,22,23,24,25    │               │
│ Nexus6P           │ Google             │ Nexus 6P                            │ VIRTUAL  │ 2560 x 1440 │ 23,24,25,26,27    │               │
│ Nexus7            │ Asus               │ Nexus 7 (2012)                      │ VIRTUAL  │ 1280 x 800  │ 19,21,22          │               │
│ Nexus7_clone_16_9 │ Generic            │ Nexus7 clone, DVD 16:9 aspect ratio │ VIRTUAL  │ 1280 x 720  │ 23,24,25,26       │ beta          │
│ Nexus9            │ HTC                │ Nexus 9                             │ VIRTUAL  │ 2048 x 1536 │ 21,22,23,24,25    │               │
│ NexusLowRes       │ Generic            │ Low-resolution MDPI phone           │ VIRTUAL  │  640 x 360  │ 23,24,25,26,27,28 │               │
│ OnePlus5          │ OnePlus            │ OnePlus 5                           │ PHYSICAL │ 1920 x 1080 │ 26                │               │
│ Pixel2            │ Google             │ Pixel 2                             │ VIRTUAL  │ 1920 x 1080 │ 26,27,28          │               │
│ a5y17lte          │ Samsung            │ Galaxy A5 2017                      │ PHYSICAL │ 1920 x 1080 │ 24                │               │
│ athene            │ Motorola           │ Moto G4 Plus                        │ PHYSICAL │ 1920 x 1080 │ 23                │               │
│ athene_f          │ Motorola           │ Moto G4                             │ PHYSICAL │ 1920 x 1080 │ 23                │               │
│ cheryl            │ Razer              │ Razer Phone                         │ PHYSICAL │ 2560 x 1440 │ 25                │               │
│ crownqlteue       │ Samsung            │ Galaxy Note 9 USA                   │ PHYSICAL │ 2220 x 1080 │ 27                │               │
│ flo               │ Asus               │ Nexus 7 (2013)                      │ PHYSICAL │ 1920 x 1200 │ 19,21             │               │
│ g3                │ LG                 │ LG G3                               │ PHYSICAL │ 2560 x 1440 │ 19                │               │
│ grandpplte        │ Samsung            │ Galaxy J2 Prime SM-G532M            │ PHYSICAL │  960 x 540  │ 23                │               │
│ griffin           │ Motorola           │ Moto Z XT1650                       │ PHYSICAL │ 2560 x 1440 │ 24                │               │
│ hammerhead        │ LG                 │ Nexus 5                             │ PHYSICAL │ 1920 x 1080 │ 21,23             │               │
│ harpia            │ Motorola           │ Moto G Play (4th Gen) XT1607        │ PHYSICAL │ 1280 x 720  │ 23                │               │
│ hero2lte          │ Samsung            │ Galaxy S7 edge                      │ PHYSICAL │ 1440 x 2560 │ 23                │               │
│ herolte           │ Samsung            │ Galaxy S7                           │ PHYSICAL │ 1440 x 2560 │ 23                │ deprecated=24 │
│ hlte              │ Samsung            │ Galaxy Note 3 Duos                  │ PHYSICAL │ 1920 x 1080 │ 19                │               │
│ htc_m8            │ HTC                │ HTC One (M8)                        │ PHYSICAL │ 1920 x 1080 │ 19                │               │
│ hwALE-H           │ Huawei             │ Huawei P8 lite                      │ PHYSICAL │ 1280 x 720  │ 21                │               │
│ j1acevelte        │ Samsung            │ Galaxy J1 ace SM-J111M              │ PHYSICAL │  800 x 480  │ 22                │               │
│ j7xelte           │ Samsung            │ Galaxy J7 (SM-J710MN)               │ PHYSICAL │ 1280 x 720  │ 23                │               │
│ lt02wifi          │ Samsung            │ Galaxy Tab 3                        │ PHYSICAL │  600 x 1024 │ 19                │               │
│ lucye             │ LG                 │ LG G6 LGUS997                       │ PHYSICAL │ 2880 x 1440 │ 24                │               │
│ lv0               │ LG                 │ LG K3                               │ PHYSICAL │  854 x 480  │ 23                │               │
│ m0                │ Samsung            │ Samsung Galaxy S3                   │ PHYSICAL │ 1280 x 720  │ 18                │               │
│ mata              │ Essential Products │ Essential PH-1                      │ PHYSICAL │ 2560 x 1312 │ 25                │               │
│ mlv1              │ LG                 │ LG K4 (LG-X230)                     │ PHYSICAL │  854 x 480  │ 23                │               │
│ potter            │ Motorola           │ Moto G (5) Plus                     │ PHYSICAL │ 1920 x 1080 │ 24                │               │
│ sailfish          │ Google             │ Pixel                               │ PHYSICAL │ 1080 x 1920 │ 25,26,27,28       │               │
│ shamu             │ Motorola           │ Nexus 6                             │ PHYSICAL │ 2560 x 1440 │ 21,22,23          │               │
│ star2qlteue       │ Samsung            │ Samsung Galaxy S9+ (US)             │ PHYSICAL │ 2220 x 1080 │ 26                │               │
│ starqlteue        │ Samsung            │ Samsung Galaxy S9 (US)              │ PHYSICAL │ 2220 x 1080 │ 26                │               │
│ taimen            │ Google             │ Pixel 2 XL                          │ PHYSICAL │ 2880 x 1440 │ 26,27             │               │
│ victara           │ Motorola           │ Moto X                              │ PHYSICAL │ 1920 x 1080 │ 19                │               │
│ walleye           │ Google             │ Pixel 2                             │ PHYSICAL │ 1920 x 1080 │ 26,27,28          │ default       │
│ zeroflte          │ Samsung            │ Galaxy S6                           │ PHYSICAL │ 2560 x 1440 │ 23                │               │
└───────────────────┴────────────────────┴─────────────────────────────────────┴──────────┴─────────────┴───────────────────┴───────────────┘
WARNING: Some devices are deprecated. Learn more at https://firebase.google.com/docs/test-lab/available-testing-devices#deprecated

and it works, tomorrow I can try to run a test and see if everything is ok.

@bootstraponline
Copy link
Contributor

It'll be interesting to see if gcloud can run the test. If so then I'll look into how to enable Flank for this use case.

@leinardi
Copy link
Author

leinardi commented Mar 21, 2019 •
edited

Hi, I did some test and gcloud works fine on the company network, same as Flank v2, last time I tried it some months ago.

This is the output of gcloud:

$ gcloud firebase test android run \
>       --use-orchestrator \
>       --type instrumentation \
>       --app /<redacted>/Workspace/git/dsa_android/PAYBACK_global_italy_api19_debug_19.03.10270-680045955-f1d1ae12f1.apk \
>       --test /<redacted>/Workspace/git/dsa_android/app-global-italy-debug-androidTest.apk \
>       --device model=NexusLowRes,version=28,locale=de,orientation=portrait

Have questions, feedback, or issues? Get support by visiting:
  https://firebase.google.com/support/

Uploading [/<redacted>/Workspace/git/dsa_android/PAYBACK_global_italy_api19_debug_19.03.10270-680045955-f1d1ae12f1.apk] to Firebase Test Lab...
Uploading [/<redacted>/Workspace/git/dsa_android/app-global-italy-debug-androidTest.apk] to Firebase Test Lab...
Raw results will be stored in your GCS bucket at [https://console.developers.google.com/storage/browser/<redacted>/]

Test [matrix-32zlychrq6kuu] has been created in the Google Cloud.
Firebase Test Lab will execute your instrumentation test on 1 device(s).
Creating individual test executions...done.                                                                                                                                     

Test results will be streamed to [https://console.firebase.google.com/project/<redacted>].
12:49:58 Test is Pending
12:50:31 Starting attempt 1.
12:50:31 Test is Running
12:51:25 Started logcat recording.
12:51:25 Preparing device.
12:51:52 Logging in to Google account on device.
12:51:52 Installing apps.
12:52:05 Retrieving Pre-Test Package Stats information from the device.
12:52:05 Retrieving Performance Environment information from the device.
12:52:05 Started crash detection.
12:52:05 Started crash monitoring.
12:52:05 Started performance monitoring.
12:52:12 Started video recording.
12:52:12 Starting instrumentation test.
13:03:23 Completed instrumentation test.
13:03:36 Stopped performance monitoring.
13:04:02 Stopped crash monitoring.
13:04:02 Stopped logcat recording.
13:04:02 Retrieving Post-test Package Stats information from the device.
13:04:02 Logging out of Google account on device.
13:04:02 Done. Test time = 666 (secs)
13:04:02 Starting results processing. Attempt: 1
13:04:28 Completed results processing. Time taken = 21 (secs)
13:04:28 Test is Finished

Instrumentation testing complete.

More details are available at [https://console.firebase.google.com/project/<redacted>].
┌─────────┬────────────────────────────┬────────────────────────────────┐
│ OUTCOME │      TEST_AXIS_VALUE       │          TEST_DETAILS          │
├─────────┼────────────────────────────┼────────────────────────────────┤
│ Failed  │ NexusLowRes-28-de-portrait │ 8 test cases failed, 62 passed │
└─────────┴────────────────────────────┴────────────────────────────────┘

@bootstraponline
Copy link
Contributor

Ok, I'll need some way to reproduce the issue so I can tell if it's fixed.

It looks like gcloud CLI does support proxies:
https://github.com/bootstraponline/gcloud_cli/blob/0752e88b155a417a18d244c242b4ab3fb9aa1c1f/google-cloud-sdk/lib/googlecloudsdk/core/http_proxy.py

@bootstraponline
Copy link
Contributor

Ping me on the Firebase Test Lab Slack (https://firebase.community) and we can try debugging what's going on.

@leinardi
Copy link
Author

leinardi commented Mar 21, 2019 •
edited

Mmm, the first thing that come to my mind is to use a local squid instance and set some firewall rules that allow only squid to access internet.

Unfortunately my company doesn't use Slack (I know...), I have to get it installed, I will let you know as soon as I'm connected.

@bootstraponline bootstraponline changed the title java.net.ConnectException: Connection refused (Connection refused) Proxy support in Flank Mar 22, 2019
@bootstraponline bootstraponline self-assigned this Mar 22, 2019
@bootstraponline bootstraponline mentioned this issue Mar 25, 2019
Open
@bootstraponline
Copy link
Contributor

This is blocked on upstream support for proxies.

@bootstraponline
Copy link
Contributor

I'm pretty sure I could fix this if I had an easy way to reproduce the problem. Closing the ticket for now as non-reproducible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bootstraponline bootstraponline

Labels
Blocked Feature
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@leinardi @bootstraponline