Issue 4977 - Request for compressed response in AndroidWebRequest

libadblockplus-android/src/org/adblockplus/libadblockplus/android/AndroidWebRequest.java

Index: libadblockplus-android/src/org/adblockplus/libadblockplus/android/AndroidWebRequest.java
diff --git a/libadblockplus-android/src/org/adblockplus/libadblockplus/android/AndroidWebRequest.java b/libadblockplus-android/src/org/adblockplus/libadblockplus/android/AndroidWebRequest.java
index 848088a909c302cc3780704a707fe1ab35e71298..29bcdc0f722e330411b6d92c7cb4e3cb0cd4f4c4 100644
--- a/libadblockplus-android/src/org/adblockplus/libadblockplus/android/AndroidWebRequest.java
+++ b/libadblockplus-android/src/org/adblockplus/libadblockplus/android/AndroidWebRequest.java
@@ -18,6 +18,7 @@
 package org.adblockplus.libadblockplus.android;
 
 import java.io.BufferedReader;
+import java.io.InputStream;
 import java.io.InputStreamReader;
 import java.net.HttpURLConnection;
 import java.net.URL;
@@ -25,6 +26,7 @@ import java.util.HashSet;
 import java.util.LinkedList;
 import java.util.List;
 import java.util.Map;
+import java.util.zip.GZIPInputStream;
 
 import org.adblockplus.libadblockplus.AdblockPlusException;
 import org.adblockplus.libadblockplus.FilterEngine;
@@ -37,19 +39,31 @@ import android.util.Log;
 
 public class AndroidWebRequest extends WebRequest
 {
+  protected static final String ENCODING_GZIP = "gzip";
+  protected static final String ENCODING_IDENTITY = "identity";
+
   public final static String TAG = Utils.getTag(WebRequest.class);
 
   private final HashSet<String> subscriptionURLs = new HashSet<String>();
   private final boolean elemhideEnabled;
-
-  public AndroidWebRequest(boolean enableElemhide)
+  private final boolean compressedStream;
+
+  /**
+   * Ctor
+   * @param enableElemhide Enable element hiding?
+   *                       Element hiding requires significantly more memory
+   *                       but allows better ad blocking
+   * @param compressedStream Request for gzip compressed stream from the server
+   */
+  public AndroidWebRequest(boolean enableElemhide, boolean compressedStream)

[Felix Dahlke, 2017/03/27 14:13:16]

I don't really see a use case for passing `compressedStream` as `false` - why
not always compress?

[anton, 2017/03/27 18:56:13]

The server can not support it or some or compressing takes more battery - who
knows why it can be undesired.
In ctor with no arguments i use `true` it seems to be the best by default.

[anton, 2017/03/28 07:18:54]

* decompressing (on user's device)

[Felix Dahlke, 2017/03/28 08:52:00]

If the server doesn't support compression it'll just return an uncompressed
response. But yeah, maybe clients will want to overwrite this for reasons we
cannot think of, fine by me if we leave it like this.

   {
     this.elemhideEnabled = enableElemhide;
+    this.compressedStream = compressedStream;
   }
 
   public AndroidWebRequest()
   {
-    this(false);
+    this(false, true);
   }
 
   private boolean isListedSubscriptionUrl(final URL url)
@@ -84,6 +98,8 @@ public class AndroidWebRequest extends WebRequest
 
       final HttpURLConnection connection = (HttpURLConnection) url.openConnection();
       connection.setRequestMethod("GET");
+      connection.setRequestProperty("Accept-Encoding",
+        (compressedStream ? ENCODING_GZIP : ENCODING_IDENTITY));

[Felix Dahlke, 2017/03/27 14:13:16]

Why not just leave the `Accept-Encoding` header out if `compressedStream` is
false? That's how the behaviour was before.

[anton, 2017/03/27 18:56:13]

Not true sometimes, previously it depended on android version:
https://android-developers.googleblog.com/2011/09/androids-http-clients.html
"In Gingerbread, we added transparent response compression"

So i decided to add ENCODING_IDENTITY if the compressed stream is not desired
for some reason.

[Felix Dahlke, 2017/03/28 08:52:00]

Acknowledged.

       connection.connect();
 
       final ServerResponse response = new ServerResponse();
@@ -91,7 +107,11 @@ public class AndroidWebRequest extends WebRequest
 
       if (response.getResponseStatus() == 200)
       {
-        final BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream(), "UTF-8"));
+        final InputStream inputStream =
+          (compressedStream && ENCODING_GZIP.equals(connection.getContentEncoding())
+            ? new GZIPInputStream(connection.getInputStream())
+            : connection.getInputStream());
+        final BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream, "UTF-8"));
         final StringBuilder sb = new StringBuilder();
 
         String line;