Issue 4954 - Implement a downloader script supporting OAuth2 authentication/authorization

sitescripts/oauth2dl/bin/oauth2dl.py

Index: sitescripts/oauth2dl/bin/oauth2dl.py
diff --git a/sitescripts/oauth2dl/bin/oauth2dl.py b/sitescripts/oauth2dl/bin/oauth2dl.py
new file mode 100644
index 0000000000000000000000000000000000000000..7925ee7ef629c70e500d6237c0de11506ae51647
--- /dev/null
+++ b/sitescripts/oauth2dl/bin/oauth2dl.py
@@ -0,0 +1,123 @@
+# This file is part of the Adblock Plus web scripts,
+# Copyright (C) 2006-present eyeo GmbH
+#
+# Adblock Plus is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# Adblock Plus is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Adblock Plus.  If not, see <http://www.gnu.org/licenses/>.
+
+
+import argparse

[Vasily Kuznetsov, 2018/07/20 21:00:01]

PEP8 recommends breaking the imports into three groups: stdlib, 3-rd party
libraries, local imports, and separating the three groups by an empty line.

[Tudor Avram, 2018/07/23 19:29:03]

Acknowledged.

[Tudor Avram, 2018/07/31 09:20:16]

Done.

+import codecs
+import os
+import sys
+from httplib2 import Http
+from oauth2client.service_account import ServiceAccountCredentials
+import json
+
+try:

[Vasily Kuznetsov, 2018/07/20 21:00:00]

I think we can assume that the root of sitescripts is in the PYTHONPATH so the
first import is sufficient.

[Tudor Avram, 2018/07/23 19:29:03]

That was to allow it to run both as `python -m ...` and as `python ___.py`. When
I run it as a module, it works, but as a simple file the import breaks. At least
on my machine 

[Tudor Avram, 2018/07/31 09:20:17]

Done.

+    from sitescripts.oauth2dl.bin import constants as cnts
+except ImportError:
+    import constants as cnts
+
+
+def write_to_file(content, path):

[Vasily Kuznetsov, 2018/07/20 21:00:00]

I'm not sure if it really makes sense to have this in a separate function, but
don't feel too strong about it.

[Tudor Avram, 2018/07/23 19:29:03]

Acknowledged.

[Tudor Avram, 2018/07/31 09:20:16]

Done.

+    """Write data to file.
+
+    Parameters
+    ----------
+    content: str
+        The data to write.
+    path: str
+        Path to the file we write to.
+    """
+    with codecs.open(path, encoding='utf-8', mode='wb') as f:
+        f.write(content)
+
+
+def download_file(url, key_file, scope):
+    """Download a file using Oauth2.
+
+    Parameters
+    ----------
+    url: str
+        The url of the file we want to download
+    key_file: str
+        Path/ url to key file used in Oauth2
+    scope: str
+        The scope used in Oauth2
+
+    Returns
+    -------
+    dict
+        Headers resulted from the HTTP request.
+    str
+        The content of the file we want to download/
+        error message if it was unsuccessful.
+    """
+    credentials = ServiceAccountCredentials.from_json_keyfile_name(key_file, scopes=[scope])

[Vasily Kuznetsov, 2018/07/20 21:00:00]

PEP8 recommends keeping the lines under 80 characters. This line can be
converted this way so I would propose to do it.

[Tudor Avram, 2018/07/23 19:29:03]

Yeah, I know. that's why I added the flake8 exception in `.tox`. I just couldn't
find a nice way to break that into multiple lines. 

I just realised that I can try something like: 

credentials = \
          ServiceAccount.....

[Tudor Avram, 2018/07/31 09:20:16]

Done.

+
+    http_auth = credentials.authorize(Http())
+
+    headers, content = http_auth.request(url)
+    try:
+        content = content.decode('utf-8')
+    finally:
+        return headers, content
+
+
+def parse_args():
+    """Set up the required arguments and returns them."""
+    parser = argparse.ArgumentParser(description='Download using Oauth2')
+
+    parser.add_argument('url', help='URL to download from')
+    parser.add_argument('-k', '--key', help='Oauth2 key file path',
+                        default=os.environ.get('OAUTH2DL_KEY'))
+    parser.add_argument('-s', '--scope', help='Oauth2 scope',
+                        default=os.environ.get('OAUTH2DL_SCOPE'))
+    parser.add_argument('-o', help='Path where to save the file.')
+
+    return parser.parse_args()
+
+
+def main():
+    args = parse_args()
+
+    if args.key is None:
+        sys.exit(cnts.KEYFILE_NOT_FOUND_ERROR)
+
+    if args.scope is None:
+        sys.exit(cnts.SCOPE_NOT_FOUND_ERROR)
+
+    try:
+        headers, content = download_file(args.url, args.key, args.scope)
+    except KeyError as err:
+        sys.exit(cnts.INVALID_KEY_FILE.format(str(err), str(args.key)))
+    except Exception as err:
+        sys.exit(err)
+
+    if headers['status'] != '200':
+        try:
+            error_json = json.loads(content, encoding='utf-8')
+            sys.exit(cnts.GOOGLE_OAUTH_ERROR.format(
+                str(error_json['error']['code']),
+                str(error_json['error']['message']),
+            ))
+        except ValueError:
+            sys.exit(content)
+
+    if args.o is None:
+        sys.stdout.write(content)
+    else:
+        write_to_file(content, args.o)
+
+
+if __name__ == '__main__':
+    main()