Issue 6945 - Add script to make filter list diffs

abp/filters/diff_script.py

+# This file is part of Adblock Plus <https://adblockplus.org/>,
+# Copyright (C) 2006-present eyeo GmbH
+#
+# Adblock Plus is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# Adblock Plus is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Adblock Plus.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Command line script for rendering Adblock Plus filter list diffs."""
+
+import argparse
+import io
+import sys
+
+from collections import namedtuple
+
+from .sources import NotFound
+from .renderer import render_diff
+__all__ = ['main']
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(description='Render a filter list diff.')
+    parser.add_argument(
+        'base', help='the older filter list that needs to be updated',
+        default='-', nargs='?')
+    parser.add_argument(
+        'latest', help='the most recent version of the filter list',
+        default='-', nargs='?')
+    parser.add_argument(
+        'outfile', help='output file for filter list diff',
+        default='-', nargs='?')
+    return parser.parse_args()
+
+
+def main(args_in=None):
+    """Entry point for the diff rendering script (diffrender)."""
+    if args_in:
+        Parse = namedtuple('Test', 'base, latest, outfile')
+        args = Parse(args_in['base'], args_in['latest'], args_in['outfile'])
+    else:
+        args = parse_args()
+
+    base = open(args.base, 'r')
+    latest = open(args.latest, 'r')

[Sebastian Noack, 2018/09/18 16:56:12]

Any reason you don't use io.open(..., encoding="utf-8") here but below in order
to write the output? I guess, it works because parse_line() implicitly decodes
the input if bytes are provided, but we might want to avoid dealing with binary
data as much as possible.

Another advantage of io.open() is that it behaves the same on Python 2 and 3,
while the semantics of open() changed in Python 3 (to the semantics of io.open).

[Sebastian Noack, 2018/09/18 16:56:12]

Please make sure all files are explicitly closed.

[rhowell, 2018/09/18 17:23:33]

Done.

+
+    try:
+        lines = render_diff(base, latest)
+        if args.outfile == '-':
+            for line in lines:
+                sys.stdout.write(line + '\n')
+        else:
+            open(args.outfile, 'w')

[rhowell, 2018/09/18 16:01:54]

This feels hacky..

[Sebastian Noack, 2018/09/18 16:56:12]

Well, this is redundant. Why did you put it here?

[rhowell, 2018/09/18 17:23:33]

Wow, you're right. I swore I was getting an error whenever I tried to write to
this file, that the file didn't exist. I read through the docs for io
(https://docs.python.org/2/library/io.html), and didn't see an option to create
the file if it doesn't exist (Like 'w+' when using `open`). Fixed.

[Sebastian Noack, 2018/09/18 19:13:18]

Both, open(..., 'w') and io.open(..., 'w') create the file if it doesn't exist.
However, 'w+' (read+write) doesn't, but there is no need to use this mode here.

+            with io.open(args.outfile, 'w', encoding='utf-8') as out_fp:
+                for line in lines:
+                    out_fp.write(line + '\n')
+    except NotFound as exc:
+        sys.exit(exc)