Issue 5250 - Add copyright update script

update-copyright/update_copyright.py

 #!/usr/bin/env python3
 
 import os
 import subprocess
 import re
 import datetime
 import shutil
 import urllib.parse
 import urllib.request
 import html.parser
@@ skipping 95 matching lines @@
     def handle_endtag(self, tag):
         if tag == 'tr':
             self.recordtr = False
             self.cell = 0
 
     def handle_data(self, data):
         if self.cell == 2 and self.recordtr is True:
             self.recordtr = False
             self.cell = 0
             # Only process the URL if the description is not Deprecated
-            if ('*DEPRECATED*' not in data and '(Deprecated)' not in data and
-               len(self.current_url) > 2):

[Sebastian Noack, 2017/07/07 16:05:27]

The indentation is off by one character here. Example from PEP-8
(https://www.python.org/dev/peps/pep-0008/#indentation) for hanging indentation:

  if (this_is_one_thing and
      that_is_another_thing):
      # Since both conditions are true, we can frobnicate.
      do_something()

Alternatively, you could revert to use a "deprecated" variable again, in order
to avoid wrapping here.

[Sebastian Noack, 2017/07/07 16:58:29]

It seems you missed this comment.

[Sebastian Noack, 2017/07/17 10:19:53]

It seems this comment still hasn't been addressed.

[rosie, 2017/07/17 13:23:16]

Done.

[rosie, 2017/07/17 13:23:16]

I was getting a flake8 error when using the proper indentation. Instead, I added
some extra indentation on the conditional continuation line, as recommended
here: https://github.com/PyCQA/pycodestyle/issues/386

Hope that's ok?

+            if ('*DEPRECATED*' not in data and
+                    '(Deprecated)' not in data and
+                    len(self.current_url) > 2):
                 self.result += [self.current_url]
                 return self.result
 
 
 def extract_urls(hg_page):
     base_url = os.path.dirname(hg_page) + '/'
     parser = Parser()
     with urllib.request.urlopen(hg_page) as response:
         parser.feed(response.read().decode('utf-8'))
     parser.close()
     repo_urls = []
     for url in parser.result:
         repo_urls.append(urllib.parse.urljoin(base_url, url))
     return repo_urls
 
 
 def main(hg_page, hg_upstream):
     for repo in extract_urls(hg_page):
         process_repo(repo, hg_upstream)
 
 
 if __name__ == '__main__':
     arg_parser = argparse.ArgumentParser()
     arg_parser.add_argument('-u', '--hg-url',
-                            help='specify which Mercurial URL site to scrape',
-                            required=True)
+                            help='specify which Mercurial URL site to scrape')
     arg_parser.add_argument('-p', '--push-url',
-                            default=None,
                             help='specify where to push the repository')
     args = arg_parser.parse_args()
-    main(args.hg_url, args.push_url)
+    if args.hg_url and not args.push_url:
+        arg_parser.error('If -u is provided, -p is mandatory')
+    main(args.hg_url or 'https://hg.adblockplus.org/',
+         args.push_url or 'ssh://hg@hg.adblockplus.org/')