Issue #6707 - Make the generated malware domain filter list encode domains as Punycode

sitescripts/subscriptions/bin/updateMalwareDomainsList.py

Index: sitescripts/subscriptions/bin/updateMalwareDomainsList.py
diff --git a/sitescripts/subscriptions/bin/updateMalwareDomainsList.py b/sitescripts/subscriptions/bin/updateMalwareDomainsList.py
index a2cfb5ff8d46bfe4534dc79f0f03f47e31257519..c2f208228717f4e3122a4bc95cbb39f6f66c8a36 100644
--- a/sitescripts/subscriptions/bin/updateMalwareDomainsList.py
+++ b/sitescripts/subscriptions/bin/updateMalwareDomainsList.py
@@ -60,7 +60,7 @@ def main():
         subprocess.check_call(['hg', '-q', 'up', '-R', tempdir, '-r', 'default'])
 
         path = os.path.join(tempdir, 'malwaredomains_full.txt')
-        file = codecs.open(path, 'wb', encoding='utf-8')
+        file = codecs.open(path, 'wb')
 
         print >>file, FILTERLIST_HEADER
 
@@ -80,7 +80,7 @@ def main():
             if not domain:
                 continue
 
-            print >>file, '||%s^' % domain.decode('idna')
+            print >>file, '||%s^' % domain.decode('idna').encode('punycode')

[Sebastian Noack, 2018/07/05 14:55:28]

This seems incorrect. Let's take "xn--fuson-1sa.org" (actual domain in the
Malware Domains List):

  >>> 'xn--fuson-1sa.org'.decode('idna').encode('punycode')
  b'fuson.org-i5a'

This is incorrect, we need 'xn--fuson-1sa.org' here (with the "xn--" annotation,
like it was in the input).

As far as I understand, this line should be just changed like this:

  print >>file, '||%s^' % domain

[Tudor Avram, 2018/07/06 11:53:16]

Done.

         file.close()
 
         if subprocess.check_output(['hg', 'stat', '-R', tempdir]) != '':