Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code

Unified Diff: filters/get_subscriptions.py

Issue 5636796054503424: Issue 1170 - [adblockplus.org Anwiki to CMS migration] Migrate content (Closed)
Patch Set: Addressed more of snoack's comments. Created Feb. 27, 2015, 2:56 p.m.
Use n/p to move between diff chunks; N/P to move between comments.
Jump to:
View side-by-side diff with in-line comments
Download patch
« no previous file with comments | « no previous file | filters/parse_interface.py » ('j') | filters/parse_interface.py » ('J')
Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
Index: filters/get_subscriptions.py
diff --git a/filters/get_subscriptions.py b/filters/get_subscriptions.py
new file mode 100644
index 0000000000000000000000000000000000000000..2cdb76b017a3d3399fe6b53be8fc9d6d5e3bae37
--- /dev/null
+++ b/filters/get_subscriptions.py
@@ -0,0 +1,42 @@
+# This file is part of the Adblock Plus website,
+# Copyright (C) 2006-2015 Eyeo GmbH
+#
+# Adblock Plus is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# Adblock Plus is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Adblock Plus. If not, see <http://www.gnu.org/licenses/>.
+
+import codecs
+import os
+import tarfile
+import urllib
+
+from sitescripts.subscriptions.subscriptionParser import parse_file, calculate_supplemented
+
+def get_subscriptions(_):
+ result = {}
+ utf8_reader = codecs.getreader('utf8')
+ source = urllib.urlopen("https://hg.adblockplus.org/subscriptionlist/archive/default.tar.gz")
+ try:
+ with tarfile.open(fileobj=source, mode="r|gz") as archive:
+ for fileinfo in archive:
+ if os.path.splitext(fileinfo.name)[1] != ".subscription":
Wladimir Palant 2015/02/27 15:37:02 Actually, I'm unsure that this is really better th
Sebastian Noack 2015/02/27 15:40:43 AFAIK splitext() is implemented identically for al
Sebastian Noack 2015/02/27 15:41:18 Operating systems, I meant.
Wladimir Palant 2015/02/27 16:19:58 Sure, it is. However, this doesn't change the fact
Sebastian Noack 2015/02/27 16:32:32 As I said, strictly it would be more correct to us
+ continue
+
+ filedata = parse_file(fileinfo.name, utf8_reader(archive.extractfile(fileinfo)))
+ if filedata.unavailable:
+ continue
+
+ result[filedata.name] = filedata
+ finally:
+ source.close()
+
+ calculate_supplemented(result)
+ return result.values()
« no previous file with comments | « no previous file | filters/parse_interface.py » ('j') | filters/parse_interface.py » ('J')

Powered by Google App Engine
This is Rietveld