Left: | ||
Right: |
OLD | NEW |
---|---|
1 # This file is part of Adblock Plus <https://adblockplus.org/>, | 1 # This file is part of Adblock Plus <https://adblockplus.org/>, |
2 # Copyright (C) 2006-present eyeo GmbH | 2 # Copyright (C) 2006-present eyeo GmbH |
3 # | 3 # |
4 # Adblock Plus is free software: you can redistribute it and/or modify | 4 # Adblock Plus is free software: you can redistribute it and/or modify |
5 # it under the terms of the GNU General Public License version 3 as | 5 # it under the terms of the GNU General Public License version 3 as |
6 # published by the Free Software Foundation. | 6 # published by the Free Software Foundation. |
7 # | 7 # |
8 # Adblock Plus is distributed in the hope that it will be useful, | 8 # Adblock Plus is distributed in the hope that it will be useful, |
9 # but WITHOUT ANY WARRANTY; without even the implied warranty of | 9 # but WITHOUT ANY WARRANTY; without even the implied warranty of |
10 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | 10 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
11 # GNU General Public License for more details. | 11 # GNU General Public License for more details. |
12 # | 12 # |
13 # You should have received a copy of the GNU General Public License | 13 # You should have received a copy of the GNU General Public License |
14 # along with Adblock Plus. If not, see <http://www.gnu.org/licenses/>. | 14 # along with Adblock Plus. If not, see <http://www.gnu.org/licenses/>. |
15 | 15 |
16 """ | 16 """ |
17 Functions for integrating with rPython. | 17 Functions for integrating with rPython. |
18 | 18 |
19 see: https://cran.r-project.org/web/packages/rPython/index.html | 19 see: https://cran.r-project.org/web/packages/rPython/index.html |
20 """ | 20 """ |
21 | 21 |
22 from __future__ import unicode_literals | 22 from __future__ import unicode_literals |
23 | 23 |
24 from abp.filters import parse_line | 24 from abp.filters import parse_line |
25 | 25 |
26 __all__ = ['line2dict'] | 26 __all__ = ['line2dict'] |
27 | 27 |
28 | 28 |
29 def parse_options(options): | |
Vasily Kuznetsov
2019/03/22 12:17:04
Maybe we should change this name to something like
rhowell
2019/03/27 19:14:06
Done.
| |
30 """Recursively parse filter options into dicts. | |
31 | |
32 Parameters | |
Vasily Kuznetsov
2019/03/22 12:17:04
Nit: The indentation of this line seems to be off.
rhowell
2019/03/27 19:14:06
Done.
| |
33 ---------- | |
34 options: A list of tuples or namedtuples | |
Vasily Kuznetsov
2019/03/22 12:17:04
We're treating them as just tuples so we probably
rhowell
2019/03/27 19:14:07
Done.
| |
35 The filter options | |
36 | |
37 Returns | |
38 ------- | |
39 dict | |
40 The resulting dictionary | |
41 | |
42 """ | |
43 result = dict(options) | |
44 if 'domain' in result: | |
45 result['domain'] = parse_options(result.get('domain')) | |
Vasily Kuznetsov
2019/03/22 12:17:04
Nit: `result.get('domain')` could be just `result[
Vasily Kuznetsov
2019/03/22 12:17:04
This conversion will lose the order of the domain
sporz
2019/03/25 17:30:03
Thank you Vasily for pointing this out - I wasn't
sporz
2019/03/25 17:30:03
Interesting choice to put this as a function on it
rhowell
2019/03/27 19:14:06
I ran into a few issues trying to call tuple2dict
rhowell
2019/03/27 19:14:07
I was able to implement this using OrderedDict and
| |
46 | |
47 return result | |
48 | |
49 | |
29 def tuple2dict(data): | 50 def tuple2dict(data): |
30 """Convert a parsed filter from a namedtuple to a dict. | 51 """Convert a parsed filter from a namedtuple to a dict. |
31 | 52 |
32 Parameters | 53 Parameters |
33 ---------- | 54 ---------- |
34 data: namedtuple | 55 data: namedtuple |
35 The parsed filter. | 56 The parsed filter. |
36 | 57 |
37 Returns | 58 Returns |
38 ------- | 59 ------- |
39 dict | 60 dict |
40 The resulting dictionary | 61 The resulting dictionary |
41 | 62 |
42 """ | 63 """ |
43 result = dict(data._asdict()) | 64 result = {} |
Vasily Kuznetsov
2019/03/22 12:17:04
Maybe we just add `result['options'] = parse_optio
rhowell
2019/03/27 19:14:07
Yeah, good idea. We also need to check that 'optio
| |
65 | |
66 for key in data._fields: | |
67 name = key | |
68 value = getattr(data, key) | |
69 if 'options' in name: | |
70 result[name] = parse_options(value) | |
71 else: | |
72 result[name] = value | |
73 | |
44 result['type'] = data.__class__.__name__ | 74 result['type'] = data.__class__.__name__ |
45 | 75 |
46 return result | 76 return result |
47 | 77 |
48 | 78 |
49 def strings2utf8(data): | 79 def strings2utf8(data): |
50 """Convert strings in a data structure to utf8 byte strings. | 80 """Convert strings in a data structure to utf8 byte strings. |
51 | 81 |
52 Parameters | 82 Parameters |
53 ---------- | 83 ---------- |
54 data: dict | 84 data: dict |
55 The data to convert. Can include nested dicts, lists and tuples. | 85 The data to convert. Can include nested dicts, lists and tuples. |
56 | 86 |
57 Returns | 87 Returns |
58 ------- | 88 ------- |
59 dict | 89 dict |
60 With all strings encoded as unicode. | 90 With all strings encoded as unicode. |
61 | 91 |
62 """ | 92 """ |
63 if isinstance(data, dict): | 93 if isinstance(data, dict): |
64 return {strings2utf8(k): strings2utf8(v) for k, v in data.items()} | 94 return {strings2utf8(k): strings2utf8(v) for k, v in data.items()} |
65 if isinstance(data, list): | |
Vasily Kuznetsov
2019/03/22 12:17:04
Unfortunately we can't remove the list branch here
rhowell
2019/03/27 19:14:06
Ah, gotcha. Since we didn't have a test using the
Vasily Kuznetsov
2019/04/02 17:36:21
Yeah, seems reasonable to keep it as a list, unles
| |
66 return [strings2utf8(v) for v in data] | |
67 if isinstance(data, tuple): | |
68 return tuple(strings2utf8(v) for v in data) | |
69 if isinstance(data, type('')): | 95 if isinstance(data, type('')): |
70 # The condition is a Python 2/3 way of saying "unicode string". | 96 # The condition is a Python 2/3 way of saying "unicode string". |
71 return data.encode('utf-8') | 97 return data.encode('utf-8') |
72 return data | 98 return data |
73 | 99 |
74 | 100 |
75 def line2dict(text, mode='body'): | 101 def line2dict(text, mode='body'): |
76 """Convert a filterlist line to a dictionary. | 102 """Convert a filterlist line to a dictionary. |
77 | 103 |
78 All strings in the output dictionary will be UTF8 byte strings. This is | 104 All strings in the output dictionary will be UTF8 byte strings. This is |
(...skipping 35 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... | |
114 ------- | 140 ------- |
115 list of dict | 141 list of dict |
116 With the parsing results and all strings converted to utf8 byte | 142 With the parsing results and all strings converted to utf8 byte |
117 strings. | 143 strings. |
118 | 144 |
119 """ | 145 """ |
120 result = [] | 146 result = [] |
121 for string in string_list: | 147 for string in string_list: |
122 result.append(line2dict(string, mode)) | 148 result.append(line2dict(string, mode)) |
123 return result | 149 return result |
OLD | NEW |