Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code

Side by Side Diff: abp/filters/rpy.py

Issue 30031558: Issue 7391 - Let rpy recursively parse filter options to dicts (Closed) Base URL: https://hg.adblockplus.org/python-abp
Patch Set: Created March 21, 2019, 11:12 p.m.
Left:
Right:
Use n/p to move between diff chunks; N/P to move between comments.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | tests/test_rpy.py » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 # This file is part of Adblock Plus <https://adblockplus.org/>, 1 # This file is part of Adblock Plus <https://adblockplus.org/>,
2 # Copyright (C) 2006-present eyeo GmbH 2 # Copyright (C) 2006-present eyeo GmbH
3 # 3 #
4 # Adblock Plus is free software: you can redistribute it and/or modify 4 # Adblock Plus is free software: you can redistribute it and/or modify
5 # it under the terms of the GNU General Public License version 3 as 5 # it under the terms of the GNU General Public License version 3 as
6 # published by the Free Software Foundation. 6 # published by the Free Software Foundation.
7 # 7 #
8 # Adblock Plus is distributed in the hope that it will be useful, 8 # Adblock Plus is distributed in the hope that it will be useful,
9 # but WITHOUT ANY WARRANTY; without even the implied warranty of 9 # but WITHOUT ANY WARRANTY; without even the implied warranty of
10 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 10 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
11 # GNU General Public License for more details. 11 # GNU General Public License for more details.
12 # 12 #
13 # You should have received a copy of the GNU General Public License 13 # You should have received a copy of the GNU General Public License
14 # along with Adblock Plus. If not, see <http://www.gnu.org/licenses/>. 14 # along with Adblock Plus. If not, see <http://www.gnu.org/licenses/>.
15 15
16 """ 16 """
17 Functions for integrating with rPython. 17 Functions for integrating with rPython.
18 18
19 see: https://cran.r-project.org/web/packages/rPython/index.html 19 see: https://cran.r-project.org/web/packages/rPython/index.html
20 """ 20 """
21 21
22 from __future__ import unicode_literals 22 from __future__ import unicode_literals
23 23
24 from abp.filters import parse_line 24 from abp.filters import parse_line
25 25
26 __all__ = ['line2dict'] 26 __all__ = ['line2dict']
27 27
28 28
29 def parse_options(options):
Vasily Kuznetsov 2019/03/22 12:17:04 Maybe we should change this name to something like
rhowell 2019/03/27 19:14:06 Done.
30 """Recursively parse filter options into dicts.
31
32 Parameters
Vasily Kuznetsov 2019/03/22 12:17:04 Nit: The indentation of this line seems to be off.
rhowell 2019/03/27 19:14:06 Done.
33 ----------
34 options: A list of tuples or namedtuples
Vasily Kuznetsov 2019/03/22 12:17:04 We're treating them as just tuples so we probably
rhowell 2019/03/27 19:14:07 Done.
35 The filter options
36
37 Returns
38 -------
39 dict
40 The resulting dictionary
41
42 """
43 result = dict(options)
44 if 'domain' in result:
45 result['domain'] = parse_options(result.get('domain'))
Vasily Kuznetsov 2019/03/22 12:17:04 Nit: `result.get('domain')` could be just `result[
Vasily Kuznetsov 2019/03/22 12:17:04 This conversion will lose the order of the domain
sporz 2019/03/25 17:30:03 Thank you Vasily for pointing this out - I wasn't
sporz 2019/03/25 17:30:03 Interesting choice to put this as a function on it
rhowell 2019/03/27 19:14:06 I ran into a few issues trying to call tuple2dict
rhowell 2019/03/27 19:14:07 I was able to implement this using OrderedDict and
46
47 return result
48
49
29 def tuple2dict(data): 50 def tuple2dict(data):
30 """Convert a parsed filter from a namedtuple to a dict. 51 """Convert a parsed filter from a namedtuple to a dict.
31 52
32 Parameters 53 Parameters
33 ---------- 54 ----------
34 data: namedtuple 55 data: namedtuple
35 The parsed filter. 56 The parsed filter.
36 57
37 Returns 58 Returns
38 ------- 59 -------
39 dict 60 dict
40 The resulting dictionary 61 The resulting dictionary
41 62
42 """ 63 """
43 result = dict(data._asdict()) 64 result = {}
Vasily Kuznetsov 2019/03/22 12:17:04 Maybe we just add `result['options'] = parse_optio
rhowell 2019/03/27 19:14:07 Yeah, good idea. We also need to check that 'optio
65
66 for key in data._fields:
67 name = key
68 value = getattr(data, key)
69 if 'options' in name:
70 result[name] = parse_options(value)
71 else:
72 result[name] = value
73
44 result['type'] = data.__class__.__name__ 74 result['type'] = data.__class__.__name__
45 75
46 return result 76 return result
47 77
48 78
49 def strings2utf8(data): 79 def strings2utf8(data):
50 """Convert strings in a data structure to utf8 byte strings. 80 """Convert strings in a data structure to utf8 byte strings.
51 81
52 Parameters 82 Parameters
53 ---------- 83 ----------
54 data: dict 84 data: dict
55 The data to convert. Can include nested dicts, lists and tuples. 85 The data to convert. Can include nested dicts, lists and tuples.
56 86
57 Returns 87 Returns
58 ------- 88 -------
59 dict 89 dict
60 With all strings encoded as unicode. 90 With all strings encoded as unicode.
61 91
62 """ 92 """
63 if isinstance(data, dict): 93 if isinstance(data, dict):
64 return {strings2utf8(k): strings2utf8(v) for k, v in data.items()} 94 return {strings2utf8(k): strings2utf8(v) for k, v in data.items()}
65 if isinstance(data, list):
Vasily Kuznetsov 2019/03/22 12:17:04 Unfortunately we can't remove the list branch here
rhowell 2019/03/27 19:14:06 Ah, gotcha. Since we didn't have a test using the
Vasily Kuznetsov 2019/04/02 17:36:21 Yeah, seems reasonable to keep it as a list, unles
66 return [strings2utf8(v) for v in data]
67 if isinstance(data, tuple):
68 return tuple(strings2utf8(v) for v in data)
69 if isinstance(data, type('')): 95 if isinstance(data, type('')):
70 # The condition is a Python 2/3 way of saying "unicode string". 96 # The condition is a Python 2/3 way of saying "unicode string".
71 return data.encode('utf-8') 97 return data.encode('utf-8')
72 return data 98 return data
73 99
74 100
75 def line2dict(text, mode='body'): 101 def line2dict(text, mode='body'):
76 """Convert a filterlist line to a dictionary. 102 """Convert a filterlist line to a dictionary.
77 103
78 All strings in the output dictionary will be UTF8 byte strings. This is 104 All strings in the output dictionary will be UTF8 byte strings. This is
(...skipping 35 matching lines...) Expand 10 before | Expand all | Expand 10 after
114 ------- 140 -------
115 list of dict 141 list of dict
116 With the parsing results and all strings converted to utf8 byte 142 With the parsing results and all strings converted to utf8 byte
117 strings. 143 strings.
118 144
119 """ 145 """
120 result = [] 146 result = []
121 for string in string_list: 147 for string in string_list:
122 result.append(line2dict(string, mode)) 148 result.append(line2dict(string, mode))
123 return result 149 return result
OLDNEW
« no previous file with comments | « no previous file | tests/test_rpy.py » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld