Issue 6685 - Offer incremental filter list downloads

abp/filters/parser.py

Index: abp/filters/parser.py
===================================================================
--- a/abp/filters/parser.py
+++ b/abp/filters/parser.py
@@ -140,9 +140,9 @@
 Include = _line_type('Include', 'target', '%include {0.target}%')
 
 
-METADATA_REGEXP = re.compile(r'!\s*(\w+)\s*:\s*(.*)')
+METADATA_REGEXP = re.compile(r'!\s*([\w-]+)\s*:\s*(.*)')
 METADATA_KEYS = {'Homepage', 'Title', 'Expires', 'Checksum', 'Redirect',
-                 'Version'}
+                 'Version', 'Diff-URL', 'Diff-Expires'}

[Sebastian Noack, 2018/08/21 19:42:45]

I would prefer if python-abp would be agnostic of the supported metadata keys.
Otherwise, we have to keep this set in sync with what is implemented in Adblock
Plus, which seems unpractical.

However, note that the regexp above would have to be changed like following so
that it won't match lines with arbitrary URLs: r'!\s*(\w+)\s*:(?!//)\s*(.*)'

[Vasily Kuznetsov, 2018/08/22 13:10:50]

Makes sense and I support it. I actually thought it was already implemented this
way before I started reviewing this change :)

BTW, the regexp should probably be r'!\s*([\w-]+)\s*:(?!//)\s*(.*)' so that we
can still have dashes in metadata field names.

[Sebastian Noack, 2018/08/22 14:31:06]

You are right, well spotted.

[rhowell, 2018/08/27 22:06:26]

It appears this was done to prevent mistaking a comment with a colon for
metadata, like in this test:

def test_parse_nonmeta():
    line = parse_line('! WrongHeader: something')
    assert line.type == 'comment'

It probably doesn't matter too much, since the parser wouldn't be able to parse
invalid metadata anyway. I'll update the test so it doesn't use a colon.

 INCLUDE_REGEXP = re.compile(r'%include\s+(.+)%')
 HEADER_REGEXP = re.compile(r'\[(Adblock(?:\s*Plus\s*[\d\.]+?)?)\]', flags=re.I)
 HIDING_FILTER_REGEXP = re.compile(r'^([^/*|@"!]*?)#([@?])?#(.+)$')