Issue 4044 - Added handling for __future__ unicode_literals import to check_quotes()

flake8-abp/flake8_abp.py

Index: flake8-abp/flake8_abp.py
===================================================================
--- a/flake8-abp/flake8_abp.py
+++ b/flake8-abp/flake8_abp.py
@@ -372,21 +372,25 @@
         return (0, 'A303 non-default file encoding')
 
 check_non_default_encoding.name = 'abp-non-default-encoding'
 check_non_default_encoding.version = __version__
 
 
 def check_quotes(logical_line, tokens, previous_logical):
     first_token = True
+    is_unicode_literals = False

[Sebastian Noack, 2016/05/20 00:04:34]

While I didnt test this change yet I have a hard time to believe that tests
actually pass. This function is called per line. And usually "from __future__
import  unicode_literals" will be placed in a line prior to the actual string
literal using that feature just as in the example used as test. So I think we
need some persistence across lines.

[Vasily Kuznetsov, 2016/05/20 08:46:24]

Funnily enough, the test does pass in the first version. I think it might be
because `is_unicode_literals` is False from the start and then it's almost
always False, so `elif not is_unicode_literals:` branch always gets executed.
But anyway, the logic is not right that way.

 
     for kind, token, start, end, _ in tokens:
         if kind == tokenize.INDENT or kind == tokenize.DEDENT:
             continue
 
+        if token is 'unicode_literals':

[Sebastian Noack, 2016/05/20 00:04:34]

It's not as simple as that. Following examples will raise false positives:

unicode_literals = 1
from foo import unicode_literals
def unicode_literals(): pass

So you have to consider the surrounding tokens.

[Jon Sonesen, 2016/05/20 02:00:01]

Yes I should be checking the logical line object which the second patch contains

+            is_unicode_literals = True
+
         if kind == tokenize.STRING:
             match = re.search(r'^(u)?(b)?(r)?((""")?.*)$',
                               token, re.IGNORECASE | re.DOTALL)
             (is_unicode, is_bytes, is_raw,
              literal, has_doc_quotes) = match.groups()
 
             if first_token and re.search(r'^(?:(?:def|class)\s|$)',
                                          previous_logical):
@@ -401,16 +405,18 @@
                     literal = re.sub(r'\\(?!{})'.format(literal[0]),
                                      '\\\\\\\\', literal)
 
                 if sys.version_info[0] >= 3:
                     if is_bytes:
                         literal = 'b' + literal
                 elif is_unicode:
                     literal = 'u' + literal
+                elif not is_unicode_literals:

[Sebastian Noack, 2016/05/20 00:04:34]

Just add it to the condition above using "or" rather than duplicating the block
body?

[Jon Sonesen, 2016/05/20 02:00:01]

Will do.

+                    literal = 'u' + literal
 
                 if ascii(eval(literal)) != literal:
                     yield (start, "A110 string literal doesn't match "
                                   '{}()'.format(ascii.__name__))
 
         first_token = False
 
 check_quotes.name = 'abp-quotes'