Issue 7042 - [XTM Integration] Solve UnicodeDecodeErrors occuring

tests/test_xtm_translations_utils.py

Index: tests/test_xtm_translations_utils.py
diff --git a/tests/test_xtm_translations_utils.py b/tests/test_xtm_translations_utils.py
index 6cd5def9bd89fd248fecb23fa6c374f4054d0f29..fa7599581034050280872430ad0c596b8ae8a2da 100644
--- a/tests/test_xtm_translations_utils.py
+++ b/tests/test_xtm_translations_utils.py
@@ -13,6 +13,8 @@
 # You should have received a copy of the GNU General Public License
 # along with Adblock Plus.  If not, see <http://www.gnu.org/licenses/>.
 
+from __future__ import unicode_literals
+
 import json
 import os
 
@@ -219,3 +221,14 @@ def test_write_data(toydir, path):
     utils.write_to_file(data, str(toydir.join(path)))
 
     assert toydir.join(path).read('rb') == data
+
+
+def test_extract_unicode_strings(temp_site):
+    """Test correct extraction of unicode strings for translation.
+
+    Testing for presence of "Ͷ" - \u0376.

[Vasily Kuznetsov, 2018/10/12 09:39:51]

Doesn't this line cause an error for you? I get:

.tox/py27/lib/python2.7/site-packages/_pytest/python.py:482: in
_importtestmodule
    mod = self.fspath.pyimport(ensuresyspath=importmode)
.tox/py27/lib/python2.7/site-packages/py/_path/local.py:668: in pyimport
    __import__(modname)
E     File
"/Users/vkuznetsov/Documents/prog/hg.adblockplus.org/cms/tests/test_xtm_translations_utils.py",
line 229
E   SyntaxError: Non-ASCII character '\xcd' in file
/Users/vkuznetsov/Documents/prog/hg.adblockplus.org/cms/tests/test_xtm_translations_utils.py
on line 230, but no encoding declared; see http://python.org/dev/peps/pep-0263/
for details

I think the right approach here would be to remove the "Ͷ" from the docstring
instead of declaring encoding -- it's better to stick to ASCII in source code
unless you really have to.

[Tudor Avram, 2018/10/12 10:37:02]

Huh, ok. It didn't give me an error, but well,  I guess it makes sense to have
ASCII only characters in the codebase. Changing it now.

+    """
+    with FileSource(temp_site) as fs:
+        strings = utils.extract_strings(fs)
+
+    assert '\u0376' in strings['translate-unicode']['simple']['message']