Noissue - Add requirement for Python 3.5 compatibility to Python section of the coding style guide

pages/coding-style.html

 title=Coding Style
 
 
 <h2 id="precedence">{{s1 Precedence}}</h2>
 
 <p>{{s2 The Adblock Plus coding style is based on others. Our rules (both general and language specific) always overrule referenced style guides. The language specific sections overrule the general section.}}</p>
 
 <h2 id="consistency">{{s3 Consistency}}</h2>
 
 <p>{{s4 Consistency is most important: Be consistent within functions, files, modules and projects. Making existing code conform with this style guide is fine, but it should happen in dedicated commits, preferably for a whole module or project at once.}}</p>
@@ skipping 26 matching lines @@
   <li>{{html-css-hexadecimal CSS color values should be specified in hexadecimal where possible.}}</li>
   <li>{{html-css-shorthand CSS shorthand properties usage is optional.}}</li>
   <li>{{html-css-order CSS rule declaration order should follow the <a href="https://make.wordpress.org/core/handbook/best-practices/coding-standards/css/#property-ordering">WordPress CSS Coding Standards</a>.}}</li>
   <li>{{html-css-units CSS number values should specify units where possible.}}</li>
   <li>{{html-css-leading-zero Don't omit the optional leading 0 for decimal numbers.}}</li>
 </ul>
 
 <h2 id="python">{{s19 Python}}</h2>
 <ul>
   <li>{{python-general Follow <a href="https://www.python.org/dev/peps/pep-0008/">PEP-8</a> and the recommendations in the offical Python documentation.}}</li>
-  <li>Make the code compatible with both Python 2.7 and Python 3.5 (see <a href="https://docs.python.org/dev/howto/pyporting.html">this guide</a>). Use <a href="https://docs.python.org/2/library/__future__.html">__future__ imports</a> to address syntactic differences but avoid <a href="https://pythonhosted.org/six/">six</a>, <a href="http://python-future.org/compatible_idioms.html">python-future</a>, etc. to not introduce additional dependencies.</li>

[Sebastian Noack, 2016/05/17 17:51:00]

You forgot to use the boilerplate to make this string translatable.

[Sebastian Noack, 2016/05/17 17:51:00]

Perhaps we shouldn't even make Python 2.7 mandatory in the style guide. As I
recently said, why must code which neither have Python 2 only dependencies nor
Python 2 users, be compatible with Python 2 and cannot even be Python 3 only?

Also I'm not sure whether we should refer to Python 3.5 explicitly. Once Python
3.6 comes out we probably want to be compatible there, and don't care about
Python 3.5 anymore unless we actually have it in production. So I think the rule
should be rather to keep code compatible with the latest Python 3 version in
addition to whatever the current requirements are.

Anyway, perhaps this is something, we should further discuss in the Python
developer meeting, tomorrow.

[Vasily Kuznetsov, 2016/05/18 08:50:50]

Oops, that's embarrassing. Fixed.

[Vasily Kuznetsov, 2016/05/18 08:50:50]

While I support the intention of dropping Python 2 support, I think we should be
careful about it. At the moment our Python code runs on our servers which have
Python 2 installed or on some developer machines where I'm not sure we can
expect to always have Python 3. So I don't think we can already declare that
we're going to Python 3 only.
My thinking was that Python 3.5 is the default version of Python 3 now and the
future versions will be backwards compatible with it. At some point we might
want to drop support for Python 3.5 and only support Python 3.6 but not
necessarily as soon as 3.6 comes out. So it seems better to have this under our
control instead of the meaning of our style guide changing as soon as new Python
version comes out.
+1

[Wladimir Palant, 2016/05/18 18:28:09]

So what we are talking about is merely the transition period, we don't want to
keep supporting both Python 2 and 3 - both servers and developer machines can be
updated. We cannot realistically switch our entire codebase to Python 3 all at
once. We will need that transition period where code (especially shared modules)
works with either Python version, so that we can make sure everything works
correctly. It's also questionable how we want to indicate that some code within
the same repository requires Python 3 whereas other code still works with Python
2.7. So it's probably easiest if we get Python 3 supported consistently before
we drop support for Python 2.
In fact, this isn't as easy as I thought originally. Problem is, Ubuntu 12.04
(which we are using on all our servers right now) only comes with Python 3.2.
Ubuntu 14.04 (which we want to migrate to eventually) offers Python 3.4. In
order to use Python 3.5 we will have to skip an LTS release and go to Ubuntu
16.04. Even if Python 3.6 comes out soon, having it as our baseline is
completely unrealistic right now.

In other words, we need to check with infrastructure which Python version we can
realistically require. Maybe it is possible to get a newer Python version on
Ubuntu 12.04 by acceptable means. Maybe we can expect using Ubuntu 16.04 before
our Python 3 migration is done (which can take a while). Maybe we have to make
compromises and change those formatting statements after all. I cannot tell.

[Wladimir Palant, 2016/05/18 18:47:47]

Matze says that there is a plan to upgrade the servers to Debian 8, end of this
year being the pessimistic estimate (let's hope that this holds true indeed). He
also says that installing Python 3.5 isn't a problem even though Debian 8 only
offers Python 3.4 at the moment. I think that we should have Python 3.5 as our
baseline explicitly, even if Python 3.6 comes out at some point - servers aren't
an environment where installing the latest and greatest is an easy thing to do.

[Sebastian Noack, 2016/05/18 19:06:32]

Whichever Python version is available on the server seems to me more like a
requirement (for those scripts that potentially run on the server), not
something that belongs into a style guide. You wouldn't list the version of
libraries used on the server either in the styleguide.

And the next time we update the server we will have a newer Python version
anyway. So I think it makes sense, if there is any newer Python version than the
one used on our servers to test/support them as well.

[Vasily Kuznetsov, 2016/05/19 09:03:31]

So we all agree that we would like to eventually migrate all the Python code to
Python 3. In the near future the more precise target for the migration is Python
3.5 until we decide that we can actually do something else. Since at the moment
everything runs on Python 2, having Python 3.5 in the style guide is a way to
indicate this migration intention and make sure that we don't add migration work
with the new code.

As for replacing Python 3.5 with "latest Python" -- there seems to be a
reasonably broad agreement that we don't want to automatically drop support for
older version immediately as the new one comes out and it needs to be a
deliberate action. If our code runs on Python 3.5, it's very likely to run and
have the same behaviour on Python 3.6 and later, Python 3 branch is supposed to
maintain backwards compatibility after all. So writing "Python 3.5" would cover
Python 3.5 and any later version, which is what we want.


Then there's the requirement of compatibility with Python 2.7. My reasoning is
that this is what we currently run on the servers and what most people would
normally use to run those scripts outside of the server environment. Before we
officially unsupport Python 2, all the code has to be compatible with it, so I
didn't want the style guide to give an impression that only Python 3 support is
needed.

[Sebastian Noack, 2016/05/19 10:10:24]

Yes, Python 3.6+ will in theory be backwards compatible with Python 3.5. But
nevertheless, as soon as Python 3.6 is out, we should test it as well (which in
most cases just means to add an env to the tox.ini). So how about simply
changing "Python 3.5" to "Python 3.5+" here?

[Vasily Kuznetsov, 2016/05/19 10:52:28]

Done.

[Wladimir Palant, 2016/05/19 11:33:26]

As long as this is about automated testing - yes, testing in newer Python
versions is a good thing. Manual testing should always happen with Python 3.5,
since that's the platform we are going to use for a long time. And it makes
sense to mention this explicitly here, since we probably want to have a
consistent baseline Python version throughout all projects. Whether it's Python
3.5 or Python 3.5+ I don't really care.

[Sebastian Noack, 2016/05/19 11:51:53]

I agree, that when manually testing it's important to test on 3.5. However, if
you take the style guide literally you would test on 2.7, 3.5 and each newer
version anyway. Plus, new code as well as code migrated to Python 3, will have
automated tests anyway. So IMO no reason to be even more explicit.

+  <li>{{python-version Make the code compatible with both Python 2.7 and Python 3.5+ (see <a href="https://docs.python.org/dev/howto/pyporting.html">this guide</a>). Use <a href="https://docs.python.org/2/library/__future__.html">__future__ imports</a> to address syntactic differences but avoid <a href="https://pythonhosted.org/six/">six</a>, <a href="http://python-future.org/compatible_idioms.html">python-future</a>, etc. to not introduce additional dependencies.}}</li>
   <li>{{python-strings Write string literals so that they match the behaviour of <a href="https://docs.python.org/2/library/functions.html#func-repr"><code><fix>repr()</fix></code></a> in Python 2, or <a href="https://docs.python.org/3/library/functions.html#ascii"><code><fix>ascii()</fix></code></a> in Python 3, i.e. use single quotes except to avoid escaping of embedded quotes and escape special and non-ascii characters. For docstrings, however, follow <a href="https://www.python.org/dev/peps/pep-0257/">PEP-257</a>.}}</li>
   <li>{{python-prefix In modules, prefix private functions and variables with a single underscore.}}</li>
   <li>{{python-concatenation Use the <code><fix>+</fix></code> operator when concatenating exactly two strings, use the <a href="https://docs.python.org/2/library/stdtypes.html#str.format"><code><fix>format()</fix></code> method</a> for more complex string formatting, use the <a href="https://docs.python.org/2/library/stdtypes.html#str.join"><code><fix>join()</fix></code> method</a> when concatenating pre-existing sequences.}}</li>
   <li>{{python-tuple-vs-list Use tuples for data that have structure, use lists for data that have order.}}</li>
   <li>{{python-builtins Don't override builtins except for <a href="https://docs.python.org/2/library/functions.html#non-essential-built-in-funcs">non-essential builtins</a> and <code><fix>file</fix></code> which is superfluos in modern code as well.}}</li>
   <li>{{python-map-filter Use list comprehensions or generator expressions instead of calling <code><fix>map()</fix></code> or <code><fix>filter()</fix></code> with a lambda function.}}</li>
   <li>{{python-regexp Use <a href="https://docs.python.org/2/library/re.html#re.search"><code><fix>re.search()</fix></code></a> instead of <code><fix>re.match()</fix></code> to avoid <a href="https://docs.python.org/2/library/re.html#search-vs-match">confusion</a> about implied beginning of the string but not the ending.}}</li>
   <li>{{python-flake8 Run <a href="https://pypi.python.org/pypi/flake8">flake8</a> with the <a href="https://hg.adblockplus.org/codingtools/file/tip/flake8-abp">flake8-abp</a> and <a href="https://pypi.python.org/pypi/pep8-naming">pep8-naming</a> extensions and fix any warning.}}</li>
   <li>{{python-tests Write tests and have them run along flake8 using <a href="https://pypi.python.org/pypi/tox">tox</a>.}}</li>
 </ul>
@@ skipping 17 matching lines @@
 <ul>
   <li>{{objc-conventions Follow Apple's <a href="https://developer.apple.com/library/ios/documentation/Cocoa/Conceptual/ProgrammingWithObjectiveC/Conventions/Conventions.html">conventions</a>}}.</li>
 </ul>
 
 <h2 id="puppet">{{s31 Puppet}}</h2>
 <ul>
   <li>{{s32 Follow the <a href="http://docs.puppetlabs.com/guides/style_guide.html">Puppet Style Guide</a>.}}</li>
   <li>{{s33 Opening braces don't go on their own line.}}</li>
   <li>{{s34 Arrows should not be aligned.}}</li>
 </ul>