Noissue - Add requirement for Python 3.5 compatibility to Python section of the coding style guide

Noissue - Add requirement for Python 3.5 compatibility to Python section of the coding style guide

[Vasily Kuznetsov, 2016-05-17 17:30:18]

I added a rule about python version compatibility that requires 2.7 and 3.5. It
doesn't specifically say that it only applies for new code, but this is the same
as for all the other rules that are sometimes broken with the code that predates
them. What do you think?

[Sebastian Noack, 2016-05-17 17:51:01]

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
File pages/coding-style.html (right):

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
pages/coding-style.html:47: <li>Make the code compatible with both Python 2.7
and Python 3.5 (see <a
href="https://docs.python.org/dev/howto/pyporting.html">this guide</a>). Use <a
href="https://docs.python.org/2/library/__future__.html">__future__ imports</a>
to address syntactic differences but avoid <a
href="https://pythonhosted.org/six/">six</a>, <a
href="http://python-future.org/compatible_idioms.html">python-future</a>, etc.
to not introduce additional dependencies.</li>
You forgot to use the boilerplate to make this string translatable.

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
pages/coding-style.html:47: <li>Make the code compatible with both Python 2.7
and Python 3.5 (see <a
href="https://docs.python.org/dev/howto/pyporting.html">this guide</a>). Use <a
href="https://docs.python.org/2/library/__future__.html">__future__ imports</a>
to address syntactic differences but avoid <a
href="https://pythonhosted.org/six/">six</a>, <a
href="http://python-future.org/compatible_idioms.html">python-future</a>, etc.
to not introduce additional dependencies.</li>
Perhaps we shouldn't even make Python 2.7 mandatory in the style guide. As I
recently said, why must code which neither have Python 2 only dependencies nor
Python 2 users, be compatible with Python 2 and cannot even be Python 3 only?

Also I'm not sure whether we should refer to Python 3.5 explicitly. Once Python
3.6 comes out we probably want to be compatible there, and don't care about
Python 3.5 anymore unless we actually have it in production. So I think the rule
should be rather to keep code compatible with the latest Python 3 version in
addition to whatever the current requirements are.

Anyway, perhaps this is something, we should further discuss in the Python
developer meeting, tomorrow.

[Vasily Kuznetsov, 2016-05-18 08:50:50]

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
File pages/coding-style.html (right):

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
pages/coding-style.html:47: <li>Make the code compatible with both Python 2.7
and Python 3.5 (see <a
href="https://docs.python.org/dev/howto/pyporting.html">this guide</a>). Use <a
href="https://docs.python.org/2/library/__future__.html">__future__ imports</a>
to address syntactic differences but avoid <a
href="https://pythonhosted.org/six/">six</a>, <a
href="http://python-future.org/compatible_idioms.html">python-future</a>, etc.
to not introduce additional dependencies.</li>
On 2016/05/17 17:51:00, Sebastian Noack wrote:
> You forgot to use the boilerplate to make this string translatable.

Oops, that's embarrassing. Fixed.

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
pages/coding-style.html:47: <li>Make the code compatible with both Python 2.7
and Python 3.5 (see <a
href="https://docs.python.org/dev/howto/pyporting.html">this guide</a>). Use <a
href="https://docs.python.org/2/library/__future__.html">__future__ imports</a>
to address syntactic differences but avoid <a
href="https://pythonhosted.org/six/">six</a>, <a
href="http://python-future.org/compatible_idioms.html">python-future</a>, etc.
to not introduce additional dependencies.</li>
On 2016/05/17 17:51:00, Sebastian Noack wrote:
> Perhaps we shouldn't even make Python 2.7 mandatory in the style guide. As I
> recently said, why must code which neither have Python 2 only dependencies nor
> Python 2 users, be compatible with Python 2 and cannot even be Python 3 only?

While I support the intention of dropping Python 2 support, I think we should be
careful about it. At the moment our Python code runs on our servers which have
Python 2 installed or on some developer machines where I'm not sure we can
expect to always have Python 3. So I don't think we can already declare that
we're going to Python 3 only.

> Also I'm not sure whether we should refer to Python 3.5 explicitly. Once
Python
> 3.6 comes out we probably want to be compatible there, and don't care about
> Python 3.5 anymore unless we actually have it in production. So I think the
rule
> should be rather to keep code compatible with the latest Python 3 version in
> addition to whatever the current requirements are.

My thinking was that Python 3.5 is the default version of Python 3 now and the
future versions will be backwards compatible with it. At some point we might
want to drop support for Python 3.5 and only support Python 3.6 but not
necessarily as soon as 3.6 comes out. So it seems better to have this under our
control instead of the meaning of our style guide changing as soon as new Python
version comes out.

> Anyway, perhaps this is something, we should further discuss in the Python
> developer meeting, tomorrow.

+1

[Sebastian Noack, 2016-05-18 16:23:30]

This is the suggested change to the coding style guide in regard of Python 3, we
briefly discussed in the meeting today. I don't have anything to add, that I
didn't already say on this review. Happy bikeshedding. ;)

[Wladimir Palant, 2016-05-18 18:28:09]

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
File pages/coding-style.html (right):

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
pages/coding-style.html:47: <li>Make the code compatible with both Python 2.7
and Python 3.5 (see <a
href="https://docs.python.org/dev/howto/pyporting.html">this guide</a>). Use <a
href="https://docs.python.org/2/library/__future__.html">__future__ imports</a>
to address syntactic differences but avoid <a
href="https://pythonhosted.org/six/">six</a>, <a
href="http://python-future.org/compatible_idioms.html">python-future</a>, etc.
to not introduce additional dependencies.</li>
On 2016/05/18 08:50:50, Vasily Kuznetsov wrote:
> While I support the intention of dropping Python 2 support, I think we should
be
> careful about it. At the moment our Python code runs on our servers which have
> Python 2 installed or on some developer machines where I'm not sure we can
> expect to always have Python 3. So I don't think we can already declare that
> we're going to Python 3 only.

So what we are talking about is merely the transition period, we don't want to
keep supporting both Python 2 and 3 - both servers and developer machines can be
updated. We cannot realistically switch our entire codebase to Python 3 all at
once. We will need that transition period where code (especially shared modules)
works with either Python version, so that we can make sure everything works
correctly. It's also questionable how we want to indicate that some code within
the same repository requires Python 3 whereas other code still works with Python
2.7. So it's probably easiest if we get Python 3 supported consistently before
we drop support for Python 2.

> My thinking was that Python 3.5 is the default version of Python 3 now and the
> future versions will be backwards compatible with it. At some point we might
> want to drop support for Python 3.5 and only support Python 3.6 but not
> necessarily as soon as 3.6 comes out. So it seems better to have this under
our
> control instead of the meaning of our style guide changing as soon as new
Python
> version comes out.

In fact, this isn't as easy as I thought originally. Problem is, Ubuntu 12.04
(which we are using on all our servers right now) only comes with Python 3.2.
Ubuntu 14.04 (which we want to migrate to eventually) offers Python 3.4. In
order to use Python 3.5 we will have to skip an LTS release and go to Ubuntu
16.04. Even if Python 3.6 comes out soon, having it as our baseline is
completely unrealistic right now.

In other words, we need to check with infrastructure which Python version we can
realistically require. Maybe it is possible to get a newer Python version on
Ubuntu 12.04 by acceptable means. Maybe we can expect using Ubuntu 16.04 before
our Python 3 migration is done (which can take a while). Maybe we have to make
compromises and change those formatting statements after all. I cannot tell.

[Wladimir Palant, 2016-05-18 18:47:47]

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
File pages/coding-style.html (right):

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
pages/coding-style.html:47: <li>Make the code compatible with both Python 2.7
and Python 3.5 (see <a
href="https://docs.python.org/dev/howto/pyporting.html">this guide</a>). Use <a
href="https://docs.python.org/2/library/__future__.html">__future__ imports</a>
to address syntactic differences but avoid <a
href="https://pythonhosted.org/six/">six</a>, <a
href="http://python-future.org/compatible_idioms.html">python-future</a>, etc.
to not introduce additional dependencies.</li>
On 2016/05/18 18:28:09, Wladimir Palant wrote:
> In other words, we need to check with infrastructure which Python version we
can
> realistically require. Maybe it is possible to get a newer Python version on
> Ubuntu 12.04 by acceptable means. Maybe we can expect using Ubuntu 16.04
before
> our Python 3 migration is done (which can take a while). Maybe we have to make
> compromises and change those formatting statements after all. I cannot tell.

Matze says that there is a plan to upgrade the servers to Debian 8, end of this
year being the pessimistic estimate (let's hope that this holds true indeed). He
also says that installing Python 3.5 isn't a problem even though Debian 8 only
offers Python 3.4 at the moment. I think that we should have Python 3.5 as our
baseline explicitly, even if Python 3.6 comes out at some point - servers aren't
an environment where installing the latest and greatest is an easy thing to do.

[Sebastian Noack, 2016-05-18 19:06:32]

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
File pages/coding-style.html (right):

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
pages/coding-style.html:47: <li>Make the code compatible with both Python 2.7
and Python 3.5 (see <a
href="https://docs.python.org/dev/howto/pyporting.html">this guide</a>). Use <a
href="https://docs.python.org/2/library/__future__.html">__future__ imports</a>
to address syntactic differences but avoid <a
href="https://pythonhosted.org/six/">six</a>, <a
href="http://python-future.org/compatible_idioms.html">python-future</a>, etc.
to not introduce additional dependencies.</li>
Whichever Python version is available on the server seems to me more like a
requirement (for those scripts that potentially run on the server), not
something that belongs into a style guide. You wouldn't list the version of
libraries used on the server either in the styleguide.

And the next time we update the server we will have a newer Python version
anyway. So I think it makes sense, if there is any newer Python version than the
one used on our servers to test/support them as well.

[Vasily Kuznetsov, 2016-05-19 09:03:31]

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
File pages/coding-style.html (right):

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
pages/coding-style.html:47: <li>Make the code compatible with both Python 2.7
and Python 3.5 (see <a
href="https://docs.python.org/dev/howto/pyporting.html">this guide</a>). Use <a
href="https://docs.python.org/2/library/__future__.html">__future__ imports</a>
to address syntactic differences but avoid <a
href="https://pythonhosted.org/six/">six</a>, <a
href="http://python-future.org/compatible_idioms.html">python-future</a>, etc.
to not introduce additional dependencies.</li>
On 2016/05/18 19:06:32, Sebastian Noack wrote:
> Whichever Python version is available on the server seems to me more like a
> requirement (for those scripts that potentially run on the server), not
> something that belongs into a style guide. You wouldn't list the version of
> libraries used on the server either in the styleguide.
> 
> And the next time we update the server we will have a newer Python version
> anyway. So I think it makes sense, if there is any newer Python version than
the
> one used on our servers to test/support them as well.

So we all agree that we would like to eventually migrate all the Python code to
Python 3. In the near future the more precise target for the migration is Python
3.5 until we decide that we can actually do something else. Since at the moment
everything runs on Python 2, having Python 3.5 in the style guide is a way to
indicate this migration intention and make sure that we don't add migration work
with the new code.

As for replacing Python 3.5 with "latest Python" -- there seems to be a
reasonably broad agreement that we don't want to automatically drop support for
older version immediately as the new one comes out and it needs to be a
deliberate action. If our code runs on Python 3.5, it's very likely to run and
have the same behaviour on Python 3.6 and later, Python 3 branch is supposed to
maintain backwards compatibility after all. So writing "Python 3.5" would cover
Python 3.5 and any later version, which is what we want.


Then there's the requirement of compatibility with Python 2.7. My reasoning is
that this is what we currently run on the servers and what most people would
normally use to run those scripts outside of the server environment. Before we
officially unsupport Python 2, all the code has to be compatible with it, so I
didn't want the style guide to give an impression that only Python 3 support is
needed.

[Sebastian Noack, 2016-05-19 10:10:24]

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
File pages/coding-style.html (right):

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
pages/coding-style.html:47: <li>Make the code compatible with both Python 2.7
and Python 3.5 (see <a
href="https://docs.python.org/dev/howto/pyporting.html">this guide</a>). Use <a
href="https://docs.python.org/2/library/__future__.html">__future__ imports</a>
to address syntactic differences but avoid <a
href="https://pythonhosted.org/six/">six</a>, <a
href="http://python-future.org/compatible_idioms.html">python-future</a>, etc.
to not introduce additional dependencies.</li>
On 2016/05/19 09:03:31, Vasily Kuznetsov wrote:
> On 2016/05/18 19:06:32, Sebastian Noack wrote:
> > Whichever Python version is available on the server seems to me more like a
> > requirement (for those scripts that potentially run on the server), not
> > something that belongs into a style guide. You wouldn't list the version of
> > libraries used on the server either in the styleguide.
> > 
> > And the next time we update the server we will have a newer Python version
> > anyway. So I think it makes sense, if there is any newer Python version than
> the
> > one used on our servers to test/support them as well.
> 
> So we all agree that we would like to eventually migrate all the Python code
to
> Python 3. In the near future the more precise target for the migration is
Python
> 3.5 until we decide that we can actually do something else. Since at the
moment
> everything runs on Python 2, having Python 3.5 in the style guide is a way to
> indicate this migration intention and make sure that we don't add migration
work
> with the new code.
> 
> As for replacing Python 3.5 with "latest Python" -- there seems to be a
> reasonably broad agreement that we don't want to automatically drop support
for
> older version immediately as the new one comes out and it needs to be a
> deliberate action. If our code runs on Python 3.5, it's very likely to run and
> have the same behaviour on Python 3.6 and later, Python 3 branch is supposed
to
> maintain backwards compatibility after all. So writing "Python 3.5" would
cover
> Python 3.5 and any later version, which is what we want.
> 
> 
> Then there's the requirement of compatibility with Python 2.7. My reasoning is
> that this is what we currently run on the servers and what most people would
> normally use to run those scripts outside of the server environment. Before we
> officially unsupport Python 2, all the code has to be compatible with it, so I
> didn't want the style guide to give an impression that only Python 3 support
is
> needed.

Yes, Python 3.6+ will in theory be backwards compatible with Python 3.5. But
nevertheless, as soon as Python 3.6 is out, we should test it as well (which in
most cases just means to add an env to the tox.ini). So how about simply
changing "Python 3.5" to "Python 3.5+" here?

[Vasily Kuznetsov, 2016-05-19 10:52:28]

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
File pages/coding-style.html (right):

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
pages/coding-style.html:47: <li>Make the code compatible with both Python 2.7
and Python 3.5 (see <a
href="https://docs.python.org/dev/howto/pyporting.html">this guide</a>). Use <a
href="https://docs.python.org/2/library/__future__.html">__future__ imports</a>
to address syntactic differences but avoid <a
href="https://pythonhosted.org/six/">six</a>, <a
href="http://python-future.org/compatible_idioms.html">python-future</a>, etc.
to not introduce additional dependencies.</li>
On 2016/05/19 10:10:24, Sebastian Noack wrote:
> On 2016/05/19 09:03:31, Vasily Kuznetsov wrote:
> > On 2016/05/18 19:06:32, Sebastian Noack wrote:
> > > Whichever Python version is available on the server seems to me more like
a
> > > requirement (for those scripts that potentially run on the server), not
> > > something that belongs into a style guide. You wouldn't list the version
of
> > > libraries used on the server either in the styleguide.
> > > 
> > > And the next time we update the server we will have a newer Python version
> > > anyway. So I think it makes sense, if there is any newer Python version
than
> > the
> > > one used on our servers to test/support them as well.
> > 
> > So we all agree that we would like to eventually migrate all the Python code
> to
> > Python 3. In the near future the more precise target for the migration is
> Python
> > 3.5 until we decide that we can actually do something else. Since at the
> moment
> > everything runs on Python 2, having Python 3.5 in the style guide is a way
to
> > indicate this migration intention and make sure that we don't add migration
> work
> > with the new code.
> > 
> > As for replacing Python 3.5 with "latest Python" -- there seems to be a
> > reasonably broad agreement that we don't want to automatically drop support
> for
> > older version immediately as the new one comes out and it needs to be a
> > deliberate action. If our code runs on Python 3.5, it's very likely to run
and
> > have the same behaviour on Python 3.6 and later, Python 3 branch is supposed
> to
> > maintain backwards compatibility after all. So writing "Python 3.5" would
> cover
> > Python 3.5 and any later version, which is what we want.
> > 
> > 
> > Then there's the requirement of compatibility with Python 2.7. My reasoning
is
> > that this is what we currently run on the servers and what most people would
> > normally use to run those scripts outside of the server environment. Before
we
> > officially unsupport Python 2, all the code has to be compatible with it, so
I
> > didn't want the style guide to give an impression that only Python 3 support
> is
> > needed.
> 
> Yes, Python 3.6+ will in theory be backwards compatible with Python 3.5. But
> nevertheless, as soon as Python 3.6 is out, we should test it as well (which
in
> most cases just means to add an env to the tox.ini). So how about simply
> changing "Python 3.5" to "Python 3.5+" here?

Done.

[Wladimir Palant, 2016-05-19 11:33:26]

LGTM

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
File pages/coding-style.html (right):

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
pages/coding-style.html:47: <li>Make the code compatible with both Python 2.7
and Python 3.5 (see <a
href="https://docs.python.org/dev/howto/pyporting.html">this guide</a>). Use <a
href="https://docs.python.org/2/library/__future__.html">__future__ imports</a>
to address syntactic differences but avoid <a
href="https://pythonhosted.org/six/">six</a>, <a
href="http://python-future.org/compatible_idioms.html">python-future</a>, etc.
to not introduce additional dependencies.</li>
On 2016/05/19 10:10:24, Sebastian Noack wrote:
> Yes, Python 3.6+ will in theory be backwards compatible with Python 3.5. But
> nevertheless, as soon as Python 3.6 is out, we should test it as well (which
in
> most cases just means to add an env to the tox.ini).

As long as this is about automated testing - yes, testing in newer Python
versions is a good thing. Manual testing should always happen with Python 3.5,
since that's the platform we are going to use for a long time. And it makes
sense to mention this explicitly here, since we probably want to have a
consistent baseline Python version throughout all projects. Whether it's Python
3.5 or Python 3.5+ I don't really care.

[Sebastian Noack, 2016-05-19 11:51:54]

LGTM

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
File pages/coding-style.html (right):

https://codereview.adblockplus.org/29342128/diff/29342129/pages/coding-style....
pages/coding-style.html:47: <li>Make the code compatible with both Python 2.7
and Python 3.5 (see <a
href="https://docs.python.org/dev/howto/pyporting.html">this guide</a>). Use <a
href="https://docs.python.org/2/library/__future__.html">__future__ imports</a>
to address syntactic differences but avoid <a
href="https://pythonhosted.org/six/">six</a>, <a
href="http://python-future.org/compatible_idioms.html">python-future</a>, etc.
to not introduce additional dependencies.</li>
On 2016/05/19 11:33:26, Wladimir Palant wrote:
> On 2016/05/19 10:10:24, Sebastian Noack wrote:
> > Yes, Python 3.6+ will in theory be backwards compatible with Python 3.5. But
> > nevertheless, as soon as Python 3.6 is out, we should test it as well (which
> in
> > most cases just means to add an env to the tox.ini).
> 
> As long as this is about automated testing - yes, testing in newer Python
> versions is a good thing. Manual testing should always happen with Python 3.5,
> since that's the platform we are going to use for a long time. And it makes
> sense to mention this explicitly here, since we probably want to have a
> consistent baseline Python version throughout all projects. Whether it's
Python
> 3.5 or Python 3.5+ I don't really care.

I agree, that when manually testing it's important to test on 3.5. However, if
you take the style guide literally you would test on 2.7, 3.5 and each newer
version anyway. Plus, new code as well as code migrated to Python 3, will have
automated tests anyway. So IMO no reason to be even more explicit.