Python is amazing. It is amazing because it has managed to stay relevant and keep growing for so many years.
For a very long time, one of the most important virtues of Python was interoperability. Thanks to that feature it didn't matter what operating system you or your customers were using. If the Python interpreter was available for a given operating system, your software written in Python would work there. And, most importantly, your software would always work the same way regardless of that operating system. However, this is now a common trait. Many modern programming languages provide similar interoperability capabilities. Also, with the advent of cloud computing, web-based applications, and reliable virtualization software, it isn't that important to have a programming language that works in many operating systems.
What seems to be increasingly more important for programmers nowadays is productivity. With a constant search for innovation, it is often important to build something that you can test in the field with real users and then iterate quickly from that point until you build a really valuable product. And Python allows programmers to iterate really fast. The Official Python Package Index is a tremendous library of software libraries and frameworks that can be easily reused in your software. It greatly reduces the amount of time and effort required to get your job done. This vast availability of community libraries together with clear and concise syntax that favors readability greatly limits the time and effort that has to be spent on creating and maintaining the software. That makes Python really shine in the area of programming productivity.
Python stayed relevant for so long because it was constantly evolving. And it still keeps evolving. That's why in this chapter we will take a brief look at the current status of Python and learn how to stay up to date with changes happening in the Python ecosystem and community.
In this chapter, we will cover the following topics:
Let's start by considering the history and development of Python, and where we're currently at.
Python isn't a young language. Its history starts somewhere in the late 1980s and the official 1.0 release happened in year 1994. We could discuss the whole timeline of major Python releases mentioned here, but we really only need to be concerned about a few dates:
So, at the time of writing, Python 3 is almost half as old as Python itself. It has also been active for longer than Python 2 was when it comes to active development of new language features.
Although Python 3 has been around for quite some time, its adoption was so slow that the initial end of life for Python 2 was postponed by 5 years. It was mostly due to a lot of backward compatibility issues that didn't always allow an easy and straightforward transition. A big part of Python's greatness comes from the vast number of freely available libraries. So, it was even harder to make the Python 3 transition if your software critically relied on a third-party Python package that wasn't compatible with Python 3 yet.
Fortunately, Python 2 is finally behind us and the Python community can finally breathe a sigh of relief. Many developers of open-source Python libraries stopped providing compatibility with Python 2 already a long time ago. Also, the official sunsetting of Python 2 provided an incentive to strategically prioritize transitions to Python 3 in corporate environments, where there is usually limited desire to do so. That incentive can be attributed mainly to the fact that there is absolutely no chance of security patches for Python 2 being delivered anymore.
We know that Python 2 is no longer officially supported by language developers. Due to the lack of security patches, it should be considered unsafe. But is it dead yet?
Probably not. Even relatively popular open-source libraries can lose the interest of their authors and contributors over time. It also happens that some better alternatives appear, and there's simply no point in developing the original library anymore. Such libraries are often left unmaintained, so no one could update them for compatibility with Python 3.
One of the important reasons why Python 3 couldn't take off for a long time was the sluggish release processes of many Linux distributions. Distribution maintainers often aren't inclined toward the fast adoption of new language versions, especially if these versions break backward compatibility and require patches of other software. Many Python programmers are Linux users, and if they don't have access to the latest version of the language through the system package repository they are less likely to upgrade it on their own.
But the place where Python 2 will definitely linger for a few more years is corporate environments. When money is at stake, it is usually hard to convince the stakeholders that something that's already been done needs to be revisited just for the sake of being up to date with upstream changes, especially if the software works.
Python 2 code probably won't be that common for the core components of actively developed software, but it can still be found quite often in utility scripts, internal tools, or services that haven't seen active development for a long time.
If you're in a situation where you still need to maintain bits of software written in Python 2 you would be better off migrating to Python 3 soon. This often can't be done overnight, and sometimes you'll have to convince others first before you're ready to proceed.
If you're in such a situation, consider the following strategy:
Usually the hardest thing to do is to get buy-in from stakeholders, especially if you're writing code professionally and need to find a way to squeeze such a project into the day-to-day development of new features. You'll need to be well prepared before raising such an issue. That's why the first two steps are essential. They allow you to estimate the amount of work required to make the change and construct a convincing reason for doing so. After all, the best way to convince someone is by presenting the list of benefits of doing a migration. The following are common positive reasons for doing a migration:
Another tool for convincing stakeholders is explaining the risks related to not migrating to a newer version of Python:
Regardless of your communication strategy, you'll have to create a reasonable tactical plan that carefully balances the promise of positive outcomes, risks, and the ability to reach team goals at a sustainable pace. For instance, investing in reducing future maintenance costs usually pays for itself only after a longer period of time. That's why it makes sense to also spread such investment over time. Known and exploitable security vulnerabilities, on the other hand, should always be prioritized.
Technology evolves constantly. People are constantly looking for tools that will allow them to solve their problems more easily than before. Every few months, either a completely new language pops up almost out of nowhere, or one of the well-established languages introduces a completely new syntax feature. This also happens to Python. We will briefly consider a few of the most important recent additions to the Python language in Chapter 3, New Things in Python.
New languages or language features drive the creation of novel libraries and frameworks. These in turn open the way for new programming paradigms and new design patterns. The success of one such pattern or paradigm in one language ecosystem often encourages programmers to adopt them in other languages. That's how new ideas spread from language to language.
We see that happening to Python too. We will be able to see how seemingly different programming languages share similar features or concepts in Chapter 4, Python in Comparison with Other Languages.
This process of language evolution is constant, inevitable, and seems to be only speeding up. Starting from Python 3.9 (released October 5, 2020), new major releases of Python will be happening annually. These releases more often than not lead to explosions of new libraries and frameworks that are trying to test and explore new tools. It's great for the Python community because it gives it constant fuel to innovate. But it can be overwhelming for anyone that wants to keep an eye on what's happening in Python. And being up to date is extremely important for every Python professional.
In the following few sections, we will discuss a few ways in which you can source information on what is happening in Python. It will allow you to better anticipate changes happening to the language and the community around it. This will keep you up to date with the latest best practices, and new tools that are worth investing in.
The Python community has a well-established way of dealing with changes. While speculative Python language ideas are mostly discussed on a specific mailing list (python-ideas@python.org), nothing major ever gets changed without the existence of a new document, called a Python Enhancement Proposal (PEP).
You can subscribe to python-ideas@python.org and other mailing lists at https://mail.python.org/mailman3/lists/.
A PEP is a formalized document that describes, in detail, the proposal of change to be made in Python. It is also the starting point for the community discussion. A few of the topics covered in this chapter were in fact already extensively described in their dedicated PEP documents:
The whole purpose, format, and workflow of these documents is also standardized in the form of a standalone PEP—the PEP 1 document.
PEP documentation is very important for Python, and, depending on the topic, it serves different purposes:
A list of all proposed PEPs is available in a living (by which we mean continuously updated) PEP 0 document. It is a great source of information for those who are wondering what direction Python is heading in but do not have time to track every discussion on Python mailing lists. It shows which ideas have been accepted, which ideas have been implemented, and also which ideas are still under consideration.
Online URLs for PEP documents take the following form:
http://www.python.org/dev/peps/pep-XXXX
XXXX
is a 4-digit PEP number with leading zeros. It should also be easy to find them through web search engines using the Python PEP XXX
search term.
The PEP 0 document is available at https://www.python.org/dev/peps/.
The index of all officially discussed proposals serves an additional purpose. Very often, people ask questions like the following:
In most such cases, the answer is already available in the specific PEP where the feature was discussed. Instead of having the same conversation about adding some language feature over and over again, people can be redirected to a specific PEP. And there are a lot of PEPs describing Python language features that have been proposed but not accepted.
When it comes to being up to date, the most important section of the PEP index is definitely open PEPs, which describes ideas that are still under active consideration. For instance, the following are selected interesting PEPs that were still open at the time of writing:
These proposals vary from relatively small extensions of the existing standard library (such as PEP 603) to completely new and complex syntax features (such as PEP 638). If you wonder what the Python community is considering including in future Python releases, then open PEPs are the best source of such information.
The non-profit organization behind Python is Python Software Foundation (PSF). It holds intellectual property rights for Python and manages its licensing. A brief excerpt from its mission statement is as follows:
The mission of the Python Software Foundation is to promote, protect, and advance the Python programming language, and to support and facilitate the growth of a diverse and international community of Python programmers.
You can find the full PSF mission statement text at https://www.python.org/psf/mission/.
Support for the community of Python programmers is important to the PSF mission because it is, after all, the community that drives the development of Python. The community does that by extending the language transparently and openly (as explained in the PEP documents section), but also through expanding and maintaining a rich ecosystem of third-party packages and frameworks. So, one of the best ways to know what's happening in Python is to get in touch with its community.
Like any other programming language, there are a lot of independent online communities dedicated to Python. They are usually focused on specific frameworks or software development areas, like web development, data science, machine learning, and so on.
One could assume that there is at least a single place online where all important discussions about the core of the language and its interpreter happen. Unfortunately, it's not that simple.
Due to many reasons, some of which are historic, the landscape of official Python mailing lists and discussion boards can be very confusing. There are numerous official and semi-official mailing lists and discussion boards where Python developers hang out. It is especially confusing when it comes to mailing lists because official mailing lists are spread over two different mailing list managers and thus have two separate sets of list archives. Those archives are:
When it comes to actual mailing lists, there's plenty of them, but sadly the majority of them aren't active anymore. Some mailing lists are dedicated to specific projects (like scikit-image@python.org) or focus on specific areas of interest of their members (like code-quality@python.org). Besides lists with very specific themes, there are few general mailing lists that should be very interesting for every Python programmer. These are:
Besides classic mailing lists, there is an official internet forum on the Discourse platform that is available at https://discuss.python.org/. It is a relatively new addition to the Python official discussion landscape and its purpose overlaps with many of the pre-existing mailing lists as it has dedicated categories for discussing new ideas, PEPs, and topics of core Python development. It has a lower entry barrier for those unfamiliar with the concept of mailing lists, and a much more modern user experience.
Unfortunately, not every new discussion happens on discuss.python.org, so if you want to know about everything that happens at the heart of Python development you will have to keep track of discussion both on the forum and mailing lists. Hopefully, these will eventually converge in a single place.
Besides the official message boards and mailing lists, there are a couple of open Python communities based on popular discussion and messaging platforms. The most notable ones are:
These three communities are open in the sense that you can freely join them as long as you are a user of their underlying platforms (which are, of course, free). Whatever you choose will probably be dictated by your preference for the specific messaging or discussion platform. The chances are high that you or one of your friends is already using one of these platforms.
The unquestionable advantage of such open communities is that they gather a very large number of members, and you can almost always find someone online that you can talk to. This provides the opportunity for ad hoc and loose discussions about various Python-related topics and allows you to seek quick help in case of simple programming problems.
The downside is that it is impossible to keep track of every discussion happening there. Fortunately, these communities often create systems of separate subchannels or tags that you can subscribe to if you want to be notified about the content of your specific interest. Also, these communities aren't officially endorsed and curated by the PSF. As a result, information found on Reddit or other online communities may sometimes be biased or inaccurate.
Reading all the new PEPs, following mailing lists, and joining communities are the best ways to know what is happening at the moment. Unfortunately, doing this consistently requires a lot of time and effort as you will have to filter through huge amounts of information. Also, sources like mailing lists, message boards, and messaging platforms can be full of emotions because discussion is essentially a human interaction. And surprisingly, some tech discussions on contentious topics can be so heated that they are almost indistinguishable from social network drama.
If you are really busy or quickly get overloaded with social interactions between online strangers, there is another way. Instead of filtering online information all by yourself, you can turn to curated content like blogs, newsletters, and so-called "awesome lists."
Newsletters are especially good if you want to be up to date. These are some interesting newsletters that are worth subscribing to:
These newsletters will keep you informed about the most important Python announcements. They will also help you discover new blogs or especially interesting discussions from other discussion platforms, like Reddit or Hacker News. Keep in mind that the content of many general Python newsletters often overlaps, so probably there is no need to subscribe to all of them.
A completely different way to source information is via awesome lists. These are curated lists of links to valuable or important resources on specific topics maintained usually as Git repositories on GitHub. These lists are often very long and split into several categories.
These are examples of popular Python-related awesome lists curated by various GitHub users:
Awesome lists usually grow over time to enormous sizes. This means they are not a good resource for keeping yourself constantly updated. This is because they are simply snapshots of what the curators considered awesome at the time. Nevertheless, if you need to jump into a completely new area, let's say artificial intelligence, they serve as a good starting point for further research.
In this chapter, we've discussed the current status of Python and the process of change that is visible throughout the history of that language. We've learned why Python is changing and why it is important to follow that change.
Staying relevant is one of the biggest and most stressful challenges that professional programmers face regardless of the programming language they choose. Due to over 30 years of Python history and the ever-growing Python community, it isn't always clear how to efficiently stay up to date with the changes in the Python ecosystem. That's why we've looked into all the places where you can follow important discussions about Python's future.
What also changes together with the language are development tools that aim to ease and improve the software development processes. In the next chapter, we will continue with the topic of change and take a look at modern development environments. We will learn how to create repeatable and consistent runtime environments both for production and development use. We will also get familiar with various productivity tools provided by the Python community.