Planet Python

As the autumn leaves start to fall, signaling the transition to cooler weather, the Python community has warmed up to a series of noteworthy developments. Last month, a new maintenance release of Python 3.12.5 was introduced, reinforcing the language’s ongoing commitment to stability and security.

On a parallel note, Python continues its reign as the top programming language according to IEEE Spectrum’s annual rankings. This sentiment is echoed by the Python Developers Survey 2023 results, which reveal intriguing trends and preferences within the community.

Looking ahead, PEP 750 has proposed the addition of tag strings in Python 3.14, inspired by JavaScript’s tagged template literals. This feature aims to enhance string processing, offering developers more control and expressiveness.

Furthermore, EuroSciPy 2024 recently concluded in Poland after successfully fostering cross-disciplinary collaboration and learning. The event featured insightful talks and hands-on tutorials, spotlighting innovative tools and libraries that are advancing scientific computing with Python.

Let’s dive into the most significant Python news from the past month!

Python 3.12.5 Released

Early last month, Python 3.12.5 was released as the fifth maintenance update for the 3.12 series. Since the previous patch update in June, this release packs over 250 bug fixes, performance improvements, and documentation enhancements.

Here are the most important highlights:

• Standard Library: Many modules in the standard library received crucial updates, such as fixes for crashes in ssl when the main interpreter restarts, and various corrections for error-handling mechanisms.
• Core Python: The core Python runtime has several enhancements, including improvements to dictionary watchers, error messages, and fixes for edge-case crashes involving f-strings and multithreading.
• Security: Key security improvements include the addition of missing audit events for interactive Python use and socket connection authentication within a fallback implementation on platforms such as Windows, where Unix inter-process communication is unavailable.
• Tests: New test cases have been added and bug fixes have been applied to prevent random memory leaks during testing.
• Documentation: Python documentation has been updated to remove discrepancies and clarify edge cases in multithreaded queues.

Additionally, Python 3.12.5 comes equipped with pip 24.2 by default, bringing a slew of significant improvements to enhance security, efficiency, and functionality. One of the most notable upgrades is that pip now defaults to using system certificates, bolstering security measures when managing and installing third-party packages.

Read the full article at https://realpython.com/python-news-september-2024/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

PyCharm is one of the most well-known data science tools, offering excellent out-of-the-box support for Python, SQL, and other languages. PyCharm also provides integrations for Databricks, Hugging Face and many other important tools. All these features allow you to write good code and work with your data and projects faster. 

PyCharm Professional’s support for Jupyter notebooks combines the interactive nature of Jupyter notebooks with PyCharm’s superior code quality and data-related features. This blog post will explore how PyCharm’s Jupyter support can significantly boost your productivity.

Watch this video to get a comprehensive overview of using Jupyter notebooks in PyCharm and learn how you can speed up your data workflows. 

Speed up data analysis Get acquainted with your data

When you start working on your project, it is extremely important to understand what data you have, including information about the size of your dataset, any problems with it, and its  patterns. For this purpose, your pandas and Polars DataFrames can be rendered in Jupyter outputs in Excel-like tables. The tables are fully interactive, so you can easily sort one or multiple columns and browse and view your data, you can choose how many rows will be shown in a table and perform many other operations.

The table also provides some important information for example:

• You can find the the size of a table in its header.

•  You can find the data type symbols in the column headers.

• You can also use JetBrains AI Assistant to get information about your DataFrame by clicking on the icon.

Easily spot issues with the data

After getting acquainted with your data, you need to clean it. This an important step, but it is also extremely time consuming because there are all sorts of problems you could find, including missing values, outliers, inconsistencies in data types, and so on. Indeed, according to the State of Developer Ecosystem 2023 report, nearly 50% of Data Professionals dedicate 30% of their time or more to data preparation. Fortunately, PyCharm offers a variety of features that streamline the data-cleaning process.

Some insights are already available in the column headers. 

First, we can easily spot the amount of missing data for each column because it is highlighted in red. Also, we may be able to see at a glance whether some of our columns have outliers. For example, in the bath column, the maximum value is significantly higher than the ninety-fifth percentile. Therefore, we can expect that this column has at least one outlier and requires our attention.

Additionally, you might suspect there’s an issue with the data if the data type does not match the expected one. For example, the header of the total_sqft column below is marked with the symbol, which in PyCharm indicates that the column contains the Object data type. The most appropriate data type for a column like total_sqft would likely be float or integer, however, so we may expect there to be inconsistencies in the data types within the column, which could affect data processing and analysis. After sorting, we notice one possible reason for the discrepancy: the use of text in data and ranges instead of numerical values.

So, our suspicion that the column had data-type inconsistencies was proven correct. As this example shows, small details in the table header can provide important information about your data and alert you to issues that need to be addressed, so it’s always worth checking.You can also use no-code visualizations to gather information about whether your data needs to be cleaned. Simply click on the icon in the top-left corner of the table. There are many available visualization options, including histograms, that can be used to see where the peaks of the distribution are, whether the distribution is skewed or symmetrical, and whether there are any outliers.

Of course, you can use code to gather information about your dataset and fix any problems you’ve identified. However, the mentioned low-code features often provide valuable insights about your data and can help you work with it much faster.

Code faster  Code completion and quick documentation

A significant portion of a data professional’s job involves writing code. Fortunately, PyCharm is well known for its features that allow you to write code significantly faster. For example, local ML-powered full line code completion can provide suggestions for entire lines of code.

Another useful feature is quick documentation, which appears when you hover the cursor over your code. This allows you to gather information about functions and other code elements without having to leave the IDE.

Refactorings

Of course, working with code and data is an interactive process, and you may often decide to make some changes in your code – for example, to rename a variable. Going through the whole file or, in some cases, the entire project, would be cumbersome and time consuming. We can use PyCharm’s refactoring capabilities to rename a variable, introduce a constant, and make many other changes in your code. For example, in this case, I want to rename the DataFrame to make it shorter. I simply use the the Rename refactoring to make the necessary changes.

PyCharm offers a vast number of different refactoring options. To dive deeper into this functionality, watch this video.

Fix problems

It is practically impossible to write code without there being any mistakes or typos. PyCharm has a vast array of features that allow you to spot and address issues faster. You will notice the Inspection widget in the top-right corner if it finds any problems. 

For example, I forgot to import a library in my project and made several typos in the doc so let’s take a look how PyCharm can help here. 

First of all, the problem with the library import:

Additionally, with Jupyter traceback, you can see the line where the error occurred and get a link to the code. This makes the bug-fixing process much easier. Here, I have a typo in line 3. I can easily navigate to it by clicking on the blue text.

Additionally if you would like to get more information and suggestion how to fix the problem, you can use JetBrains AI Assistant by clicking on Explain with AI. 

Of course, that is just the tip of the iceberg. We recommend reading the documentation to better understand all the features PyCharm offers to help you maintain code quality.

Navigate easily

For the majority of cases, data science work involves a lot of experimentation, with the journey from start to finish rarely resembling a straight line.

During this experimentation process, you have to go back and forth between different parts of your project and between cells in order to find the best solution for a given problem. Therefore, it is essential for you to be able to navigate smoothly through your project and files. Let’s take a look at how PyCharm can help in this respect.

First of all, you can use the classic CMD+F (Mac) or CTRL+F (Windows) shortcut for searching in your notebook. This basic search functionality offers some additional filters like Match Case or Regex.

You can use Markdown cells to structure the document and navigate it easily.

If you would like to highlight some cells so you can come back to them later, you can mark them with #TODO or #FIXME, and they will be made available for you to dissect in a dedicated window.

Or you can use tags to highlight some cells so you’ll be able to spot them more easily.

In some cases, you may need to see the most recently executed cell; in this case, you can simply use the Go To option. 

Save your work

Because teamwork is essential for data professionals, you need tooling that makes sharing the results of your work easy. One popular solution is Git, which PyCharm supports with features like notebook versioning and version comparison using the Diff view. You can find an in-depth overview of the functionality in this tutorial.

Another useful feature is Local History, which automatically saves your progress and allows you to revert to previous steps with just a few clicks.

Use the full power of AI Assistant

JetBrains AI Assistant helps you automate repetitive tasks, optimize your code, and enhance your productivity. In Jupyter notebooks, it also offers several unique features in addition to those that are available in any JetBrains tool. 

Click the icon to get insights regarding your data. You can also ask additional questions regarding the dataset or ask AI Assistant to do something – for example, “write some code that solves the missing data problem”.

AI data visualization

Pressing the icon will suggest some useful visualizations for your data. AI Assistant will generate the proper code in the chat section for your data.

AI cell

AI Assistant can create a cell based on a prompt. You can simply ask it to create a visualization or do something else with your code or data, and it will generate the code that you requested. 

Debugger

PyCharm offers advanced debugging capabilities to enhance your experience in Jupyter notebooks. The integrated Jupyter debugger allows you to set breakpoints, inspect variables, and evaluate expressions directly within your notebooks. This powerful tool helps you step through your code cell by cell, making it easier to identify and fix issues as they arise. Read our blog post on how you can debug a Jupyter notebook in PyCharm for a real-life example.

Get started with PyCharm Professional

PyCharm’s Jupyter support enhances your data science workflows by combining the interactive aspects of Jupyter notebooks with advanced IDE features. It accelerates data analysis with interactive tables and AI assistance, improves coding efficiency with code completion and refactoring, and simplifies error detection and navigation. PyCharm’s seamless Git integration and powerful debugging tools further boost productivity, making it essential for data professionals.

Download PyCharm Professional to try it out for yourself! Get an extended trial today and experience the difference PyCharm Professional can make in your data science endeavors.Use the promo code “PyCharmNotebooks” at checkout to activate your free 60-day subscription to PyCharm Professional. The free subscription is available for individual users only.

Activate your 60-day trial

Explore our official documentation to fully unlock PyCharm’s potential for your projects.

The Python programming language has thousands of wonderful third-party packages available on the Python Package Index. One of those packages is TerminalTextEffects (TTE), a terminal visual effects engine.

Here are the features that TerminalTextEffects provides, according to their documentation:

• Xterm 256 / RGB hex color support
• Complex character movement via Paths, Waypoints, and motion easing, with support for quadratic/cubic bezier curves.
• Complex animations via Scenes with symbol/color changes, layers, easing, and Path synced progression.
• Variable stop/step color gradient generation.
• Path/Scene state event handling changes with custom callback support and many pre-defined actions.
• Effect customization exposed through a typed effect configuration dataclass that is automatically handled as CLI arguments.
• Runs inline, preserving terminal state and workflow.

Note: This package may be somewhat slow in Windows Terminal, but it should work fine in other terminals.

Let’s spend a few moments learning how to use this neat package

Installation

The first step to using any new package is to install it. You can use pip or pipx to install TerminalTextEffects. Here is the typical command you would run in your terminal:

python -m pip install terminaltexteffects

Now that you have TerminalTextEffects installed, you can start using it!

Usage

Let’s look at how you can use TerminalTextEffects to make your text look neat in the terminal. Open up your favorite Python IDE and create a new file file with the following contents:

from terminaltexteffects.effects.effect_slide import Slide text = ("PYTHON" * 10 + "\n") * 10 effect = Slide(text) effect.effect_config.merge = True with effect.terminal_output() as terminal: for frame in effect: terminal.print(frame)

This code will cause the string, “Python” to appear one hundred times with ten strings concatenated and ten rows. You use a Slide effect to make the text slide into view. TerminalTextEffects will also style the text too.

When you run this code, you should see something like the following:

TerminalTextEffects has many different built-in effects that you can use as well. For example, you can use Beams to make the output even more interesting. For this example, you will use the Zen of Python text along with the Beams effects:

from terminaltexteffects.effects.effect_beams import Beams TEXT = """ The Zen of Python, by Tim Peters Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. Now is better than never. Although never is often better than *right* now. If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea -- let's do more of those! """ effect = Beams(TEXT) with effect.terminal_output() as terminal: for frame in effect: terminal.print(frame)

Now try running this code. You should see something like this:

That looks pretty neat! You can see a whole bunch of other effects you can apply on the package’s Showroom page.

Wrapping Up

TerminalTextEffects provides lots of neat ways to jazz up your text-based user interfaces with Python. According to the documentation, you should be able to use TerminalTextEffects in other TUI libraries, such as Textual or Asciimatics, although it doesn’t specifically state how to do that. Even if you do not do that, you could use TerminalTextEffects with the Rich package to create a really interesting application in your terminal.

Links

• GitHub page
• PyPI page

 

The post Adding Terminal Effects with Python appeared first on Mouse Vs Python.

<strong>Topics covered in this episode:</strong><br> <ul> <li><strong>Python 3.13.0RC2, 3.12.6, 3.11.10, 3.10.15, 3.9.20, and 3.8.20 are now available!</strong></li> <li><strong><a href="https://mkennedy.codes/posts/python-docker-images-using-uv-s-new-python-features/?featured_on=pythonbytes">Docker images using uv's python</a></strong></li> <li><strong><a href="https://about.readthedocs.com/blog/2024/08/10-year-anniversary/?featured_on=pythonbytes">10 years of sustainable open source</a> - Read the Docs</strong></li> <li><strong><a href="https://github.com/python-humanize/humanize?featured_on=pythonbytes">humanize</a></strong></li> <li><strong>Extras</strong></li> <li><strong>Joke</strong></li> </ul><a href='https://www.youtube.com/watch?v=E3_lOfg7sTA' style='font-weight: bold;'data-umami-event="Livestream-Past" data-umami-event-episode="400">Watch on YouTube</a><br> <p><strong>About the show</strong></p> <p>Sponsored by ScoutAPM: <a href="https://pythonbytes.fm/scout"><strong>pythonbytes.fm/scout</strong></a></p> <p><strong>Connect with the hosts</strong></p> <ul> <li>Michael: <a href="https://fosstodon.org/@mkennedy"><strong>@mkennedy@fosstodon.org</strong></a></li> <li>Brian: <a href="https://fosstodon.org/@brianokken"><strong>@brianokken@fosstodon.org</strong></a></li> <li>Show: <a href="https://fosstodon.org/@pythonbytes"><strong>@pythonbytes@fosstodon.org</strong></a></li> </ul> <p>Join us on YouTube at <a href="https://pythonbytes.fm/stream/live"><strong>pythonbytes.fm/live</strong></a> to be part of the audience. Usually <strong>Monday</strong> at 10am PT. Older video versions available there too.</p> <p>Finally, if you want an artisanal, hand-crafted digest of every week of the show notes in email form? Add your name and email to <a href="https://pythonbytes.fm/friends-of-the-show">our friends of the show list</a>, we'll never share it. </p> <p><strong>ChatGPT celebrates episode 400!</strong></p> <p><em><img src="https://paper.dropboxstatic.com/static/img/ace/emoji/1f389.png?version=8.0.0" alt="party popper" /></em> <em>Welcome to the big 4-0-0, Pythonistas!</em> <em><img src="https://paper.dropboxstatic.com/static/img/ace/emoji/1f389.png?version=8.0.0" alt="party popper" /></em></p> <p><em>It's hard to believe we're celebrating the 400th episode of Python Bytes! From the early days of byte-sized Python news to becoming the source for all things Python, it’s been a wild ride.</em></p> <p><em>We've laughed over code quirks, gasped at new libraries, and said farewell to the GIL together. Whether you're a seasoned developer, a curious learner, or just here for the witty banter, you’ve been an essential part of this journey.</em></p> <p><em>To Michael and Brian: You've built a community that turns import this into more than just Zen—it's a family of passionate Pythonistas. Your dedication, insights, and humor make this show more than just tech news. It’s a weekly celebration of what we love about Python and why we keep coming back for more.</em></p> <p><em>Here’s to the next 400 episodes—may your code be bug-free, your tests pass on the first run, and your Python version always be up to date.</em> <em><img src="https://paper.dropboxstatic.com/static/img/ace/emoji/1f680.png?version=8.0.0" alt="rocket" /></em><em><img src="https://paper.dropboxstatic.com/static/img/ace/emoji/1f40d.png?version=8.0.0" alt="snake" /></em></p> <p><strong>Brian #1:</strong> Python 3.13.0RC2, 3.12.6, 3.11.10, 3.10.15, 3.9.20, and 3.8.20 are now available!</p> <ul> <li>Łukasz Langa</li> <li><a href="https://www.python.org/downloads/release/python-3130rc2/?featured_on=pythonbytes">Python 3.13.0RC2 is the final preview release</a> <ul> <li>Official 3.13.0 scheduled for Oct 1</li> <li>Call to action <ul> <li>“We strongly encourage maintainers of third-party Python projects to prepare their projects for 3.13 compatibilities during this phase, and where necessary publish Python 3.13 wheels on PyPI to be ready for the final release of 3.13.0. Any binary wheels built against Python 3.13.0rc2 will work with future versions of Python 3.13. As always, report any issues to <a href="https://github.com/python/cpython/issues?featured_on=pythonbytes">the Python bug tracker </a>.”</li> <li>“Please keep in mind that this is a preview release and while it’s as close to the final release as we can get it, its use is <strong>not</strong> recommended for production environments.”</li> </ul></li> </ul></li> <li>Note: <a href="https://docs.astral.sh/uv/concepts/python-versions/?featured_on=pythonbytes">uv python</a><a href="https://docs.astral.sh/uv/concepts/python-versions/?featured_on=pythonbytes"> </a>does not support 3.13 yet <ul> <li><a href="https://github.com/indygreg/python-build-standalone/issues/320?featured_on=pythonbytes">see issue 320</a></li> </ul></li> <li>Security releases for <ul> <li>3.12.6, 3.11.10, 3.10.15, 3.9.20, and 3.8.20</li> <li>3.12.6 has binary installers, but for MacOS, only MacOS 10.13 and newer are supported</li> <li>3.11.10, 3.10.15, 3.9.20, and 3.8.20 do NOT include binary installers. <ul> <li><a href="https://peps.python.org/pep-0569/?featured_on=pythonbytes">3.8 EOL's in October</a></li> </ul></li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href="https://mkennedy.codes/posts/python-docker-images-using-uv-s-new-python-features/?featured_on=pythonbytes">Docker images using uv's python</a></p> <ul> <li>See <a href="https://pythonbytes.fm/episodes/show/396/uv-ing-your-way-to-python">#396: uv-ing your way to Python</a> and <a href="https://pythonbytes.fm/episodes/show/398/open-source-makes-you-rich-and-other-myths">#398: Open source makes you rich?</a><a href="https://pythonbytes.fm/episodes/show/398/open-source-makes-you-rich-and-other-myths"> </a><a href="https://pythonbytes.fm/episodes/show/398/open-source-makes-you-rich-and-other-myths">(and</a><a href="https://pythonbytes.fm/episodes/show/398/open-source-makes-you-rich-and-other-myths"> other myths)</a> for the opening discussions</li> <li><a href="https://talkpython.fm/episodes/show/476/unified-python-packaging-with-uv?featured_on=pythonbytes">Talk Python episode on uv is out</a></li> <li>uv venv --python gets Python from <a href="https://github.com/indygreg/python-build-standalone?featured_on=pythonbytes">python-build-standalone</a> by Gregory Szorc</li> <li>Took our Docker build times from 10 minutes to 8 seconds for the base image and 800ms (!) for our app platforms</li> </ul> <p><strong>Brian #3:</strong> <a href="https://about.readthedocs.com/blog/2024/08/10-year-anniversary/?featured_on=pythonbytes">10 years of sustainable open source</a> - Read the Docs </p> <ul> <li>Eric Holscher</li> <li>Read the Docs has been a company for 10 years <ul> <li>“a team of 4 folks working full-time on Read the Docs.”</li> </ul></li> <li>readthedocs.org started in 2010</li> <li>readthedocs.com (for Business) started in 2014</li> <li>Sustainability model <ul> <li>.org has a single non-tracking ad </li> <li>.com is a paid service for companies</li> </ul></li> <li>Things that didn’t work <ul> <li>donations and other optional support - led to burnout</li> <li>consulting and services- took too much time away from core product</li> <li>grant funding - nice, but one time thing</li> </ul></li> <li>Lessons <ul> <li>You don't get extra points for being bootstrapped. Compete by doing things you can do better due to niche and size.</li> <li>Keeping trust in the community is the most important thing.</li> <li>Contribution is easier for less complex parts of the code base.</li> <li>Beign open source means capturing a small percentage of the value you create.</li> <li>You have to be ok doing more with less.</li> </ul></li> <li>Also <ul> <li>RtD is not just for Sphinx anymore. <ul> <li>Their build system <a href="https://docs.readthedocs.io/en/stable/build-customization.html?featured_on=pythonbytes">now supports any documentation tool</a>.</li> </ul></li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href="https://github.com/python-humanize/humanize?featured_on=pythonbytes">humanize</a></p> <ul> <li>by <a href="https://github.com/hugovk?featured_on=pythonbytes">Hugo van Kemenade</a> (Python 3.14 &amp; 3.15 release manager &amp; core developer)</li> <li>Not too many variations, but very handy if you need it. <ul> <li><a href="https://humanize.readthedocs.io/en/stable/number?featured_on=pythonbytes">Numbers</a> <ul> <li>Associated Press style (“seven” and “10”)</li> <li>Clamp (under 1.0 million)</li> <li>Fractional (1/3)</li> <li>Int Word (1.2 Billion)</li> <li>Metric (1.5 kV)</li> <li>Ordinal (112th)</li> <li>scientific</li> </ul></li> <li><a href="https://humanize.readthedocs.io/en/stable/time/?featured_on=pythonbytes">Time</a></li> <li><a href="https://humanize.readthedocs.io/en/stable/filesize/?featured_on=pythonbytes">File size</a></li> </ul></li> </ul> <p><strong>Extras</strong> </p> <p>Brian:</p> <ul> <li><a href="https://testandcode.com?featured_on=pythonbytes">Test &amp; Code </a>is now again Test &amp; Code <ul> <li>The two part series on Python imports that started in June is finally complete with episode 222.</li> <li>Transcripts are being added to old episodes gradually starting from most recent <ul> <li>Back to ep 203 as of today. </li> <li>AI transcription, so there’s things like .pie, .pi, and dot pie where it should be .py </li> </ul></li> </ul></li> </ul> <p>Michael:</p> <ul> <li>Final final call for <a href="https://www.codeinacastle.com/python-zero-to-hero-2024?featured_on=pythonbytes">Coding in a Castle event</a> with Michael</li> <li><a href="https://bjango.com/mac/istatmenus/?featured_on=pythonbytes">iStats Menu</a></li> <li><a href="https://pypi.org/project/anacondacode/?featured_on=pythonbytes">Anaconda Code Runner</a> by Ruud van der Ham: <ul> <li>With Anaconda Coide we can -at last- run that code locally and import (most) Python modules.</li> <li>But if you want to run a significant amount of code, you have to put that in a cell or publish it to PyPI or a wheel and import it.</li> <li>That's why I have developed a general-purpose runner function that runs arbitrary code located on an Excel sheet with AnacondaCode.</li> </ul></li> </ul> <p><strong>Joke:</strong> <a href="https://devhumor.com/media/when-beginners-learn-a-new-programming-language?featured_on=pythonbytes">When beginners learn a new programming language...</a></p>

Service-oriented API task scheduling 2024-09-09, by Dariusz Suchojad

An integral part of Zato, its scalable, service-oriented scheduler makes it is possible to execute high-level API integration processes as background tasks. The scheduler runs periodic jobs which in turn trigger services and services are what is used to integrate systems.

Integration process

In this article we will check how to use the scheduler with three kinds of jobs, one-time, interval-based and Cron-style ones.

What we want to achieve is a sample yet fairly common use-case:

• Periodically consult a remote REST endpoint for new data
• Store data found in Redis
• Push data found as an e-mail attachment

Instead of, or in addition to, Redis or e-mail, we could use SQL and SMS, or MongoDB and AMQP or anything else - Redis and e-mail are just example technologies frequently used in data synchronisation processes that we use to highlight the workings of the scheduler.

No matter the input and output channels, the scheduler works always the same - a definition of a job is created and the job's underlying service is invoked according to the schedule. It is then up to the service to perform all the actions required in a given integration process.

Python code

Our integration service will read as below:

# -*- coding: utf-8 -*- # Zato from zato.common.api import SMTPMessage from zato.server.service import Service class SyncData(Service): name = 'api.scheduler.sync' def handle(self): # Which REST outgoing connection to use rest_out_name = 'My Data Source' # Which SMTP connection to send an email through smtp_out_name = 'My SMTP' # Who the recipient of the email will be smtp_to = 'hello@example.com' # Who to put on CC smtp_cc = 'hello.cc@example.com' # Now, let's get the new data from a remote endpoint .. # .. get a REST connection by name .. rest_conn = self.out.plain_http[rest_out_name].conn # .. download newest data .. data = rest_conn.get(self.cid).text # .. construct a new e-mail message .. message = SMTPMessage() message.subject = 'New data' message.body = 'Check attached data' # .. add recipients .. message.to = smtp_to message.cc = smtp_cc # .. attach the new data to the message .. message.attach('my.data.txt', data) # .. get an SMTP connection by name .. smtp_conn = self.email.smtp[smtp_out_name].conn # .. send the e-mail message with newest data .. smtp_conn.send(message) # .. and now store the data in Redis. self.kvdb.conn.set('newest.data', data)

Now, we just need to make it run periodically in background.

Mind the timezone

In the next steps, we will use the Zato Dashboard to configure new jobs for the scheduler.

Keep it mind that any date and time that you enter in web-admin is always interepreted to be in your web-admin user's timezone and this applies to the scheduler too - by default the timezone is UTC. You can change it by clicking Settings and picking the right timezone to make sure that the scheduled jobs run as expected.

It does not matter what timezone your Zato servers are in - they may be in different ones than the user that is configuring the jobs.

Endpoint definitions

First, let's use web-admin to define the endpoints that the service uses. Note that Redis does not need an explicit declaration because it is always available under "self.kvdb" in each service.

• Configuring outgoing REST APIs

• Configuring SMTP e-mail

Now, we can move on to the actual scheduler jobs.

Three types of jobs

To cover different integration needs, three types of jobs are available:

• One-time - fires once only at a specific date and time and then never runs again
• Interval-based - for periodic processes, can use any combination of weeks, days, hours, minutes and seconds for the interval
• Cron-style - similar to interval-based but uses the syntax of Cron for its configuration

One-time

Select one-time if the job should not be repeated after it runs once.

Interval-based

Select interval-based if the job should be repeated periodically. Note that such a job will by default run indefinitely but you can also specify after how many times it should stop, letting you to express concepts such as "Execute once per hour but for the next seven days".

Cron-style

Select cron-style if you are already familiar with the syntax of Cron or if you have some Cron tasks that you would like to migrate to Zato.

Running jobs manually

At times, it is convenient to run a job on demand, no matter what its schedule is and regardless of what type a particular job is. Web-admin lets you always execute a job directly. Simply find the job in the listing, click "Execute" and it will run immediately.

Extra context

It is very often useful to provide additional context data to a service that the scheduler runs - to achieve it, simply enter any arbitrary value in the "Extra" field when creating or an editing a job in web-admin.

Afterwards, that information will be available as self.request.raw_request in the service's handle method.

Reusability

There is nothing else required - all is done and the service will run in accordance with a job's schedule.

Yet, before concluding, observe that our integration service is completely reusable - there is nothing scheduler-specific in it despite the fact that we currently run it from the scheduler.

We could now invoke the service from command line. Or we could mount it on a REST, AMQP, WebSocket or trigger it from any other channel - exactly the same Python code will run in exactly the same fashion, without any new programming effort needed.

More resources

➤ Python API integration tutorial
➤ What is an integration platform?
➤ Python Integration platform as a Service (iPaaS)
➤ What is an Enterprise Service Bus (ESB)? What is SOA?

More blog posts➤

Python's comments start with an octothorpe character.

Table of contents

1. Writing a comment in Python
2. Inline comments in Python
3. Best practices for commenting in Python
4. Comment as needed, but not too much

Writing a comment in Python

We have a Python program that prints out Hello!, pauses for a second, and then prints Goodbye! on the same line:

from time import sleep print("Hello!", end="", flush=True) sleep(1) # ANSI code to clear current line print("\r\033[K", end="") print("Goodbye!")

It prints Hello!:

~ $ python3 hello.py Hello!

And then one second later it overwrites Hello! with Goodbye!:

~ $ python3 hello.py Goodbye!

It does this using an ANSI escape code (that \033[K string).

The line above the print call in our code is called a comment:

# ANSI code to clear current line print("\r\033[K", end="")

Python's comments all start with the # character.

I call this character an octothorpe, though it goes by many names. Some of the more common names for # are hashmark, number sign, and pound sign.

You can write a comment in Python by putting an octothorpe character (#) at the beginning of a line, and then writing your comment. The comment stops at the end of the line, meaning the next line is code... unless you write another octothorpe character!

Here we've written more details and added an additional line to note that this code doesn't yet work on Windows:

# ANSI code to clear current line: \r moves to beginning, \033[K erases to end. # Note: This will not work on Windows without code to enable ANSI escape codes. print("\r\033[K", end="")

This is sometimes called a block comment because it's a way to write a block of text that represents a comment.

Unlike some programming languages, Python has no multiline comment syntax. If you think you've seen a multiline comment, it may have been a docstring or a multiline string. More on that in multiline comments in Python.

Inline comments in Python

Comments don't need to be …

Read the full article: https://www.pythonmorsels.com/commenting-in-python/

Now that uv is rapidly advancing I have started to dive back into making multi-version imports for Python work. The goal here is to enable multiple resolutions from the solver in uv so that two incompatible versions of a library can be installed and used simultaniously.

Simplified speaking it should be possible for a library to depend on both pydantic 1.x and 2.x simultaniously.

I have not made it work yet, but I have I think found all of the pieces that stand in the way. This post mostly exists to share how it could be done with the least amount of changes to Python.

Basic Operation

Python's import system places modules in a module cache. This cache is exposed via sys.modules. Every module that is imported is placed in that container prior to initialization. The key is the import path of the module. This in some ways presents the first issue.

Note on Terms for Packages, Modules and Distributions

Python's terms for packages are super confusing. Here is what I will use in this article:

• foo.py: this is a python “module”. It gets registered in sys.modules as 'foo' and has an attribute __name__ set to 'foo'.
• foo/__init__.py: declares also a Python “module” named 'foo' but it is simultaniously a “package”. Unlike a normal module it also has two extra attributes: __path__ which is set to ['./foo'] so that sub modules can be found and it has an attribute __package__ which is also set to 'foo' which marks it as package.
• Additionally on PyPI one can register things. These things were called packages at one point and are now mostly called "projects". Within Python however they are not called Projects but “distribution packages”. For instance this is what you see when you try to use the importlib.metadata API. For now I will just call this a “distribution”.

Note that a distribution can ship both modules and multiple at once. You could have a package called whatever and it reports a foo.py file and a bar/baz.py file which in turn would make foo and bar.baz be importable.

Say you have two Python distributions both of which provide the same toplevel package. In that case they are going to clash in sys.modules. As there is actually relationship of the distribution name to the entry in sys.modules this is a problem that does not just exist with multi version imports but it's one that does not happen all that much.

So let's say we have two distributions: foo@1.0.0 and foo@2.0.0. Both expose a toplevel module called foo which is a true Python package with a single __init__.py file. The installer would already fail to place these because one fully overrides the other.

So step 1 would be to place these modules in different places. So where they normally would be in site-packages, in this case we might want to not have these packages there. That solves us the file system clashes.

So we might place them in some extra cache that looks like this:

.venv/ multi-version-packages/ foo@1.0.0/ foo/ __init__.py foo@2.0.0/ foo/ __init__.py

Now that package is entirely non-importable since nothing looks at multi-version-packages. We will need a custom import hook to get them imported. That import hook will also need to change the name of what's stored in sys.modules.

So instead of registering foo as sys.modules['foo'] we might want to try to register it as sys.modules['foo@1.0.0'] and sys.modules['foo@2.0.0'] instead. There is however a catch and that is this very common pattern:

import sys def import_module(name): __import__(name) return sys.modules[name]

That poses a bit of a problem because someone is probably going to call this as import_module('foo') and now we would not find the entry in sys.modules.

This means that in addition to the new entries in sys.modules we would also need to register some proxies that “redirect” us to the real names. These proxies however would need to know if they point to 1.0.0 or 2.0.0.

Metadata

So let's deal with this problem first. How do we know if we need 1.0.0 or 2.0.0? The answer is most likely a package's dependenices. Instead of allowing a package to depend simultaniously on two different versions of the same dependency we can start with a much simpler problem and say that each package can only depend on one version. So that means if I have a myapp package it would have to pick between foo@1.0.0 or foo@2.0.0. However if it were to depended on another package (say slow-package) that one could depend on a different version of foo than myapp:

myapp v0.1.0 ├── foo v2.0.0 └── slow-package v0.1.0 └── foo v1.0.0

In that case when someone tries to import foo we would be consulting the package metadata of the calling package to figure out which version is attempted.

There are two challenges with this today and they come from the history of Python:

1. the import hook does not (always) know which module triggered the import
2. python modules do not know their distribution package

Let's look at these in detail.

Import Context

The goal is that when slow_package/__init__.py imports foo we get foo@1.0.0 version, when myapp/__init__.py improts foo we get the foo@2.0.0 version. What is needed for this to work is that the import system understands not just what is imported, but who is importing. In some sense Python has that. That's because __import__ (which is the entry point to the import machinery) gets the module globals. Here is what an import statement roughly maps to:

# highlevel import from foo import bar # under the hood _rv = __import__('foo', globals(), locals(), ['bar']) bar = _rv.bar

The name of the package that is importing can be retrieved by inspecting the globals(). So in theory for instance the import system could utilize this information. globals()['__name__'] would tell us slow_package vs myapp. There however is a catch and that is that the import name is not the distribution name. The PyPI package could be called mycompany-myapp and it exports a python package just called myapp. This happens very commonly in all kinds of ways. For instance on PyPI one installs Scikit-learn but the python package installed is sklearn.

There is however another problem and that is interpreter internals and C/Rust extensions. We have already established that Python packages will pass globals and locals when they import. But what do C extensions do? The most common internal import API is called PyImport_ImportModule and only takes a module name. Is this a problem? Do C extensions even import stuff? Yes they do. Here is an example from pygame:

MODINIT_DEFINE (color) { PyObject *colordict; colordict = PyImport_ImportModule ("pygame.colordict"); if (colordict) { PyObject *_dict = PyModule_GetDict (colordict); PyObject *colors = PyDict_GetItemString (_dict, "THECOLORS"); /* TODO */ } else { MODINIT_ERROR; } /* snip */ }

And that makes sense. A sufficiently large python package will have inter dependencies between the stuff written in C and Python. It's also complicated by the fact that the C module does initialize a module, but it does not have a natural module scope. The way the C extension initializes the module is with the PyModule_Create API:

static struct PyModuleDef module_def = { PyModuleDef_HEAD_INIT, "foo", /* name of module */ NULL, -1, SpamMethods }; PyMODINIT_FUNC PyInit_foo(void) { return PyModule_Create(&module_def); }

So both the name of the module created as well as the name of what is imported is entirely hardcoded. A C extension does not “know” what the intended name is, it must know this on its own.

In some sense this is already a bit of a disconnect beween the Python and C world. Python for instance has relative imports (from .foo import bar). This is implemented by inspecting the globals. There is however no API to do these relative imports on the C layer.

The only workaround I know right now would be to perform stack walking. That way one would try to isolate the shared library that triggered the import to understand which module it comes from. An alternative would be to carry the current C extension module that is active on the interpreter state, but that would most likely be quite expensive.

The goal would be to find out which .so/.dylib file triggered the import. Stack walking is a rather expensive operation and it can be incredibly brittle but there might not be a perfect way around it. Ideally Python would at any point know which c extension module is active.

Distributions from Modules

So let's say that we have the calling python module figured out: now we need to figure out the associated PyPI distribution name. Unfortunately such a mapping does not exist at all. Ideally when a sys.module entry is created, we either record a special attribute there (say __distribution__) which carries the name of the PyPI distribution name so we can call importlib.metadata.distribution(__distribution__).requires to get the requirements or we have some other API to map it.

In the absence of that, how could we get it? There is an expensive way to get a reverse mapping (importlib.metadata.packages_distributions) but unfortunately it has some limitations:

1. it's very slow
2. it has situations where it does not manage to reveal the distribution for a package
3. it can reveal more than one distribution for a package

Because of namespace packages in particular it can return more than one distribution that provides a package such as foo (eg: foo-bar provides foo.bar and foo-baz provides foo.baz. In that case it will just return both foo-bar and foo-baz for foo).

The solution here might just be that installers like uv start materializing the distribution name onto the modules in one way or another.

Putting it Together

The end to end solution might be this:

1. install multi-version packages outside of site-packages
2. materialize a __distribution__ field onto modules or provide an API that maps import names to their PyPI distribution name so that meta data (requirements) can be discovered.
3. patch __import__ to resolve packages to their fully-qualified, multi version name based on who imports it
   • via globals() for python code
   • via stack-walking for C extensions (unless a better option is found)
4. register proxy entries in sys.modules that have a dynamic __getattr__ which redirects to the fully qualified names if necessary. This would allow someone to access sys.modules['foo'] and automatically proxy it to foo@1.0.0 or foo@2.0.0 respectively.

There are lots of holes with this approach unfortunately. That's in parts because people patch around in sys.modules. Interestingly enough sys.modules can be manipulated but it can't be replaced. This might make it possible to replace that dictionary with some more magical dictionary in future versions of Python potentially.

In this episode we're talking about importing part of a package into another part of the same package.

We'll look at: `from . import module` and `from .module import something`

and also:  `import package` to access the external API from with the package.

Why would we use `import package` if `from . import api` would work fine?

 Learn pytest

• pytest is the number one test framework for Python.
• Learn the basics super fast with Hello, pytest!
• Then later you can become a pytest expert with The Complete pytest Course
• Both courses are at courses.pythontest.com

<p>In this episode we're talking about importing part of a package into another part of the same package.</p><p><br></p><p>We'll look at: `from . import module` and `from .module import something`</p><p>and also:  `import package` to access the external API from with the package.</p><p><br></p><p>Why would we use `import package` if `from . import api` would work fine?</p> <br><p><strong> Learn pytest</strong></p><ul><li>pytest is the number one test framework for Python.</li><li>Learn the basics super fast with <a href="https://courses.pythontest.com/hello-pytest">Hello, pytest!</a></li><li>Then later you can become a pytest expert with <a href="https://courses.pythontest.com/the-complete-pytest-course">The Complete pytest Course</a></li><li>Both courses are at <a href="https://courses.pythontest.com/">courses.pythontest.com</a></li></ul>

Is Python pass by reference or pass by value? This question intrigues every Python programmer coming from a C or Java background. In this tutorial, you will get its answer and learn the meaning of “pass by reference” and “pass by value” along with their difference in Python. Python Pass by Reference vs Pass by […]

The post Pass by Reference vs Pass by Value in Python appeared first on TechBeamers.

Hi there!
A big joint release today. Mostly security fixes but we also have the final release candidate of 3.13 so let’s start with that!

Python 3.13.0RC2

Final opportunity to test and find any show-stopper bugs before we bless and release 3.13.0 final on October 1st.

Get it here: Python Release Python 3.13.0rc2 | Python.org

Call to action

We strongly encourage maintainers of third-party Python projects to prepare their projects for 3.13 compatibilities during this phase, and where necessary publish Python 3.13 wheels on PyPI to be ready for the final release of 3.13.0. Any binary wheels built against Python 3.13.0rc2 will work with future versions of Python 3.13. As always, report any issues to the Python bug tracker.

Please keep in mind that this is a preview release and while it’s as close to the final release as we can get it, its use is not recommended for production environments.

Core developers: time to work on documentation now

• Are all your changes properly documented?
• Are they mentioned in What’s New?
• Did you notice other changes you know of to have insufficient documentation?

As a reminder, until the final release of 3.13.0, the 3.13 branch is set up so that the Release Manager (@thomas) has to merge the changes. Please add him (@Yhg1s on GitHub) to any changes you think should go into 3.13.0. At this point, unless something critical comes up, it should really be documentation only. Other changes (including tests) will be pushed to 3.13.1.

New features in Python 3.13

• A new and improved interactive interpreter, based on PyPy’s, featuring multi-line editing and color support, as well as colorized exception tracebacks.
• An experimental free-threaded build mode, which disables the Global Interpreter Lock, allowing threads to run more concurrently. The build mode is available as an experimental feature in the Windows and macOS installers as well.
• A preliminary, experimental JIT, providing the ground work for significant performance improvements.
• The locals() builtin function (and its C equivalent) now has well-defined semantics when mutating the returned mapping, which allows debuggers to operate more consistently.
• The (cyclic) garbage collector is now incremental, which should mean shorter pauses for collection in programs with a lot of objects.
• A modified version of mimalloc is now included, optional but enabled by default if supported by the platform, and required for the free-threaded build mode.
• Docstrings now have their leading indentation stripped, reducing memory use and the size of .pyc files. (Most tools handling docstrings already strip leading indentation.)
• The dbm module has a new dbm.sqlite3 backend that is used by default when creating new files.
• The minimum supported macOS version was changed from 10.9 to 10.13 (High Sierra). Older macOS versions will not be supported going forward.
• WASI is now a Tier 2 supported platform. Emscripten is no longer an officially supported platform (but Pyodide continues to support Emscripten).
• iOS is now a Tier 3 supported platform, with Android on the way as well.

Python 3.12.6

This is an expedited release for 3.12 due to security content. The schedule returns back to regular programming in October.

One notable change for macOS users: as mentioned in the previous release of 3.12, this release drops support for macOS versions 10.9 through 10.12. Versions of macOS older than 10.13 haven’t been supported by Apple since 2019, and maintaining support for them has become too difficult. (All versions of Python 3.13 have already dropped support for them.)

Get it here: Python Release Python 3.12.6 | Python.org

92 commits.

Python 3.11.10

Python 3.11 joins the elite club of security-only versions with no binary installers.

Get it here: Python Release Python 3.11.10 | Python.org

28 commits.

Python 3.10.15

Get it here: Python Release Python 3.10.15 | Python.org

24 commits.

Python 3.9.20

Get it here: Python Release Python 3.9.20 | Python.org

22 commits.

Python 3.8.20

Python 3.8 is very close to End of Life (see the Release Schedule). Will this be the last release of 3.8 ever? We’ll see… but now I think I jinxed it.

Get it here: Python Release Python 3.8.20 | Python.org

22 commits.

Security content in today’s releases

• gh-123678 and gh-116741: Upgrade bundled libexpat to 2.6.3 to fix CVE-2024-28757, CVE-2024-45490, CVE-2024-45491 and CVE-2024-45492.
• gh-118486: os.mkdir() on Windows now accepts mode of 0o700 to restrict the new directory to the current user. This fixes CVE-2024-4030 affecting tempfile.mkdtemp() in scenarios where the base temporary directory is more permissive than the default.
• gh-123067: Fix quadratic complexity in parsing "-quoted cookie values with backslashes by http.cookies. Fixes CVE-2024-7592.
• gh-113171: Fixed various false positives and false negatives in IPv4Address.is_private, IPv4Address.is_global, IPv6Address.is_private, IPv6Address.is_global. Fixes CVE-2024-4032.
• gh-67693: Fix urllib.parse.urlunparse() and urllib.parse.urlunsplit() for URIs with path starting with multiple slashes and no authority. Fixes CVE-2015-2104.
• gh-121957: Fixed missing audit events around interactive use of Python, now also properly firing for python -i, as well as for python -m asyncio. The event in question is cpython.run_stdin.
• gh-122133: Authenticate the socket connection for the socket.socketpair() fallback on platforms where AF_UNIX is not available like Windows.
• gh-121285: Remove backtracking from tarfile header parsing for hdrcharset, PAX, and GNU sparse headers. That’s CVE-2024-6232.
• gh-114572: ssl.SSLContext.cert_store_stats() and ssl.SSLContext.get_ca_certs() now correctly lock access to the certificate store, when the ssl.SSLContext is shared across multiple threads.
• gh-102988: email.utils.getaddresses() and email.utils.parseaddr() now return ('', '') 2-tuples in more situations where invalid email addresses are encountered instead of potentially inaccurate values. Add optional strict parameter to these two functions: use strict=False to get the old behavior, accept malformed inputs. getattr(email.utils, 'supports_strict_parsing', False) can be use to check if the strict paramater is available. This improves the CVE-2023-27043 fix.
• gh-123270: Sanitize names in zipfile.Path to avoid infinite loops (gh-122905) without breaking contents using legitimate characters. That’s CVE-2024-8088.
• gh-121650: email headers with embedded newlines are now quoted on output. The generator will now refuse to serialize (write) headers that are unsafely folded or delimited; see verify_generated_headers. That’s CVE-2024-6923.
• gh-119690: Fixes data type confusion in audit events raised by _winapi.CreateFile and _winapi.CreateNamedPipe.
• gh-116773: Fix instances of <_overlapped.Overlapped object at 0xXXX> still has pending operation at deallocation, the process may crash.
• gh-112275: A deadlock involving pystate.c’s HEAD_LOCK in posixmodule.c at fork is now fixed.

Stay safe and upgrade!

Upgrading is highly recommended to all users of affected versions.

Thank you for your support

Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation.

–
Łukasz Langa @ambv
on behalf of your friendly release team,

Ned Deily @nad
Steve Dower @steve.dower
Pablo Galindo Salgado @pablogsal
Łukasz Langa @ambv
Thomas Wouters @thomas

 

 

Python programs are created in code editors, and can be run using system command prompt.

Table of contents

1. The Python REPL
2. Definitions: program, script, command-line, editor
3. Creating a Python program in a code editor
4. Running our Python program
5. Printing to see program output
6. Python programs are not launched from the REPL
7. Different operating systems
8. Python scripts are run from the command prompt or terminal

The Python REPL

This is the Python REPL, also known as the Interactive Python Interpreter:

>>>

How can I tell it's the Python REPL?

Well, the prompt shows three greater than signs (>>>), which indicates that we're in the Python REPL.

Typing all of your code in the Python REPL is pretty limiting.

Our code disappears right after we've typed it. Also, if we'd like someone to be able to import our code as a module, they can't.

Definitions: program, script, command-line, editor

Let's make a Python program.

Read the full article: https://www.pythonmorsels.com/creating-python-programs/

We are thrilled to announce that Djangonaut Space, a mentorship program, is open for applicants for our next cohort!

Djangonaut Space is holding a third session this year! This session will start on October 14th, 2024. We are accepting applications until September 14th, 2024. More details can be found in the website.

Djangonaut Space is a free, 8-week group mentoring program where individuals will work self-paced in a semi-structured learning environment. It seeks to help members of the community who wish to level up their current Django code contributions and potentially take on leadership roles in Django in the future.

“I signed up for this program with the goal of starting my journey as a contributor, but I ended up gaining so much more. In this community, I found incredible people who not only guide you toward solutions but also encourage and celebrate every achievement along the way.” - Raffaella, Djangonaut

If you have questions, they are holding an AMA session on Zoom next week. See their social media account for more details:

• Twitter/X
• Mastodon

We’re excited to announce the September 2024 release of the Python and Jupyter extensions for Visual Studio Code!

This release includes the following announcements:

• Django unit test support
• Go to definition from inlay hints with Pylance

If you’re interested, you can check the full list of improvements in our changelogs for the Python, Jupyter and Pylance extensions.

Django unit test support

We are excited to announce support for one of our most requested features: you can now discover and run Django unit tests through the Test Explorer!

In order to enable this feature, you will need to add a MANAGE_PY_PATH environment variable, pointing to your Django application’s manage.py file. To do so, you can follow these steps:

1. Set "python.testing.unittestEnabled": true, in your settings.json file.
2. Add MANAGE_PY_PATH as an environment variable:
   1. Create a .env file at the root of your project.
   2. Add MANAGE_PY_PATH='<path-to-manage.py>' to the .env file, replacing <path-to-manage.py> with the path to your application’s manage.py file.

      Tip: You can copy the path by right clicking on the file in the Explorer view and selecting Copy Path.

3. Add Django test arguments to "python.testing.unittestArgs": [] in the settings.json file as needed, and remove any arguments that are not compatible with Django.

Note: By default, the Python extension looks for and loads .env files at the project root. If your .env file is not at the project root or you are using VS Code variable substitution, add "python.envFile": "${workspaceFolder}/<path-to-.env>" to your settings.json file, so the Python extension can load the environment variables in this file when running and discovering tests. See our Python environment variables docs for more information on environment variables.

Navigate to the Testing view, and select the Refresh Tests button to have your Django tests displayed!

For troubleshooting tips, please see our Django testing docs. As you explore this newly added feature, please provide feedback and report any issues in our vscode-python repo or by using the Python: Report Issue command.

Go to definition from inlay hints with Pylance

When enabling inlay hints with Pylance, you can now more conveniently navigate to a type’s definition through Ctrl+Click or Cmd+Click when hovering over it.

Other Changes and Enhancements

We have also added small enhancements and fixed issues requested by users that should improve your experience working with Python and Jupyter Notebooks in Visual Studio Code. Some notable changes include:

• You can now access the VS Code Native REPL for Python from the Command Palette (Ctrl/Cmd + Shift + P) using Python: Start Native REPL(@vscode-python#23727)
• VS Code Native REPL for Python now starts at the project folder (@vscode-python#23821)
• Strings are now normalized when sending commands to the VS Code Native REPL (@vscode-python#23743)
• You can now restart the debugger when debugging tests through the debug control widget (@vscode-python#23752)

Call for Community Feedback

As we are planning and prioritizing future work, we value your feedback! Below are a few issues we would love feedback on:

• Design proposal for test coverage in (@vscode-python#22827)

Try out these new improvements by downloading the Python extension and the Jupyter extension from the Marketplace, or install them directly from the extensions view in Visual Studio Code (Ctrl + Shift + X or ⌘ + ⇧ + X). You can learn more about Python support in Visual Studio Code in the documentation. If you run into any problems or have suggestions, please file an issue on the Python VS Code GitHub page.

The post Python in Visual Studio Code – September 2024 Release appeared first on Python.

Are you interested in practicing your Python skills while learning how to solve astrophysics and astronomy problems? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

In this episode, we continued working with the Kamal deploy tool. Last time, we build a demo image. With this session, we created a Digital Ocean Droplet (i.e., VM) and deployed to that. Along the journey, we learned a lot about how Kamal operates.

Last year the Python Software Foundation was announced as a CVE Numbering Authority (CNA) to manage and assign CVE IDs for CPython and pip. Becoming a CVE Numbering Authority allows the PSF to provide expertise about Python in the CVE ecosystem, ensuring that users have accurate and up-to-date information about vulnerabilities affecting key projects.
Today, the PSF is expanding our CNA scope to also include Pallets projects, such as Flask, Jinja, Click, and Quart. For a complete list, see the Pallets organization on GitHub. Please report any security vulnerabilities for these projects following the Pallets security policy.
 This work is being done to learn how the PSF can better serve Python's large ecosystem of projects in the context of the CVE ecosystem. The PSF previously published a guide on how open source projects can become their own CVE Numbering Authorities. You can learn more about the CVE CNA program on the CVE website.

Pallets is a fiscal sponsoree of the Python Software Foundation. Fiscal sponsorship is a key plank of the PSF’s mission in supporting the Python community. The PSF supports 20 fiscal sponsorees including regional PyCons, Python Meetup and User Groups, and Python projects. Learn more about our Fiscal Sponsorees on our website and consider supporting the groups with a US-tax deductible donation.

Translation tools have existed for many years and are incredibly useful if you're learning a new language or wanting to read foreign websites. One of the most popular tools is Google Translate , but there is now another alternative: using OpenAI's ChatGPT tool to translate text.

In this tutorial, we'll build a desktop translator application to translate natural language using ChatGPT APIs. We'll be building the UI using the Tkinter library from the Python standard library:

Example translation of text via OpenAI

Table of Contents

• Installing the Required Packages
• Building the Window
• Creating the GUI for the Translator App
• Getting a List of Languages
  • Adding the Interface
• Building the Input UI
• Getting an OpenAI API Key
• Implementing the Translation Functionality
• Conclusion

Installing the Required Packages

Our Translator uses the openai library to perform the actual translation via OpenAI's ChatGPT tool. Tkinter is already available in the standard library.

The first task will be to set up a Python virtual environment. Open the terminal, and run the following commands:

• Windows
• macOS/Linux

bat > mkdir translator > cd translator > python -m venv venv > .\venv\Scripts\activate > python -m pip install openai sh $ mkdir translator $ cd translator/ $ python -m venv venv $ source venv/bin/activate (venv) $ python -m pip install openai

Working through these instructions, first we create a root directory for the Translator app. Next we create and activate a Python virtual environment for the project. Finally, we install the openai package.

Next, create a file named translator.py in the root of your project. Also add a folder called images/ where you'll store the icons for the application. The folder structure should look like this:

python translator/ &boxv &boxvr&boxh&boxh images/ &boxv &boxvr&boxh&boxh arrow.png &boxv &boxur&boxh&boxh logo.png &boxv &boxur&boxh&boxh translator.py

The images for this project can be downloaded here.

The images/ folder contains the two icons that you'll use for the application. The translator.py is the app's source file.

Building the Window

Open the translator.py file with your favorite Python code editor. We'll start by creating our main window:

python import tkinter as tk class TranslatorApp(tk.Tk): def __init__(self): super().__init__() self.title("Language Translator") self.resizable(width=False, height=False) if __name__ == "__main__": app = TranslatorApp() app.mainloop()

This code imports Tkinter and then defines the application's main class, which we have called TranslatorApp. This class will hold the application's main window and allow us to run the main loop.

Importing tkinter under the alias tk is a common convention in Tkinter code.

Inside the class we define the __init__() method, which handles initialization of the class. In this method, we first call the initializer __init__() of the parent class, tk.Tk, to initialize the app's window. Then, we set the window's title using the title() method. To make the window unresizable, we use the resizable() method with width and height set to False.

At the bottom of the code, we have the if __name__ == "__main__" idiom to check whether the file is being run directly as an executable program. Inside the condition block we first create an instance of TranslatorApp and then run the application's main loop or event loop.

If you run this code, you'll get an empty Tkinter window on your desktop:

python $ python translator.py

The empty Tkinter window

Creating the GUI for the Translator App

Now that the main window is set up, let's start adding widgets to build the GUI. To do this, we'll create a method called setup_ui(), as shown below:

python import tkinter as tk class TranslatorApp(tk.Tk): def __init__(self): super().__init__() self.title("Language Translator") self.resizable(width=False, height=False) self.setup_ui() def setup_ui(self): frame = tk.Frame(self) frame.pack(padx=10, pady=10) if __name__ == "__main__": app = TranslatorApp() app.mainloop()

The setup_ui() method will define the application's GUI. In this method, we first create a frame widget using the tk.Frame class whose master argument is set to self (the application's main window). Next, we position the frame inside the main window using the pack() geometry manager, using padx and pady arguments to set some padding around the frame.

Finally, we add the call to self.setup_ui() to the __init__() method.

We'll continue to develop the UI by adding code to the setup_ui() method.

Net, we'll add the app's logo. In the setup_ui() method add the following code below the frame definition:

python import tkinter as tk class TranslatorApp(tk.Tk): def __init__(self): super().__init__() self.title("Language Translator") self.resizable(width=False, height=False) self.setup_ui() def setup_ui(self): frame = tk.Frame(self) frame.pack(padx=10, pady=10) self.logo = tk.PhotoImage(file="images/logo.png") tk.Label(frame, image=self.logo).grid(row=0, column=0, sticky="w") if __name__ == "__main__": app = TranslatorApp() app.mainloop()

This code loads the logo using the tk.PhotoImage class. To resize it, we use the subsample() method. Then, we add the logo to the frame using a tk.Label widget. The label takes the frame and the logo as arguments. Finally, to position the logo, we use the grid() geometry manager with appropriate values for the row, column, and sticky arguments.

The sticky argument determines which side of a cell the widget should align -- North (top), South (bottom), East (right) or West (left). Here we're aligning it on the Wiest or left of the cell with "w":

Tkinter window with the OpenAI logo in it

Getting a List of Languages

We need list of languages to shown in the dropdown. There are various lists available online. But since we're using OpenAI for the translations, why not use it to give us the list of languages too. Since this is just for testing purposes, lets grab the top 20 human languages (by first and second language speakers).

We can prompt ChatGPT with something like:

Give me a list of the top 20 human languages with the most first and second language speakers in Python list format

..and it will return the following list:

python languages = [ "English", "Mandarin Chinese", "Hindi", "Spanish", "French", "Standard Arabic", "Bengali", "Russian", "Portuguese", "Urdu" ]

I'm going to add Dutch to the list, because it's my second language. Feel free to add your own languages to the list.

Adding the Interface

Let's start adding some inputs to the UIs. First we'll create the language selection drop down boxes:

python import tkinter as tk import tkinter.ttk as ttk LANGUAGES = [ "English", "Mandarin Chinese", "Hindi", "Spanish", "French", "Standard Arabic", "Bengali", "Russian", "Portuguese", "Urdu", "Dutch", # Gekoloniseerd. ] DEFAULT_SOURCE = "English" DEFAULT_DEST = "Dutch" class TranslatorApp(tk.Tk): def __init__(self): super().__init__() self.title("Language Translator") self.resizable(width=False, height=False) self.setup_ui() def setup_ui(self): frame = tk.Frame(self) frame.pack(padx=10, pady=10) self.logo = tk.PhotoImage(file="images/logo.png") tk.Label(frame, image=self.logo).grid(row=0, column=0, sticky="w") # Source language combobox self.from_language = ttk.Combobox(frame, values=LANGUAGES) self.from_language.current(LANGUAGES.index(DEFAULT_SOURCE)) self.from_language.grid(row=1, column=0, sticky="we") # Arrow icon self.arrows = tk.PhotoImage(file="images/arrow.png").subsample(15, 15) tk.Label(frame, image=self.arrows).grid(row=1, column=1) # Destination language combobox self.to_language = ttk.Combobox(frame, values=LANGUAGES) self.to_language.current(LANGUAGES.index(DEFAULT_DEST)) self.to_language.grid(row=1, column=2, sticky="we") if __name__ == "__main__": app = TranslatorApp() app.mainloop()

We have added our language list as the constant LANGUAGES. We also define the default languages for when the application starts up, using constants DEFAULT_SOURCE and DEFAULT_DEST.

Next, we create two combo boxes to hold the list of source and destination languages. The combo boxes are created using the ttk.Combobox class. One to the left and another to the right. Between the combo boxes, we've also added an arrow icon loaded using the tk.PhotoImage class. Again, we've added the icon to the app's window using ttk.Label.

Both combo boxes take frame and values as arguments. The values argument populates the combo boxes with languages. To specify the default language, we use the current() method, looking up the position of our default languages in the languages list with .index().

To position the combo boxes inside the frame, we use the grid() geometry manager with the appropriate arguments. Run the application, and you will see the following window:

Source and destination languages

With the source and destination combo boxes in place, let's add three more widgets: two scrollable text widgets and a button. The scrollable text on the left will hold the source text, while the scrollable text on the right will hold the translated text. The button will allow us to run the actual translation.

Building the Input UI

Get back to the code editor and update the setup_ui() method as follows. Note that we also need to import the ScrollText class:

python import tkinter as tk import tkinter.ttk as ttk from tkinter.scrolledtext import ScrolledText LANGUAGES = [ "English", "Mandarin Chinese", "Hindi", "Spanish", "French", "Standard Arabic", "Bengali", "Russian", "Portuguese", "Urdu", "Dutch", ] DEFAULT_SOURCE = "English" DEFAULT_DEST = "Dutch" class TranslatorApp(tk.Tk): def __init__(self): super().__init__() self.title("Language Translator") self.resizable(width=False, height=False) self.setup_ui() def setup_ui(self): frame = tk.Frame(self) frame.pack(padx=10, pady=10) self.logo = tk.PhotoImage(file="images/logo.png").subsample(5, 5) tk.Label(frame, image=self.logo).grid(row=0, column=0, sticky="w") # Source language combobox languages = [lang.title() for lang in LANGUAGES.values()] self.from_language = ttk.Combobox(frame, values=languages) self.from_language.current(languages.index(DEFAULT_SOURCE)) self.from_language.grid(row=1, column=0, sticky="we") # Arrow icon self.arrows = tk.PhotoImage(file="images/arrow.png").subsample(15, 15) tk.Label(frame, image=self.arrows).grid(row=1, column=1) # Destination language combobox self.to_language = ttk.Combobox(frame, values=languages) self.to_language.current(languages.index(DEFAULT_DEST)) self.to_language.grid(row=1, column=2, sticky="we") # Source text self.from_text = ScrolledText( frame, font=("Dotum", 16), width=50, height=20, ) self.from_text.grid(row=2, column=0) # Translated text self.to_text = ScrolledText( frame, font=("Dotum", 16), width=50, height=20, state="disabled", ) self.to_text.grid(row=2, column=2) # Translate button self.translate_button = ttk.Button( frame, text="Translate", command=self.translate, ) self.translate_button.grid(row=3, column=0, columnspan=3, pady=10) def translate(self): pass if __name__ == "__main__": app = TranslatorApp() app.mainloop()

In the code snippet, we use the ScrolledText class to create the two scrolled text areas. Both text areas take frame, font, width, and height as arguments. The second text area also takes state as an additional argument. Setting state to "disabled" allows us to create a read-only text area.

Then, we use the ttk.Button class to create a button with frame, text, and command as arguments. The command argument allows us to bind the button's click event to the self.translate() method, which we will define in a moment. For now, we've added a placeholder.

To position all these widgets on the app's window, we use the grid() geometry manager. Now, the app will look something like the following:

Translator app's GUI

Our translation app's GUI is ready! Finally, we can start adding functionality to the application.

Getting an OpenAI API Key

You can use OpenAPI's APIs for free, with some limitations. To get an OpenAI API key you will need to create an account. Once you have created an account go ahead and get an API key.

Click "Create new secret key" in the top right hand corner to create a key. Give the key a name (it doesn't matter what you use) and then click "Create secret key". Copy the resulting key and keep it safe. You'll need it in the next step.

Implementing the Translation Functionality

We'll implement the language translation functionality in the translate() method. This gets the current text from the UI and then uses openai to perform the translation. We need a few more imports, and to create the OpenAI client instance at the top of the application:

python import tkinter as tk import tkinter.ttk as ttk from tkinter.messagebox import showerror from tkinter.scrolledtext import ScrolledText import httpcore from openai import OpenAI client = OpenAI( api_key="<YOUR API KEY HERE>" )

Here we've imported the showerror helper for displaying error boxes in our application. We've imported httpcore which we'll use to handle HTTP errors when accessing the API. Finally, we've added an import for the OpenAI class from openai. This is what handles the actual translation.

To use it, we create an instance of the class as OpenAI. Replace <YOUR API KEY HERE> with the API key you generated on OpenAI just now.

We'll continue by implementing the translate() method. Below we're just showing the function itself:

python class TranslatorApp(tk.Tk): # ... def translate(self): source_language = self.from_language.get() destination_language = self.to_language.get() text = self.from_text.get(1.0, tk.END).strip() try: completion = client.chat.completions.create( messages=[ {"role": "system", "content": "You are a language interpreter."}, { "role": "user", "content": ( f"Translate the following text from {source_language} " f"to {destination_language}, only reply with the text: " f"{text}" ), }, ], model="gpt-3.5-turbo", ) reply = completion.choices[0].message.content except httpcore.ConnectError: showerror( title="Error", message="Make sure you have an internet connection", ) return except Exception as e: showerror( title="Error", message=f"An unexpected error occurred: {e}", ) return self.to_text.config(state="normal") self.to_text.delete(1.0, tk.END) self.to_text.insert(tk.END, reply) self.to_text.config(state="disabled")

The translate() method handles the entire translation process. It starts by retrieving the source and destination languages from the corresponding combo boxes, and the input text from the box on the left.

If any of these are not defined, we use a showerror dialog to inform the user of the problem.

Once we have the source and destination language and some text to translate, we can perform the actual translation through ChatGPT. First, we give the language model a hint about what we want it to do -- interpret language:

python {"role": "system", "content": "You are a language interpreter."},

Next we build the message we want it to respond to. We ask it to translate the provided text from the source to destination language, and to respond with only the translated text. If we don't specify this, we'll get some additional description or context.

You might want to experiment with asking for the text and context separately, as that is often helpful when learning languages.

python { "role": "user", "content": ( f"Translate the following text from {source_language} " f"to {destination_language}, only reply with the text: " f"{text}" ), },

The created completion is submitted to the API and we can retrieve the resulting text from the object:

python reply = completion.choices[0].message.content

If the call to translate() finds a connection error, then we tell the user to check their internet connection. To handle any other exceptions, we catch the generic Exception class and display an error message with the exception details.

If the translation is successful, then we enable the destination scrolled area, display the translated text, and disable the area again so it remains read-only.

The complete final code is shown below:

python import tkinter as tk import tkinter.ttk as ttk from tkinter.messagebox import showerror from tkinter.scrolledtext import ScrolledText import httpcore from openai import OpenAI client = OpenAI( api_key="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" ) LANGUAGES = [ "English", "Mandarin Chinese", "Hindi", "Spanish", "French", "Standard Arabic", "Bengali", "Russian", "Portuguese", "Urdu", "Dutch", ] DEFAULT_SOURCE = "English" DEFAULT_DEST = "Dutch" class TranslatorApp(tk.Tk): def __init__(self): super().__init__() self.title("Language Translator") self.resizable(width=False, height=False) self.setup_ui() def setup_ui(self): frame = tk.Frame(self) frame.pack(padx=10, pady=10) self.logo = tk.PhotoImage(file="images/logo.png") tk.Label(frame, image=self.logo).grid(row=0, column=0, sticky="w") # Source language combobox self.from_language = ttk.Combobox(frame, values=LANGUAGES) self.from_language.current(LANGUAGES.index(DEFAULT_SOURCE)) self.from_language.grid(row=1, column=0, sticky="we") # Arrow icon self.arrows = tk.PhotoImage(file="images/arrow.png").subsample(15, 15) tk.Label(frame, image=self.arrows).grid(row=1, column=1) # Destination language combobox self.to_language = ttk.Combobox(frame, values=LANGUAGES) self.to_language.current(LANGUAGES.index(DEFAULT_DEST)) self.to_language.grid(row=1, column=2, sticky="we") # Source text self.from_text = ScrolledText( frame, font=("Dotum", 16), width=50, height=20, ) self.from_text.grid(row=2, column=0) # Translated text self.to_text = ScrolledText( frame, font=("Dotum", 16), width=50, height=20, state="disabled", ) self.to_text.grid(row=2, column=2) # Translate button self.translate_button = ttk.Button( frame, text="Translate", command=self.translate, ) self.translate_button.grid(row=3, column=0, columnspan=3, pady=10) def translate(self): source_language = self.from_language.get() destination_language = self.to_language.get() text = self.from_text.get(1.0, tk.END).strip() try: completion = client.chat.completions.create( messages=[ {"role": "system", "content": "You are a language interpreter."}, { "role": "user", "content": ( f"Translate the following text from {source_language} " f"to {destination_language}, only reply with the text: " f"{text}" ), }, ], model="gpt-3.5-turbo", ) reply = completion.choices[0].message.content except httpcore.ConnectError: showerror( title="Error", message="Make sure you have an internet connection", ) return except Exception as e: showerror( title="Error", message=f"An unexpected error occurred: {e}", ) return self.to_text.config(state="normal") self.to_text.delete(1.0, tk.END) self.to_text.insert(tk.END, reply) self.to_text.config(state="disabled") if __name__ == "__main__": app = TranslatorApp() app.mainloop()

The finished app is shown below:

The completed Translator app

Conclusion

In this tutorial we built a Translator application using the Tkinter GUI library from the Python standard library. We worked step by step through building the UI using a grid layout, and then implemented the language translation functionality with openai & ChatGPT.

Try and take what you've learnt in this tutorial & applying it to your own projects!

I am happy to announce two upcoming public events focused on helping the scientific Python ecosystem develop their accessibility skills before the new year.

Pre-commit hooks are a great way to help maintain code quality. However, some of your code quality standards may be specific to your project, and therefore, not covered by existing code linting and formatting tools. In this article, I will show you how to incorporate custom checks into your `pre-commit` setup.

In Python, lists and tuples are versatile and useful data types that allow you to store data in a sequence. You’ll find them in virtually every nontrivial Python program. Learning about them is a core skill for you as a Python developer.

In this tutorial, you’ll:

• Get to know lists and tuples
• Explore the core characteristics of lists and tuples
• Learn how to define and manipulate lists and tuples
• Decide when to use lists or tuples in your code

To get the most out of this tutorial, you should know the basics of Python programming, including how to define variables.

Get Your Code: Click here to download the free sample code that shows you how to work with lists and tuples in Python.

Take the Quiz: Test your knowledge with our interactive “Lists vs Tuples in Python” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

Lists vs Tuples in Python

Challenge yourself with this quiz to evaluate and deepen your understanding of Python lists and tuples. You'll explore key concepts, such as how to create, access, and manipulate these data types, while also learning best practices for using them efficiently in your code.

Getting Started With Python Lists and Tuples

In Python, a list is a collection of arbitrary objects, somewhat akin to an array in many other programming languages but more flexible. To define a list, you typically enclose a comma-separated sequence of objects in square brackets ([]), as shown below:

Python >>> colors = ["red", "green", "blue", "yellow"] >>> colors ['red', 'green', 'blue', 'yellow'] Copied!

In this code snippet, you define a list of colors using string objects separated by commas and enclose them in square brackets.

Similarly, tuples are also collections of arbitrary objects. To define a tuple, you’ll enclose a comma-separated sequence of objects in parentheses (()), as shown below:

Python >>> person = ("Jane Doe", 25, "Python Developer", "Canada") >>> person ('Jane Doe', 25, 'Python Developer', 'Canada') Copied!

In this example, you define a tuple with data for a given person, including their name, age, job, and base country.

Up to this point, it may seem that lists and tuples are mostly the same. However, there’s an important difference:

Feature List Tuple Is an ordered sequence ✅ ✅ Can contain arbitrary objects ✅ ✅ Can be indexed and sliced ✅ ✅ Can be nested ✅ ✅ Is mutable ✅ ❌

Both lists and tuples are sequence data types, which means they can contain objects arranged in order. You can access those objects using an integer index that represents their position in the sequence.

Even though both data types can contain arbitrary and heterogeneous objects, you’ll commonly use lists to store homogeneous objects and tuples to store heterogeneous objects.

Note: In this tutorial, you’ll see the terms homogeneous and heterogeneous used to express the following ideas:

• Homogeneous: Objects of the same data type or the same semantic meaning, like a series of animals, fruits, colors, and so on.
• Heterogeneous: Objects of different data types or different semantic meanings, like the attributes of a car: model, color, make, year, fuel type, and so on.

You can perform indexing and slicing operations on both lists and tuples. You can also have nested lists and nested tuples or a combination of them, like a list of tuples.

The most notable difference between lists and tuples is that lists are mutable, while tuples are immutable. This feature distinguishes them and drives their specific use cases.

Essentially, a list doesn’t have a fixed length since it’s immutable. Therefore, it’s natural to use homogeneous elements to have some structure in the list. A tuple, on the other hand, has a fixed length so the position of elements can have meaning, supporting heterogeneous data.

Creating Lists in Python

In many situations, you’ll define a list object using a literal. A list literal is a comma-separated sequence of objects enclosed in square brackets:

Python >>> countries = ["United States", "Canada", "Poland", "Germany", "Austria"] >>> countries ['United States', 'Canada', 'Poland', 'Germany', 'Austria'] Copied!

In this example, you create a list of countries represented by string objects. Because lists are ordered sequences, the values retain the insertion order.

Read the full article at https://realpython.com/python-lists-tuples/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]