Planet Python

When choosing a tool for profiling Python code performance, should it focus on the CPU, GPU, memory, or individual lines of code? What if it looked at all those factors and didn't alter code performance while measuring it? This week on the show, we talk about Scalene with Emery Berger, Professor of Computer Science at the University of Massachusetts Amherst.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

This week we talk about code smells.

Listen here:

Also available on our YouTube channel:

While there, like and subscribe to get similar content regularly …

Code smells are characteristics in the code that might indicate deeper issues or potential problems. While they’re not necessarily bugs, they can be a sign of poor code quality or maintainability issues.

We distilled 15 common smells ranging from generic programming to Python specific issues. We hope it will make your more conscious of your code as well as code you’ll review.

If you have any feedback, hit us up on:
– LinkedIn
– X
– Email
(Also for any podcast topic requests …)

Mentioned Dictionary Dispatch Pattern video

And to write cleaner, more maintainable code, in the context of (complex) real world applications, check out our 1:1 coaching options.

Chapters:
00:00 Intro music
00:20 What are code smells?
01:11 1. Long functions or classes
01:46 2. Duplicated code
02:25 3. Data Clumps
03:13 4. Using the global space
03:52 5. Magic numbers
04:38 6. Primitive obsession
05:06 7. Overusing comments
06:23 8. Too deep nesting
07:36 9. Switch statement or long if-elif-elif-else chains
08:41 10. Too deep inheritance
09:45 11. Dead code
10:21 12. Misusing (nested) listcomps
11:03 13. Single letter variable names
12:03 14. Mutable Default Arguments
13:05 15. Error Silencing
14:04 Wrap up
14:56 Outro music

Thanks for tuning in as always and next week we’ll be back with a brand new episode …

Introduction

Let's say you have a list of home addresses and want to see which ones reside on a "Street", "Ave", "Lane", etc. Given the variability of physical addresses, you'd probably want to use a regular expression to do the matching. But how do you apply a regex to a list? That's exactly what we'll be looking at in this Byte.

Why Match Lists with Regular Expressions?

Regular expressions are one of best, if not the best, ways to do pattern matching on strings. In short, they can be used to check if a string contains a specific pattern, replace parts of a string, and even split a string based on a pattern.

Another reason you may want to use a regex on a list of strings: you have a list of email addresses and you want to filter out all the invalid ones. You can use a regular expression to define the pattern of a valid email address and apply it to the entire list in one go. There are an endless number of examples like this as to why you'd want to use a regex over a list of strings.

Python's Regex Module

Python's re module provides built-in support for regular expressions. You can import it as follows:

import re

The re module has several functions to work with regular expressions, such as match(), search(), and findall(). We'll be using these functions to check if any element in a list matches a regular expression.

Link: For more information on using regex in Python, check out our article, Introduction to Regular Expressions in Python

Using the match() Function

To check if any element in a list matches a regular expression, you can use a loop to iterate over the list and the re module's match() function to check each element. Here's an example:

import re # List of strings list_of_strings = ['apple', 'banana', 'cherry', 'date'] # Regular expression pattern for strings starting with 'a' pattern = '^a' for string in list_of_strings: if re.match(pattern, string): print(string, "matches the pattern")

In this example, the match() function checks if each string in the list starts with the letter 'a'. The output will be:

apple matches the pattern

Note: The ^ character in the regular expression pattern indicates the start of the string. So, ^a matches any string that starts with 'a'.

This is a basic example, but you can use more complex regular expression patterns to match more specific conditions. For example, here is a regex for matching an email address:

([A-Za-z0-9]+[.-_])*[A-Za-z0-9]+@[A-Za-z0-9-]+(\.[A-Z|a-z]{2,})+ Using the search() Function

While re.match() is great for checking the start of a string, re.search() scans through the string and returns a MatchObject if it finds a match anywhere in the string. Let's tweak our previous example to find any string that contains "Hello".

import re my_list = ['Hello World', 'Python Hello', 'Goodbye World', 'Say Hello'] pattern = "Hello" for element in my_list: if re.search(pattern, element): print(f"'{element}' matches the pattern.")

The output will be:

'Hello World' matches the pattern. 'Python Hello' matches the pattern. 'Say Hello' matches the pattern.

As you can see, re.search() found the strings that contain "Hello" anywhere, not just at the start.

Using the findall() Function

The re.findall() function returns all non-overlapping matches of pattern in string, as a list of strings. This can be useful when you want to extract all occurrences of a pattern from a string. Let's use this function to find all occurrences of "Hello" in our list.

import re my_list = ['Hello Hello', 'Python Hello', 'Goodbye World', 'Say Hello Hello'] pattern = "Hello" for element in my_list: matches = re.findall(pattern, element) if matches: print(f"'{element}' contains {len(matches)} occurrence(s) of 'Hello'.")

The output will be:

'Hello Hello' contains 2 occurrence(s) of 'Hello'. 'Python Hello' contains 1 occurrence(s) of 'Hello'. 'Say Hello Hello' contains 2 occurrence(s) of 'Hello'. Working with Nested Lists

What happens if our list contains other lists? Python's re module functions won't work directly on nested lists, just like it wouldn't work with the root list in the previous examples. We need to flatten the list or iterate through each sub-list.

Let's consider a list of lists, where each sub-list contains strings. We want to find out which strings contain "Hello".

import re my_list = [['Hello World', 'Python Hello'], ['Goodbye World'], ['Say Hello']] pattern = "Hello" for sub_list in my_list: for element in sub_list: if re.search(pattern, element): print(f"'{element}' matches the pattern.")

The output will be:

'Hello World' matches the pattern. 'Python Hello' matches the pattern. 'Say Hello' matches the pattern.

We first loop through each sub-list in the main list. Then for each sub-list, we loop through its elements and apply re.search() to find the matching strings.

Working with Mixed Data Type Lists

Python lists are versatile and can hold a variety of data types. This means you can have a list with integers, strings, and even other lists. This is great for a lot of reasons, but it also means you have to deal with potential issues when the data types matter for your operation. When working with regular expressions, we only deal with strings. So, what happens when we have a list with mixed data types?

import re mixed_list = [1, 'apple', 3.14, 'banana', '123', 'abc123', '123abc'] regex = r'\d+' # matches any sequence of digits for element in mixed_list: if isinstance(element, str) and re.match(regex, element): print(f"{element} matches the regex") else: print(f"{element} does not match the regex or is not a string")

In this case, the output will be:

1 does not match the regex or is not a string apple does not match the regex or is not a string 3.14 does not match the regex or is not a string banana does not match the regex or is not a string 123 matches the regex abc123 does not match the regex or is not a string 123abc matches the regex

We first check if the current element is a string. Only then do we check if it matches the regular expression. This is because the re.match() function expects a string as input. If you try to use it on an integer or a float, Python will throw an error.

Conclusion

Python's re module provides several functions to match regex patterns in strings. In this Byte, we learned how to use these functions to check if any element in a list matches a regular expression. We also saw how to handle lists with mixed data types. Regular expressions can be complex, so take your time to understand them. With a bit of practice, you'll find that they can be used to solve many problems when working with strings.

Introduction

Working with any language, you've probably come across warnings - and lots of them. In Python, our warnings are the yellow-highlighted messages that appear when code runs. These warnings are Python's way of telling us that, while our code is technically correct and will run, there's something in it that's not quite right or could eventually lead to issues. Sometimes these warnings are helpful, and sometimes they're not. So what if we want to disable them?

In this Byte, we'll show a bit more about what Python warnings are, why you might want to disable them, and how you can do it.

Python Warnings

Python warnings are messages that the Python interpreter throws when it encounters unusual code that may not necessarily result in an error, but is probably not what you intended. These warnings can include things like deprecation warnings, which tell you when a Python feature is being phased out, or syntax warnings, which alert you to weird but syntactically correct code.

Here's an example of a warning you might see:

import warnings def fxn(): warnings.warn("fxn() is deprecated", DeprecationWarning, stacklevel=2) warnings.simplefilter('always', DeprecationWarning) fxn()

When you run this code, Python will output a DeprecationWarning:

$ python warnings.py warnings.py:9: DeprecationWarning: fxn() is deprecated fxn()

Note: We had to add the warnings.simplefilter('always', DeprecationWarning) line in order to get the warning to show. Otherwise DeprecationWarnings are ignored by default.

Why disable them?

That's a good question. Warnings are indeed useful, but there are times when you might want to disable them.

For example, if you're working with a large codebase and you're aware of the warnings but have decided to ignore them for now, having them constantly pop up can be not only annoying but also cause you to miss more important output from your code. In the same vein, if you're running a script that's outputting to a log file, you might not want warnings cluttering up your logs.

How to Disable Python Warnings

There are a few ways to disable warnings in Python, and we'll look at three of them: using the warnings module, using command line options, and using environment variables.

Using the warnings Module

Python's warnings module provides a way to control how warnings are displayed. You can use the filterwarnings function to ignore all warnings programmatically:

import warnings warnings.filterwarnings("ignore")

This will suppress all warnings. If you want to suppress only a specific type of warning, you can do so by specifying the warning class:

warnings.filterwarnings("ignore", category=DeprecationWarning)

In this case, only DeprecationWarnings will be suppressed.

Using Command Line Options

If you're running your Python script from the command line, you can use the -W option followed by ignore to suppress all warnings:

$ python -W ignore your_script.py Using Environment Variables

You can also use the PYTHONWARNINGS environment variable to control the display of warnings. To ignore all warnings, you can set this variable to ignore:

$ export PYTHONWARNINGS="ignore" $ python your_script.py

This will suppress all warnings for the entire session. If you want to suppress warnings for all sessions, you can add the export PYTHONWARNINGS="ignore" line to your shell's startup file (like .bashrc or .bash_profile for bash) so that this setting is always set.

Risks of Disabling Warnings

While there are several ways to disable warnings in Python, you should also understand the risks associated with doing this.

For example, a DeprecationWarning alerts you that a function you're using is slated for removal in a future version of Python or a library. If you ignore this warning, your code may suddenly stop working when you upgrade to a new version.

As a general rule, it's best to just fix the issues causing the warnings, instead of simply suppressing the warnings. There are, however, situations where removing warnings is actually the most practical solution, like when you're using a library that generates warnings you can't control and aren't actually helpful. In these cases, it's best to just suppress only the specific warnings you need to, and avoid using a blanket "ignore all" command.

Conclusion

Warnings are there for a reason, like signaling potential issues in the code that might lead to bugs or unexpected behavior, so it's best not to suppress them. However, there are times when you may want to disable these warnings, whether to clean up your console output or because you're aware of the warning and have decided it's not relevant to your particular situation.

In this Byte, we've learned about Python's warnings, how to suppress them, along with the potential risks of doing so.

The PSF is pleased to announce its second batch of PSF Fellows for 2023! Let us welcome the new PSF Fellows for Q2! The following people continue to do amazing things for the Python community:

Esteban Maya Cadavid Twitter, LinkedIn, Github, InstagramMartijn Pieters Stack Overflow, GitHub, WebsitePhilip JonesMastodon, GitHub, WebsiteYifei WangGitHub 

Thank you for your continued contributions. We have added you to our Fellow roster online.

The above members help support the Python ecosystem by being phenomenal leaders, sustaining the growth of the Python scientific community, maintaining virtual Python communities, maintaining Python libraries, creating educational material, organizing Python events and conferences, starting Python communities in local regions, and overall being great mentors in our community. Each of them continues to help make Python more accessible around the world. To learn more about the new Fellow members, check out their links above.

Let's continue recognizing Pythonistas all over the world for their impact on our community. The criteria for Fellow members is available online: https://www.python.org/psf/fellows/. If you would like to nominate someone to be a PSF Fellow, please send a description of their Python accomplishments and their email address to psf-fellow at python.org. Quarter 3 nominations are currently in review. We are accepting nominations for Quarter 4 through November 20, 2023.

Are you a PSF Fellow and want to help the Work Group review nominations? Contact us at psf-fellow at python.org.

Introduction

When dealing with large amounts of data or files, you might find yourself needing to compress files into a more manageable format. One of the best ways to do this is by creating a zip archive.

In this article, we'll be exploring how you can create a zip archive of a directory using Python. Whether you're looking to save space, simplify file sharing, or just keep things organized, Python's zipfile module provides a to do this.

Creating a Zip Archive with Python

Python's standard library comes with a module named zipfile that provides methods for creating, reading, writing, appending, and listing contents of a ZIP file. This module is useful for creating a zip archive of a directory. We'll start by importing the zipfile and os modules:

import zipfile import os

Now, let's create a function that will zip a directory:

def zip_directory(directory_path, zip_path): with zipfile.ZipFile(zip_path, 'w') as zipf: for root, dirs, files in os.walk(directory_path): for file in files: zipf.write(os.path.join(root, file), os.path.relpath(os.path.join(root, file), os.path.join(directory_path, '..')))

In this function, we first open a new zip file in write mode. Then, we walk through the directory we want to zip. For each file in the directory, we use the write() method to add it to the zip file. The os.path.relpath() function is used so that we store the relative path of the file in the zip file, instead of the absolute path.

Let's test our function:

zip_directory('test_directory', 'archive.zip')

After running this code, you should see a new file named archive.zip in your current directory. This zip file contains all the files from test_directory.

Note: Be careful when specifying the paths. If the zip_path file already exists, it will be overwritten.

Python's zipfile module makes it easy to create a zip archive of a directory. With just a few lines of code, you can compress and organize your files.

In the following sections, we'll dive deeper into handling nested directories, large directories, and error handling. This may seem a bit backwards, but the above function is likely what most people came here for, so I wanted to show it first.

Using the zipfile Module

In Python, the zipfile module is the best tool for working with zip archives. It provides functions to read, write, append, and extract data from zip files. The module is part of Python's standard library, so there's no need to install anything extra.

Here's a simple example of how you can create a new zip file and add a file to it:

import zipfile # Create a new zip file zip_file = zipfile.ZipFile('example.zip', 'w') # Add a file to the zip file zip_file.write('test.txt') # Close the zip file zip_file.close()

In this code, we first import the zipfile module. Then, we create a new zip file named 'example.zip' in write mode ('w'). We add a file named 'test.txt' to the zip file using the write() method. Finally, we close the zip file using the close() method.

Creating a Zip Archive of a Directory

Creating a zip archive of a directory involves a bit more work, but it's still fairly easy with the zipfile module. You need to walk through the directory structure, adding each file to the zip archive.

import os import zipfile def zip_directory(folder_path, zip_file): for folder_name, subfolders, filenames in os.walk(folder_path): for filename in filenames: # Create complete filepath of file in directory file_path = os.path.join(folder_name, filename) # Add file to zip zip_file.write(file_path) # Create a new zip file zip_file = zipfile.ZipFile('example_directory.zip', 'w') # Zip the directory zip_directory('/path/to/directory', zip_file) # Close the zip file zip_file.close()

We first define a function zip_directory() that takes a folder path and a ZipFile object. It uses the os.walk() function to iterate over all files in the directory and its subdirectories. For each file, it constructs the full file path and adds the file to the zip archive.

The os.walk() function is a convenient way to traverse directories. It generates the file names in a directory tree by walking the tree either top-down or bottom-up.

Note: Be careful with the file paths when adding files to the zip archive. The write() method adds files to the archive with the exact path you provide. If you provide an absolute path, the file will be added with the full absolute path in the zip archive. This is usually not what you want. Instead, you typically want to add files with a relative path to the directory you're zipping.

In the main part of the script, we create a new zip file, call the zip_directory() function to add the directory to the zip file, and finally close the zip file.

Working with Nested Directories

When working with nested directories, the process of creating a zip archive is a bit more complicated. The first function we showed in this article actually handles this case as well, which we'll show again here:

import os import zipfile def zipdir(path, ziph): for root, dirs, files in os.walk(path): for file in files: ziph.write(os.path.join(root, file), os.path.relpath(os.path.join(root, file), os.path.join(path, '..'))) zipf = zipfile.ZipFile('Python.zip', 'w', zipfile.ZIP_DEFLATED) zipdir('/path/to/directory', zipf) zipf.close()

The main difference is that we're actually creating the zip directory outside of the function and pass it as a parameter. Whether you do it within the function itself or not is up to personal preference.

Handling Large Directories

So what if we're dealing with a large directory? Zipping a large directory can consume a lot of memory and even crash your program if you don't take the right precautions.

Luckily, the zipfile module allows us to create a zip archive without loading all files into memory at once. By using the with statement, we can ensure that each file is closed and its memory freed after it's added to the archive.

import os import zipfile def zipdir(path, ziph): for root, dirs, files in os.walk(path): for file in files: with open(os.path.join(root, file), 'r') as fp: ziph.write(fp.read()) with zipfile.ZipFile('Python.zip', 'w', zipfile.ZIP_DEFLATED) as zipf: zipdir('/path/to/directory', zipf)

In this version, we're using the with statement when opening each file and when creating the zip archive. This will guarantee that each file is closed after it's read, freeing up the memory it was using. This way, we can safely zip large directories without running into memory issues.

Error Handling in zipfile

When working with zipfile in Python, we need to remember to handle exceptions so our program doesn't crash unexpectedly. The most common exceptions you might encounter are RuntimeError, ValueError, and FileNotFoundError.

Let's take a look at how we can handle these exceptions while creating a zip file:

import zipfile try: with zipfile.ZipFile('example.zip', 'w') as myzip: myzip.write('non_existent_file.txt') except FileNotFoundError: print('The file you are trying to zip does not exist.') except RuntimeError as e: print('An unexpected error occurred:', str(e)) except zipfile.LargeZipFile: print('The file is too large to be compressed.')

FileNotFoundError is raised when the file we're trying to zip doesn't exist. RuntimeError is a general exception that might be raised for a number of reasons, so we print the exception message to understand what went wrong. zipfile.LargeZipFile is raised when the file we're trying to compress is too big.

Note: Python's zipfile module raises a LargeZipFile error when the file you're trying to compress is larger than 2 GB. If you're working with large files, you can prevent this error by calling ZipFile with the allowZip64=True argument.

Common Errors and Solutions

While working with the zipfile module, you might encounter several common errors. Let's explore some of these errors and their solutions:

FileNotFoundError

This error happens when the file or directory you're trying to zip does not exist. So always check if the file or directory exists before attempting to compress it.

IsADirectoryError

This error is raised when you're trying to write a directory to a zip file using ZipFile.write(). To avoid this, use os.walk() to traverse the directory and write the individual files instead.

PermissionError

As you probably guessed, this error happens when you don't have the necessary permissions to read the file or write to the directory. Make sure you have the correct permissions before trying to manipulate files or directories.

LargeZipFile

As mentioned earlier, this error is raised when the file you're trying to compress is larger than 2 GB. To prevent this error, call ZipFile with the allowZip64=True argument.

try: with zipfile.ZipFile('large_file.zip', 'w', allowZip64=True) as myzip: myzip.write('large_file.txt') except zipfile.LargeZipFile: print('The file is too large to be compressed.')

In this snippet, we're using the allowZip64=True argument to allow zipping files larger than 2 GB.

Compressing Individual Files

With zipfile, not only can it compress directories, but it can also compress individual files. Let's say you have a file called document.txt that you want to compress. Here's how you'd do that:

import zipfile with zipfile.ZipFile('compressed_file.zip', 'w') as myzip: myzip.write('document.txt')

In this code, we're creating a new zip archive named compressed_file.zip and just adding document.txt to it. The 'w' parameter means that we're opening the zip file in write mode.

Now, if you check your directory, you should see a new zip file named compressed_file.zip.

Extracting Zip Files

And finally, let's see how to reverse this zipping by extracting the files. Let's say we want to extract the document.txt file we just compressed. Here's how to do it:

import zipfile with zipfile.ZipFile('compressed_file.zip', 'r') as myzip: myzip.extractall()

In this code snippet, we're opening the zip file in read mode ('r') and then calling the extractall() method. This method extracts all the files in the zip archive to the current directory.

Note: If you want to extract the files to a specific directory, you can pass the directory path as an argument to the extractall() method like so: myzip.extractall('/path/to/directory/').

Now, if you check your directory, you should see the document.txt file. That's all there is to it!

Conclusion

In this guide, we focused on creating and managing zip archives in Python. We explored the zipfile module, learned how to create a zip archive of a directory, and even dove into handling nested directories and large directories. We've also covered error handling within zipfile, common errors and their solutions, compressing individual files, and extracting zip files.

Mariatta has been a contributor to Python for many years and is a very inspiring public speaker.

Some of what we talk about:

• Python Documentation Working Group
• GitHub bots, There's an API for that
• PyLadies
• PyLadiesCon
• Typo of the Day (maintainerd, verbossity, work-lie balance, etc.)
• A fish aquarium
• Cooking, Baking
• History of Tempura
• Working with APIs with Python
• Public Speaking / Giving Talks
• The power of seeing other women give talks

★ Support this podcast while learning ★

• Python Testing with Pytest, the course, is a great way to learn pytest quickly
• Part 1, pytest Primary Power is now available. 
• Start testing effectively and efficiently today.

★ Support this podcast on Patreon ★ <p>Mariatta has been a contributor to Python for many years and is a very inspiring public speaker.</p><p>Some of what we talk about:</p><ul> <li>Python Documentation Working Group</li> <li>GitHub bots, <a href="https://mariatta.ca/posts/talks/theres-an-api-for-that/">There's an API for that</a> </li> <li><a href="https://pyladies.com">PyLadies</a></li> <li><a href="https://conference.pyladies.com">PyLadiesCon</a></li> <li>Typo of the Day (maintainerd, verbossity, work-lie balance, etc.)</li> <li>A fish aquarium</li> <li>Cooking, Baking</li> <li>History of Tempura</li> <li>Working with APIs with Python</li> <li>Public Speaking / Giving Talks</li> <li>The power of seeing other women give talks</li> </ul> <br><p><strong>★ Support this podcast while learning ★</strong></p><ul> <li> <a href="https://pythontest.com/courses/">Python Testing with Pytest, the course</a>, is a great way to learn pytest quickly</li> <li>Part 1, <a href="https://testandcode.teachable.com/p/pytest-primary-power">pytest Primary Power</a> is now available. </li> <li>Start testing effectively and efficiently today.</li> </ul> <strong> <a href="https://www.patreon.com/PythonPeople" rel="payment" title="★ Support this podcast on Patreon ★">★ Support this podcast on Patreon ★</a> </strong>

I wrote a tiny script that creates an RSS feed for all the audio files it finds in a folder.
I call it derb.

My mother gets devotional songs and sermons on cds, which I rip to MP3 files and then dump on her phone for her.1
She listens to them all the time, and now three of her friends want to do the same too.
I thought of just sticking them in my self hosted Jellyfin instance,2 but then I realised, all of them have erratic, slow internet. So the idea of self hosting a podcast feed really appealed to me.

So I quickly used Feedgenerator in conjunction with Tinytag, to whip up a script that’d help me do just that. The code’s up on Github, if you want to go install and play with it yourselves.

Here’s a quick walk through derb.py.3

Setting up house

We set up a place to accept a path containing the files. The feed will ultimately be placed as a feed.xml in the same folder as well We then walk through the folder (after a really basic sanity check) and gather all the files into a list.
The base_url is where the feeds (along with the audio) will be served from.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 base_url = "https://ab.janusworx.com" book_folder = input("Paste path to audiobook folder here: ") book_out_path = base_url + (book_folder.split("/")[-1]) file_list = os.walk(book_folder) # Do a basic check on the validity of the path we get, # before we build the audio file list. try: all_files = (list(file_list)[0][2]) except IndexError as e: print(f"\n\n" f"---\n" f"ERROR!: {e}\n" f"Have you typed in the right path?\n" f"---\n") sys.exit("Quitting script!") audio_files = [] for each_file in all_files: each_file = Path(each_file) if each_file.suffix in ['.mp3', '.m4a', '.m4b']: audio_files.append(str(each_file)) audio_files = sorted(audio_files) Creating a feed

We now go about the business of setting up the feed proper.
To begin with, we grab the first file we can get our grubby paws on, and create a TinyTag object that’ll give us a lot of metadata. (If there isn’t any, we quit.)
Oh, and by the way, how do I know what data I’d need to create a feed? I just cribbed everything from the Feedgenerator’s excellent documentation. I also looked at the widely linked to, RSS reference page for clarification if I got confused.

We then, instantiate create a feedgenerator object along with the podcast extension.
Following which, we supply the feed a title, an id4, feed author details, a language, podcast category and description.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 # Setup a feed ## Grab feed metadata from the first audio file feed_metadata_file = TinyTag.get(Path(book_folder, audio_files[0])) ## Grab title from metadata file. ## At the same time, break out if there isn’t any. if not feed_metadata_file.album: sys.exit("\n---\nStopping feed creation.\n Setup audio file metadata with a tag editor") feed_title = feed_metadata_file.album ## Creating feed instance audio_book_feed = FeedGenerator() audio_book_feed.load_extension("podcast") ## Setting up more stuff on the feed proper audio_book_feed.id(base_url) audio_book_feed.title(feed_title) audio_book_feed.author({"name": "Jason Braganza", "email": "feedback@janusworx.com"}) audio_book_feed.link(href=f'{book_out_path}', rel='self') audio_book_feed.language('en') audio_book_feed.podcast.itunes_category('Private') audio_book_feed.description(feed_title) Adding episodes and writing out the file

After which it’s then a matter of looping through that audio file list we created and adding them as entries to the feed object we created.
Once again we grab the metadata from each file, using Tinytag, and then set each feed entry’s details (title, id and enclosure).
I’ve hardcoded the mime types, since I know I only have two basic type of audio files. If you don’t know what kind of audio, you might be serving, the mimetypes-magic package should help.
Finally we write it all out to a file.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 # Loop the file list and create entries in the feed for each_file in audio_files: each_file_metadata = TinyTag.get(Path(book_folder, each_file)) episode_file_path = Path(each_file) episode_suffix = episode_file_path.suffix episode_mime_type = 'audio/mpeg' if episode_suffix == '.mp3' else 'audio/x-m4a' episode_title = each_file_metadata.title episode_size = str(each_file_metadata.filesize) episode_link = f"{book_out_path}/{each_file}" audio_episode = audio_book_feed.add_entry() audio_episode.title(episode_title) audio_episode.id(episode_link) audio_episode.enclosure(episode_link, episode_size, episode_mime_type) # Write the rss feed to the same folder as the source audio files audio_book_feed.rss_file(f"{book_folder}/feed.xml") Serving

Once done, I moved it all to my trusty Pi, which runs barebones Nginx with a single bare page, protected by basic auth.
I decided not to publish the feeds publicly.
Rather I’m going to just set it up in their podcast players, when I meet them, or pass it over to their kids over Signal.5
It’s already worked with three of them, so everyone’s happy and here’s me hoping, fingers crossed, it’ll be easy to support in the long run.

Feedback on this post? Mail me at feedback@janusworx.com

P.S. Subscribe to my mailing list!
Forward these posts and letters to your friends and get them to subscribe!
P.P.S. Feed my insatiable reading habit.

1. Yes, while it’s slowly moving to Youtube, that world still mostly depends on CDs and USB sticks. ↩︎

2. the audio files, not my mother’s friends. ↩︎

3. Large parts are elided. Please look at Github for the actual file. ↩︎

4. normally, the site it’ll be served from ↩︎

5. Why should I be the only one doing family tech support? ↩︎

Kivy is an open-source Python software library for developing graphical user interfaces. It supports cross-platform development for the desktop as well as the creation of multi-touch apps for mobile devices.

Kivy apps can run across several platforms, including Windows, Linux, macOS, Android, and IOS. One place where Kivy particularly shines is in game development. By combining Kivy's 2D physics capabilities with a simple physics engine, you can create impressive 2D simulations and games.

In this article, you'll learn about using Kivy to develop Python apps. We will go through an introduction to Kivy and what it can do. You'll learn how to create a simple Kivy app in Python and learn the basics of Kivy's styling language, known as Kv. Finally, you'll use Kivy's graphics module to draw 2D shapes on the Kivy canvas.

To get the most out of this tutorial, you should have basic knowledge of Python. Previous knowledge of general concepts of GUI programming, such as event loops, widgets, layouts, and forms, is also a plus.

Table of Contents

• Installing Kivy
• Writing Your First Kivy GUI App in Python
• Exploring Widgets and Layouts
  • Widgets
  • Layouts
• Using Widgets and Layouts: A Practical Example
• Drawing Shapes in Kivy: The canvas Property
• Styling Your GUIs With the Kivy Language
  • Relying on the Automatic Widget Loading
  • Loading Widgets Through the Builder Class
• More Resources
• Conclusion

There are many different Python GUI libraries available, and choosing one for your project can be a really tough and confusing decision to make. For advice see our guide to Python GUI libraries.

Let's get started. We'll first take a few moments to install and set up Kivy on your computer.

Installing Kivy

Before using a third-party library like Kivy, we must install it in our working environment. Installing Kivy is as quick as running the python -m pip install kivy command on your terminal or command line. This command will install the library from the Python package index (PyPI).

Note that as of the time of writing this tutorial, Kivy only officially supports Python versions up to 3.10. For detailed information about installing Kivy, visit the official installation guide.

However, when working with third-party libraries, it's good practice to create a Python virtual environment, which is a self-contained Python installation for a particular version of Python that you can use to isolate the dependencies of a given project.

To create a virtual environment, you'll typically use Python's venv module from the standard library. Fire up a command-line window and type in the following command in your working directory.

sh $ python -m venv kivy_env

This command will create a folder called kivy_env containing a Python virtual environment. The Python version in this environment is the same as you get when you run python --version on your command line.

Next, we need to activate the virtual environment. Use the appropriate command, depending on whether you're on Windows, macOS, or Linux:

• Windows
• macOS
• Linux

sh C:/> .\kivy_env\Scripts\activate sh $ source kivy_env/bin/activate sh $ source kivy_env/bin/activate

Once that's confirmed to be working, you can then install Kivy within the virtual environment you just created by running the following:

sh (kivy_env) $ python -m pip install kivy

With this command, you'll install Kivy in your active Python virtual environment, so you're now ready to go.

You can also install Kivy by downloading its source code directly from GitHub and doing a manual installation on the command line. For more information on following this installation path, check out the section about installing Kivy from source in the documentation.

Writing Your First Kivy GUI App in Python

Without further ado, let's get right into creating our first app with Kivy and Python. For this app, we will use a Label object to display the traditional "Hello, World!" message on our screen. To write a minimal Kivy GUI app, we need to run a few steps:

1. Subclassing the App class
2. Implementing its build() method, which returns a Widget instance
3. Instantiating this class and calling its run() method

Let's start by importing the required classes. For our example, we only need the App and Label classes. Create a Python file called app.py and add the following imports:

python from kivy.app import App from kivy.uix.label import Label

The App class provides the base functionality required to create GUI apps with Kivy, such as managing the event loop. Meanwhile, the Label class will work as the root visual element or widget for our GUI.

Next, we can create our subclass of App. We have called it MainApp here. However, you can call it whatever you like:

python from kivy.app import App from kivy.uix.label import Label class MainApp(App): def build(self): return Label(text="Hello, World!")

This subclass uses the concept of inheritance in object-oriented programming (OOP) in Python. All the attributes and methods defined in the superclass, App, are automatically inherited by the subclass, MainApp.

In order for our app to create a UI, we need to define a build() method. In build(), we create and return either a widget or layout, which will be the root object in our UI structure.

The build() method is the entry point to whatever will be drawn on the screen. In our example, it creates and returns a label with the "Hello, World!" text on it.

Finally, we need to create an instance of MainApp and call its run() method:

python from kivy.app import App from kivy.uix.label import Label class MainApp(App): def build(self): return Label(text="Hello, World!") MainApp().run()

In the final line, we create an instance of MainApp and call its run() method. This method launches the application and runs its main loop. That's it! We're ready to run our first Kivy app. Open your command line and run the following command:

sh $ python app.py

You'll see the following window on your screen:

First Kivy GUI Application

Great! You've just written your first Kivy GUI app using Python. It shows a black window with the message "Hello, World!" In its center. Note that the window's title bar shows the title Main, which comes from the name of your App subclass.

The next step is to explore some other essential features of Kivy that will allow you to write fully-functional GUI apps with this library.

Exploring Widgets and Layouts

In the previous section, we mentioned widgets and layouts a few times -- you may be wondering what they are! A widget is an element of a GUI that displays information or provides a specific function. They allow your users to interact with your app's GUI.

A layout, on the other hand, provides a way of arranging widgets into a particular structure in your application's windows. A layout can also give certain behaviors to widgets that belong to it, like the ScatterLayout, which enables multi-touch resizing of a child widget.

In Kivy, you'll find widget and layout classes in their corresponding module under the kivy.uix module. For example, to import the Button class, we can use:

python from kivy.uix.button import Button

In Kivy, widgets and layout classes are usually located in modules named after the class itself. However, the class uses CamelCase, and the containing module uses lower casing.

For example, take the following imports:

python # Widgets from kivy.uix.label import Label from kivy.uix.image import Image # Layouts from kivy.uix.boxlayout import BoxLayout from kivy.uix.floatlayout import FloatLayout

You'll find some exceptions to this naming convention. For example:

python from kivy.uix.image import AsyncImage from kivy.uix.screenmanager import FadeTransition

This commonly happens with modules that define multiple and closely related classes, such as Image and AsyncImage.

Widgets

Widgets are the building blocks of Kivy-based GUIs. Some of the most commonly used GUI widgets in Kivy apps include the following:

• Widget is the base class required for creating widgets.
• Label is used for rendering text on windows and dialogs.
• TextInput provides a box for editable plain text.
• Button triggers actions when the user presses it.
• CheckBox provides a two-state button that can be either checked or unchecked.
• Image is used to display an image on your GUIs.
• ProgressBar visualizes the progress of some tasks.
• DropDown provides a versatile drop-down list that can list different widgets.

With these widgets and some others that Kivy provides, you can build complex and user-friendly interfaces for your applications.

Layouts

Kivy also has a rich set of layout classes that allows you to arrange your widgets coherently and functionally to build up GUIs. Some examples of common layouts include:

• BoxLayout arranges widgets sequentially in either a vertical or horizontal fashion.
• FloatLayout arranges widgets in a specific position on the containing window.
• RelativeLayout arranges child widgets according to relative positions.
• GridLayout arranges widgets in a grid defined by the rows and columns.
• PageLayout creates multi-page layouts in a way that allows flipping from one page to another.
• ScatterLayout positions its child widgets similarly to a RelativeLayout.
• StackLayout stacks in a left-to-right and then top-to-bottom order, or top-to-bottom then left-to-right order.

You can combine and nest layouts together to build complex user interfaces.

Using Widgets and Layouts: A Practical Example

As an example of how to use widgets and layouts in Kivy, let's look at a commonly used layout class: the GridLayout. With this class, we can create a grid of rows and columns. Each cell of the grid has a unique pair of zero-based coordinates. Consider the following example:

python from kivy.app import App from kivy.uix.button import Button from kivy.uix.gridlayout import GridLayout ROWS = COLS = 3 class GridApp(App): def build(self): root = GridLayout(rows=ROWS, cols=COLS) for i in range(ROWS): for j in range(COLS): root.add_widget(Button(text=f"({i}, {j})")) return root GridApp().run()

In the build() method, we instantiate the GridLayout with three rows and three columns. Then use a for loop to add button widgets to the layout using the add_widget() method.

When we run this app, we get the window that is shown below:

Grid Layout in Kivy

Each button on the grid shows its corresponding pair of coordinates. The first coordinate represents the row, while the second represents the column. Like the rest of the layout classes, GridLayout can take several arguments that you can use to fine-tune its behavior.

Drawing Shapes in Kivy: The canvas Property

To deeply customize a GUI or design a 2D video game, we may need to draw 2D shapes, such as a rectangle, circle, ellipse, or triangle. Doing this is straightforward in Kivy. The library provides a rich set of shape classes that you can find in the kivy.graphics package. Some of these classes include:

• Ellipse
• Line
• Rectangle
• Triangle

To draw a shape on the screen with Kivy, we need to use the canvas property of a Widget object. This property holds an instance of the Canvas class, which lives in the kivy.graphics package.

Let's see how this works with an example of a white square drawn on the screen:

python from kivy.app import App from kivy.core.window import Window from kivy.graphics import Rectangle from kivy.uix.widget import Widget class CanvasApp(App): def build(self): root = Widget() size = 200 width, height = Window.size pos_x = 1/2 * (width - size) pos_y = 1/2 * (height - size) with root.canvas: Rectangle(size=[size, size], pos=[pos_x, pos_y]) return root CanvasApp().run()

Inside build(), we create the root widget and define the size of our shape. It'll be a square shape, so each side is equal.

Next, we compute the coordinates to center our shape on the window. The coordinates passed when creating the shape are for the top left corner of the window.

To calculate the correct values, we take the width and height of our main window, halving these values to get the center. We then subtract half of the width or height of our shape to position the center of our shape in the middle of the window. This can be simplified to 1/2 * (width - size) or 1/2 * (height - size). We store the resulting top left coordinates in pos_x and pos_y.

Next, we use the canvas property of our root window to draw the shape. This property supports the with statement, which provides the appropriate context for creating our shapes. Inside the with block, we define our Rectangle instance with the size and pos arguments.

Finally, we return the root widget as expected. The final line of code creates the app instance and calls its run() method. If you run this app from your command line, then you'll get the following window on the screen:

Drawing Shapes in Kivy With Canvas

Cool! You've drawn a square on your Kivy app. The computed coordinates place the square in the center of the window. The default color is white. However, we can change it:

python # ... from kivy.graphics import Color, Rectangle from kivy.uix.widget import Widget # ... class CanvasApp(App): def build(self): # ... with root.canvas: Color(1, 1, 0, 1) Rectangle(size=[side, side], pos=[pos_x, pos_y]) # ...

In this code snippet, we have added an import for the Color class from the graphics package. The Color class accepts four numeric arguments between 0 and 1 representing the red, green, blue, and transparency components of our target color.

For example, the values (1, 0, 0, 1) represent an entirely red and fully opaque color. The value (0, 1, 0, 0.5) is fully green, half opaque, and half transparent. Consequently, the value (1, 1, 0, 1) gives a fully opaque yellow color. So, if you run the app, then you'll get the following output:

Drawing Shapes in Color With Kivy

We can experiment with different color values and also with different shape classes, which is cool.

Finally, note that to see the effect of the Color() on the drawn rectangle, the Color class must be instantiated before the Rectangle class. You can think of this as dipping your paintbrush on a palette before using it to paint on your canvas! Interestingly, any drawing that comes after the Color instance is painted accordingly so long as a different color has not been applied.

Using the with statement is pretty convenient and facilitates working with shapes. Alternatively, we can use the canvas.add() method:

python root.canvas.add(Color(1, 1, 0, 1)) root.canvas.add( Rectangle(size=[side, side], pos=[pos_x, pos_y]) )

These statements are equivalent to the statements we have in the with block. Go ahead and give it a try yourself.

Styling Your GUIs With the Kivy Language

Kivy also provides a declarative language known as the Kv language, which aims at separating your application's GUI design and business logic. In this tutorial, we will not go deep into using the Kv language. However, we will highlight some of its main features and strengths.

With Kv language, you can declare and style the widgets and graphical components of your GUI apps. You will put your Kv code in files with the .kv extension. Then you can load the content of these files into your app to build the GUI. You'll have at least two ways to load the content of a .kv file:

• Relying on the automatic loading mechanism
• Using the Builder class for manual loading

In the following sections, you'll learn the basics of these two ways of using the Kv language to build the GUI of your Kivy apps.

Relying on the Automatic Widget Loading

As stated earlier, the Kv language helps you separate business logic from GUI design. Let's illustrate this possibility with an updated version of our "Hello, World!" app:

python from kivy.app import App from kivy.uix.label import Label class CustomLabel(Label): pass class MainApp(App): def build(self): root = CustomLabel() return root MainApp().run()

As you can see we have subclassed the Label class to create a new CustomLabel haven't made any modifications to the subclass, so it functions exactly like the Label class but with a different name. We add a pass statement, which is a Python placeholder statement which makes the code syntactically valid.

Next, create a file called main.kv alongside your app's file. Define a label using the following code:

kv <CustomLabel>: text: "Hello, World!"

Note that your label must have the same name as your custom Python class in the app's file. Additionally, the .kv file must have the same name as your subclass of App, but without the App suffix and in lowercase. In this example, your subclass is named MainApp, so your .kv file must be main.kv.

Now you can run the app from your command line. You'll get the following window on your screen:

Kivy Application Using the Kv Language

The Kv language, also known as kivy language or just kvlang, allows us to create widget trees in a declarative way. It also lets you bind widget properties to each other or to callbacks.

Loading Widgets Through the Builder Class

When your Kivy project grows, your .kv file will grow as well. So, it is recommended that you split up the file into different files for readability. In such cases, you will end up with multiple .kv files, and the automatic loading mechanism will not be sufficient. You'll have to use the Builder class from kivy.lang.Builder.

To explore how to use Builder, let's build a sample GUI consisting of a label and button in a BoxLayout. The label will be provided in the labels.kv file, while the buttons will live in the buttons.kv file.

Here's the Python code for this app:

python from kivy.app import App from kivy.lang import Builder from kivy.uix.boxlayout import BoxLayout from kivy.uix.button import Button from kivy.uix.label import Label Builder.load_file("labels.kv") Builder.load_file("buttons.kv") class CustomLabel(Label): pass class CustomButton(Button): pass class MainApp(App): def build(self): root = BoxLayout(orientation="vertical") root.add_widget(CustomLabel()) root.add_widget(CustomButton()) return root MainApp().run()

After importing the required classes, we call the load_file() method. This method takes the filename of a .kv file as an argument and loads it into your app.

Next, you create the custom label and button following the pattern used in the previous section. Inside build(), you create a BoxLayout and add the two widgets to it. Now you need to provide the required .kv files.

Go ahead and create a labels.kv file with the following content:

kv <CustomLabel>: text: "This is a custom label!" font_size: 50 bold: True

This file provides a label with the text "This is a custom label!". Its font will have a size of 50 pixels and will be bold.

The buttons.kv will have the following code:

kv <CustomButton>: text: "Click me!"

Your custom button will be quite minimal. It'll only have the text "Click me!" on it. Go ahead and run the app from your command line. You'll get the following window on your screen:

Kivy Application Using the Kv Language With Multiple kv Files

In addition to using the load_file() to build Kv language files, you can also parse and load Kv language directly in a multi-line string in your Python file:

python Builder.load_string(""" <CustomLabel>: text: "This is a custom label!" font_size: 50 bold: True """) Builder.load_string(""" <CustomButton>: text: "Click me!" """)

These calls to load_string() are completely equivalent to the corresponding calls to load_file() in our original code example.

Let's take a look at a final example of using the Kv language. This time we'll use the language to draw shapes. Create a rectangle.py file with the following content:

python from kivy.app import App from kivy.uix.widget import Widget class CustomRectangle(Widget): pass class MainApp(App): def build(self): return CustomRectangle() MainApp().run()

Now go ahead and create another file in the same directory and save it as main.kv. Then add the following content:

kv <CustomRectangle>: canvas: Color: rgba: 1, 1, 0, 1 Rectangle: size: 200, 200 pos: 0, 0

If you run the rectangle.py file, then you will see a 200×200 pixels rectangle ---square in this case--- at the lower left corner of your window! For more guidelines on using Kv Language, check out its official documentation.

More Resources

For some more examples of what you can do with Kivy, take a look at the Kivy examples section in the documentation. Depending on your interest, you can also the other resources. For example:

• If you're interested in 3D, then the Kivy 3D demo gives a good demonstration of the framework's rendering abilities.
• If you're interested in using Kivy to develop for mobile, you can write functional Android apps (APKs) with Python and pack them using tools like Buildozer and Python-For-Android without learning Java.
• If you want a complete vision of where you can use Kivy, then check out the gallery of examples provided by the Kivy community.

Conclusion

What you've learned in this tutorial is just the tip of the Kivy iceberg. There's so much more to Kivy than what meets the eye. It's a powerful GUI library that provides a well-structured hierarchy of classes and objects that you can use to create modern and cross-platform GUIs for your applications.

I am very happy to announce that my Python GUI programming book Create GUI Applications with Python & Qt6 / PyQt6 Edition is now available in Korean from Acorn Publishing

It's more than a little mind-blowing to see a book I've written translated into another language -- not least one I cannot remotely understand! When I started writing this book a few years ago I could never have imagined it would end up on book shelves in Korea, never mind in Korean. This is just fantastic.

파이썬과 Qt6로 GUI 애플리케이션 만들기 파이썬 애플리케이션 제작 실습 가이드

If you're in Korea, you can also pick up a copy at any of the following bookstores: Kyobobook, YES24 or Aladin

Thanks again to Acorn Publishing for translating my book and making it available to readers in Korea.

When writing GUI applications you often need to accept data from users. A reliable application must verify and validate all its input data before taking any further action or doing any processing. Input validation is the process of examining the data your users provide to your applications to ensure its validity.

In this tutorial, we'll look at some examples of common input data validation strategies you can use when developing GUI applicatipons with the Tkinter UI library.

Table of Contents

• Input Validation Strategies in GUI Apps
• Form-Level Input Validation in Tkinter
• Widget-Level Input Validation in Tkinter
• Conclusion

Input Validation Strategies in GUI Apps

Input data validation is commonly needed in dialog windows where we provide input widgets for the user to enter information that our program requires to work properly. When validating input data in a dialog or any other component of our app's GUI, there are two main approaches we can take:

1. Real-time validation: This strategy is also called widget-level validation and involves validating the user's input as soon as they enter it in the app's GUI.
2. Batch validation: This strategy is also known as form-level validation and involves validating all input fields at once, usually when the user clicks the submit button on a dialog.

Real-time validation works best in situations where input data is continuously entered and needs to be verified quickly. For example, when we're using a table widget to update a table in a database in real time.

Batch validation is suitable when there are interdependencies between different pieces of input. In this case, the input data needs to be validated all at once. A good example of this type of validation is a login form which typically verifies the user and the password at once when we click the Login button.

In the following sections, you'll learn how to implement these two validation strategies in a Tkinter application.

Form-Level Input Validation in Tkinter

Tkinter is a popular GUI library in Python. It provides a basic set of widgets that you can use for building GUI applications quickly. The library comes with a few widgets for data entry, including Entry, Text, Spinbox, and others.

To demonstrate the form-level validation strategy, we'll use the Entry widget, which presents a simple text box to the user. For this example, we'll create a form with an Entry and a Button. The Entry widget will only accept numeric values. We'll run the input data validation when the user clicks on the button to submit the information for processing.

Below is the starter code for our small Tkinter app:

python from tkinter import Tk, ttk # Create the app's main window root = Tk() root.title("Form-Level Input Validation") root.geometry("490x100") # Add widgets entry = ttk.Entry(root, width=35) entry.grid(row=0, column=0, padx=5, pady=5) button = ttk.Button(root, text="Validate", command=validate_numbers) button.grid(row=0, column=1, padx=5, pady=5) label = ttk.Label(root, text="Display") label.grid(row=1, column=0, columnspan=2, padx=5, pady=5) # Run the app's main loop root.mainloop()

In the first line of the code, we import Tk and ttk from the tkinter package. Tk is Tkinter's main GUI class which provides the application window and main loop. The ttk module provides styled widgets for you to create modern Tkinter apps.

Next, we create the parent window, root, by instantiating the Tk class. The window will have a title and a geometry (shape), which we set using the title() and geometry() methods.

Then, we create three different widgets: an entry, a label, and a button using the ttk.Entry, ttk.Label and ttk.Button classes, respectively.

To create the Entry widget, we use the parent window, root, and width as arguments. To place the entry widget in the parent window, we use the grid() geometry manager. This lays widgets out in a grid, at the specified row and column positions, with padx and pady (horizontal and vertical padding) in pixels.

Then we create a button using the ttk.Button class with the parent window and text as arguments. Again, we use grid() to place the button on the parent window, on the same row as the Entry but now in column 1.

Finally, we create a ttk.Label widget using a parent window and text as arguments. Using the grid() method, we place this on the next row in column 0, using colspan=2 to stretch it across two columns.

With our application complete, we run it by calling mainloop() on root. You'll get the following output:

An example Tkinter application with Entry, Button, and Label widgets

Great! We have a working Tkinter app with a simple form.

Let's jump into input validation. So far, the button doesn't do anything when we click it. We want the button to run code for validating the data provided through the entry widget. Let's create a function for validating numeric values:

python from tkinter import Tk, ttk # Create the app's main window root = Tk() root.title("Form-Level Input Validation") root.geometry("490x100") # Create a validation function def validate_numbers(): input_data = entry.get() if input_data: try: float(input_data) label.config( text=f"Valid numeric value: {input_data}", foreground="green", ) except ValueError: label.config( text=f'Numeric value expected, got "{input_data}"', foreground="red", ) else: label.config( text="Entry is empty", foreground="red", ) # ...

Here we have implemented a validate_numbers() function to test if the user's input is a valid number. Inside the function, we get the input data from the entry widget using the get() method, storing it in the variable input_data.

First, we use an if statement to check whether input_data contains anything at all. If input_data is an empty string, which is falsey, the else clause will change the label's text to "Entry is empty" in red.

If input_data contains some text, we further validate this by attempting to cast the value to a float using the built-in float() type. If the value is not a valid float, the code will throw a ValueError exception. If this occurs, we catch this exception and display an informative message in red.

Finally, in the case where input_data contains something which is a valid numeric value, a success message will be shown in green.

Now that we have implemented our validation function, we can bind it to our button by passing it as the command argument. Update the Button instantiation as follows:

python # Add widgets # ... button = ttk.Button(root, text="Validate", command=validate_numbers) # ...

Here, the command argument points to the function that we want to trigger when the button is clicked. If you run the program again and enter valid numbers, you'll get an output similar to the following when clicking the Validate button:

Entry box with a valid number, showing the success message

Great, the app validated the input successfully! What will happen if we provide an invalid input? Check out the form below:

Entry box with an invalid number, showing the failure message

Great! Everything works! If you enter something which is not a number, your app will now correctly identify the invalid input.

Widget-Level Input Validation in Tkinter

With the widget-level validation strategy, we validate the input data using widget-specific events. For example, an input validation function can run when an entry widget gets or loses focus. This approach comes in handy when we want to validate input data in real time, widget by widget.

To see this strategy in action, let's create a window with two entry widgets. The code for that is as follows:

python from tkinter import Tk, ttk # Create the app's main window root = Tk() root.title("Widget-Level Validation") root.geometry("490x120") # Add widgets name_label = ttk.Label(root, text="Name:") name_label.grid(row=0, column=0, padx=5, pady=5) name_entry = ttk.Entry(root, width=35) name_entry.grid(row=0, column=1, padx=5, pady=5) age_label = ttk.Label(root, text="Age:") age_label.grid(row=1, column=0, padx=5, pady=5) age_entry = ttk.Entry(root, width=35) age_entry.grid(row=1, column=1, padx=5, pady=5) label = ttk.Label(root, text="Display") label.grid(row=2, column=0, columnspan=2, padx=5, pady=5) # Run the app's main loop root.mainloop()

In this example, we are creating a window with two entry widgets and a label for displaying a message after the validation process.

If you run this example, you'll get the following window:

Window showing two Entry boxes with a validation message

In this next example, we'll validate the age input when the user moves the focus out of the age entry. First, let's create the function for validating input:

python from tkinter import Tk, ttk # Create the app's main window root = Tk() root.title("Widget-Level Validation") root.geometry("490x120") # Create a validation function def validate_age(): age = age_entry.get() if age: if age.isdigit() and int(age) in range(1, 151): label.config( text=f"Valid age: {age}", foreground="green", ) return True else: label.config( text="Age must be a number between 1 and 150", foreground="red", ) return False else: label.config( text="Entry is empty", foreground="red", ) return False # ...

The validate_age() function gets the input using the get() method, storing the value in the variable age.

We again use an if statement to check whether the input value is empty or not. If the entry is empty, the else clause runs, showing the failure message.

If the user has provided a value, then we validate the input data using the isdigit() method and the built-in range() function.

It's important to note that functions used for widget-level validation must be predicate functions, returning either True or False.

Finally, we need to bind this function to the entry widget which input we want to validate. To do this, we need to pass two new arguments to the Entry() constructor:

python # Add widgets # ... age_entry = ttk.Entry( root, width=35, validatecommand=validate_age, validate="focusout" ) # ...

The validatecommand argument specifies the function to be called for input validation, and the validate command specifies on which event this function should be called. In this example, we have used "focusout". However, the validate argument can take any of the following values:

• "focus" triggers the validation whenever the widget receives or loses focus.
• "focusin" triggers the validation whenever the widget receives focus.
• "focusout" triggers the validation whenever the widget loses focus.
• "key" triggers the validation whenever any keystroke changes the widget's content.
• "all" triggers the validation in all the above situations.
• "none" turns the validation off.

Go ahead and run your program. Enter a name and play with different input values for the age. After entering the age value, move the focus to the name entry to trigger the validation. Here's how the app works with a valid age:

Window showing validation of a valid age

When you move the focus to the Name entry, the age entry triggers the age validation function. Now let's try an invalid age:

Window showing validation of an invalid age

The validation works as expected! Whenever you enter an invalid age value and move the focus out of the age entry, you get an error message pointing out the problem. That's great!

Conclusion

In this tutorial, you've learned about input validation in GUI applications. You now know how to validate user input in Tkinter using two different validation strategies: form-level validation and widget-level validation.

You can now apply these two strategies to validate user input in your own applications.

Graphical User Interfaces (GUIs) allow users to interact with software through intuitive and user-friendly graphical elements such as buttons, icons, text boxes, and windows. GUIs provide a simple way to perform complex tasks and navigate a software system using the mouse and keyboard and without needing to remember complex commands. By using familiar visual concepts across different tools and platforms, GUIs help make new software feel intuitive and user-friendly, even for beginners.

While Python is more commonly used for command-line tools, data science, and web apps, it is also perfectly capable of building graphical desktop applications. The Python ecosystem makes it possible to build almost anything, from small user-friendly interfaces for your scripts to more complex data analysis or engineering tools. Whether you're a Python beginner or a seasoned developer, learning how to build GUI apps with Python is an excellent addition to your skill set.

Table of Contents

• An Introduction to Python GUI Libraries in Python
• The Tkinter GUI Library
  • How Do You Install Tkinter?
  • How Can You Write a Tkinter App?
  • What Is Tkinter Commonly Used for?
  • Is Tkinter Outdated?
  • Why Does Tkinter Look Old-Fashioned?
  • How Can You Design GUIs for Tkinter?
  • What Are Tkinter's Pros & Cons?
• The PyQt GUI Framework
  • How Do You Install PyQt?
  • How Can You Write a PyQt App?
  • What Is PyQt Commonly Used for?
  • Can You Use PyQt for Open-Source and Commercial Apps?
  • How Can You Design GUIs for PyQt Apps?
  • What Are PyQt's Pros and Cons?
• Tkinter vs. PyQt: A Feature Comparison
• Decision Time: How to Choose the Best GUI Library for Your Project
  • What Are Your Goals?
  • Do You Need a GUI Library or a GUI Framework?
  • Is Tkinter Easier to Learn Than PyQt?
  • Which One Should You Learn First, Tkinter or PyQt?
• What Next?

An Introduction to Python GUI Libraries in Python

One of Python's core strengths is the rich ecosystem of libraries and frameworks available. GUI programming libraries are not an exception -- you'll find several GUI libraries for Python.

While having multiple choice is great, it means that if you want to learn how to write GUI applications with Python, the first question you will need to answer is -- Which library should I use?

Even though many GUI libraries are available, none has the widespread adoption of Tkinter and PyQt. So, if you're starting with GUI programming in Python, it'll make sense to start with one of these. There are far more tutorials, help & resources available to get you up and running, and you're more likely to find help if you need it.

Google Trends Plot Ranking Tkinter, PyQt, and Other Python GUI Libraries

The popularity of Tkinter largely stems from it being bundled with Python and, therefore, being the default Python GUI library. Most beginners will find this library first.

PyQt is often seen as the next logical step in your GUI journey when you want to start building real applications or commercial-quality software with Python.

Whether you choose Tkinter or PyQt will largely depend on your goals for writing GUI applications.

In this article, we'll explore and compare Tkinter and PyQt. We'll assess their pros & cons for different use cases and help you decide which library to use for your project.

The Tkinter GUI Library

Best for simple tool GUIs, small portable applications.

Tkinter is the default GUI library for Python. It comes bundled with Python on both Windows and macOS. On Linux, you may have to install additional packages to get it set up.

The library is a wrapper around the Tcl/Tk GUI toolkit. Its name is an amalgamation of the words Tk and Interface.

Tkinter supports standard layouts and basic widgets, as well as more complex widgets, such as tabbed views & progress bars. Tkinter is a pure GUI library rather than a GUI framework. It only provides a set of graphical components for building GUIs.

It doesn't provide support for GUIs-driven data sources, databases, or for displaying or manipulating multimedia or hardware. However, if you need to make something simple that doesn't require any additional dependencies, Tkinter may be what you are looking for.

Tkinter is a cross-platform GUI library. It was first released in 1990, and it has continued to evolve until today. For example, the addition of the Themed Tk extensions in 2009 improved the appearance of widgets, giving a native look & feel to your Tkinter GUIs.

Tkinter is a limited library with a friendly API (application programming interface) that makes it easy to understand and learn. Because of this, Tkinter tends to be the first choice for creating quick GUIs for relatively small Python programs.

To learn more about how to use Tkinter, check out Build your own desktop apps with Python & Tkinter.

How Do You Install Tkinter?

Tkinter comes installed by default with Python on macOS and Windows. On some Linux systems, you may need to install an additional package. Tkinter also runs fine on a Raspberry Pi, although depending on your distribution, you may need to install it first.

How Can You Write a Tkinter App?

A minimal Hello, World! GUI application in Tkinter will look something like this:

python import tkinter as tk # Create the app's main window window = tk.Tk() window.title("Hello, World!") def handle_button_press(): window.destroy() button = tk.Button(text="My simple app.", command=handle_button_press) button.pack() # Start the event loop window.mainloop()

In this example, we first import tkinter as tk, a common practice when using Tkinter. We then create our app's main window by instantiating the Tk class. The title() method allows us to give a descriptive title to the app's windows. Next we write a function to responding to a click on the button. In this case our method closes the application window & terminates it.

To create the button, we use the Button class with an appropriate text. In order to trigger our handle_button_press() function when the button is pressed, we pass the function to Button as the command argument.

Next, we call pack() on our button object. This organizes our widget into the window layout -- although in this case, there is only a single widget. Finally, we run the app by calling mainloop() on our window object.

If you run the example, you'll see a simple window like follows.

A Tkinter App Using Tk Standard Widgets

As you can see, by default, Tkinter has very rudimentary-looking widgets.

We can modify the example to use Themed Tk widgets, which give a native appearance to widgets. The changes are minor, adding a from tkinter import ttk import and using ttk.<Widget> when constructing widgets.

python import tkinter as tk from tkinter import ttk # Create the app's main window window = tk.Tk() window.title("Hello, World!") def handle_button_press(): window.destroy() button = ttk.Button(text="My simple app.", command=handle_button_press) button.pack() # Start the event loop window.mainloop()

If you run this example, the simple window will appear again but now using platform-native widgets (what you see will depend on your own system).

A Tkinter App Using Themed Tk Widgets

What Is Tkinter Commonly Used for?

Tkinter is typically used for simple GUIs. For example, small proof-of-concept, research, or educational apps built by Python hobbyists, and the like. The non-native appearance of Tkinter apps and lack of a real application framework make this library unsuitable for building professional applications, particularly commercial software.

While it's possible to put something simple together with Tkinter, as your application grows or your requirements become more complex, you'll likely end up spending more time reinventing the wheel than you saved by using a "simple" system. So, if you expect your project to grow in the future, then you should start with PyQt instead.

Is Tkinter Outdated?

The complaints of Tkinter being outdated largely stem from how the apps built with the library look on modern systems compared to other apps. Another important complaint driver is the library's limited set of widgets.

Tk, the library around which Tkinter was built, was first released in 1991. However, it's been continuously developed & maintained. In 2009, Tkinter added support for Tk 8.5 "Themed Tk" (Ttk), which allows Tk widgets to be more easily themed to look like the native on different desktop environments. Ttk also added some additional widgets, such as combo boxes, progress bars, tree views, notebooks, separators, and size grips, which weren't available in default Tk.

Why Does Tkinter Look Old-Fashioned?

It doesn't have to! Tkinter's reputation for looking bad stems from earlier versions of Tk without the native platform-theming support. These early versions used an old cross-platform "Motif" style, which was blocky and a bit ugly.

Tk 8.5 added support for desktop native theming, so your applications now follow the style of the desktop they are running on. This new look is provided through Themed Tk, which was added to Tkinter in 2009.

If you don't want a native appearance but still want to improve how your Tkinter applications look, there are other options too. For example, Tkinter Designer allows you to design graphical interfaces using Figma. Once you have the interface ready, then you can export it into a working Tkinter app.

How Can You Design GUIs for Tkinter?

There isn't an official design tool for Tkinter GUIs. However, you can find some tools available online. For example, Visual Tk allows you to build a GUI using a drag-drop interface in your browser & will then generate the Python code for you to create the interface in Tkinter itself.

Alternatively, Tkinter Designer will take a design drawn in the Figma software and use it to build a Tkinter-based GUI.

What Are Tkinter's Pros & Cons?

You can find plenty of reasons to consider Tkinter for your next project and reasons you may want to try something else. Let's take a look at some of the pros & cons of using Tkinter to build GUI applications with Python so that you can be in a better position to decide.

The pros of using Tkinter include the following:

• It is part of the Python standard library and comes installed by default (on Windows and macOS). Once you install Python, you're ready to start building GUIs with Tkinter.
• It doesn't have additional dependencies for your applications unless you need third-party libraries for additional functionality.
• It is relatively simple, meaning there isn't much to take in while learning to use it.
• It is a cross-platform library
• It has a lot of documentation and tutorials available.
• It can be used freely for commercial software because its license is the same as Python's.

On the other hand, the cons of using Tkinter include the following:

• It doesn't come with advanced widgets, such as data-driven views, database interfaces, vector graphics canvas, or multimedia GUI elements. To implement any of these in your applications, you need to write them yourself. This is achievable but will add to your development & maintenance burden.
• It has no official GUI designer application. You'll find a few third-party options available with varying degrees of completeness and flexibility.
• It lacks bundled features, which means that you're more likely to need third-party libraries to complete your applications. Luckily, there is no shortage of those in the Python ecosystem!
• It doesn't have a native look & feel by default. The default appearance of Tkinter applications is old-fashioned. However, this is largely fixed by the Themed Tk extensions.

Now that you have a better idea of what Tkinter is and what are its main features and limitations, it's time for you to know its closest competitor, PyQt.

The PyQt GUI Framework

Best for desktop applications, multimedia, scientific, and engineering software.

PyQt is a Python GUI framework built around the C++ Qt framework, which is developed and maintained by the Qt Company. It allows us to create modern interfaces that look right at home on any platform, including Windows, macOS, Linux, and even Android. It also has solid tooling, with the most notable being Qt Creator, which includes a WYSIWYG editor for designing GUI interfaces quickly and easily.

PyQt is developed and maintained by Riverbank Computing and was first released in 1998, four years after Tkinter.

The free-to-use version of PyQt is licensed under GNU General Public License (GPL) v3. This means that PyQt is limited to GPL-licensed applications unless you purchase its commercial license.

The Qt Company has its own Python binding for Qt, which is called PySide. This library was released in 2009. The main difference between PyQt and PySide is in licensing. PySide is licensed under GNU Lesser General Public License (LGPL), which means that you use PySide in non-GPL applications without any additional fee.

Qt, and by extension PyQt, is not just a GUI library. It's a complete GUI application development framework. In addition to standard GUI components, such as widgets and layouts, Qt provides:

• MVC (model-view-controller) data-driven views (spreadsheets, tables)
• Database interfaces and models
• Graph plotting
• Vector graphics visualization
• Multimedia playback
• Sound effects and playlists
• Interfaces for hardware, such as printing devices

Qt's signals and slots mechanism for event programming allows us to properly architect complex applications from reusable and isolated components.

To learn more about how to use PyQt, check out Build your own desktop apps with Python & PyQt6.

While other toolkits can work great when building small Python tools, PyQt really comes into its own for building commercial-quality applications where we benefit from the pre-built components. These benefits come at the expense of a steep learning curve but don't feel overwhelmed yet, you can just focus on the parts your project needs.

PyQt-based applications use platform-native widgets to ensure that they look and feel native on Windows, macOS, and Qt-based Linux desktop environments.

How Do You Install PyQt?

PyQt v6.4.2 is the latest version of the library. To run it on your computer, make sure you have Python v3.6.1 or later installed. To install it, just run the following command:

sh $ python -m pip install pyqt6

This command will install the PyQt6 library for your platform and your version of Python. The library will be automatically downloaded from PyPI.

If you want to use Qt's own official Python library, you can install PySide with python -m pip install pyside6

As of writing, only PyQt5 is currently supported on Raspberry Pi. But you can use both the Qt Widgets (standard) and QML/Qt Quick (declarative) APIs. You can use QML to build modern touchscreen interfaces with animations, transitions, and effects.

To install PyQt5 on a Raspberry Pi, run the following command:

sh $ sudo apt-get install python3-pyqt5

This command will install PyQt5 from your Raspberry Pi current repository. Note that the Qt framework will also be installed as a dependency.

For other installation options, see our complete installation guides.

How Can You Write a PyQt App?

A minimal Hello, World! GUI application in PyQt6, using the Qt Widgets is shown below:

python from PyQt6.QtWidgets import QApplication, QMainWindow, QPushButton # Create the app's main window class MainWindow(QMainWindow): def __init__(self): super().__init__() self.setWindowTitle("Hello, World!") button = QPushButton("My simple app.") button.pressed.connect(self.close) self.setCentralWidget(button) self.show() # Create the app, the main window, and run the app app = QApplication([]) window = MainWindow() app.exec()

In this example, we first import the required classes from PyQt6.QtWidgets. We then define the MainWindow class, which will provide the app's main window. The window will have a title and a button. This button is connected to the close() method inherited from QMainWindow.

Finally, we instantiated QApplication and MainWindow. To run the application, we call the exec() method on the app instance.

When run, you'll get a window with the title "Hello, World!" and containing a single push button that says "My simple app."

A Windowed App With a Push Button

That's a very simple example. To showcase the real capabilities of PyQt and the Qt framework, below is a more complex example consisting of a minimal but working web browser:

python from PyQt6.QtCore import QUrl from PyQt6.QtWebEngineWidgets import QWebEngineView from PyQt6.QtWidgets import QApplication, QMainWindow class MainWindow(QMainWindow): def __init__(self): super().__init__() self.browser = QWebEngineView() self.browser.setUrl(QUrl("https://www.google.com")) self.setCentralWidget(self.browser) self.show() app = QApplication([]) window = MainWindow() app.exec()

To make this code work, you need the Qt WebEngine extensions. This is a core part of Qt's library but is installed separately due to its size. Run the following to install this from the command line.

bash pip install PyQt6-WebEngine

With PyQt6-WebEngine installed, you can now use the QWebEngineView class in your own applications.

If you run the code above, you'll get a minimal web browser that loads up Google & lets you browse from there.

A Minimal Web Browser, Written in PyQt

For a final example, we'll create a quick video player. This example uses layouts to build a simple GUI with a viewer and a button. You can press the button to launch a platform native file picker dialog:

python from PyQt6.QtCore import QSize, QUrl from PyQt6.QtMultimedia import QMediaPlayer from PyQt6.QtMultimediaWidgets import QVideoWidget from PyQt6.QtWidgets import ( QApplication, QFileDialog, QPushButton, QVBoxLayout, QWidget, ) class Window(QWidget): def __init__(self): super().__init__() self.setWindowTitle("Video Player") self.viewer = QVideoWidget() self.viewer.setMinimumSize(QSize(800, 400)) self.player = QMediaPlayer() self.loadVideoBtn = QPushButton("Open video file...") self.loadVideoBtn.pressed.connect(self.openVideFile) layout = QVBoxLayout() layout.addWidget(self.viewer) layout.addWidget(self.loadVideoBtn) self.setLayout(layout) def openVideFile(self): filename, _ = QFileDialog.getOpenFileName( self, caption="Open video file", filter="MP4 Video (*.mp4)", ) if filename: video = QUrl(filename) self.player.setSource(video) self.player.setVideoOutput(self.viewer) self.player.play() app = QApplication([]) window = Window() window.show() app.exec()

When you run this example you'll get a very basic video player that's capable of loading and playing videos in MP4 format.

A Minimal MP4 Video Player, Written in PyQt

The above examples should give you an idea of how powerful PyQt actually is. As you can see, there is much more to it than just a set of simple GUI widgets.

For some more demo PyQt applications, see our library of examples.

What Is PyQt Commonly Used for?

PyQt is most commonly used for, and particularly well suited for, building full-featured GUI desktop applications. As you already learned, PyQt supports (MVC-like) data-driven views, vector graphics, animations & transitions, databases, and threading/concurrency.

Qt Designer, the GUI creator provided by Qt, allows you to build professional quality software in no time. The signals and slots mechanism makes it possible to properly decouple the components of an application, allowing for robust and maintainable system architectures.

You can also use PyQt for building touchscreen user interfaces for Raspberry Pi-powered hardware -- both using the Qt Widgets and QML/Qt Quick APIs. While PyQt can technically be used to build apps for mobile devices, this type of apps is rarely seen outside of the hobbyist space.

Can You Use PyQt for Open-Source and Commercial Apps?

Yes, you can! PyQt is free to use for personal development and GPL-licensed open-source software. For non-GPL software, such as commercially licensed software, a license is required from Riverbank Computing.

The Qt for Python framework ---PySide--- by the Qt Company is licensed under the LGPL. If you use PySide, then you don't need to license your software under the GPL (or LGPL). As a result, you don't have to share your application's source code. So, if you don't want to buy a commercial license for PyQt, then you can use PySide, which doesn't require a license for use in commercial software.

How Can You Design GUIs for PyQt Apps?

The Qt framework provides Qt Designer, which is a drag-drop UI editor. You can use Qt Designer to design modern and intuitive interfaces for your PyQt applications quickly. The interfaces generated using Qt Designer can be either loaded as-is in your applications or converted to Python code that you can then import into your apps.

On Windows, you can install Qt Designer and other Qt tools by using pip as follows:

bash $ python -m pip install pyqt6-tools

This command installs Qt Designer from PyPI. To run the GUI editor, you can execute the designer.exe app from your Script directory.

If you use PySide, then Qt Designer will be installed by default with the library.

Finally, you can also build GUIs from scratch using Python code. Whether you use Qt Designer or code is entirely up to you. The best choice will largely depend on your project.

What Are PyQt's Pros and Cons?

There are a number of reasons you may want to choose PyQt for your project and reasons you may not. Let's take a look at some of the pros and cons of using PyQt to build GUI applications with Python.

To kick things off, let's start with the pros:

• It is a powerful & modern GUI framework that is suitable for building professional applications.
• It includes several advanced widgets, including data-driven views, charts, database interfaces, vector graphics canvas, video playback, and a fully-functional web browser component.
• It can take advantage of Qt Designer, which allows you to design GUIs using a graphical drag-and-drop editor.
• It is cross-platform and can run on Windows, Linux, macOS, and mobile devices.
• It provides modern and native-looking GUI components out of the box in all the major platforms. These components can be largely customized if required.
• It is a batteries-included library, which means that you can accomplish many things with PyQt directly. This characteristic means less need for third-party dependencies.
• It has plenty of support and online learning resources, including our own complete PyQt tutorial.
• It allows the creation of touchscreen interfaces with the QML/Qt Quick API.

Even though PyQt has many neat features and advantages, it also has some cons:

• It can be intimidating to beginners. The size of the library and its complex feature set can make it overwhelming, to begin with.
• It has poor and incomplete Python documentation. As an alternative, you can use the official Qt documentation. However, this documentation is for the C++ library and can be hard to translate.
• It will take time to fully learn the framework and how it works.
• It allows for a GPL or commercial license only.

Up to this point, you've learned a lot about Tkinter and PyQt. To make your life more pleasant, the following section provides a quick and summarized comparison between these two GUI tools.

Tkinter vs. PyQt: A Feature Comparison

Now that we have a good understanding of Tkinter and PyQt, let's put them head-to-head on some key features:

Feature Tkinter PyQt Installation & Setup It's part of the Python installer on Windows and macOS. It may require additional packages on Linux. It needs to be installed separately using pip. It can also be installed from the source. License It uses the Python license. It uses GPL or a commercial license. GUI Builder Tools It has no standard GUI builder app, but some third-party tools are available. It can use Qt Designer for building GUIs. Widgets Set It has a basic set of widgets for most common applications. It has basic and advanced widgets for building quite complex interfaces. Application framework It's not a framework, so you'll have to use third-party Python libraries to implement many features. It's a full GUI framework that includes databases, plotting, vector graphics, threading, multimedia, networking, and more.

As you can see, PyQt is the most feature-rich of the two libraries. It is more capable of both building the GUI and the backend of your application. That said, if you're building simple GUI tools, then Tkinter may still be more than enough. It'll all depend on your project.

Decision Time: How to Choose the Best GUI Library for Your Project

So far, we've looked in detail at each library, seen some example code, and explored the pros and cons of using each library. We've also addressed some common questions that come up when looking at the two libraries.

By now, you have everything you need to decide which is best for your project. However, you may still be stuck on making the decision. In the following sections, we'll recap what we've learned about PyQt and Tkinter in the context of your own project and goals.

What Are Your Goals?

Which GUI library you pick is heavily dependent on your goals in writing GUI applications. If you are learning Python and just want to experiment, then Tkinter is a perfectly decent way to do that. On the other hand, if you are learning GUI development with a view to building professional applications, then it will make more sense to start with PyQt.

Qt, which PyQt is built on, is a complete GUI application framework that provides tools and components for building modern applications. This feature offloads a lot of the manual and repetitive work, helping you build well-architected systems.

Take, for example, loading a CSV file and displaying it in a table. In PyQt, you can write a small model to interface between the data source and the built-in table view. PyQt does all the work for you. It takes the data and displays it in an efficient way.

In Tkinter, you will need to create the table by yourself, widget by widget, in a grid layout. Then, you would have to use some external tool for loading the data from the CSV file. Finally, you would have to find a way to display the data in your custom Tkinter table.

In PyQt, you can have millions of rows in a table without problems using a single widget created to render the entire table. In Tkinter, each cell will be an individual widget object ---even those outside the view--- meaning you'll quickly encounter problems if you work with large data.

Do You Need a GUI Library or a GUI Framework?

Tkinter is a GUI library, while PyQt is a GUI framework. While both allow you to build graphical user interfaces, they differ in the scope of what they include and what parts of your application they help you build.

While a GUI library will help you add, position, and draw widgets in your application's GUI and hook those widgets up to your own code, a GUI framework provides additional functionalities, which are commonly required when building applications.

For example, PyQt provides components for connecting to databases and creating semi-automatic model-views of database entries. It provides a vector graphics canvas, plotting, 3D rendering, networking, threading/concurrency, and more.

Even though many of these features are already available in the Python standard library or in third-party libraries, most of them will not integrate as cleanly into a GUI program as those provided natively by the framework itself.

The trade-off is between using a single, big dependency (PyQt) vs. lots of small dependencies (standard-library or third-party) to build your apps. Using a framework can speed up the development of complex projects because much of what you need is available out of the box. As mentioned, whether you get any benefit from the PyQt framework will largely depend on your specific project.

Is Tkinter Easier to Learn Than PyQt?

It can be. Previously, there was a lack of beginner tutorials available for PyQt. This condition made it difficult to get started. That's no longer the case. Now you'll find lots of PyQt5 and PyQt6 tutorials available online.

Our own PyQt6 tutorial takes you from the absolute basics of GUI development concepts to building relatively complex applications. We're adding new tutorials regularly, covering both basic and advanced topics.

There are more beginner-friendly tutorials available for Tkinter than PyQt. Basic Tkinter examples tend to avoid using subclasses, while PyQt examples default to using them, which can imply more reasoning if you're not familiar with object-oriented programming and inheritance.

PyQt introduces additional and complex concepts, such as signals and slots, which can be confusing for new developers. However, they are one of the most powerful parts of the framework, making it possible to build well-architected and maintainable software. The time taken to understand these things is well rewarded.

Which One Should You Learn First, Tkinter or PyQt?

There is little benefit in starting with Tkinter if you plan to switch to PyQt later. While PyQt is a large framework with thousands of classes and features, you don't need to learn all of it at once. While some of the basic GUI concepts you learn with Tkinter -- widgets, layouts, event-based programming -- will transfer over to PyQt, there are many other concepts that won't.

As a developer, you'll be looking for tools that are easy to use, well-designed for the job, and at the same time, can grow with your projects. If you want to move on to develop professional or commercial software with Python, you don't want to start over again with an entirely new stack. If you think you are likely to want to migrate to PyQt later, then you may as well start there first.

What Next?

Now you have enough knowledge to make an informed decision between using PyQt or Tkinter for your GUI projects. On Python GUIs, we have lots of tutorials to help you take the next step of actually building something great!

Our PyQt6 tutorial is pretty complete and takes you from basic examples to complex applications. This way, you'll get the required skills to build multimedia, scientific, and engineering software at a professional level.

On the other hand, if you're just looking to create simple GUIs for your automation and utility tools, then our Tkinter tutorial will give you the basics you need to get a window on the screen and start experimenting.

In any case, you will have fun building GUIs!

Introduction

Python provides some very convenient ways to work with file-like objects, including the with feature. But what if we need to open multiple files in this way? Your could wouldn't exactly be "clean" if you had a bunch of nested with open statements. In this article, we'll show you how to open multiple files using the with open statement in Python.

What is 'with' in Python?

The with statement in Python is used in exception handling to make the code cleaner and much easier to understand. It simplifies the management of common resources like file streams. Let's take a quick look at how it works:

with open('example.txt', 'r') as file: data = file.read()

In the above code snippet, with open('example.txt', 'r') as file: is the with statement we're referring to. The open() function is used to open a file, and 'example.txt' is the name of the file we want to open. 'r' is the mode in which we want to open the file. In this case, it's read mode. The as keyword is used to create an alias - in this case, file.

The code then reads the file example.txt and assigns its content to the variable data. One of the benefits of using with is that it automatically takes care of closing the file once it's no longer needed, even if exceptions were raised. This makes it much easier to handle files manually.

Note: The with statement is not only used for opening files, but can also be used with other objects like threads, locks, and network connections. The main idea is to manage resources that can be open and closed.

The with open statement is similar to the following:

try: file = open('example.txt', 'r') data = file.read() finally: file.close()

This code is more verbose and you're more likely to forget to close the file. Using with is a more Pythonic way of handling files.

Why Open Multiple Files at Once?

There are a few reasons why you might want to open multiple files at once in Python. One of the most common scenarios is when you're working with a large dataset that's been split across multiple files. Instead of opening, reading, and closing each file one by one, you can streamline the process by opening all the files at once, reading in the data, and then closing the files.

This not only makes your code more efficient, but it's also cleaner and easier to read. Plus, it saves you from the potential headache of having to keep track of which files you've already opened/closed and which ones you haven't.

Wait! While opening multiple files at once can make your code more efficient, it can also consume more memory. So just be careful when working with a large number of files or very large files.

Opening Multiple Files

The basic method of opening multiple files in Python involves using the with open() function in combination with Python's built-in zip() function. Here's how you can do it:

with open('file1.txt', 'r') as file1, open('file2.txt', 'r') as file2: for line1, line2 in zip(file1, file2): print(line1.strip(), line2.strip())

In this code, we're opening two files, 'file1.txt' and 'file2.txt', for reading. The with open() function will help us make sure that the files are properly closed after they're no longer needed. We use the zip() function to read the lines from both files simultaneously, which is nice to have if the files are related in some way (for example, if one file contains questions and the other contains answers).

The output of this code will be the lines from 'file1.txt' and 'file2.txt' printed side by side.

This is just a basic method of opening multiple files in Python. There are other ways to do it that might be better suited to your specific use case, like using list comprehension or opening the files in a loop.

One thing to point out is that as of Python 3.10, you can now group the multiple open calls in parentheses and on multiple lines, which may help with readability:

with ( open('file1.txt', 'r') as file1, open('file2.txt', 'r') as file2 ): for line1, line2 in zip(file1, file2): print(line1.strip(), line2.strip()) Using 'with open' in a Loop

Another way to open multiple files is to just use a loop and open each file one at a time so that no two files are open simultaneously. This can be helpful if you're working with very large files and can't have multiple files open due to memory constraints.

filenames = ['file1.txt', 'file2.txt', 'file3.txt'] for file in filenames: with open(file, 'r') as f: print(f.read())

Here, we loop over our list of filenames. For each one, we use with open to open the file and assign it to the variable f. We then print out the contents of the file. Once the with block is exited, the file is automatically closed.

Using List Comprehension to Open Multiple Files

List comprehension is a popular feature in Python that allows you to create lists in a very readable and compact way. It can be used to open multiple files in a single line of code. Note that this method does not use with, so you'll need to manually close the files and handle errors, but it still might be helpful in some cases.

filenames = ['file1.txt', 'file2.txt', 'file3.txt'] files = [open(file) for file in filenames]

In this example, we have a list of filenames. We use list comprehension to iterate over the list, opening each file, and storing the resulting file object in a new list, files.

Handling Errors and Issues When Opening Multiple Files Using with

When working with files in any programming language, it's common to have to handle errors since so much can go wrong. Python's with keyword and try/except blocks can work together to better handle these errors

Let's take a look at an example:

filenames = ["file1.txt", "file2.txt", "nonexistent.txt", "file3.txt"] for name in filenames: try: with open(name, 'r') as f: print(f.read()) except FileNotFoundError: print(f"{name} not found.") except PermissionError: print(f"You don't have permission to read {name}.")

In this example, we attempt to open four files with a loop. We've wrapped the file opening operation inside a with block which is itself enclosed in a try/except block. This way, if the file doesn't exist, a FileNotFoundError is caught and a custom error message is printed. We use it this way to specifically handle certain scenarios and letting with handle the closing of the file, if it was ever opened.

Output:

$ python open_multiple_files_with_with.py ...contents of file1.txt... ...contents of file2.txt... nonexistent.txt not found. ...contents of file3.txt... Conclusion

In this article, we've explored various ways to open multiple files using with open in Python. We started with the basic method of opening multiple files, and then moved on to slightly more advanced techniques like using loops and list comprehension. We also briefly touched on how to handle errors and issues when opening multiple files.

Unlocking Python’s true potential in terms of speed through shared-memory parallelism has traditionally been limited and challenging to achieve. That’s because the global interpreter lock (GIL) doesn’t allow for thread-based parallel processing in Python. Fortunately, there are several work-arounds for this notorious limitation, which you’re about to explore now!

In this tutorial, you’ll learn how to:

• Run Python threads in parallel on multiple CPU cores
• Avoid the data serialization overhead of multiprocessing
• Share memory between Python and C runtime environments
• Use different strategies to bypass the GIL in Python
• Parallelize your Python programs to improve their performance
• Build a sample desktop application for parallel image processing

To get the most out of this advanced tutorial, you should understand the difference between concurrency and parallelism. You’ll benefit from having previous experience with multithreading in programming languages other than Python. Finally, it’s best if you’re eager to explore uncharted territory, such as calling foreign Python bindings or writing bits of C code.

Don’t worry if your knowledge of parallel processing is a bit rusty, as you’ll have a chance to quickly refresh your memory in the upcoming sections. Also, note that you’ll find all the code samples, image files, and the demo project from this tutorial in the supporting materials, which you can download below:

Get Your Code: Click here to download the free sample code that shows you how to bypass the GIL and achieve parallel processing in Python.

Recall the Fundamentals of Parallel Processing

Before dipping your toes into specific ways of bypassing the GIL in Python, you might want to revisit some related topics. Over the following few sections, you’ll familiarize yourself with different computer processing models, task types, abstractions over modern CPUs, and some history. If you already know this information, then feel free to jump ahead to the classic mechanism for parallelization in Python.

What’s Parallel Processing?

Under Flynn’s taxonomy, the most common types of parallel processing allow you to run the same (SIMD) or different code fragments (MIMD) in separate execution streams at the same time:

Parallel Execution of Tasks

Here, two independent tasks or jobs execute alongside each other. To run more than one piece of code simultaneously like this, you need a computer equipped with a central processing unit (CPU) comprising multiple physical cores, which is the norm nowadays. While you could alternatively access a cluster of geographically distributed machines, you’ll consider only the first option in this tutorial.

Parallel processing is a particular form of concurrent processing, which is a broader term encompassing context switching between multiple tasks. It means that a currently running task might voluntarily suspend its execution or be forcibly suspended to give a slice of the CPU time to another task:

Concurrent Execution of Tasks

In this case, two tasks have been sliced into smaller and intertwined chunks of work that share a single core of the same processing unit. This is analogous to playing chess against multiple opponents at the same time, as shown in one of the scenes from the popular TV miniseries The Queen’s Gambit. After each move, the player proceeds to the next opponent in a round-robin fashion, trying to remember the state of the corresponding game.

Note: Context switching makes multitasking possible on single-core architectures. However, multi-core CPUs also benefit from this technique when the tasks outnumber the available processing power, which is often the case. Therefore, concurrent processing usually involves spreading the individual task slices over many CPUs, combining the power of context switching and parallel processing.

While it takes time for people to switch their focus, computers take turns much quicker. Rapid context switching gives the illusion of parallel execution despite using only one physical CPU. As a result, multiple tasks are making progress together.

Because of time-sharing, the total time required to finish your intertwined tasks running concurrently is longer when compared to a genuinely parallel version. In fact, context switching has a noticeable overhead that makes the execution time even worse than if you run your tasks one after another using sequential processing on a single CPU. Here’s what sequential processing looks like:

Sequential Execution of Tasks

With sequential processing, you don’t start another task until the previous one finishes, so you don’t have the costs of switching back and forth. This situation corresponds to playing an entire chess game with one opponent before moving on to the next one. Meanwhile, the remaining players must sit tight, patiently waiting for their turn.

On the other hand, playing multiple games concurrently can maximize your throughput. When you prioritize games with players who make quick decisions over those who need more time to think, then you’ll finish more games sooner. Therefore, intertwining can improve the latency, or response times, of the individual tasks, even when you only have one stream of execution.

As you can probably tell, choosing between parallel, concurrent, and sequential processing models can feel like plotting out your next three moves in chess. You have to consider several factors, so there’s no one-size-fits-all solution.

Whether context switching actually helps will depend on how you prioritize your tasks. Inefficient task scheduling can lead to starving shorter tasks of CPU time.

Additionally, the types of tasks are critical. Two broad categories of concurrent tasks are CPU-bound and I/O-bound ones. The CPU-bound tasks will only benefit from truly parallel execution to run faster, whereas I/O-bound tasks can leverage concurrent processing to reduce latency. You’ll learn more about these categories’ characteristics now.

How Do CPU-Bound and I/O-Bound Tasks Differ? Read the full article at https://realpython.com/python-parallel-processing/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Introduction

Python, somewhat similar ot Node, uses a system of installed modules/packages. But as you continue to install more and more modules, it might get a bit tricky to keep track of all of them. In this Byte, we'll explore how to get a list of all locally installed Python modules, which can be very helpful in managing your Python environment.

Python Modules

A Python module is basically just a file (or directory of files) containing Python definitions and statements. These modules can define functions, classes, and variables that can be utilized in other Python code. It's like a toolkit filled with various tools, each designed for a specific task. Python comes with a standard library of modules, but the beauty of Python actually lies in the vast array of 3rd party modules that are available. These can be installed locally and used as needed.

For instance, if you're working on a web scraping project, you might install the BeautifulSoup module. Or, if you're dealing with data analysis, the pandas module is popular for many data-related tasks.

How to List Installed Python Modules

Python provides a few different ways to list all the modules installed in your local environment. The most common method is using the pip command, which is the standard package manager for Python.

Using pip list

The pip list command is a quick way to see all the Python modules installed in your current environment. Open up your terminal or command line interface and type in the following command:

$ pip list

This will provide you with a list of all installed packages, along with their respective versions.

Package Version --------------- ------- beautifulsoup4 4.9.3 numpy 1.19.5 pandas 1.1.5 pip 21.0.1 setuptools 54.1.2

Note: Remember that the list of modules you see will be specific to your current Python environment. If you're using a virtual environment, only the modules installed in that environment will be shown.

That's it for the pip list command. It's a pretty simple way to get a quick overview of your installed Python modules. In the next section of this Byte, we'll see another useful command, pip freeze, and see how it differs from pip list.

Using pip freeze

pip freeze is another command that you can use to list all installed Python modules. But unlike pip list, pip freeze returns the list of modules in a format that pip can consume. This means each line of the output is a valid argument for pip install.

Let's run pip freeze in the command line and see what it gives us:

$ pip freeze

This will return something like:

asn1crypto==0.24.0 certifi==2018.1.18 cffi==1.11.5 chardet==3.0.4 cryptography==2.1.4 idna==2.6 pycparser==2.18 PySocks==1.6.8 requests==2.18.4 six==1.11.0 urllib3==1.22

Each line in the output is a module along with its installed version. This is super helpful when you need to replicate your environment elsewhere or create your requirements.txt file.

Differences Between pip list and pip freeze

Now that you've seen both pip list and pip freeze in action, you might be wondering what the difference is, other than some simple formatting.

And the main difference between the two commands really is just the formatting of their output. pip list outputs a slightly more human-readable format of installed packages, which is great when you're quickly checking what's installed. On the other hand, pip freeze outputs a list of packages in a format that pip can use in other commands. This is particularly useful when you want to replicate your environment, as you can simply redirect the output of pip freeze to a requirements file, then use pip install -r requirements.txt on another machine to install the same packages.

Note: Both pip list and pip freeze will list all installed packages, regardless of where they were installed from. This includes packages installed via pip, setup.py, and other package managers.

Listing Modules in a Virtual Environment

Working in a virtual environment can help you manage dependencies for different Python projects and keep them separate. When you activate a virtual environment, pip list and pip freeze will only show the packages installed in that environment.

To illustrate, let's create a new virtual environment and install a package:

$ python3 -m venv myenv $ source myenv/bin/activate (myenv) $ pip install requests

Now, if we run pip list or pip freeze, only the requests package and its dependencies will be listed:

(myenv) $ pip list Package Version ---------- ------- certifi 2021.5.30 chardet 4.0.0 idna 2.10 requests 2.25.1 urllib3 1.26.6 Conclusion

To sum up, Python provides a few ways to list all installed modules, whether you're working in a global or virtual environment. pip list and pip freeze are two commands that not only list the installed packages but also provide additional info, namely the version of the packages. Remember that the list of installed packages can vary depending on your active environment, so always double-check which environment you're in before installing or listing packages.

In the spirit of my Previously Unpublicised Code series, today I’m going to share Pandas-FSDR. This is a simple library with one function which finds significant differences between two columns in a pandas DataFrame.

For example, imagine you had the following data frame:

Subject UK World Biology 50 40 Geography 75 80 Computing 100 50 Maths 1500 1600

You may be interested in the differences between the values for the UK and the World (these could be test scores or something similar). Pandas-FSDR will tell you – by running one function you can get output like this:

• Maths is significantly smaller for UK (1500 for UK compared to 1600 for World)
• Computing is significantly larger for UK (100 for UK compared to 50 for World)

Differences are calculated in absolute and relative terms, and all thresholds can be altered by changing parameters to the function. The function will even output pre-formatted Markdown text for display in an IPython notebook, inclusion in a dashboard or similar. The output above was created by running this code:

result = FSDR(df, 'UK', 'World', rel_thresh=30, abs_thresh=75)

This is a pretty simple function, but I thought it might be worth sharing. I originally wrote it for some contract data science work I did years ago, where I was sharing the output of Jupyter Notebooks with clients directly, and wanted something that would ‘write the text’ of the comparisons for me, so it could be automatically updated when I had new data. If you don’t want it to write anything then it’ll just output a list of row indices which have significant differences.

Anyway, it’s nothing special but someone may find it useful.

The code and full documentation are available in the Pandas-FSDR Github repository

#594 – SEPTEMBER 12, 2023
View in Browser »

Playing With Genetic Algorithms in Python

A Genetic Algorithm (GA) is an AI technique where random code is mutated and tested for fitness iteratively until a solution is found. This article shows you a couple of problems solved using GAs in Python. Associated HN discussion.
JOSEP RUBIÓ PIQUÉ

Generate Beautiful QR Codes With Python

In this tutorial, you’ll learn how to use Python to generate QR codes, from your standard black-and-white QR codes to beautiful ones with your favorite colors. You’ll learn how to format QR codes, rotate them, and even replace the static background with moving images.
REAL PYTHON

Finally—Pandas Practice That Isn’t Boring

You won’t get fluent with Pandas doing boring, irrelevant, toy exercises. Bamboo Weekly poses questions about current events, using real-world data sets—and offers clear, comprehensive solutions in Jupyter notebooks. Challenge yourself, and level up your Pandas skills every Wednesday →
BAMBOO WEEKLY sponsor

I’m Mr. Null. My Name Makes Me Invisible to Computers

NULL is a magic word in many computer languages. This article is by someone who has Null as a last name, and the consequences that entails. See also this Radiolab Podcast Episode for a similar topic.
CHRISTOHPER NULL

2023 Django Developers Survey

DJANGO SOFTWARE FOUNDATION

Python 3.12.0 Release Candidate 2 Available

CPYTHON DEV BLOG

Pandas 2.1.0 Released

PYDATA.ORG

Discussions Why Prefer Indentation Over Block Markers?

STACKEXCHANGE.COM

Articles & Tutorials Launching an HTTP Server in One Line of Python Code

In this tutorial, you’ll learn how to host files with a single command using an HTTP server built into Python. You’ll also extend it by making a miniature web framework able to serve dynamic content from HTML templates. Along the way, you’ll run CGI scripts and use encryption over HTTPS.
REAL PYTHON

Introducing flake8-logging

The Python standard library’s logging module is a go-to for adding observability to applications, but there are right and wrong ways to use it. This article is about a new linter that explicitly looks for problems with your logging calls.
ADAM JOHNSON

Fully Managed Postgres + Great Support

Crunchy Bridge is a different support experience. Our team is passionate about Postgres education giving you all the information and tools to make informed choices. If you’re a developer starting out or a more seasoned expert in databases, you’ll appreciate thorough, timely, and in-depth responses →
CRUNCHY DATA sponsor

Apple Vision Framework via PyObjC for Text Recognition

Learn how to use PyObjC to interface with the Apple Vision Framework and create a script to detect text in images. Become familiar with how PyObjC works and how it maps functions and methods from Objective C to Python.
YASOOB KHALID • Shared by Yasoob Khalid

Class Concepts: Object-Oriented Programming in Python

Python uses object-oriented programming to group data and associated operations together into classes. In this video course, you’ll learn how to write object-oriented code with classes, attributes, and methods.
REAL PYTHON course

Switching to Hatch

Oliver used Poetry for most of his projects, but recently tried out Hatch instead. This blog post covers what it took to get things going and what features he used, including how he ditched tox.
OLIVER ANDRICH

The Python Dictionary Dispatch Pattern

The dictionary dispatch pattern is when you keep references to functions in a dictionary and change code behavior based on keys. Learn how to use this pattern in practice.
JAMES GALLAGHER

Analysing and Parsing the Contents of PyPI

High-level statistics gathered from PyPI, including how popular language features are, project sizes (tensorflow accounts for 16% of the data on PyPI!) and growth.
TOM FORBES • Shared by Tom Forbes

Writing a C Compiler in 500 Lines of Python

This post details how to build a C compiler, step-by-step, using Python. A great intro to compilers. The target source is WASM, so learn a bit about that too.
THEIA VOGEL

Filters in Django: filter(A, B) vsfilter(A).filter(B)

An advanced dive into the Django ORM, how it handles joins, and what that means for your code.
APIROBOT.ME • Shared by Denis

My Favorite Python Tricks for LeetCode Questions

A collection of intermediate-level Python tricks and tools. Write more Pythonic code!
JJ BEHRENS

What Is Wrong With TOML?

Some YAML people talk about why TOML is too limited.
HITCHDEV.COM

Projects & Code StrictYAML: Type-Safe, Restricted Subset of the YAML

HITCHDEV.COM

dara: Create Interactive Web Apps in Pure Python

GITHUB.COM/CAUSALENS

JobSpy: Scraper for LinkedIn, Indeed & ZipRecruiter

GITHUB.COM/CULLENWATSON

krypton: Data Encryption at Rest and IAM for Python

GITHUB.COM/KRPTN

iommi: Your First Pick for a Django Power Chord

GITHUB.COM/IOMMIROCKS

Events Weekly Real Python Office Hours Q&A (Virtual)

September 13, 2023
REALPYTHON.COM

PyData Amsterdam 2023

September 14 to September 17, 2023
PYDATA.ORG

Python Atlanta

September 14 to September 15, 2023
MEETUP.COM

Kiwi PyCon 2023

September 15 to September 18, 2023
KIWIPYCON.NZ

PyCon CZ 2023

September 15 to September 18, 2023
PYCON.ORG

PyData Seattle: Language Creators Charity Fundraiser

September 19 to September 20, 2023
PYDATA.ORG

PyCon Uganda

September 21 to September 24, 2023
PYCON.ORG

PyCon UK 2023

September 22 to September 26, 2023
PYCONUK.ORG

Happy Pythoning!
This was PyCoder’s Weekly Issue #594.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

In most systems I come across, private keys are all over the place, made available as secrets to the workloads that need them. The trouble is not just that sometimes, secrets are not really secret, but more fundamentally, that private keys are something that we don't need to handle directly at all, they're basically too hot to handle 🔥.

Instead, we can use a remote service to manage our private keys such as Azure Key Vault, obviating the need to handle them ourselves.

Ideally, the keys managed by the service are in fact generated there, too, such that at no point in time is the key ever available to you. We can ask the service to sign or decrypt some data, but the actual key is never revealed.

But before we talk about how we can switch to such managed keys, let's look at a few example use-cases:

• Authentication

  Lots of systems are integrated using public/private key pairs. For example, when we commit to GitHub, many developers use SSH to connect, using key pair-based authentication; or when we connect to a database, the authentication protocol might use key pair-based authentication instead of a password, for an extra layer of transport security.

• Session signing

  Most web applications require the signing of session cookies, a token that proves to the web application that you're logged in, or simply that you have an "open session". Typically, this is configured using a locally available signing key, but again, we can improve security by using a managed key (and it's possible to be clever here and amortize the cost of using the managed key over a period of time or number of sessions created).

The operation we need in both of these cases is the ability to sign a payload. Managed keys are perfect for this!

The Snowflake client library for Python has an authentication system that supports multiple methods. Working on a data platform solution for Grundfos, a Danish company and also the world's largest pump manufacturer, I contributed a change to extend this authentication system to accomodate managed keys (the existing system supported only concrete private keys such as a file on disk).

For Python, the cryptography package has become the defacto standard for encryption. The RSAPrivateKey interface represents an abstract private key that uses traditional RSA asymmetric encryption.

With the change linked above, it's now possible to implement a custom private key object which defers operations to a managed key service since the Snowflake library now takes bytes | RSAPrivateKey as input.

I'm collaborating with Microsoft to provide this functionality out of the box with the azure-keyvault-keys package, a feature that's likely to go into the next release. In code, this would look like the following:

from azure.keyvault.keys import KeyClient from azure.keyvault.keys.crypto import ManagedRsaKey from snowflake.connector import connect key_client = KeyClient( "https://<keyvault-name>.vault.azure.net/", credential=AZURE_CREDENTIAL ) ctx = connect( # Here goes the usual connection parameters ... private_key=ManagedRsaKey(KEY_NAME, key_client) )

The prerequisite to have this work is to use ALTER USER ... SET RSA_PUBLIC_KEY = '<key-data>'. You can download the public key in PEM-format (suitable as key data here) using the portal or Azure CLI:

$ az keyvault key download \ --vault-name <keyvault-name> \ -n <key-name> \ --encoding PEM \ -f /dev/stdout

Using managed keys is not free. There is a cost for each operation, but for Azure Key Vault for example, using standard RSA 2048-bit keys, these operations are as cheap as reading a secret. As mentioned previously, in some cases, we can be clever and use a surrogate signing key for a period of time. This also reduces the operational dependence on the remote service.

These costs (and infrastructure complexity) should be easily offset by the much lessened burden of compliance with security protocol. If you never saw a private key, there is no risk having compromised it. Access to using a managed key can be granted on a higher level and revoked just as easily.

Introduction

Python is not necessarily known for its speed, but there are certain things that can help you squeeze out a bit more performance from your code. Surprisingly, one of these practices is running code in a function rather than in the global scope. In this article, we'll see why Python code runs faster in a function and how Python code execution works.

Python Code Execution

To understand why Python code runs faster in a function, we need to first understand how Python executes code. Python is an interpreted language, which means it reads and executes code line by line. When Python executes a script, it first compiles it to bytecode, an intermediate language that's closer to machine code, and then the Python interpreter executes this bytecode.

def hello_world(): print("Hello, World!") import dis dis.dis(hello_world) 2 0 LOAD_GLOBAL 0 (print) 2 LOAD_CONST 1 ('Hello, World!') 4 CALL_FUNCTION 1 6 POP_TOP 8 LOAD_CONST 0 (None) 10 RETURN_VALUE

The dis module in Python disassembles the function hello_world into bytecode, as seen above.

Note: The Python interpreter is a virtual machine that executes the bytecode. The default Python interpreter is CPython, which is written in C. There are other Python interpreters like Jython (written in Java), IronPython (for .NET), and PyPy (written in Python and C), but CPython is the most commonly used.

Why Python Code Runs Faster in a Function

Consider a simplified example with a loop that iterates over a range of numbers:

def my_function(): for i in range(100000000): pass

When this function is compiled, the bytecode might look something like this:

SETUP_LOOP 20 (to 23) LOAD_GLOBAL 0 (range) LOAD_CONST 3 (100000000) CALL_FUNCTION 1 GET_ITER FOR_ITER 6 (to 22) STORE_FAST 0 (i) JUMP_ABSOLUTE 13 POP_BLOCK LOAD_CONST 0 (None) RETURN_VALUE

The key instruction here is STORE_FAST, which is used to store the loop variable i.

Now let's consider the bytecode if the loop is at the top level of a Python script:

SETUP_LOOP 20 (to 23) LOAD_NAME 0 (range) LOAD_CONST 3 (100000000) CALL_FUNCTION 1 GET_ITER FOR_ITER 6 (to 22) STORE_NAME 1 (i) JUMP_ABSOLUTE 13 POP_BLOCK LOAD_CONST 2 (None) RETURN_VALUE

Notice the STORE_NAME instruction is used here, rather than STORE_FAST.

The bytecode STORE_FAST is faster than STORE_NAME because in a function, local variables are stored in a fixed-size array, not a dictionary. This array is directly accessible via an index, making variable retrieval very quick. Basically, it's just a pointer lookup into the list and an increase in the reference count of the PyObject, both of which are highly efficient operations.

On the other hand, global variables are stored in a dictionary. When you access a global variable, Python has to perform a hash table lookup, which involves calculating a hash and then retrieving the value associated with it. Though this is optimized, it's still inherently slower than an index-based lookup.

Benchmarking and Profiling Python Code

Want to test this for yourself? Try benchmarking and profiling your code.

Benchmarking and profiling are important practices in performance optimization. They help you understand how your code behaves and where the bottlenecks are.

Benchmarking is where you time your code to see how long it takes to run. You can use Python's built-in time module, as we'll show later, or use more sophisticated tools like timeit.

Profiling, on the other hand, provides a more detailed view of your code's execution. It shows you where your code spends most of its time, which functions are called, and how often. Python's built-in profile or cProfile modules can be used for this.

Here's an example of how you can profile your Python code:

import cProfile def loop(): for i in range(10000000): pass cProfile.run('loop()')

This will output a detailed report of all the function calls made during the execution of the loop function.

Note: Profiling adds quite a bit of overhead to your code execution, so the execution time shown by the profiler will be longer than the actual execution time.

Benchmarking Code in a Function vs. Global Scope

In Python, the speed of code execution can vary depending on where the code is executed - in a function or in the global scope. Let's compare the two using a simple example.

Consider the following code snippet that calculates the factorial of a number:

def factorial(n): result = 1 for i in range(1, n + 1): result *= i return result print(factorial(20))

Now let's run the same code but in the global scope:

n = 20 result = 1 for i in range(1, n + 1): result *= i print(result)

To benchmark these two pieces of code, we can use the timeit module in Python, which provides a simple way to time small bits of Python code.

import timeit def benchmark(): start = timeit.default_timer() # your code here end = timeit.default_timer() print(end - start) # benchmark function code benchmark(factorial(20)) # benchmark global scope code benchmark(result)

You'll find that the function code executes faster than the global scope code. This is because Python executes function code faster due to a variety of reasons we'll discuss in other sections.

Profiling Code in a Function vs. Global Scope

Python provides a built-in module called cProfile for this purpose. Let's use it to profile our factorial code in a function and in the global scope.

import cProfile def profile(): pr = cProfile.Profile() pr.enable() # your code here pr.disable() pr.print_stats() # profile function code profile(factorial(20)) # profile global scope code profile(result)

From the profiling results, you'll see that the function code is more efficient in terms of time and space. This is due to the way Python's interpreter, CPython, handles function calls, which we'll discuss in the next section.

Optimizing Python Function Performance

Given that Python functions tend to run faster than equivalent code in the global scope, it's worth looking into how we can further optimize our function performance.

Of course, because of what we saw earlier, one strategy is to use local variables instead of global variables. Here's an example:

import time # Global variable x = 5 def calculate_power_global(): for i in range(1000000): y = x ** 2 # Accessing global variable def calculate_power_local(x): for i in range(1000000): y = x ** 2 # Accessing local variable start = time.time() calculate_power_global() end = time.time() print(f"Execution time with global variable: {end - start} seconds") start = time.time() calculate_power_local(x) end = time.time() print(f"Execution time with local variable: {end - start} seconds")

In this example, calculate_power_local will typically run faster than calculate_power_global, because it's using a local variable instead of a global one.

Another optimization strategy is to use built-in functions and libraries whenever possible. Python's built-in functions are implemented in C, which is much faster than Python. Similarly, many Python libraries, such as NumPy and Pandas, are also implemented in C or C++, making them faster than equivalent Python code.

For example, consider the task of summing a list of numbers. You could write a function to do this:

def sum_numbers(numbers): total = 0 for number in numbers: total += number return total

However, Python's built-in sum function will do the same thing, but faster:

numbers = [1, 2, 3, 4, 5] total = sum(numbers)

Try timing these two code snippets yourself and figure out which one is faster!

Conclusion

In this article, we've explored the interesting world of Python code execution, specifically focusing on why Python code tends to run faster when encapsulated in a function. We briefly looked into the concepts of benchmarking and profiling, providing practical examples of how these processes can be carried out in both a function and the global scope.

We also discussed a few ways to optimize your Python function performance. While these tips can certainly make your code run faster, you should use certain optimizations carefully as it's important to balance readability and maintainability with performance.

Everything you need to get started with pytest and use it effectively. Learn about: Test Functions Structure functions effectively Fixtures setup, teardown, and so much more Builtin Fixtures many common testing tasks are pre-built and ready to use Parametrization turn one test into many test cases Markers Builtin markers provided by pytest Custom markers for test selection Combining markers and fixtures to alter pytest behavior Back To School Special To celebrate wrapping up the first course, I’m offering pytest Primary Power for $49 for a limited time, and the bundle for $99.