Planet Python

DjangoCon US 2023 is just about here!

While the conference isn’t until Oct 16-20, we’re sharing this now because discounted hotel rooms are filling up fast and will go away entirely on September 15th.

If you can’t make it in person this year, we also have online tickets: https://ti.to/defna/djangocon-us-2023.

All talks will be available online via the LoudSwarm platform, so you won’t miss any action. We have eleven exclusive talks for our online attendees, so you get more content. Online tickets are a great and affordable way to access the treasure trove of knowledge being shared without making a larger time or financial commitment.

Tutorials are virtual on October 8th

This year’s conference will have the tutorials the week before the conference. There wasn’t space at the venue to hold them in-person. You can find the full schedule on our website: https://2023.djangocon.us/tutorials/. This year, we are excited to have tutorials ranging from mastering test driven development to supercharging your Django development environment with VS Code and dev containers.

The In-person DjangoCon US Experience

On Monday, October 16th, we kick off with the main conference schedule packed full of talks. Again, that full schedule is live so that you can start planning which room you will be in during the event: https://2023.djangocon.us/talks/

Two particular talks we’re excited about are Natalia Bidart’s talk “Inside Out: My Journey of Understanding Inclusion” and Thibaud Colas’ talk “Django’s accessibility track record”.

After two days of talks, on Wednesday the 18th, we will have a full day’s worth of deep-dive sessions, ending with a career development panel. Finally, on Thursday and Friday, we will be hosting two full days of sprints where anyone can get involved making contributions to the Django ecosystem: https://2023.djangocon.us/sprints/. Note that sprints are included in the main ticket prices, but we do ask that you register ahead of time, so we know how many folks will be coming to each day’s session: https://ti.to/defna/djangocon-us-2023

Beyond The Talks and Sprints

There is so much happening at DjangoCon US beyond the main conference agenda. We will have board game sessions, impromptu meetups, breakout sessions for those interested in various topics, and so much more! All of the things that are traditionally very difficult to do online, we plan to facilitate in-person so that attendees get the most out of their conference experience.

Plus, you can explore Durham, North Carolina with the great people you meet here. We think you’ll find it a diverse, welcoming spot with lots to do within walking distance of the conference!

We are incredibly excited to see everyone online and in person. We hope that you will join us at this year’s conference!

Introduction

In Python, you may have come across a string literal prefixed with the 'b' character and wondered what it means. This Byte aims to shed light on this feature of Python strings, its connection with Unicode and binary strings, and how it can be used in your code.

Binary Strings in Python

The 'b' prefix in Python string literals is used to create a bytes literal. When you prefix a string with 'b', you're telling Python to treat the string as binary data. This can be useful when you need to work with data at the byte level.

# Python binary string with 'b' prefix bs = b'Hello, Python!' print(bs)

Output:

b'Hello, Python!'

In the example above, the 'b' prefix tells Python that 'Hello, Python!' is a bytes literal, not a regular string. When you print it, Python shows the 'b' prefix in the output to indicate that it's a bytes object, not a str object.

One thing to remember is that a bytes object is immutable. This means that once you've defined a bytes object, you can't change its elements. However, you can convert a bytes object to a bytearray object, which is mutable.

# Python bytearray ba = bytearray(bs) ba[0] = 74 # Change 'H' (72 in ASCII) to 'J' (74 in ASCII) print(ba)

Output:

bytearray(b'Jello, Python!')

In the example above, we first convert the bytes object to a bytearray object. Then we change the first byte of the bytearray. When we print it, we see that the first character has changed from 'H' to 'J'.

The 'b' Prefix with Different Data Types

Now, let's explore how the 'b' prefix works with different data types in Python. When you put a 'b' in front of a string literal, Python interprets the string as bytes. This is particularly useful when you're dealing with binary data, such as data read from a binary file or received over a network.

Let's take an example. Try running the following code:

b_string = b'Hello, StackAbuse readers!' print(b_string)

You'll see this output:

b'Hello, StackAbuse readers!'

You can see that the 'b' prefix is preserved in the output, indicating that this is a bytes object, not a regular string.

Note: When you're using the 'b' prefix, you can only include ASCII characters in your string. If you try to include a non-ASCII character, Python will raise a SyntaxError.

For example:

b_string = b'Hello, StackAbuse readers! 👋' print(b_string)

This will raise a SyntaxError:

File "<stdin>", line 1 SyntaxError: bytes can only contain ASCII literal characters.

This is because the 'b' prefix tells Python to interpret the string as bytes, and bytes can only represent ASCII characters.

Why use binary strings?

Let's see some use-cases for opting for byte strings over other data types:

Memory Usage

Byte strings are more memory-efficient when you're dealing with binary data that doesn't require the additional features offered by regular strings. In Python, a regular string is Unicode and therefore can require multiple bytes to represent a single character. On the other hand, a byte string only uses a single byte per element, which can be particularly useful when working with large datasets or when manipulating binary files.

I/O Operations

If you're dealing with file I/O that involves binary files, like images or executables, byte strings are almost a necessity. When you read a binary file, the data is read into memory as bytes. If you were to use regular strings in this scenario, you'd have to perform additional operations to convert the binary data to a Unicode string representation, which could be both time-consuming and error-prone.

Network Programming

In the realm of network programming, byte strings can significantly simplify tasks. Most networking protocols expect data to be sent in a bytes-like format. By using byte strings, you can easily construct packets without having to worry about text encodings and character sets.

Compatibility with C Libraries

Another scenario where byte strings come in handy is when interfacing with C libraries. The C programming language often employs byte arrays for strings. If you're using Python's ctypes library or other similar methods to call C functions, you'll find that byte strings provide a hassle-free way to pass string data between Python and C.

Simplified Binary Operations

Byte strings allow for easier bitwise operations, like bitwise AND, OR, and XOR, which might be cumbersome when working with regular strings. If you're working on encryption algorithms, data compression, or other tasks that require bitwise manipulation, byte strings make these operations much easier.

Conclusion

In this Byte, we've explored the use of the 'b' prefix in Python string literals. We've learned that this prefix tells Python to interpret the string as bytes, rather than a regular string. This is useful when working with binary data, but it also means that you can only include ASCII characters in your string.

Python is a powerful language with a lot of features designed to make your life as a programmer easier. The 'b' prefix is just one of these features, but it's a helpful one to understand if you're working with binary data.

From restaurant e-menus to airline boarding passes, QR codes have numerous applications that impact your day-to-day life and enrich the user’s experience. Wouldn’t it be great to make them look good, too? With the help of this tutorial, you’ll learn how to use Python to generate beautiful QR codes for your personal use case.

In its most basic format, a QR code contains black squares and dots on a white background, with information that any smartphone or device with a dedicated QR scanner can decode. Unlike a traditional bar code, which holds informationation horizontally, a QR code holds the data in two dimensions, and it can hold over a hundred times more information.

In this tutorial, you’ll learn how to:

• Generate a basic black-and-white QR code
• Change the size and margins of the QR code
• Create colorful QR codes
• Rotate the QR code
• Replace the static background with an animated GIF

Traditionally, QR codes have been predominantly black-and-white. Now, thanks to creative innovations, QR codes come in all sorts of shapes, sizes, and colors. If you’d like to learn how to create not only two-tone QR codes but also colorful and artistic ones, then this is the tutorial for you.

Free Bonus: Click here to download the sample code that shows you how to generate a beautiful code with Python.

Using Python to Generate a Basic QR Code

To begin with, you’re going to create a black-and-white QR code that encodes some content. To follow along with this tutorial, you’ll need to install Segno, which is a popular Python library for generating QR codes. To install Segno, you can run pip from your command line:

$ python -m pip install segno

Once you’ve installed segno, create a Python file named basic_qrcode.py. To create a black-and-white QR code object that encodes some content, you’ll have to use the make_qr() function. This ensures that you’re creating a full-size QR code, and the only mandatory argument you’ll need to pass is the information that you want to encode.

Note: You can also use the make() function to create a QR code object, but depending on the content you’re encoding, it might create a micro QR code. For the sake of this tutorial, you can think of that as a smaller QR code. To enforce the creation of a full-size QR code, you should use the make_qr() function.

QR codes are capable of handling all types of data, such as alphanumeric characters, symbols, and even URLs. To begin your journey of generating QR codes in Python, you’ll start by creating a QR code object that will encode the text "Hello, World":

# basic_qrcode.py import segno segno.make_qr("Hello, World")

At this stage, you’ve written the code to encode the text "Hello, World" by passing this to the make_qr() function. This will greet your user with that text, and it might also offer the option of searching for "Hello, World" on the Internet.

Now, to generate a black-and-white QR code that you can actually view and scan, you’ll need to store the encoded content as a variable and use the .save() method to save your QR code as an image. Here’s an example of how you can create a variable called qrcode that encodes the text "Hello, World" as a black-and-white QR code object, which you then save as a PNG file called basic_qrcode.png:

# basic_qrcode.py import segno qrcode = segno.make_qr("Hello, World") qrcode.save("basic_qrcode.png")

The .save() method serializes the QR code into a file format of your choice, as long as the chosen format is supported. When you apply .save() to the variable that you’ve created with the encoded content, you need to specify the filename including an optional file path. In the example above, you’re saving the QR code image as a file named basic_qrcode.png in the same directory where you’ll be executing your code, so you don’t specify a file path.

If you’d like to save the image in a different directory, then you can specify the desired file path together with the filename as an argument in the .save() method.

Once you’ve finished writing the steps for generating a black-and-white QR code in basic_qrcode.py, you’ll need to run the Python script from your command line:

$ python basic_qrcode.py

Congratulations, you’ve just created a black-and-white QR code that encodes the text "Hello, World" using make_qr() and .save(). You can now scan your QR code image, which you’ll find in the same directory that you’re running your code from. Or, you can scan the QR code image below:

You may have found that the QR code image is a little difficult to view or read because of its size, so the next item that you’re going to adjust is the size of your basic QR code.

Changing the Size the QR Code

When trying to scan your QR code image or the QR code in the tutorial, you might have found it difficult to read because of its size.

Read the full article at https://realpython.com/python-generate-qr-code/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Introduction

In Matplotlib, there are a number of ways to close plots, including the functions cla(), clf(), and close(). These functions are helpful tools for managing our plots. But when should we use each one? Let's take a look.

Using cla() to Clear a Plot

The cla() function stands for "clear axis". It is used to clear the current axis in a figure, basically removing all plotted data, labels, titles, or other elements from the axis, while leaving the figure itself intact. It's like erasing a drawing but leaving the canvas untouched. Here's a simple example:

import matplotlib.pyplot as plt plt.plot([1, 2, 3, 4]) plt.ylabel('Numbers') plt.cla() plt.show()

After calling plt.cla(), the plot is cleared, but the figure is still open. You can add new plots to the same figure without having to create a new one. Here's what that would look like with the previous example:

Note: cla() only clears the current axis. If you have multiple axes in your figure, you'll need to select each one and call cla() to clear them all.

Using clf() to Clear a Figure

clf(), or "clear figure", is a step up from cla(). Instead of just clearing the axis, clf() clears the entire figure. This includes all axes, titles, labels, and plotted data. It's like throwing away the canvas and getting a brand new one. Let's see it how it works:

import matplotlib.pyplot as plt plt.figure(1) # first figure plt.subplot(211) # the first subplot in the first figure plt.plot([1, 2, 3]) plt.subplot(212) # the second subplot in the first figure plt.plot([4, 5, 6]) plt.clf() plt.show()

After calling plt.clf(), the figure and all their subplots are cleared. You can then start fresh with a new plot.

Using close() to Close a Window or Figure

Finally, we have close(). This function closes the window, along with any figures and plots it contains. It's like not just throwing away the canvas, but also leaving the studio. Here's how you use it:

import matplotlib.pyplot as plt plt.plot([1, 2, 3, 4]) plt.ylabel('Numbers') plt.show(block=False) plt.pause(3) plt.close()

After calling plt.close(), the figure window will close completely after 3 seconds. Any attempt to plot to it will result in an error, unless you create a new figure.

Note: The trick here is to use the param block=False so that the code can continue to execute past plt.show() and eventually close using the plt.close() method. If you don't set the block param, the execution will stop there and not reach the close() method.

Differences Between cla(), clf(), and close()

In the previous sections, we've looked at how cla(), clf(), and close() can be used to manage our plots and figures in Matplotlib. However, it's important to understand the differences between these three functions to use them in the right circumstances.

cla() or clear axis essentially clears an axis, leaving an empty plot or subplot behind, but keeping the axis in tact. It's useful when you need to plot new data on the same axis. Here's an example:

clf() or clear figure clears the entire figure including all its axes, but keeps the window opened, as if it was just created. So, if you're planning to create a completely new plot, clf() is the way to go.

Finally, close() closes a window completely, which will be helpful when you're done with your plot and want to free up the memory.

While cla() and clf() are used inside a plotting script, close() is generally used at the end of the script. The cla() and clf() methods are often used when you're showing an interactive plot or one that changes over time.

Conclusion

In this Byte, we've explored the differences between cla(), clf(), and close() in Matplotlib. These functions provide us with different levels of control over how we clear our plots, figures, and windows. Understanding when and how to use these commands can help you manage your plots better.

Remember, cla() clears an axis, clf() clears an entire figure, and close() closes a window completely. So, whether you're updating a single plot, creating a new figure, or wrapping up your work, Matplotlib likely has a method for your use-case.

<strong>Topics covered in this episode:</strong><br> <ul> <li><a href="https://github.com/sybrenjansen/mpire"><strong>mpire</strong></a></li> <li><a href="https://pypi.org/project/mopup/"><strong>mopup</strong></a> - the macOS Python.org Updater</li> <li><a href="https://engineering.fb.com/2023/08/15/developer-tools/immortal-objects-for-python-instagram-meta/"><strong>Immortal Objects for Python</strong></a></li> <li><a href="https://stackabuse.com/common-docstring-formats-in-python/"><strong>Common Docstring Formats in Python</strong></a></li> <li><strong>Extras</strong></li> <li><strong>Joke</strong></li> </ul><a href='https://www.youtube.com/watch?v=Qr0fTmiK5O8' style='font-weight: bold;'>Watch on YouTube</a><br> <p><strong>About the show</strong></p> <p>Sponsored by Sentry: <a href="https://pythonbytes.fm/sentry"><strong>pythonbytes.fm/sentry</strong></a></p> <p><strong>Connect with the hosts</strong></p> <ul> <li>Michael: <a href="https://fosstodon.org/@mkennedy"><strong>@mkennedy@fosstodon.org</strong></a></li> <li>Brian: <a href="https://fosstodon.org/@brianokken"><strong>@brianokken@fosstodon.org</strong></a></li> <li>Show: <a href="https://fosstodon.org/@pythonbytes"><strong>@pythonbytes@fosstodon.org</strong></a></li> </ul> <p>Join us on YouTube at <a href="https://pythonbytes.fm/stream/live"><strong>pythonbytes.fm/live</strong></a> to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too.</p> <p><strong>Michael #1:</strong> <a href="https://github.com/sybrenjansen/mpire"><strong>mpire</strong></a></p> <ul> <li>A Python package for easy multiprocessing, but faster than multiprocessing</li> <li><code>MPIRE</code> is faster in most scenarios, packs more features, and is generally more user-friendly than the default multiprocessing package.</li> <li>Tons of <a href="https://github.com/sybrenjansen/mpire#features">features</a>.</li> <li><a href="https://towardsdatascience.com/mpire-for-python-multiprocessing-is-really-easy-d2ae7999a3e9">Nice intro article</a> with benchmarks.</li> </ul> <p><strong>Brian #2:</strong> <a href="https://pypi.org/project/mopup/"><strong>mopup</strong></a> - the macOS Python.org Updater</p> <ul> <li>Glyph Lefkowitz</li> <li>On a mac, install Python with the standard Python.org installer.</li> <li>Then, periodically, update with <code>python3 -m mopup</code> </li> <li>I just did it and went from Python 3.11.4 to 3.11.5</li> <li>See also <a href="https://blog.glyph.im/2023/08/get-your-mac-python-from-python-dot-org.html">Get Your Mac Python From Python.org for reasons to use python.org</a> over other ways, also from Glyph.</li> </ul> <p><strong>Michael #3:</strong> <a href="https://engineering.fb.com/2023/08/15/developer-tools/immortal-objects-for-python-instagram-meta/"><strong>Immortal Objects for Python</strong></a></p> <ul> <li>Instagram has introduced Immortal Objects – <a href="https://peps.python.org/pep-0683/">PEP-683</a> – to Python. </li> </ul> <p><strong>Brian #4:</strong> <a href="https://stackabuse.com/common-docstring-formats-in-python/"><strong>Common Docstring Formats in Python</strong></a></p> <ul> <li>Scott Robinson</li> <li>I don’t mean to disrespect Scott, but I’m honestly curious if this is really common.</li> <li>I like docstrings for the “why” of a function. And prefer type hints for types.</li> <li>Let me know what you use, at <a href="https://fosstodon.org/@brianokken">@brianokken@fosstodon.org</a></li> </ul> <p><strong>Extras</strong> </p> <p>Brian:</p> <ul> <li>In search for a working retro Lunar Lander in Python</li> </ul> <p>Michael:</p> <ul> <li><a href="https://fosstodon.org/@RhetTbull/110978885928495695"><strong>Releases follow up</strong></a></li> <li><a href="https://www.bleepingcomputer.com/news/security/lazarus-hackers-deploy-fake-vmware-pypi-packages-in-vmconnect-attacks/"><strong>North Korean hackers behind malicious VMConnect PyPI campaign</strong></a> </li> </ul> <p><strong>Joke:</strong> <a href="https://devhumor.com/media/okay-it-s-bingo-time-let-s-see-who-get-s-a-perfect-bingo"><strong>It’s Bingo Time!</strong></a></p>

I'm pleased to announce the release of Python 3.12 release candidate 2.

https://www.python.org/downloads/release/python-3120rc2/

 This is the second release candidate of Python 3.12.0

This release, 3.12.0rc2, is the last release preview for Python 3.12.

There will be no ABI changes from this point forward in the 3.12 series. The intent is for the final release of 3.12.0, scheduled for Monday, 2023-10-02, to be identical to this release candidate. This is the last chance to find critical problems in Python 3.12.

Call to action

We strongly encourage maintainers of third-party Python projects to prepare their projects for 3.12 compatibilities during this phase, and where necessary publish Python 3.12 wheels on PyPI to be ready for the final release of 3.12.0. Any binary wheels built against Python 3.12.0rc2 will work with future versions of Python 3.12. As always, report any issues to the Python bug tracker.

Please keep in mind that this is a preview release and while it’s as close to the final release as we can get it, its use is not recommended for production environments.

Core developers: time to work on documentation now

• Are all your changes properly documented?
• Are they mentioned in What’s New?
• Did you notice other changes you know of to have insufficient documentation?

 Major new features of the 3.12 series, compared to 3.11  New features

• More flexible f-string parsing, allowing many things previously disallowed (PEP 701).
• Support for the buffer protocol in Python code (PEP 688).
• A new debugging/profiling API (PEP 669).
• Support for isolated subinterpreters with separate Global Interpreter Locks (PEP 684).
• Even more improved error messages. More exceptions potentially caused by typos now make suggestions to the user.
• Support for the Linux perf profiler to report Python function names in traces.
• Many large and small performance improvements (like PEP 709), delivering an estimated 5% overall performance improvementcitation needed.

Type annotations

• New type annotation syntax for generic classes (PEP 695).
• New override decorator for methods (PEP 698).

Deprecations

• The deprecated wstr and wstr_length members of the C implementation of unicode objects were removed, per PEP 623.
• In the unittest module, a number of long deprecated methods and classes were removed. (They had been deprecated since Python 3.1 or 3.2).
• The deprecated smtpd and distutils modules have been removed (see PEP 594 and PEP 632. The setuptools package continues to provide the distutils module.
• A number of other old, broken and deprecated functions, classes and methods have been removed.
• Invalid backslash escape sequences in strings now warn with SyntaxWarning instead of DeprecationWarning, making them more visible. (They will become syntax errors in the future.)
• The internal representation of integers has changed in preparation for performance enhancements. (This should not affect most users as it is an internal detail, but it may cause problems for Cython-generated code.)

(Hey, fellow core developer, if a feature you find important is missing from this list, let Thomas know.)

For more details on the changes to Python 3.12, see What’s new in Python 3.12. The next scheduled release of Python 3.12 will be 3.12.0, the final release, currently scheduled for 2023-10-02.

 More resources

• Online Documentation.
• PEP 693, the Python 3.12 Release Schedule.
• Report bugs via GitHub Issues.
• Help fund Python and its community.

 Enjoy the new release
Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation.
Your release team,Thomas WoutersNed DeilySteve DowerŁukasz Langa

The DSF is once again partnering with JetBrains to run the 2023 Django Developers Survey.

Please take a moment to fill it out. The survey is an important metric of Django usage and helps guide future technical and community decisions.

The survey will be open until October 1st, 2023 Anywhere on Earth (AoE). After the survey is over, the aggregated results will be published.

JetBrains will randomly choose 10 winners (from those who complete the survey in its entirety), who will each receive a $100 Amazon Gift Card or a local equivalent. Full terms and conditions are available on the survey website.

#593 – SEPTEMBER 5, 2023
View in Browser »

Create a Python Wordle Clone With Rich

In this step-by-step project, you’ll build your own Wordle clone with Python. Your game will run in the terminal, and you’ll use Rich to ensure your word-guessing app looks good. Learn how to build a command-line application from scratch and then challenge your friends to a wordly competition!
REAL PYTHON course

Create a Beautiful Polar Histogram With Python and Matplotlib

“Polar histograms are great when you have too many values for a standard bar chart. The circular shape where each bar gets thinner towards the middle allows us to cram more information into the same area.” Learn how to create one using Python and Matplotlib.
OSCAR JOHANSSON

You Write Great Python Code but How Do You Know It’s Secure Code

If you’re not a security expert, consider Semgrep. Trusted by Slack, Gitlab, Snowflake, and thousands of engineers, it acts like a security spellchecker for your code. Simply point Semgrep to your code; it identifies vulnerabilities and even checks code dependencies and helps you ship secure code →
SEMGREP sponsor

An Intro to Protocol Buffers With Python

Protocol buffers are a data serialization format that is language agnostic, analogous to Python’s pickle format. Learn how to write them in Python and communicate with other languages that support the protocol.
MIKE DRISCOLL

Django Security Releases Issued: 4.2.5, 4.1.11, and 3.2.21

DJANGO SOFTWARE FOUNDATION

Discussions Why Did Python Win?

HACKER NEWS

Python Proposal (Inspired by Lua)

NED BATCHELDER

Articles & Tutorials Finding the Right Coding Font for Programming in Python

What should you consider when picking a font for coding in Python? What characters and their respective glyphs should you check before making your decision? This week on the show, we talk with Real Python author and core team member Philipp Acsany about his recent article, Choosing the Best Coding Font for Programming.
REAL PYTHON podcast

Hidden Features of Python

Python is a powerful programming language that’s easy to learn and fun to play with. But beyond the basics, there are plenty of hidden features and tricks. This article covers debugging regexes, the ellipsis, dir(), chaining comparisons, and more little nuggets you may not know.
SCOTT ROBINSON

Build Your Web Application In No Time No Development Knowledge Required

Quickly create interactive and customizable multi-page dashboards with Taipy, the open-source Python library, with its simple syntax. This web application builder generates highly interactive interfaces without requiring any knowledge of web development →
TAIPY sponsor

Testing With Hypothesis

Hypothesis is a property-based testing library. This style of testing uses ranges of scenarios rather than a single value, and then the tool explores the results. See also the intro article to strategy based testing.
HYPOTHESIS.READTHEDOCS.IO

PSF Authorized as a CVE Numbering Authority

The Common Vulnerabilities and Exposures program identifies, catalogs, and discloses cybersecurity vulnerabilities. The Python Software Foundation has recently been added as a numbering authority, improving Python’s ability to disclose and respond to security issues.
PYTHON SOFTWARE FOUNDATION

Deploying a Machine Learning Model to AWS Lambda

This tutorial teaches how to deploy a machine learning (ML) model to AWS Lambda, via Serverless Framework, and execute it using Boto3. It also creates a CI/CD pipeline with GitHub Actions to automate the deployment process and run end-to-end tests.
JAN GIACOMELLI • Shared by Jan Giacomelli

Python Basics: Pathlib and File System Operations

In this video course, you’ll learn how to use the pathlib module to carry out file path operations with Python. These operations include creating, iterating over, searching for, moving, and deleting files and folders.
REAL PYTHON course

Disable a Direct Push to GitHub Main Branch

If your development team uses the main branch for production and pull requests for feature management, you may want to disable the ability to (accidentally) push to main. This article shows you how.
JOHNNY METZ

5 Ways to Get Started in Open Source

This article shares ideas for finding and making your first open source contribution, using examples from contributions the author has made to various projects.
STEFANIE MOLIN • Shared by Stefanie Molin

Introduction to Python’s Functools Module

This article introduces you to the functions in Python’s functools module with real world examples to help show you how and when to use each feature.
FLORIAN DAHLITZ

Heaps in Python

Learn about both the min heap and max heap data structures using Python.
SHIVALI BHADANIYA • Shared by cody

Projects & Code compress: Text Compression to Generate Keystroke Expansion

GITHUB.COM/ESCHLUNTZ

communitynotes: Docs & Code Powering Twitter’s Notes

GITHUB.COM/TWITTER

mljar: Create Plots in Matplotlib With ChatGPT

GITHUB.COM/MLJAR • Shared by Piotr

ML-Recipes: Collection of Machine Learning Recipes

GITHUB.COM/ROUGIER

A DictWriter Interface for Google Spreadsheets

JACOB KAPLAN-MOSS

Events Cloud Builder: Python Conf

September 6 to September 7, 2023
CLOUD-BUILDERS.TECH

Weekly Real Python Office Hours Q&A (Virtual)

September 6, 2023
REALPYTHON.COM

PyCon Estonia 2023

September 7 to September 9, 2023
PYCON.EE

PyCon Portugal 2023

September 7 to September 10, 2023
PYCON.PT

PyData Amsterdam 2023

September 14 to September 17, 2023
PYDATA.ORG

Kiwi PyCon 2023

September 15 to September 18, 2023
KIWIPYCON.NZ

PyCon CZ 2023

September 15 to September 18, 2023
PYCON.ORG

Happy Pythoning!
This was PyCoder’s Weekly Issue #593.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

We’re thrilled to announce Azure Developers – Python Day! Join us on September 7th for a full day of online training and discover the latest services and features in Azure designed specifically for Python developers. You’ll learn cutting-edge cloud development techniques that can save you time and money while providing your customers with the best experience possible.

September 7, 2023, from 9:00 am – 4:00 pm (Pacific Time) / 17:00 – 00:00 (UTC)

Save the date!

During the event, you will hear directly from the experts behind the latest features in Azure designed for Python developers, techniques to save time and money, and a special session on our recently announced Python in Excel.

Whether you’re a beginner or an experienced Python developer, this event is for you. We will cover six main topic areas: App Dev, Artificial Intelligence, Cloud Native, Data Services, Serverless, and Developer Productivity.

Agenda Session Title Theme Speaker Time( PT/ UTC) Welcome to Azure Developers – Python Day Dawn Wages, Hailey Huber, & Jayme Singleton 9:00 AM / 16:00 Now we’re cooking: Building a restaurant app with Azure Cosmos DB App Development Emmanuel Deletang 9:00 AM / 16:00 Host your gRPC workloads on App Service using Python and LangChain App Development Jeff Martinez, Byron Tardif 9:30 AM / 16:30 Speedrun your Python deployments with azd Developer Productivity Savannah Ostrowski 10:00 AM / 17:00 Simple, Fast, Idiomatic: An Intro to the Azure SDK for Python Developer Productivity Rohit Ganguly 10:30 AM / 17:30 Use Python in Excel to enhance your data science Developer Productivity Sarah Kaiser 11:00 AM / 18:00 Python in Excel Data Services John Lam, Keyur Patel, Joe McDaid 11:30 AM / 18:30 Use ALL your data with Microsoft Fabric Data Services Sarah Kaiser 12:00 PM / 19:00 BREAK 12:30 PM / 19:30 Building intelligent Python apps and microservices with Azure Container Apps Serverless Anthony Chu, Pamela Fox 1:00 PM / 20:00 Building geo-distributed API with Azure Functions – developing, testing, and deploying Serverless Anton Boyko 1:30 PM / 20:30 Let’s code a drone using Azure OpenAI Services AI Bruno Capuano 2:00 PM / 21:00 How to build a practical AI app with Python, Redis, and OpenAI AI Kyle Teegarden, Shruti Pathak 2:30 PM / 21:30 Create your first copilot solutions AI Kinfey Lo 3:00 PM / 22:00 Azure Vector Search solutions with Cosmos DB AI Chris Joakim 3:30 PM / 22:30

Don’t miss out on this opportunity to build the best applications with Python. Join us on September 7th on the Azure Developers YouTube and Twitch channels. See you there!

The post Announcing: Azure Developers – Python Day appeared first on Python.

Introduction

One of the challenges you may encounter when working with file paths is extracting the file name from the path, which can vary depending on the operating system and the format of the path.

In this Byte, we'll explore how to tackle this problem using Python's built-in os.path module. We'll also look at how to handle different path formats used in Windows and Unix-style operating systems.

Using the os.path Module in Python

Python's os.path module provides a set of functions to manipulate file paths. One of these functions is basename(), which returns the last component of a pathname, which is usually the filename.

Let's see how it works:

import os path = '/home/user/documents/file.txt' filename = os.path.basename(path) print(filename)

In this case, the output will be:

file.txt

The basename() function works by splitting the path at the last slash (/) and returning the part after it. This works regardless of whether the path ends with a slash.

The os.path module functions are designed to be used with paths in the format used by the operating system where your Python code is running. This means that if you're working with paths from a different operating system, you may run into issues. For example, if your code is running on a Unix-like OS, it may have issues parsing a Windows path.

Dealing with Windows File Paths

Windows file paths can be a bit tricky because they use backslashes (\) instead of slashes (/). If you're working with Windows paths in Python, you'll need to escape the backslashes by using double backslashes (\\), or by using raw strings (r'path\to\file').

Here's how you can extract a file name from a Windows path using the os.path module:

import os path = 'C:\\Users\\user\\Documents\\file.txt' filename = os.path.basename(path) print(filename)

And the output will be:

file.txt Dealing with Unix-style File Paths

Unix-style file paths, used by Linux and macOS, use slashes (/). This makes them a bit easier to work with in Python.

Here's an example of how to extract a file name from a Unix-style path:

import os path = '/home/user/documents/file.txt' filename = os.path.basename(path) print(filename)

And, as before, the output will be:

file.txt Handling Special Characters in File Names

While working with file paths, you might come across some special characters in file names. These characters can be problematic if not handled correctly. Fortunately, Python's os.path module provides us with the tools we need to handle these special characters effectively.

Let's say we have a file path with special characters like this:

path = "/home/user/Documents/My\ Project\ #1/file.txt"

In this case, the backslashes (\) are used to escape spaces and the hash symbol (#). If we try to extract the file name from this path without considering these special characters, we might end up with an incorrect result.

To handle this, we can use the os.path.basename() function along with the raw string literal (r). The r prefix before the string literal converts the normal string to a raw string. In a raw string, backslashes (\) are treated as literal characters and not as escape characters.

Here's how you can do it:

import os path = r"/home/user/Documents/My\ Project\ #1/file.txt" filename = os.path.basename(path) print(filename) # Outputs: file.txt

In this example, the basename() function correctly identifies file.txt as the file name, despite the presence of special characters in the path.

Note: When dealing with file paths in Python, it's always a good idea to use raw strings (r) to avoid issues with escape characters.

Conclusion

In this Byte, we've seen how to extract file names from file paths in Python, regardless of the operating system or path format. We've explored how to use the os.path module to handle Windows and Unix-style file paths. We've also seen how to handle special characters in file names to help avoid hard-to-find errors when working with these kinds of files.

Python includes mechanisms for doing object-oriented programming, where the data and operations on that data are structured together. The class keyword is how you create these structures in Python. Attributes are the data values, and methods are the function-like operations that you can perform on classes. In this course, you’ll explore writing code using class and its associated attributes and methods.

In this video course, you’ll learn about:

• Why you write object-oriented code
• How to write a class
• What attributes and methods are
• How to use the descriptor protocol

This course is the first in a three-part series. Part two covers how to write reusable hierarchies of classes using inheritance, while part three dives deeper into the philosophy behind writing good object-oriented code.

Download

Sample Code (.zip)

5.2 KB

Download

Course Slides (.pdf)

1013.9 KB

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Introduction

When working with data in Python, Pandas is a library that often comes to the rescue, especially when dealing with large datasets. One of the most common tasks you'll be performing with Pandas is data indexing and selection. This Byte will introduce you to two powerful tools provided by Pandas for this purpose: iloc and loc. Let's get started!

Indexing in Pandas

Pandas provides several methods to index data. Indexing is the process of selecting particular rows and columns of data from a DataFrame. This can be done in Pandas through explicit index and label-based index methods. This Byte will focus on the latter, specifically on the loc and iloc functions.

What is iloc?

iloc is a Pandas function used for index-based selection. This means it indexes based on the integer positions of the rows and columns. For instance, in a DataFrame with n rows, the index of the first row is 0, and the index of the last row is n-1.

Note: iloc stands for "integer location", so it only accepts integers.

Example: Using iloc

Let's create a simple DataFrame and use iloc to select data.

import pandas as pd # Creating a simple DataFrame data = {'Name': ['John', 'Anna', 'Peter', 'Linda'], 'Age': [28, 24, 35, 32], 'Profession': ['Engineer', 'Doctor', 'Lawyer', 'Writer']} df = pd.DataFrame(data) print(df)

This will output:

Name Age Profession 0 John 28 Engineer 1 Anna 24 Doctor 2 Peter 35 Lawyer 3 Linda 32 Writer

Let's use iloc to select the first row of this DataFrame:

first_row = df.iloc[0] print(first_row)

This will output:

Name John Age 28 Profession Engineer Name: 0, dtype: object

Here, df.iloc[0] returned the first row of the DataFrame. Similarly, you can use iloc to select any row or column by its integer index.

What is loc?

loc is another powerful data selection method provided by Pandas. It's works by allowing you to do label-based indexing, which means you select data based on the data's actual label, not its position. It's one of the two primary ways of indexing in Pandas, along with iloc.

Unlike iloc, which uses integer-based indexing, loc uses label-based indexing. This can be a string, or an integer label, but it's not based on the position. It's based on the label itself.

Note: Label-based indexing means that if your DataFrame's index is a list of strings, for example, you'd use those strings to select data, not their position in the DataFrame.

Example: Using loc

Let's look at a simple example of how to use loc to select data. First, we'll create a DataFrame:

import pandas as pd data = { 'fruit': ['apple', 'banana', 'cherry', 'date'], 'color': ['red', 'yellow', 'red', 'brown'], 'weight': [120, 150, 10, 15] } df = pd.DataFrame(data) df.set_index('fruit', inplace=True) print(df)

Output:

color weight fruit apple red 120 banana yellow 150 cherry red 10 date brown 15

Now, let's use loc to select data:

print(df.loc['banana'])

Output:

color yellow weight 150 Name: banana, dtype: object

As you can see, we used loc to select the row for "banana" based on its label.

Differences Between iloc and loc

The primary difference between iloc and loc comes down to label-based vs integer-based indexing. iloc uses integer-based indexing, meaning you select data based on its numerical position in the DataFrame. loc, on the other hand, uses label-based indexing, meaning you select data based on its label.

Another key difference is how they handle slices. With iloc, the end point of a slice is not included, just like with regular Python slicing. But with loc, the end point is included.

Conclusion

In this short Byte, we showed examples of using the loc method in Pandas, saw it in action, and compared it with its couterpart, iloc. These two methods are both useful tools for selecting data in Pandas, but they work in slightly different ways.

News and updates

• 🚀 We started testing a new flyout menu as part of our beta test for documentation addons. The beta test is currently limited to projects using the build.commands configuration key.

• 🛣️ We continue to have a number deprecations in progress. We announced this month deprecations of installing using system packages, the configuration key build.image, and installation of pinned versions of Sphinx and MkDocs. Keep an eye on your email for any deprecation notifications, as we will continue to notify maintainers of projects that might be impacted.

• 📚 The Read the Docs Sphinx theme package, sphinx-rtd-theme, had two releases. Version 1.3.0 was released, adding support for Sphinx 7.0. Version 2.0.0rc2 is also now out. This is a release candidate that will remove support for HTML4 output and will drop support for Sphinx versions prior to 5.0. We will be announcing the release candidate more this month and will be looking to have feedback from users.

• 🔐 A security advisory involving symlink abuse during project builds was raised and patched.

• 📉 Changes to our request handling resulted in a 30% reduction in response times for 404 error responses.

  Request times for 404 handling have dropped 30% since last release.

• 🏁 Updates to Proxito are now fully rolled out to Read the Docs Community and Read the Docs for Business.

• ✨ We upgraded our application to use Django 4.2.

Upcoming changes

• ⚠️ In line with our deprecation plans for builds without a configuration file, projects will be required to specify a configuration file to continue building their documentation starting September 25th. Our last brownout date is September 4th, lasting 48 hours. To avoid any problems building your project, ensure it has a configuration file before then.

• 🚢️ We will be continuing our work on for our beta test of Read the Docs Addons. Our focus is still on improving the new flyout menu, search, and adding more new features.

• 🛠️ Our new dashboard, currently available for beta testing at https://beta.readthedocs.org, will receive some small feature additions, and we are working towards a beta of the new dashboard for Read the Docs for Business as well. We expect to have more news here in the coming months.

Want to follow along with our development progress? View our full roadmap 📍️

Possible issues

• ⚠️ Make sure to follow directions from notifications regarding deprecations. We have notified project maintainers for any project that could be affected by one of our ongoing deprecations. Updating your package ahead of brownout dates and final deprecation cutoff dates will ensure your project continues to successfully build.

Questions? Comments? Ideas for the next newsletter? Contact us!

Introduction

Data processing is a common task in any data analysis codebase. And in Python, the Pandas library is one of the most popular tools for data analysis, which also provides high-performance, easy-to-use data structures and data analysis tools, one of which is the DataFrame. In this Byte, we're going to explore how to delete rows in a DataFrame based on the value in a specific column.

DataFrames in Pandas

A DataFrame is a two-dimensional labeled data structure with columns of potentially different types. You can think of it like a spreadsheet or SQL table, or a dictionary of Series objects. It is generally the most commonly used Pandas object. Let's create a simple DataFrame:

import pandas as pd data = { 'Name': ['John', 'Anna', 'Peter', 'Linda'], 'Age': [28, 24, 35, 32], 'Country': ['USA', 'USA', 'Canada', 'USA'] } df = pd.DataFrame(data) print(df)

Output:

Name Age Country 0 John 28 USA 1 Anna 24 USA 2 Peter 35 Canada 3 Linda 32 USA Why Delete Rows Based on Column Value?

Data comes in all shapes and sizes, and not all of it is useful or relevant. Sometimes, you might want to remove rows based on a specific column value to clean your data or focus on a subset of it. For instance, you might want to remove all rows related to a particular country in a dataset of global statistics.

How to Delete Rows Based on Column Value

There are several ways to delete rows in a DataFrame based on column value, which we'll explore here.

Method 1: Using Boolean Indexing

Boolean indexing is a powerful feature in Pandas that allows us to select data based on the actual values in the DataFrame. It's a method that should be easy for most to understand, making it a great starting point for our discussion.

Let's say we have a DataFrame of student grades and we want to delete all rows where the grade is below 60. Here's how we can do it with Boolean indexing:

import pandas as pd # Create a simple dataframe df = pd.DataFrame({ 'name': ['Alice', 'Bob', 'Charlie', 'David', 'Eve'], 'grade': [85, 55, 65, 70, 50] }) # Create a boolean mask for grades >= 60 mask = df['grade'] >= 60 # Apply the mask to the dataframe df = df[mask] print(df)

Output:

name grade 0 Alice 85 2 Charlie 65 3 David 70

As you can see, the rows for Bob and Eve, who scored below 60, have been removed.

Method 2: Using the drop() Function

Pandas' drop() function has another way to remove rows from a DataFrame. This method requires a bit more setup than Boolean indexing, but it can be more intuitive for some users.

First, we need to identify the index values of the rows we want to drop. In our case, we want to drop rows where the grade is below 60. Here's how we can do it:

import pandas as pd # Create a simple dataframe df = pd.DataFrame({ 'name': ['Alice', 'Bob', 'Charlie', 'David', 'Eve'], 'grade': [85, 55, 65, 70, 50] }) # Identify indices of rows with grade < 60 drop_indices = df[df['grade'] < 60].index # Drop these indices from the dataframe df = df.drop(drop_indices) print(df)

Output:

name grade 0 Alice 85 2 Charlie 65 3 David 70

Again, Bob and Eve's rows have been removed, just like in our first method.

Method 3: Using the query() Function

The query() function in Pandas helps us to filter data using a query string syntax, much like SQL. This can be a popular choice, especially for those with experience in SQL.

Let's use query() to delete rows where the grade is below 60:

import pandas as pd # Create a simple dataframe df = pd.DataFrame({ 'name': ['Alice', 'Bob', 'Charlie', 'David', 'Eve'], 'grade': [85, 55, 65, 70, 50] }) # Filter dataframe using query() function df = df.query('grade >= 60') print(df)

Output:

name grade 0 Alice 85 2 Charlie 65 3 David 70

Once again, we have successfully removed Bob and Eve's rows from our DataFrame.

Conclusion

And there you have it - three different methods for deleting rows in a Pandas DataFrame based on column values. Each method has its own strengths and use cases, and which one you choose to use may depend on your specific needs and personal preference.

“Chapter 4, Builtin Fixtures” is now available in the pytest course. This chapter covers: tmp_path and tmp_path_factory for temporary files and directories. capsys for testing stdout and stderr. monkeypatch as a lightweight mock for patching the system under test or the OS environment. Design for Testability - A short discussion of ways to change your code to make it easier to test.

Introduction

Even with a "simple" language like Python, it's not immune to performance issues. As your codebase grows, you may start to notice that certain parts of your code are running slower than expected. This is where profiling comes into play. Profiling is an important tool in every developer's toolbox, allowing you to identify bottlenecks in your code and optimize it accordingly.

Profiling and Why You Should Do It

Profiling, in the context of programming, is the process of analyzing your code to understand where computational resources are being used. By using a profiler, you can gain insights into which parts of your code are running slower than expected and why. This can be due to a variety of reasons like inefficient algorithms, unnecessary computations, bugs, or memory-intensive operations.

Note: Profiling and debugging are very different operations. However, profiling can be used in the process of debugging as it can both help you optimize your code and find issues via performance metrics.

Let's consider an example. Suppose you've written a Python script to analyze a large dataset. The script works fine with a small subset of data, but as you increase the size of the dataset, the script takes an increasingly long time to run. This is a classic sign that your script may need optimization.

Here's a simple Python script that calculates the factorial of a number using recursion:

def factorial(n): if n == 0: return 1 else: return n * factorial(n-1) print(factorial(5))

When you run this script, it outputs 120 which is the factorial of 5. However, if you try to calculate the factorial of a very large number, say 10000, you'll notice that the script takes a considerable amount of time to run. This is a perfect candidate for profiling and optimization.

Overview of Python Profiling Tools

Profiling is a crucial aspect of software development, particularly in Python where the dynamic nature of the language can sometimes lead to unexpected performance bottlenecks. Fortunately, Python provides a rich ecosystem of profiling tools that can help you identify these bottlenecks and optimize your code accordingly.

The built-in Python profiler is cProfile. It's a module that provides deterministic profiling of Python programs. A profile is a set of statistics that describes how often and for how long various parts of the program executed.

Note: Deterministic profiling means that every function call, function return, exception, and other CPU-intensive tasks are monitored. This can provide a very detailed view of your application's performance, but it can also slow down your application.

Another popular Python profiling tool is line_profiler. It is a module for doing line-by-line profiling of functions. Line profiler gives you a line-by-line report of time execution, which can be more helpful than the function-by-function report that cProfile provides.

There are other profiling tools available for Python, such as memory_profiler for profiling memory usage, py-spy for sampling profiler, and Py-Spy for visualizing profiler output. The choice of which tool to use depends on your specific needs and the nature of the performance issues you're facing.

How to Profile a Python Script

Now that we've covered the available tools, let's move on to how to actually profile a Python script. We'll take a look at both cProfile and line_profiler.

Using cProfile

We'll start with the built-in cProfile module. This module can either be used as a command line utility or within your code directly. We'll first look at how to use it in your code.

First, import the cProfile module and run your script within its run function. Here's an example:

import cProfile import re def test_func(): re.compile("test|sample") cProfile.run('test_func()')

When you run this script, cProfile will output a table with the number of calls to each function, the time spent in each function, and other useful information.

The ouptut might look something like this:

234 function calls (229 primitive calls) in 0.001 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.001 0.001 <stdin>:1(test_func) 1 0.000 0.000 0.001 0.001 <string>:1(<module>) 1 0.000 0.000 0.001 0.001 re.py:192(compile) 1 0.000 0.000 0.001 0.001 re.py:230(_compile) 1 0.000 0.000 0.000 0.000 sre_compile.py:228(_compile_charset) 1 0.000 0.000 0.000 0.000 sre_compile.py:256(_optimize_charset) 1 0.000 0.000 0.000 0.000 sre_compile.py:433(_compile_info) 2 0.000 0.000 0.000 0.000 sre_compile.py:546(isstring) 1 0.000 0.000 0.000 0.000 sre_compile.py:552(_code) 1 0.000 0.000 0.001 0.001 sre_compile.py:567(compile) 3/1 0.000 0.000 0.000 0.000 sre_compile.py:64(_compile) 5 0.000 0.000 0.000 0.000 sre_parse.py:138(__len__) 16 0.000 0.000 0.000 0.000 sre_parse.py:142(__getitem__) 11 0.000 0.000 0.000 0.000 sre_parse.py:150(append) # ...

Now let's see how we can use it as a command line utility. Assume we have the following script:

def calculate_factorial(n): if n == 1: return 1 else: return n * calculate_factorial(n-1) def main(): print(calculate_factorial(10)) if __name__ == "__main__": main()

To profile this script, you can use the cProfile module from the command line as follows:

$ python -m cProfile script.py

The output will show how many times each function was called, how much time was spent in each function, and other useful information.

Using Line Profiler

While cProfile provides useful information, it might not be enough if you need to profile your code line by line. This is where the line_profiler tool comes in handy. It's an external tool that provides line-by-line profiling statistics for your Python programs.

First, you need to install it using pip:

$ pip install line_profiler

Let's use line_profiler to profile the same script we used earlier. To do this, you need to add a decorator to the function you want to profile:

from line_profiler import LineProfiler def profile(func): profiler = LineProfiler() profiler.add_function(func) return profiler(func) @profile def calculate_factorial(n): if n == 1: return 1 else: return n * calculate_factorial(n-1) def main(): print(calculate_factorial(10)) if __name__ == "__main__": main()

Now, if you run your script, line_profiler will output statistics for each line in the calculate_factorial function.

Remember to use the @profile decorator sparingly, as it can significantly slow down your code.

Profiling is an important part of optimizing your Python scripts. It helps you to identify bottlenecks and inefficient parts of your code. With tools like cProfile and line_profiler, you can get detailed statistics about the execution of your code and use this information to optimize it.

Interpreting Profiling Results

After running a profiling tool on your Python script, you'll be presented with a table of results. But what do these numbers mean? How can you make sense of them? Let's break it down.

The results table typically contains columns like ncalls for the number of calls, tottime for the total time spent in the given function excluding calls to sub-functions, percall referring to the quotient of tottime divided by ncalls, cumtime for the cumulative time spent in this and all subfunctions, and filename:lineno(function) providing the respective data of each function.

Here's a sample output from cProfile:

5 function calls in 0.000 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.000 0.000 <ipython-input-1-9e8e3c5c3b72>:1(<module>) 1 0.000 0.000 0.000 0.000 {built-in method builtins.exec} 1 0.000 0.000 0.000 0.000 {built-in method builtins.len} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}

The tottime and cumtime columns are particularly important as they help identify which parts of your code are consuming the most time.

Note: The output is sorted by the function name, but you can sort it by any other column by passing the sort parameter to the print_stats method. For example, p.print_stats(sort='cumtime') would sort the output by cumulative time.

Optimization Techniques Based on Profiling Results

Once you've identified the bottlenecks in your code, the next step is to optimize them. Here are some general techniques you can use:

• Avoid unnecessary computations: If your profiling results show that a function is called multiple times with the same arguments, consider using memoization techniques to store and reuse the results of expensive function calls.

• Use built-in functions and libraries: Built-in Python functions and libraries are usually optimized for performance. If you find that your custom code is slow, see if there's a built-in function or library that can do the job faster.

• Optimize data structures: The choice of data structure can greatly affect performance. For example, if your code spends a lot of time searching for items in a list, consider using a set or a dictionary instead, which can do this much faster.

Let's see an example of how we can optimize a function that calculates the Fibonacci sequence. Here's the original code:

def fib(n): if n <= 1: return n else: return(fib(n-1) + fib(n-2))

Running a profiler on this code will show that the fib function is called multiple times with the same arguments. We can optimize this using a technique called memoization, which stores the results of expensive function calls and reuses them when needed:

def fib(n, memo={}): if n <= 1: return n else: if n not in memo: memo[n] = fib(n-1) + fib(n-2) return memo[n]

With these optimizations, the fib function is now significantly faster, and the profiling results will reflect this improvement.

Remember, the key to efficient code is not to optimize everything, but rather focus on the parts where it really counts - the bottlenecks. Profiling helps you identify these bottlenecks, so you can spend your optimization efforts where they'll make the most difference.

Conclusion

After reading this article, you should have a good understanding of how to profile a Python script. We've discussed what profiling is and why it's crucial for optimizing your code. We've also introduced you to a couple of Python profiling tools, namely cProfile, a built-in Python profiler, and Line Profiler, an advanced profiling tool.

We've walked through how to use these tools to profile a Python script and how to interpret the results. Based on these results, you've learned some optimization techniques that can help you improve the performance of your code.

Just remember that profiling is a powerful tool, but it's not a silver bullet. It can help you identify bottlenecks and inefficient code, but it's up to you to come up with the solutions.

In my experience, the time invested in learning and applying profiling techniques has always paid off in the long run. Not only does it lead to more efficient code, but it also helps you become a more proficient and knowledgeable Python programmer.

Recursion is when a function calls itself. Loops are usually preferable to recursion, but recursion is excellent for traversing tree-like structures.

Table of contents

1. Recursive functions call themselves
2. Use a base case to stop recursion
3. Recursion works thanks to the call stack
4. Using for loops instead of recursion
5. Recursion's most common use case
6. Loops are great, but recursion does have its uses

Recursive functions call themselves

Here's a Python script that counts up to a given number:

from argparse import ArgumentParser def count_to(number): for n in range(1, number+1): print(n) def parse_args(): parser = ArgumentParser() parser.add_argument("stop", type=int) return parser.parse_args() def main(): args = parse_args() count_to(args.stop) if __name__ == "__main__": main()

Note that the main function in that program calls the parse_args function as well as the count_to function.

Functions can call other functions in Python. But functions can also call themselves!

Here's a function that calls itself:

def factorial(n): if n < 0: raise ValueError("Negative numbers not accepted") if n == 0: return 1 return n * factorial(n-1)

A function that calls itself is called a recursive function.

It might seem like a bad idea for a function to call itself and it often is a bad idea... but not always.

Use a base case to stop recursion

If a function calls itself …

Read the full article: https://www.pythonmorsels.com/what-is-recursion/

In August 2023, Python 3.12.0rc1 came out! With several exciting features, improvements, and optimizations, this release is only two steps away from the final release scheduled for October. If you want to stay on the cutting edge, then you must give it a try. But note that you shouldn’t use it in production.

Another exciting release was Python in Excel, which allows you to leverage the power of Python inside your Excel workbooks. You’ll be able to use Python’s data science ecosystem while you remain in your Excel comfort zone with known formulas, charts, and more.

But that’s not all! The Python Software Foundation (PSF) announced a new roster of fellows and a safety and security engineer for PyPI. Some key package maintainers were busy with the Python DataFrame Summit 2023, and several key libraries released new versions.

Let’s dive into the most exciting Python news from August 2023!

Join Now: Click here to join the Real Python Newsletter and you'll never miss another Python tutorial, course update, or post.

Python 3.12.0 Release Candidate Arrives

This August, Python put out its first release candidate version, 3.12.0rc1. This version is only two steps away from the final release, 3.12.0, which is scheduled for October 2. Before that, Python will deliver another release candidate, planned for September 4.

As Python ventures into the release candidate stage, only confirmed bug fixes will be accepted into the codebase. The goal is to have as few code changes as possible. Most likely, the core team will focus on:

• Polishing and documenting all the changes
• Updating the What’s New document

As you can read in the release notes, 3.12 will have the following list of new features compared to Python 3.11:

• More flexible f-string parsing, allowing you to do more with your f-strings than you previously could (PEP 701)
• Support for the buffer protocol in Python code (PEP 688)
• A new debugging/profiling API (PEP 669)
• Support for isolated subinterpreters with separate global interpreter locks (PEP 684)
• Even more improved error messages, meaning that you get suggestions for more exceptions potentially caused by typos
• Support for the Linux perf profiler to report Python function names in traces
• Many large and small performance improvements, such as PEP 709, which deliver an estimated 5 percent overall performance improvement

If you’d like to learn more about some of these improvements, then check out Real Python’s previews of more intuitive and consistent f-strings, ever better error messages, and support for the Linux perf profiler.

As usual, this version also brings several deprecations that you may need to consider. For a detailed list of changes, additions, and removals, you can check the changelog document.

Just like other pre-release versions, Python 3.12.0rc1 is intended for experimentation and testing purposes only and isn’t recommended for use in production.

Python Makes Its Way Into Microsoft Excel

On August 22, Microsoft announced Python in Excel, a new and exciting feature that combines the flexibility of Excel and the power of Python. This combination may have a considerable impact on the data science industry.

This is a huge announcement, and even Guido van Rossum himself has been helping with the integration of both tools:

Image source

Python in Excel allows you to natively use Python inside an Excel workbook without any additional setup requirements. You only need the new PY function, which lets you input Python code directly into Excel cells:

Read the full article at https://realpython.com/python-news-august-2023/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

This week we welcome Franek Magiera as our PyDev of the Week! Franek is a core developer of the Python programming language.

If you’d like to see what else Franek has been working on, you can check out Franek’s GitHub profile. Now let’s spend a few moments getting to know Franeky better!

Can you tell us a little about yourself (hobbies, education, etc):

I am interested in a lot of different things – playing the guitar, chess, science, computers, mountain hiking and sports. I used to be a competitive powerlifter, but now I am more into climbing and endurance sports – running and road cycling especially.

I got my bachelor’s in Automatic Control and Robotics and my master’s in Computer Science. My engineering and master’s theses are about applications of genetic programming, which was quite interesting to me when I was a student – I even co-authored a paper on genetic programming applications in dimensionality reduction. For a while I was considering a career in research, but ended up as a Java software developer.

Why did you start using Python?

When I was a student, most of my courses featured MATLAB. It is a very powerful tool and I liked working with it, but I didn’t like that when my student license would end, I wouldn’t be able to use it for my personal projects. I was also worried that there weren’t many job offers that required MATLAB knowledge.

I had heard that some of the older students were using Python and that it was quite similar to MATLAB, but it was open source and there were quite a few job offers that listed Python knowledge as one of the requirements. Also I had heard that a lot of people use it for machine learning, which I was interested in at the time. So that’s when I decided to learn Python and picked it up for one of my university projects.

What other programming languages do you know and which is your favorite?

I’ve played around with quite a lot of different programming languages. In high school I was using Pascal. In university, I was coding in C, C++ and MATLAB. Then, I’ve learnt Python and I’ve been heavily relying on it for my personal projects ever since.

I have also spent some time programming in Scala for a couple of MOOCs and in Scheme when going through SICP. I went through the Tour of Go on two separate occasions, but have never used the language for anything else.

I’ve used JavaScript and TypeScript during my internships and now at work I’m using Java exclusively. Recently, I’ve become interested in and started learning Rust.

That’s quite a range of different programming languages, but I think nowadays I’m somewhat productive only in Python and Java, since those are the two I use most often.

As for my favorite – it’s really hard to decide, but I think that’s Python. It’s the one I feel most comfortable with and the community around it is great. For what it’s worth, I’ve also had a lot of fun programming in Rust, Scala, TypeScript, Scheme and C.

What projects are you working on now?

Unfortunately, I haven’t been doing much programming and OSS development in my free time lately. Whenever I find the time and motivation to do that, I am trying to finish certain aspects of PEP 692. I also try to contribute to urllib3 from time to time, but it’s been a while since I’ve done that.

I’ve mentioned earlier that I started learning Rust – I’m spending some of my free time on writing a LISP interpreter in Rust as a way to learn the language.

I have a couple of other ideas for side projects, but it’s been hard for me to get round to actually work on them. The vast majority of my programming time is devoted to the stuff I do at work.

Which Python libraries are your favorite (core or 3rd party)?

I love numpy, pandas, matplotlib and seaborn combo for data analysis. I also liked working with DEAP when doing my research at university. As for core – typing module is my favorite.

How did you get into doing core Python development?

I just wanted to start by saying I haven’t done a lot of core Python development apart from the very small changes related to PEP 692. Anyway, here’s how I got there.

After one of my internships I got a job offer, but wanted to get my master’s degree before starting full time employment. That meant I had still one more year as a student that I wanted to enjoy before starting work full time. Then, everything got locked down due to the pandemic, which meant I had a lot of free time that I had to spend at home. Almost all of my friends were already working in industry and I was feeling like they are learning much more than I was. At the same time, no one wanted to hire me knowing that I’d leave for another job in a year. So, I decided I’d start contributing to open source projects to gain some experience and learn new things.

I started going through GitHub looking for interesting projects. That’s how I found urllib3. I’ve done a few simple PRs that got merged. Then I started working on adding type hints to the codebase. It was then when I learnt that there is no way to precisely type hint functions that use `**kwargs` – an annotation on `**kwargs` would mean that all of the keyword arguments are of that type, which in real life is rarely the case. I’ve seen an issue regarding that on mypy’s GitHub page that seemed to gain a lot of interest and has been opened for quite some time and I thought – why don’t I try fixing that?

The problem was, I knew nothing about interpreters, compilers or type checkers – it was not part of an Automatic Control and Robotics curriculum and my master’s in Computer Science was focused on data analysis. So I started reading Crafting Interpreters by Bob Nystrom to learn more about all that stuff. After that, I more or less understood how mypy was supposed to work and I could go through the codebase and understand what certain parts were responsible for. I’ve created a simple PR that probably got a lot of stuff wrong, but it worked for the cases I was interested in. I got a response from the maintainers, that the best way to move with this feature forward would be to propose a PEP. This was around the time I started full time employment, so my free time became more limited, but I still wanted to continue working on that.

I created a topic about the problem on Python’s dev mailing list. I got invited to a typing meetup where I proposed an approach to solve that problem and mostly got positive feedback. After that, I started working on the PEP, which with my chaotic approach took me around half a year to write. Then, there was a review process by the community and the Steering Council. In the end, the PEP got accepted and that’s how I ended up doing the small changes in cpython codebase.

What is your favorite obscure Python library and why?

It’s pygraphviz because it made drawing really nice graphs easy when I needed them.

Is there anything else you’d like to say?

I just wanted to say thanks for letting me be part of the series. Python’s community is very welcoming and supportive. I probably wouldn’t have gotten as far in my open source journey if it wasn’t for the help from all the people I’ve met on my path and I’m really grateful for that.

Thanks for doing the interview, Franek!

The post PyDev of the Week: Franek Magiera appeared first on Mouse Vs Python.

In accordance with our security release policy, the Django team is issuing Django 4.2.5, Django 4.1.11, and Django 3.2.21. These releases addresses the security issue detailed below. We encourage all users of Django to upgrade as soon as possible.

CVE-2023-41164: Potential denial of service vulnerability in django.utils.encoding.uri_to_iri()

django.utils.encoding.uri_to_iri() was subject to potential denial of service attack via certain inputs with a very large number of Unicode characters.

Thanks MProgrammer for the report.

This issue has severity "moderate" according to the Django security policy.

Affected supported versions

• Django main branch
• Django 4.2
• Django 4.1
• Django 3.2

Resolution

Patches to resolve the issue have been applied to Django's main branch and the 4.2, 4.1, and 3.2 release branches. The patches may be obtained from the following changesets:

• On the main branch
• On the 4.2 release branch
• On the 4.1 release branch
• On the 3.2 release branch

The following releases have been issued:

• Django 4.2.5 (download Django 4.2.5 | 4.2.5 checksums)
• Django 4.1.11 (download Django 4.1.11 | 4.1.11 checksums)
• Django 3.2.21 (download Django 3.2.21 | 3.2.21 checksums)

The PGP key ID used for this release is Mariusz Felisiak: 2EF56372BA48CD1B.

General notes regarding security reporting

As always, we ask that potential security issues be reported via private email to security@djangoproject.com, and not via Django's Trac instance or the django-developers list. Please see our security policies for further information.