Planet Python

Hi everyone,

Growth as a software engineer is an ongoing journey. Looking back, a few key principles helped me progress during the early days of my career. These lessons shaped my path, and many of them continue to guide me today, even though I’m no longer an individual contributor:

1. Drive

   Ambition isn’t a skill — it’s a will. You either have it or you don’t. To grow, you need that inner drive pushing you forward. Sometimes it’s a conscious choice, and other times, you just can’t help it — something inside you refuses to stand still, driving you to keep learning and moving forward. I remember diving into Python and CI/CD back in the day, teaching it even when I was still learning myself.

2. Delivering results

   You might have gaps in your technical or soft skills, but if you consistently deliver, that goes a long way.  I always gave extra effort (probably leaning a bit on the workaholic side), especially on projects that excited me. Delivering results also helps you build your reputation and credibility — a win-win.

3. Choosing the right projects

   Whenever possible, work on projects that have the highest impact for the company and that interest you personally. There are two benefits: high impact projects give you the visibility and future opportunities you need, and personal interest helps you push forward when the going gets tough, and the going will get tough at some point — pretty much guaranteed.

4. Craft

   Delivering high impact with speed and quality requires deep expertise in your field. Which, in turn, requires understanding what’s going on under the hood. Expanding the breadth of your knowledge while diving deep into specific technologies (the so-called T-shaped expertise) served me well.

5. Teaching

   Teaching is a great way to solidify your knowledge and identify gaps in your understanding. Teaching also boosts your visibility and can establish you as a go-to person. Sharing what you know helps others and also helps you deepen your understanding. Personally, I view well-done code reviews as a form of teaching too.

6. Tolerance for conflict

   Criticism comes with the territory. Sometimes it’s called feedback, and other times it’s just plain criticism. I took many classes at the University of Hard Knocks on this one: I missed deadlines, over-engineered technical solutions, cut corners, and delivered feedback in a less-than-perfect way at that stage of my career. My advice: start building a thick skin sooner, but be smart about it. Know when to stand your ground and when to let things slide. Above all, don’t be a jerk.

7. Learning how business works

   This one’s underrated, and it took me a long, long time to learn. I wish I’d figured it out sooner, but better late than never.  While you can succeed at a certain level without focusing on this, understanding how the business works helps you identify high-impact projects and stand out as a valued partner — not just another tech person from a cost center.

As the saying goes, “The only way to do great work is to love what you do.” I’d add — love the challenges, love the process, and love the growth that comes with them.

Stay curious,

Ruslan

What hurdles must be cleared when starting an international organization? How do you empower others in a community by sharing responsibilities? This week on the show, we speak with Jay Miller about Black Python Devs.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Conda-forge has kindly been providing support for PyPy since 2019. The conda-forge team has been very patient and generous with resources, but it seems the uptake of PyPy has not justified the effort. Major packages still are not available on PyPy, others find it hard to update versions. We don't get much feedback at all about people using PyPy, and even less about PyPy on conda-forge. The conda-forge team has proposed sunsetting PyPy going forward, which means current packages would remain but no new packages would be built. If you have an opinion, you can comment on that PR, or on this blog post.

Since conda-forge supports PyPy3.9 but not PyPy3.10, we have continued releasing PyPy3.9 even though we typically support only one version of PyPy3. With the sunsetting proposal, we will not release any more updates to PyPy3.9. I opened a poll about the intention to drop PyPy3.9. If you have an opinion, please chime in.

In this episode, we are taking a break from JourneyInbox and exploring what kind of Go app we can make by just using the Go standard library. This is the final continuation after an internet disconnection.

It’s August! For many of us, that means it’s time for hot weather and perhaps even a vacation.

But if you’re a Python/Pandas nerd like me, it’s is the perfect time to level up your programming skills. And in the coming weeks, I’ll be offering 13 (!) live, online courses on a wide variety of Python and Pandas topics, including some I’ve never before taught online.

Here’s what you can expect in all of these courses:

• In-depth explanations that go beyond the syntax
• Examples, including practical connections to real-world problems
• Exercises that help you to solidify the ideas I’ve taught
• Live coding into Jupyter, rather than teaching with slides
• Plenty of time for you to ask questions
• Lots of dad jokes

All of the classes will be recorded, so if you cannot make them at the official time, you can always watch later. I’ll also provide the Jupyter notebook and any other files I used, so that you can replay the lesson at home.

Here’s what I’ve scheduled so far; I expect to release more dates and topics in the coming weeks:

• Wednesday, August 14th: Data cleaning in Pandas
• Wednesday, August 14th: Complex Pandas queries with method chaining
• Thursday, August 15th: Design patterns in Python
• Sunday, August 18th: Python decorators
• Monday, August 19th: Textual data in Pandas
• Monday, August 19th: CSV files in Pandas
• Tuesday, August 20th: Command-line tools in Python
• Wednesday, August 21st: Grouping and pivot tables in Pandas
• Sunday, August 25th: Polars
• Monday, August 26th: Scraping with BeautifulSoup and Scrapy
• Tuesday, August 27th: Intro to FastAPI
• Sunday, September 1st: Intro to machine learning with sklearn
• Tuesday, September 3rd: asyncio

You can, of course, buy these courses individually. But they’re included in my membership programs, via LernerPython.com. The Python and Web courses are available to all members, while the Pandas and machine-learning courses are available to people who join my Python+Data membership level.

Which means: If you join me at LernerPython.com, then you get access to these courses, plus my entire catalog, plus a forum for discussion and questions, plus office hours where you can ask any questions you have. If you get a Python+Data membership, then you’ll get Bamboo Weekly, as well, with weekly Pandas challenges based on current events.

Note that my normal discounts for students, retirees/pensioners, and people in non-rich countries all apply; e-mail me at reuven@lerner.co.il if you qualify for any of these.

I’m super excited to be offering these courses, and will be back in a few weeks with even more offerings.

Meanwhile, I’m always happy to hear your questions and thoughts at reuven@lerner.co.il. All messages go straight to my personal inbox!

The post Level up your Python skills this August appeared first on Reuven Lerner.

I started writing blog 20 years ago, not on this domain, but this blog still has all the old posts starting from 8th August 2004. Though I used to write mostly one line blog posts, which is equivalent of Mastodon posts these days.

I started writing another blog, but in Swedish. So that I can feel less scared with the language.

Tools used

• Started on blogspot in 2004
• Moved to Wordpress in 2007
• Moved to Nikola, first time for me in static blogging system in 2012.
• Moved to Shonku my Golang based static blogging tool in 2013.
• Moved to khata moved to my Rust based blogging tool in 2019.

Hopefully I will write more in the coming months. But, who knows :)

As we wrap up PyCon US 2024, we can’t express enough gratitude to everyone who joined us, whether in-person or online, and made our first time together in Pittsburgh, PA a special and unforgettable experience. Not to mention, a record-breaking year - for the first time since before 2020, PyCon US sold out in-person tickets with over 2,700 tickets sold!

We had an amazing and diverse group of community members join us for PyCon US 2024, attending from 95 different countries! By the numbers, we saw a total attendance of 2,991 – with 2,551 attendees joining us in person and 440 joining us online. We couldn’t be more grateful for all who supported the Python ecosystem and helped make PyCon US 2024 a huge success.

Check out a full comprehensive recap of this year’s PyCon US conference here:

Find more photos from PyCon US 2024, captured by PSF Board Member, Kushal Das, here. 

We are also excited to announce that all PyCon US 2024 recordings are now available on the PyCon US YouTube channel! Be sure to subscribe to our channel for notifications of any new content.

We send the biggest thank you to all the presenters and speakers for their time, energy, and efforts in providing the wonderful content presented at PyCon US 2024, as well as to our incredible AV team, Altitude C, for their hard work and attention to capture recordings and provide AV this year.

The attendees, volunteers, speakers, staff, and sponsors truly make PyCon US what it is! The work of the Python Software Foundation is only possible with you all. A huge heartfelt thank you to the whole community!

We can’t wait to see you all back in Pittsburgh, PA for PyCon US 2025! If you’d like to be notified when the CFP opens and when tickets go on sale, you can watch this blog or subscribe to PyCon US News. Until then, if you have any questions or feedback, please reach out to pycon-reg@python.org.

In lieu of a formal announcement about the Third Edition, how about a progress update?

Core technology updates: Django + Python

Embarrassment-Driven Development

One of the main motivations for a third edition was that the 2e is based on Django 1.11, which dropped out of support back in 2017, and that's been a big turnoff for readers for a while, and quite embarrassing really.

So, the plan is to upgrade to Django 5.x, and progress is good -- I've already updated most of the core chapters to Django 4.2, and upgrade Python to 3.12 while I was at it. Django 5 is next, and I'm hoping/assuming it will be a smaller leap that 1->4 was, so that won't be far behind.

New Deployment Technologies: Docker + Ansible

I've always been proud that the book includes several chapters on how to actually deploy our app to production, and make the app live on the actual public Internet. But the deployment process from the first and second editions--broadly speaking, SSH in to your server, hack about to figure out how to get your app deployed manually, and then automate what you did with glorified shellscripts, aka Fabric--was starting to look less and less like what modern deployment looks like, or my experience of it at least.

I uhmmed and ahhed about it for a while, but in the end I decided to go with a deployment process that looks like this:

• Package up our app into a Docker container, and use our tests to confirm it really works
• Use Ansible to automate pushing that container onto a server and running it.

Check out the latest version of the deployment chapters here:

• Chapter 9: Containerization aka Docker
• Chapter 10: Making our App Production-Ready
• Chapter 11: Infrastructure As Code: Automated Deployments With Ansbile

I think I like how it's turned out, a lot of the fiddliness and debugging of deployment/production-readiness can now happen locally (in Docker containers on your own machine), so I think that tightens and speeds up the feedback loop a fair bit.

JavaScript

The Javascript chapter was another head-scratcher. I wanted to move away from QUnit, and include some more modern/ES6 syntax. In the end, I decided to go with Jasmine, which is old but still popular, but to keep the browser-based test runner, which is a bit of an unconventional choice, but it does mean we can avoid the whole Node.js and node_modules learning curve.

Aside from that, I've wound down the "JavaScript is such a nightmare" jokes, because they're really not fair any more, and were probably never that funny besides.

Check out the new version here:

• Chapter 16: A Gentle Excursion Into JavaScript

Some changes of emphasis

The other main changes to the book are going to be around how I talk about some of the tradeoffs involved in the use of mocking, and unit vs integration vs functional/e2e tests. I think the first and second editions were perhaps a little too opinionated on this front (I still cringe to think how defensive I was when I first wrote the Hot Lava chapter, sorry CaseY!!), and my thinking has evolved a lot since I wrote my second book with Bob.

That's still very much on the drawing board though, so you'll have to watch this space for updates on that front.

Anyways, all the latest versions of the 3e chapters are live here on the site, and also as an Early Release on O'Reilly Learning, so do dive in and let me know what you think!

When you write asynchronous code in Python, you’ll likely need to create asynchronous iterators and iterables at some point. Asynchronous iterators are what Python uses to control async for loops, while asynchronous iterables are objects that you can iterate over using async for loops.

Both tools allow you to iterate over awaitable objects without blocking your code. This way, you can perform different tasks asynchronously.

In this tutorial, you’ll:

• Learn what async iterators and iterables are in Python
• Create async generator expressions and generator iterators
• Code async iterators and iterables with the .__aiter__() and .__anext__() methods
• Use async iterators in async loops and comprehensions

To get the most out of this tutorial, you should know the basics of Python’s iterators and iterables. You should also know about Python’s asynchronous features and tools.

Get Your Code: Click here to download the free sample code that you’ll use to learn about asynchronous iterators and iterables in Python.

Take the Quiz: Test your knowledge with our interactive “Asynchronous Iterators and Iterables in Python” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

Asynchronous Iterators and Iterables in Python

Take this quiz to test your understanding of how to create and use Python async iterators and iterables in the context of asynchronous code.

Getting to Know Async Iterators and Iterables in Python

Iterators and iterables are fundamental components in Python. You’ll use them in almost all your programs where you iterate over data streams using a for loop. Iterators power and control the iteration process, while iterables typically hold data that you want to iterate over.

Python iterators implement the iterator design pattern, which allows you to traverse a container and access its elements. To implement this pattern, iterators need the .__iter__() and .__next__() special methods. Similarly, iterables are typically data containers that implement the .__iter__() method.

Note: To dive deeper into iterators and iterables, check out the Iterators and Iterables in Python: Run Efficient Iterations tutorial.

Python has extended the concept of iterators and iterables to asynchronous programming with the asyncio module and the async and await keywords. In this scenario, asynchronous iterators drive the asynchronous iteration process, mainly powered by async for loops and comprehensions.

Note: In this tutorial, you won’t dive into the intricacies of Python’s asynchronous programming. So, you should be familiar with the related concepts. If you’re not, then you can check out the following tutorials:

• Getting Started With Async Features in Python
• Async IO in Python: A Complete Walkthrough

In these tutorials, you’ll gain the required background to prepare for exploring asynchronous iterators and iterables in more depth.

In the following sections, you’ll briefly examine the concepts of asynchronous iterators and iterables in Python.

Async Iterators

Python’s documentation defines asynchronous iterators, or async iterators for short, as the following:

An object that implements the .__aiter__() and .__anext__() [special] methods. .__anext__() must return an awaitable object. [An] async for [loop] resolves the awaitables returned by an asynchronous iterator’s .__anext__() method until it raises a StopAsyncIteration exception. (Source)

Similar to regular iterators that must implement .__iter__() and .__next__(), async iterators must implement .__aiter__() and .__anext__(). In regular iterators, the .__iter__() method usually returns the iterator itself. This is also true for async iterators.

To continue with this parallelism, in regular iterators, the .__next__() method must return the next object for the iteration. In async iterators, the .__anext__() method must return the next object, which must be awaitable.

Python defines awaitable objects as described in the quote below:

An object that can be used in an await expression. [It] can be a coroutine or an object with an .__await__() method. (Source)

In practice, a quick way to make an awaitable object in Python is to call an asynchronous function. You define this type of function with the async def keyword construct. This call creates a coroutine object.

Note: You can also create awaitable objects by implementing the .__await__() special method in a custom class. This method must return an iterator that yields control back to the event loop until the awaited result is ready. This topic is beyond the scope of this tutorial.

When the data stream runs out of data, the method must raise a StopAsyncIteration exception to end the asynchronous iteration process.

Here’s an example of an async iterator that allows iterating over a range of numbers asynchronously:

Read the full article at https://realpython.com/python-async-iterators/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

We are excited to announce the continuation of Seth Larson’s work in the Security Developer-in-Residence role through the end of 2024 thanks to continued support from Alpha-Omega. (This six month extension is intended to align the renewal period for this role with the calendar year going forward).

The first year of the Security Developer-in-Residence initiative has been a success, seeing multiple improvements to the Python ecosystem's security posture. These improvements include authorizing the PSF as a CVE Numbering Authority, migrating the CPython release process to an isolated hosted build platform, and generating comprehensive Software Bill-of-Materials documents for CPython artifacts.

Open source software security continues to evolve, this year saw new regulations for software security like the EU Cyber Resiliency Act (CRA) and evolving threats to open source like the backdoor of xz-utils.

The PSF is looking forward to continuing our investment in the security of the Python ecosystem and everyone who depends on Python software. For the remainder of 2024, priorities for Security Developer-in-Residence role include:

• Formalization of the Python Security Response Team (PSRT) and processes for handling vulnerability reports and fixes.
• Developing a strategy for Software Bill-of-Materials documents and Python packages.
• Completing the migration of the CPython release process and generation of SBOM documents for the macOS installer.
• Continued engagement with the Python community promoting security best-practices and standards.

For updates on these and other projects, check out Seth’s blog.

The PSF is a non-profit whose mission is to promote, protect, and advance the Python programming language, and to support and facilitate the growth of a diverse and international community of Python programmers. The PSF supports the Python community using corporate sponsorships, grants, and donations. Are you interested in sponsoring or donating to the PSF so it can continue supporting Python and its community? Check out our sponsorship program, donate directly here, or contact our team!

The Django team is happy to announce the release of Django 5.1.

The release notes showcase a kaleidoscope of improvements. A few highlights are:

• Easier guardrails for authentication: the new and shiny LoginRequiredMiddleware, when added to MIDDLEWARE, enforces authentication for all views by default.
• A more inclusive framework: Django 5.1 includes several accessibility enhancements, such as improved screen reader support in the admin interface, more semantic HTML elements, and better association of help text and labels with form fieldsets.
• The second oldest ticket fixed in this release provides the long awaited querystring template tag, which greatly simplifies the handling of query strings when building URLs in templates.

(If you are curious about the oldest ticket fixed in this release, check out Ticket #10743.)

You can get Django 5.1 from our downloads page or from the Python Package Index. The PGP key ID used for this release is Natalia Bidart: 2EE82A8D9470983E.

With the release of Django 5.1, Django 5.0 has reached the end of mainstream support. The final minor bug fix release, 5.0.8, was issued yesterday. Django 5.0 will receive security and data loss fixes until April 2025. All users are encouraged to upgrade before then to continue receiving fixes for security issues.

See the downloads page for a table of supported versions and the future release schedule.

Test your understanding of how to create and use Python async iterators and iterables in the context of asynchronous code.

You can take this quiz after reading the Asynchronous Iterators and Iterables in Python tutorial.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

 

I'm pleased to announce the release of Python 3.12.5:

https://www.python.org/downloads/release/python-3125/

 This is the fifth maintenance release of Python 3.12

Python 3.12 is the newest major release of the Python programming language, and it contains many new features and optimizations. 3.12.5 is the latest maintenance release, containing more than 250 bugfixes, build improvements and documentation changes since 3.12.4.

This version of Python 3.12 also comes with pip 24.2 by default. However, due to an incompatibility with older macOS versions, macOS 10.9 through 10.12 will downgrade their version of pip to 24.1.2 during the installation process (in the Install Certificates step). See the installer ReadMe and the pip issue on the matter for more information. Versions of macOS older than 10.13 haven’t been supported by Apple since 2019, and maintaining support for them is becoming increasingly difficult. While this release of 3.12 still supports them, it is likely that we will be forced to drop support for macOS 10.12 and older in a future 3.12 release. (Python 3.13 has already dropped support for them.)

 

 Major new features of the 3.12 series, compared to 3.11  New features

• More flexible f-string parsing, allowing many things previously disallowed (PEP 701).
• Support for the buffer protocol in Python code (PEP 688).
• A new debugging/profiling API (PEP 669).
• Support for isolated subinterpreters with separate Global Interpreter Locks (PEP 684).
• Even more improved error messages. More exceptions potentially caused by typos now make suggestions to the user.
• Support for the Linux perf profiler to report Python function names in traces.
• Many large and small performance improvements (like PEP 709 and support for the BOLT binary optimizer), delivering an estimated 5% overall performance improvement.

Type annotations

• New type annotation syntax for generic classes (PEP 695).
• New override decorator for methods (PEP 698).

Deprecations

• The deprecated wstr and wstr_length members of the C implementation of unicode objects were removed, per PEP 623.
• In the unittest module, a number of long deprecated methods and classes were removed. (They had been deprecated since Python 3.1 or 3.2).
• The deprecated smtpd and distutils modules have been removed (see PEP 594 and PEP 632. The setuptools package continues to provide the distutils module.
• A number of other old, broken and deprecated functions, classes and methods have been removed.
• Invalid backslash escape sequences in strings now warn with SyntaxWarning instead of DeprecationWarning, making them more visible. (They will become syntax errors in the future.)
• The internal representation of integers has changed in preparation for performance enhancements. (This should not affect most users as it is an internal detail, but it may cause problems for Cython-generated code.)

For more details on the changes to Python 3.12, see What’s new in Python 3.12.

 More resources

• Online Documentation.
• PEP 693, the Python 3.12 Release Schedule.
• Report bugs via GitHub Issues.
• Help fund Python directly or via GitHub Sponsors, and support the Python community.

 Enjoy the new releases

Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation.

Your release team,
Thomas Wouters
Łukasz Langa
Ned Deily
Steve Dower

Continuous Integration / Continuous Delivery (or Deployment), CI/CD, is a set of practices used by engineering organizations to improve the quality of software they deliver, how fast they deliver that software, and detect issues with that software before they affect end users. Unfortunately, the term can mean a lot of different ideas and approaches. So, in this talk we’re going to try to unravel some of those ideas to give you some ideas on how you too can deliver software better.

#641 – AUGUST 6, 2024
View in Browser »

Setuptools Breaks Things, Then Fixes Them

This post is Bite Code’s monthly summary, but the lead story happened just days ago. In line with a 7 year old deprecation, setuptools finally removed the ability to call its test command. Many packages promptly broke. The following day the change was undone.
BITE CODE!

How to Write an Installable Django App

In this step-by-step tutorial, you’ll learn how to create an installable Django app. You’ll cover everything you need to know, from extracting your app from a Django project to turning it into a package that’s available on PyPI and installable through pip.
REAL PYTHON

Prod Alerts? You Should be Autoscaling

Let Judoscale solve your scaling issues. We support Django, Flask, and FastAPI, and we also autoscale your Celery and RQ task queues. Traffic spike? Scaled up. Quiet night? Scaled down. Work queue backlog? No problem →
JUDOSCALE sponsor

State of Flask and Pallets in 2024

Talk Python interviews David Lord, the lead maintainer of the Pallets open source organization which is responsible for Flask, Jinja, and Click. They talk about the latest for the org and Flask.
TALK PYTHON podcast

Python 3.13.0 Release Candidate 1 Released

CPYTHON DEV BLOG

Quiz: Python Strings and Character Data

REAL PYTHON

Quiz: How to Use Generators and yield in Python

REAL PYTHON

Articles & Tutorials Tips for Navigating an Issue Tracker on GitHub

Scouring an open source project’s issues can lead to an open source contribution, but there is often an overwhelming amount of issues to sift through. In this article, Stefanie shares her tips for navigating an open source project’s issue tracker to find something to work on.
STEFANIE MOLIN • Shared by Stefanie Molin

Working With JSON Data in Python

In this tutorial, you’ll learn how to read and write JSON-encoded data in Python. You’ll begin with practical examples that show how to use Python’s built-in “json” module and then move on to learn how to serialize and deserialize custom data.
REAL PYTHON

Simulate a Text File in Python

Testing an application that reads files from a disk can be complicated. It may depend on the machine, require special access, or be frustratingly slow. This course shows you how to simulate a text file using Python to simplify testing.
REAL PYTHON course

How I Use Python to Organize My Data Analyses

This is a description of how Joshua uses Python in a package-centric way to organize his approach to data analyses. This is a system he has evolved while working on his computational biology Ph.D. and working in industry.
JOSHUA COOK • Shared by Joshua Cook

Python String Formatting: Available Tools and Their Features

In this tutorial, you’ll learn about the main tools for string formatting in Python, as well as their strengths and weaknesses. These tools include f-strings, the .format() method, and the modulo operator.
REAL PYTHON

Why Does python -M json Not Work? Why Is It json.tool?

Python’s json.tool command-line interface pretty prints your JSON. Have you ever wondered why it is in json.tool instead of the module directly? This article explains the history behind this module.
TREY HUNNER

PSF Bylaws Change

Recently, the PSF board was alerted to a flaw in the bylaws that could expose the foundation to unbounded financial liability. As such, a board driven change has been instituted.
PYTHON SOFTWARE FOUNDATION

Fear of Over-Engineering Has Killed Engineering Altogether

This opinion piece outlines why the culture of rapid delivery has eroded quality engineering. It talks about how we got where we are and what should be done instead.
PAO RAMEN

Tracing the Evolution of a Python Function With git log

The git log command has arguments you can use to examine just part of a commit. This article shows you how to trace the changes to a single Python function.
JOËL PERRAS

tea-tasting: Statistical Analysis of A/B Tests

This post introduces you to the new tea-tasting project that lets you do statistical analysis on your A/B tests.
EVGENY IVANOV

Projects & Code pygamelib: Text-Mode Game Development With Python

GITHUB.COM/PYGAMELIB

git-authorship: Who Wrote Each Line in Your Repo

GITHUB.COM/THEHALE

saa: Translate Times Into Spoken Expressions

GITHUB.COM/PROTEUSIQ

python-vendorize: Vendorize Packages From PyPI

GITHUB.COM/MWILLIAMSON

django-slick-reporting: Dashboard and Reporting Engine

GITHUB.COM/RAMEZISSAC

Events Weekly Real Python Office Hours Q&A (Virtual)

August 7, 2024
REALPYTHON.COM

Python Atlanta

August 8 to August 9, 2024
MEETUP.COM

Python Nordeste 2024

August 9 to August 11, 2024
PYTHONNORDESTE.ORG

Python Communities

August 10 to August 11, 2024
NOKIDBEHIND.ORG

PyDelhi User Group Meetup

August 10, 2024
MEETUP.COM

DFW Pythoneers 2nd Saturday Teaching Meeting

August 10, 2024
MEETUP.COM

Happy Pythoning!
This was PyCoder’s Weekly Issue #641.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

There’s an amazing amount of data available on the Web. Many web services, like YouTube and GitHub, make their data accessible to third-party applications through an application programming interface (API). One of the most popular ways to build APIs is the REST architecture style. Python provides some great tools not only to get data from REST APIs but also to build your own Python REST APIs.

In this video course, you’ll learn:

• What REST architecture is
• How REST APIs provide access to web data
• How to consume data from REST APIs using the requests library
• What steps to take to build a REST API
• What some popular Python tools are for building REST APIs

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

In accordance with our security release policy, the Django team is issuing releases for Django 5.0.8 and Django 4.2.15. These releases address the security issues detailed below. We encourage all users of Django to upgrade as soon as possible.

CVE-2024-41989: Memory exhaustion in django.utils.numberformat.floatformat()

The floatformat template filter is subject to significant memory consumption when given a string representation of a number in scientific notation with a large exponent.

Thanks to Elias Myllymäki for the report.

This issue has severity "moderate" according to the Django security policy.

CVE-2024-41990: Potential denial-of-service in django.utils.html.urlize()

The urlize() and urlizetrunc() template filters are subject to a potential denial-of-service attack via very large inputs with a specific sequence of characters.

Thanks to MProgrammer for the report.

This issue has severity "moderate" according to the Django security policy.

CVE-2024-41991: Potential denial-of-service vulnerability in django.utils.html.urlize() and AdminURLFieldWidget

The urlize and urlizetrunc template filters, and the AdminURLFieldWidget widget, are subject to a potential denial-of-service attack via certain inputs with a very large number of Unicode characters.

Thanks to Seokchan Yoon for the report.

This issue has severity "moderate" according to the Django security policy.

CVE-2024-42005: Potential SQL injection in QuerySet.values() and values_list()

QuerySet.values() and values_list() methods on models with a JSONField are subject to SQL injection in column aliases via a crafted JSON object key as a passed *arg.

Thanks to Eyal Gabay of EyalSec for the report.

This issue has severity "moderate" according to the Django security policy.

Affected supported versions

• Django main branch
• Django 5.1 (currently at release candidate status)
• Django 5.0
• Django 4.2

Resolution

Patches to resolve the issue have been applied to Django's main, 5.1, 5.0, and 4.2 branches. The patches may be obtained from the following changesets.

CVE-2024-41989: Memory exhaustion in django.utils.numberformat.floatformat()

• On the main branch
• On the 5.1 branch
• On the 5.0 branch
• On the 4.2 branch

CVE-2024-41990: Potential denial-of-service in django.utils.html.urlize()

• On the main branch
• On the 5.1 branch
• On the 5.0 branch
• On the 4.2 branch

CVE-2024-41991: Potential denial-of-service vulnerability in django.utils.html.urlize() and AdminURLFieldWidget

• On the main branch
• On the 5.1 branch
• On the 5.0 branch
• On the 4.2 branch

CVE-2024-42005: Potential SQL injection in QuerySet.values() and values_list()

• On the main branch
• On the 5.1 branch
• On the 5.0 branch
• On the 4.2 branch

The following releases have been issued

• Django 5.0.8 (download Django 5.0.8 | 5.0.8 checksums)
• Django 4.2.15 (download Django 4.2.15 | 4.2.15 checksums)

The PGP key ID used for this release is Sarah Boyce: 3955B19851EA96EF

General notes regarding security reporting

As always, we ask that potential security issues be reported via private email to security@djangoproject.com, and not via Django's Trac instance, nor via the Django Forum, nor via the django-developers list. Please see our security policies for further information.

Having issues with your `pre-commit` setup? In this troubleshooting guide, I've collected the most common errors `pre-commit` users face and provided explanations and guidance for fixing them.

How to convert inconsistent datetime strings into datetime objects.

Recently I've been working on yet another rewrite of my blog, this time to FastHTML. Thanks to the power and ease of that framework, that took about 45 minutes to replicate all the web pages of my blog. Wahoo!

Alas, the atom/rss feeds took quite a bit longer.

For the atom/rss feeds I chose to use the venerable Feedgen library. The challenge there is that Feedgen is rightfully particular about the datetime objects it accepts. And over the years as this site has had 650 posts added the timestamps have become rather inconsistent in their format. On that issue I fully blame the author, who unfortunately is me.

In any case, I wrote a little Python function that handles it in a timezone aware way using the dateutils.parser() functon that I learned.

# Python stdlib from datetime import datetime from dateutils import parser # You'll need to install the pytz dependency import pytz def convert_dtstr_to_dt(date_str: str) -> datetime: """ Convert a naive or non-naive date/datetime string to a datetime object. Naive datetime strings are assumed to be in GMT (UTC) timezone. """ try: dt = parser.parse(date_str) if dt.tzinfo is None: # If the datetime object is naive, set it to GMT (UTC) dt = dt.replace(tzinfo=pytz.UTC) return dt except (ValueError, TypeError) as e: Raise Exception(f"Error parsing date string: {e}")

Original source code here.

Note: As of publishing, this article is still on my old blog. The DNS switchover to the FastHTML version of my blog happens later this week.

<strong>Topics covered in this episode:</strong><br> <ul> <li><strong><a href="https://py-free-threading.github.io?featured_on=pythonbytes">py-free-threading.github.io</a></strong></li> <li><strong><a href="https://pyfound.blogspot.com/2024/07/pythons-supportive-and-welcoming.html?featured_on=pythonbytes">Python’s Supportive and Welcoming Environment is Tightly Coupled to Its Progress</a></strong></li> <li><strong><a href="https://uptimekuma.talkpython.fm/status/talk-python?featured_on=pythonbytes">Status pages for sites</a>!</strong></li> <li><strong><a href="https://peps.python.org/pep-0751?featured_on=pythonbytes">PEP 751 – A file format to list Python dependencies for installation reproducibility</a></strong></li> <li><strong>Extras</strong></li> <li><strong>Joke</strong></li> </ul><a href='https://www.youtube.com/watch?v=Ay2u2UoTfmE' style='font-weight: bold;'data-umami-event="Livestream-Past" data-umami-event-episode="395">Watch on YouTube</a><br> <p><strong>About the show</strong></p> <p>Sponsored by us! Support our work through:</p> <ul> <li>Our <a href="https://training.talkpython.fm/?featured_on=pythonbytes"><strong>courses at Talk Python Training</strong></a></li> <li><a href="https://courses.pythontest.com/p/the-complete-pytest-course?featured_on=pythonbytes"><strong>The Complete pytest Course</strong></a></li> <li><a href="https://www.patreon.com/pythonbytes"><strong>Patreon Supporters</strong></a></li> </ul> <p><strong>Connect with the hosts</strong></p> <ul> <li>Michael: <a href="https://fosstodon.org/@mkennedy"><strong>@mkennedy@fosstodon.org</strong></a></li> <li>Brian: <a href="https://fosstodon.org/@brianokken"><strong>@brianokken@fosstodon.org</strong></a></li> <li>Show: <a href="https://fosstodon.org/@pythonbytes"><strong>@pythonbytes@fosstodon.org</strong></a></li> </ul> <p>Join us on YouTube at <a href="https://pythonbytes.fm/stream/live"><strong>pythonbytes.fm/live</strong></a> to be part of the audience. Usually Tuesdays at 10am PT. Older video versions available there too.</p> <p>Finally, if you want an artisanal, hand-crafted digest of every week of the show notes in email form? Add your name and email to <a href="https://pythonbytes.fm/friends-of-the-show">our friends of the show list</a>, we'll never share it.</p> <p><strong>Michael #1:</strong> <a href="https://py-free-threading.github.io?featured_on=pythonbytes">py-free-threading.github.io</a></p> <ul> <li>Track the status of compatibility for free-threaded Python </li> <li>See the <a href="https://py-free-threading.github.io/tracking/?featured_on=pythonbytes">Compatibility status tracking page</a> for what you can use</li> <li>Lots of resources for getting your package tested and available for <em>pythont</em></li> </ul> <p><strong>Brian #2:</strong> <a href="https://pyfound.blogspot.com/2024/07/pythons-supportive-and-welcoming.html?featured_on=pythonbytes">Python’s Supportive and Welcoming Environment is Tightly Coupled to Its Progress</a></p> <ul> <li>“Python is as popular as it is today because we have gone above and beyond to make this a welcoming community. Being a friendly and supportive community is part of how we are perceived by the wider world and is integral to the wide popularity of Python. We won a “Wonderfully Welcoming Award” last year at GitHub Universe. Over and over again, the tech press refers to Python as a supportive community.”</li> <li>Some communication recently, with the recent bylaws change, didn’t live up to our promise to be welcoming</li> <li>Please read the article for more details.</li> <li>Another quote: “We have a moral imperative – as one of the very best places to bring new people into tech and into open source – to keep being good at welcoming new people. If we do not rise and continue to rise every day to this task, then we are not fulfilling our own mission, “to support and facilitate the growth of a diverse and international community of Python programmers.” Technical skills are a game-changer for the people who acquire them and joining a vast global network of people with similar interests opens many doors. Behavior that contributes to a hostile environment around Python or throws up barriers and obstacles to those who would join the Python community must be addressed because it endangers what we have built here.”</li> </ul> <p><strong>Michael #3:</strong> <a href="https://uptimekuma.talkpython.fm/status/talk-python?featured_on=pythonbytes">Status pages for sites</a>!</p> <ul> <li>Based on <a href="https://uptime.kuma.pet?featured_on=pythonbytes">Uptime Kuma</a> I covered last week</li> <li><a href="https://uptimekuma.talkpython.fm/status/python-bytes?featured_on=pythonbytes">Python Bytes</a> status</li> <li><a href="https://uptimekuma.talkpython.fm/status/talk-python?featured_on=pythonbytes">Talk Python</a> status</li> </ul> <p><strong>Brian #4:</strong> <a href="https://peps.python.org/pep-0751?featured_on=pythonbytes">PEP 751 – A file format to list Python dependencies for installation reproducibility</a></p> <ul> <li>Brett Cannon</li> <li>Motivation <ul> <li>Currently, no standard exists to: <ul> <li>Specify what top-level dependencies should be installed into a Python environment.</li> <li>Create an immutable record, such as a lock file, of which dependencies were installed.</li> </ul></li> <li>Considering there are at least five well-known solutions to this problem in the community (pip freeze, <a href="https://pypi.org/project/pip-tools/?featured_on=pythonbytes">pip-tools</a>, <a href="https://github.com/astral-sh/uv?featured_on=pythonbytes">uv</a>, <a href="https://python-poetry.org/?featured_on=pythonbytes">Poetry</a>, and <a href="https://pypi.org/project/pdm/?featured_on=pythonbytes">PDM</a>), there seems to be an appetite for lock files in general.</li> </ul></li> <li>Rationale <ul> <li>The format is designed so that a <em>locker</em> which produces the lock file and an <em>installer</em> which consumes the lock file can be separate tools. …</li> <li>The file format is designed to be human-readable. …Finally, the format is designed so that viewing a diff of the file is easy by centralizing relevant details.</li> <li>The file format is also designed to not require a resolver at install time. …</li> </ul></li> </ul> <p><strong>Extras</strong> </p> <p>Brian:</p> <ul> <li><a href="https://courses.pythontest.com?featured_on=pythonbytes">Hello, pytest! </a>course is going well, and is purchasable as in pre-release mode. <ul> <li>Planning on Aug 19 (or before) deadline.</li> <li>Not sure what the final price will be, but I’m starting with $10. <ul> <li>I want people to want to watch it even just so see if they want to recommend to co-workers so the people around them can ramp up on pytest quickly.</li> </ul></li> </ul></li> </ul> <p>Michael:</p> <ul> <li><a href="https://mypy-lang.blogspot.com/2024/07/mypy-111-released.html?featured_on=pythonbytes">Mypy 1.11 Released</a></li> <li><a href="https://fastht.ml?featured_on=pythonbytes">FastHTML</a> (more next week)</li> <li>Coming up on the final chance to be part of <a href="https://codeinacastle.com/python-zero-to-hero-2024?utm_source=pythonbytes">the Code in a Castle event</a>.</li> </ul> <p><strong>Joke:</strong> <a href="https://devhumor.com/media/open-ai?featured_on=pythonbytes">Open source OpenAI?</a></p>