Planet Python

In this quiz, you’ll test your understanding of Python Class Constructors.

By working through this quiz, you’ll revisit the internal instantiation process, object initialization using .__init__(), and fine-tuning object creation by overriding .__new__().

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

#652 – OCTOBER 22, 2024
View in Browser »

Structural Pattern Matching in Python

In this tutorial, you’ll learn how to harness the power of structural pattern matching in Python. You’ll explore the new syntax, delve into various pattern types, and find appropriate applications for pattern matching, all while identifying common pitfalls.
REAL PYTHON

Combinatoric Iterators From itertools

The itertools module offers four combinatoric iterators that generate different combined outputs from one or more iterable. This post covers all of them: product, permutations, combinations, and combinations_with_replacement.
JUHA-MATTI SANTALA

Scrape Web Data at Scale with a 98.7% Success Rate

Extract all the data you need from any website without getting blocked with ZenRows’ Scraper API – a complete toolkit with premium proxies, anti-CAPTCHA, cloud-based scalable browsers, and more. Start your free trial now →
ZENROWS sponsor

CPython Internals: Your Guide to the Python 3 Interpreter

Unlock the inner workings of the Python language, compile the Python interpreter from source code, and participate in the development of CPython. Guido van Rossum, the creator of Python, says: “I can recommend CPython Internals to anyone who wants to get going with hacking on CPython” →
ANTHONY SHAW sponsor

SSH Scripting With Fabric and Python

Reading and writing files is a basic task that most software applications need to do, but what if you need to do that on remote machines? This tutorial introduces you to Fabric and how to connect over SSH in Python.
MIKE DRISCOLL

Python 3.14.0 Alpha 1 Released

CPYTHON DEV BLOG

Quiz: Structural Pattern Matching

REAL PYTHON

Quiz: Iterators and Iterables in Python

REAL PYTHON

Quiz: Python import: Advanced Techniques and Tips

REAL PYTHON

Articles & Tutorials Mind Your Image Metadata

Most devices record a variety of metadata when generating images. While some of that information may be innocuous, you could end up exposing the GPS coordinates to your home if you aren’t careful. In this article, Stefanie provides a brief introduction to image metadata, and then shows you how to remove it with exif-stripper.
STEFANIEMOLIN.COM • Shared by Stefanie Molin

Open Source in Python and JavaScript 2024

Python vs. JavaScript: Which open-source community is leading the way? This analysis of 36,000 GitHub repositories explores the evolution of Python and JavaScript ecosystems, highlighting key trends and popular topics. Discover how open-source communities of Python and JavaScript have shaped the tech landscape.
PYCHALLENGER.COM • Shared by Erik Nogueira Kückelheim

Accelerate Edge Devices with High-Performance AI Power

Experience the power of Edge AI—delivering lightning-fast, real-time processing where it matters. Optimize your applications to push performance and accuracy beyond limits with Intel’s OpenVINO toolkit.
INTEL CORPORATION sponsor

Using Type Hints for Multiple Return Types in Python

In this video course, you’ll learn how to define multiple return types using type hints in Python. This course covers working with single or multiple pieces of data, defining type aliases, and performing type checking using a third-party static type checker tool.
REAL PYTHON course

Narwhals: Expanding DataFrame Compatibility

How does a Python tool support all types of DataFrames and their various features? Could a lightweight library be used to add compatibility for newer formats like Polars or PyArrow? This week on the show, we speak with Marco Gorelli about his project, Narwhals.
REAL PYTHON podcast

Syntactic Sugar: Why Python Is Sweet and Pythonic

In this tutorial, you’ll learn what syntactic sugar is and how Python uses it to help you create more readable, descriptive, clean, and Pythonic code. You’ll also learn how to replace a given piece of syntactic sugar with another syntax construct.
REAL PYTHON

Entering Text in the Terminal Is Complicated

Julia asked some folks on Mastodon what they found confusing about working in a terminal. It turns out that entering text in the terminal is complicated. This post talks about why that is and how to understand it better.
JULIA EVANS

4 Lessons From Small Teams That Ship Fast

Software engineering provides a lot of leverage and small teams can do a large amount of work. This post talks about several common examples in the industry where a small group created a big product.
LEONARDO CREED

Perks of Being a Python Core Developer

Mariatta has been a Python Core Developer since 2017. If you want to know just what that means, this post talks about all the things she gets to do.
MARIATTA

Custom Dictionary Types in Pydantic

Pydantic lets you create custom types. This post talks about how to create a custom dictionary type using root models and Enums.
BRYAN ANTHONIO

How to Use Lambda Functions in Python

This article looks at some examples and best practices when using Lambda functions in Python.
FEDERICO TROTTA • Shared by AppSignal

Projects & Code ryp: R Inside Python

GITHUB.COM/WAINBERG

pyglove: Symbolic OO for Python

GITHUB.COM/GOOGLE

pipreqs: Generate requirements.txt Based on Imports

GITHUB.COM/BNDR

Bowler: Safe Code Refactoring for Modern Python

GITHUB.COM/FACEBOOKINCUBATOR

nanodjango: Full Django in a Single File

GITHUB.COM/RADIAC

Events Weekly Real Python Office Hours Q&A (Virtual)

October 23, 2024
REALPYTHON.COM

October Oslo Python Meetup

October 24, 2024
MEETUP.COM

PyCon APAC 2024

October 25 to October 27, 2024
PYCON.ID

PyCon Korea 2024

October 25 to October 28, 2024
PYCON.KR

PythonHo Conference 2024

October 26 to October 28, 2024
PYTHONHO.COM

PythOnRio Meetup

October 26, 2024
PYTHON.ORG.BR

Django Girls Aba

October 27, 2024
DJANGOGIRLS.ORG

PyCon FR 2024

October 31 to November 3, 2024
PYCON.FR

PyCon Zimbabwe

October 31 to November 3, 2024
PYCON.ORG

Happy Pythoning!
This was PyCoder’s Weekly Issue #652.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

The Python Global Interpreter Lock or GIL, in simple words, is a mutex (or a lock) that allows only one thread to hold the control of the Python interpreter.

This means that only one thread can be in a state of execution at any point in time. The impact of the GIL isn’t visible to developers who execute single-threaded programs, but it can be a performance bottleneck in CPU-bound and multi-threaded code.

Since the GIL allows only one thread to execute at a time even in a multi-threaded architecture with more than one CPU core, the GIL has gained a reputation as an “infamous” feature of Python.

In this video course you’ll learn how the GIL affects the performance of your Python programs, and how you can mitigate the impact it might have on your code.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

In this quiz, you’ll test your understanding of how to define your own Python function.

You’ll revisit theoretical knowledge about passing values to functions, when to divide your program into separate user-defined functions, and all the tools you’ll need to define complex and powerful functions in Python.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

PythonAnywhere has been around for over 10 years, and as our platform continues to grow with thousands of users, we’re committed to keeping it in top shape. Part of this involves upgrading some of the older parts of our infrastructure, with a special focus on our file storage servers—some of the oldest systems we have.

Today's exercice is just about turning a very nice example of the python soundevice module into something that works for me© to help me tune my bass.

Long story short, I suck at tuning my instrument and just lost my tuner...

This will require the python module soundevice and matplotlib.

So in order to tune my guitar I indeed need a spectrosonogram that displays the frequencies captured in real time by an audio device with an output readable enough I can actually know if I am nearing a legit frequency called a Note.

The frequencies for the notes are pretty arbitrary and I chose to only show the frequency for E, A , D, G, B since I have a 5 strings bass.
I chose the frequency between 100 and 2000 knowing that anyway any frequency below will trigger harmonics and above will trigger reasonance in the right frequency frame.

Plotting a spectrogram is done by tweaking the eponym matplotlib grapher with values chosen to fit my need and show me a laser thin beam around the right frequency. #!/usr/bin/env python3 """Show a text-mode spectrogram using live microphone data.""" import argparse import math import shutil import matplotlib.pyplot as plt from multiprocessing import Process, Queue import matplotlib.animation as animation import numpy as np import sounddevice as sd usage_line = ' press enter to quit,' def int_or_str(text): """Helper function for argument parsing.""" try: return int(text) except ValueError: return text try: columns, _ = shutil.get_terminal_size() except AttributeError: columns = 80 parser = argparse.ArgumentParser(add_help=False) parser.add_argument( '-l', '--list-devices', action='store_true', help='show list of audio devices and exit') args, remaining = parser.parse_known_args() if args.list_devices: print(sd.query_devices()) parser.exit(0) parser = argparse.ArgumentParser( description=__doc__ + '\n\nSupported keys:' + usage_line, formatter_class=argparse.RawDescriptionHelpFormatter, parents=[parser]) parser.add_argument( '-b', '--block-duration', type=float, metavar='DURATION', default=50, help='block size (default %(default)s milliseconds)') parser.add_argument( '-d', '--device', type=int_or_str, help='input device (numeric ID or substring)') parser.add_argument( '-g', '--gain', type=float, default=10, help='initial gain factor (default %(default)s)') parser.add_argument( '-r', '--range', type=float, nargs=2, metavar=('LOW', 'HIGH'), default=[50, 4000], help='frequency range (default %(default)s Hz)') args = parser.parse_args(remaining) low, high = args.range if high <= low: parser.error('HIGH must be greater than LOW') q = Queue() try: samplerate = sd.query_devices(args.device, 'input')['default_samplerate'] def plot(q): global samplerate fig, ( ax,axs) = plt.subplots(nrows=2) plt.ioff() def animate(i,q): data = q.get() ax.clear() axs.clear() axs.plot(data) ax.set_yticks([ 41.20, 82.41, 164.8, 329.6, 659.3, # E 55.00, 110.0, 220.0, 440.0, 880.0, # A 73.42, 146.8, 293.7, 587.3, # D 49.00, 98.00, 196.0, 392.0, 784.0, #G 61.74, 123.5, 246.9, 493.9, 987.8 ])#B ax.specgram(data[:,-1],mode="magnitude", Fs=samplerate*2, scale="linear",NFFT=9002) ax.set_ylim(150,1000) ani = animation.FuncAnimation(fig, animate,fargs=(q,), interval=500) plt.show() plotrt = Process(target=plot, args=(q,)) plotrt.start() def callback(indata, frames, time, status): if any(indata): q.put(indata) else: print('no input') with sd.InputStream(device=args.device, channels=1, callback=callback, blocksize=int(samplerate * args.block_duration /50 ), samplerate=samplerate) as sound: while True: response = input() if response in ('', 'q', 'Q'): break for ch in response: if ch == '+': args.gain *= 2 elif ch == '-': args.gain /= 2 else: print('\x1b[31;40m', usage_line.center(args.columns, '#'), '\x1b[0m', sep='') break except KeyboardInterrupt: parser.exit('Interrupted by user') except Exception as e: parser.exit(type(e).__name__ + ': ' + str(e))

With Python’s property(), you can create managed attributes in your classes. You can use managed attributes when you need to modify an attribute’s internal implementation and don’t want to change the class’s public API. Providing stable APIs will prevent you from breaking your users’ code when they rely on your code.

Properties are arguably the most popular way to create managed attributes quickly and in the purest Pythonic style.

In this tutorial, you’ll learn how to:

• Create managed attributes or properties in your classes
• Perform lazy attribute evaluation and provide computed attributes
• Make your classes Pythonic using properties instead of setter and getter methods
• Create read-only and read-write properties
• Create consistent and backward-compatible APIs for your classes

You’ll also write practical examples that use property() for validating input data, computing attribute values dynamically, logging your code, and more. To get the most out of this tutorial, you should know the basics of object-oriented programming, classes, and decorators in Python.

Get Your Code: Click here to download the free sample code that shows you how to use Python’s property() to add managed attributes to your classes.

Take the Quiz: Test your knowledge with our interactive “Python's property(): Add Managed Attributes to Your Classes” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

Python's property(): Add Managed Attributes to Your Classes

In this quiz, you'll test your understanding of Python's property(). With this knowledge, you'll be able to create managed attributes in your classes, perform lazy attribute evaluation, provide computed attributes, and more.

Managing Attributes in Your Classes

When you define a class in an object-oriented programming language, you’ll probably end up with some instance and class attributes. In other words, you’ll end up with variables that are accessible through the instance, class, or even both, depending on the language. Attributes represent and hold the internal state of a given object, which you’ll often need to access and mutate.

Typically, you have at least two ways to access and mutate an attribute. Either you can access and mutate the attribute directly or you can use methods. Methods are functions attached to a given class. They provide the behaviors and actions that an object can perform with its internal data and attributes.

If you expose attributes to the user, then they become part of the class’s public API. This means that your users will access and mutate them directly in their code. The problem comes when you need to change the internal implementation of a given attribute.

Say you’re working on a Circle class and add an attribute called .radius, making it public. You finish coding the class and ship it to your end users. They start using Circle in their code to create a lot of awesome projects and applications. Good job!

Now suppose that you have an important user that comes to you with a new requirement. They don’t want Circle to store the radius any longer. Instead, they want a public .diameter attribute.

At this point, removing .radius to start using .diameter could break the code of some of your other users. You need to manage this situation in a way other than removing .radius.

Programming languages such as Java and C++ encourage you to never expose your attributes to avoid this kind of problem. Instead, you should provide getter and setter methods, also known as accessors and mutators, respectively. These methods offer a way to change the internal implementation of your attributes without changing your public API.

Note: Getter and setter methods are often considered an anti-pattern and a signal of poor object-oriented design. The main argument behind this proposition is that these methods break encapsulation. They allow you to access and mutate the components of your objects from the outside.

These programming languages need getter and setter methods because they don’t have a suitable way to change an attribute’s internal implementation when a given requirement changes. Changing the internal implementation would require an API modification, which can break your end users’ code.

The Getter and Setter Approach in Python

Technically, there’s nothing that stops you from using getter and setter methods in Python. Here’s a quick example that shows how this approach would look:

Python point_v1.py class Point: def __init__(self, x, y): self._x = x self._y = y def get_x(self): return self._x def set_x(self, value): self._x = value def get_y(self): return self._y def set_y(self, value): self._y = value Copied!

In this example, you create a Point class with two non-public attributes ._x and ._y to hold the Cartesian coordinates of the point at hand.

Note: Python doesn’t have the notion of access modifiers, such as private, protected, and public, to restrict access to attributes and methods. In Python, the distinction is between public and non-public class members.

If you want to signal that a given attribute or method is non-public, then you have to use the well-known Python convention of prefixing the name with an underscore (_). That’s the reason behind the naming of the attributes ._x and ._y.

Note that this is just a convention. It doesn’t stop you and other programmers from accessing the attributes using dot notation, as in obj._attr. However, it’s bad practice to violate this convention.

To access and mutate the value of either ._x or ._y, you can use the corresponding getter and setter methods. Go ahead and save the above definition of Point in a Python module and import the class into an interactive session. Then run the following code:

Python >>> from point_v1 import Point >>> point = Point(12, 5) >>> point.get_x() 12 >>> point.get_y() 5 >>> point.set_x(42) >>> point.get_x() 42 >>> # Non-public attributes are still accessible >>> point._x 42 >>> point._y 5 Copied!

With .get_x() and .get_y(), you can access the current values of ._x and ._y. You can use the setter method to store a new value in the corresponding managed attribute. From the two final examples, you can confirm that Python doesn’t restrict access to non-public attributes. Whether or not you access them directly is up to you.

Read the full article at https://realpython.com/python-property/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

<strong>Topics covered in this episode:</strong><br> <ul> <li><a href="https://opensourcepledge.com?featured_on=pythonbytes"><strong>Open Source Pledge</strong></a></li> <li><strong>Jeff Triplet's <a href="https://djangotv.com?featured_on=pythonbytes">DjangoTV</a></strong></li> <li><strong><a href="https://peps.python.org/pep-0735/?featured_on=pythonbytes">PEP 735 – Dependency Groups in pyproject.toml</a></strong></li> <li><strong><a href="https://livereload.readthedocs.io/en/latest/index.html?featured_on=pythonbytes">livereload</a></strong></li> <li><strong>Extras</strong></li> <li><strong>Joke</strong></li> </ul><a href='https://www.youtube.com/watch?v=XzXG6cRP2Mk' style='font-weight: bold;'data-umami-event="Livestream-Past" data-umami-event-episode="406">Watch on YouTube</a><br> <p><strong>About the show</strong></p> <p>Sponsored by ScoutAPM: <a href="https://pythonbytes.fm/scout"><strong>pythonbytes.fm/scout</strong></a></p> <p><strong>Connect with the hosts</strong></p> <ul> <li>Michael: <a href="https://fosstodon.org/@mkennedy"><strong>@mkennedy@fosstodon.org</strong></a></li> <li>Brian: <a href="https://fosstodon.org/@brianokken"><strong>@brianokken@fosstodon.org</strong></a></li> <li>Show: <a href="https://fosstodon.org/@pythonbytes"><strong>@pythonbytes@fosstodon.org</strong></a></li> </ul> <p>Join us on YouTube at <a href="https://pythonbytes.fm/stream/live"><strong>pythonbytes.fm/live</strong></a> to be part of the audience. Usually <strong>Monday</strong> at 10am PT. Older video versions available there too.</p> <p>Finally, if you want an artisanal, hand-crafted digest of every week of the show notes in email form? Add your name and email to <a href="https://pythonbytes.fm/friends-of-the-show">our friends of the show list</a>, we'll never share it. </p> <p><strong>Brian #1:</strong> <a href="https://opensourcepledge.com?featured_on=pythonbytes"><strong>Open Source Pledge</strong></a></p> <ul> <li>Learned about this because of this post <ul> <li><a href="https://www.djangoproject.com/weblog/2024/oct/08/why-django-supports-the-open-source-pledge/?featured_on=pythonbytes">Why Django supports the Open Source Pledge</a></li> </ul></li> <li>Steps <ul> <li>Pay Open Source maintainers. <ul> <li>Min to participate is 2k/year/dev at your company</li> </ul></li> <li>Self-report annually <ul> <li>Publish a blog post outlining your payments</li> </ul></li> </ul></li> <li><a href="https://lucumr.pocoo.org/2024/10/14/mixing-oss-and-money/?featured_on=pythonbytes">Armin’s post</a> about launching Open Source Pledge and mixing money with open source</li> </ul> <p><strong>Michael #2:</strong> Jeff Triplet's <a href="https://djangotv.com?featured_on=pythonbytes">DjangoTV</a></p> <ul> <li>A nice aggregation of lots of Django conference talks</li> <li>Filter by conference</li> <li>Good search as well</li> </ul> <p><strong>Brian #3:</strong> <a href="https://peps.python.org/pep-0735/?featured_on=pythonbytes">PEP 735 – Dependency Groups in pyproject.toml</a></p> <ul> <li>Author: Stephen Rosen, Sponsor: Brett Cannon, PEP-Delegate: Paul Moore</li> <li>Accepted. Resolotion Oct 10, 2024</li> <li>“This PEP specifies a mechanism for storing package requirements in pyproject.toml files such that they are not included in any built distribution of the project.”</li> <li>Allow us to define named groups of dependencies that can be independent of the main project.</li> <li><p>ex:</p> <pre><code>[dependency-groups] test = ["pytest", "coverage"] docs = ["sphinx", "sphinx-rtd-theme"] typing = ["mypy", "types-requests"] typing-test = [{include-group = "typing"}, {include-group = "test"}, "useful-types"] </code></pre></li> <li><p>“might” work like this: pip install --dependency-groups=test,typing</p> <ul> <li>but tool venders are able to define how they use groups. Of course.</li> </ul></li> <li>Similar solutions <ul> <li>multiple requirements.txt files: requirements_test.txt, requirements_docs.txt, etc. <ul> <li>no standard naming convention, not standardized </li> </ul></li> <li>package extras: <ul> <li>not gauranteed to be statically defined (TIL)</li> <li>additional to main dependencies, so not independent</li> </ul></li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href="https://livereload.readthedocs.io/en/latest/index.html?featured_on=pythonbytes">livereload</a></p> <ul> <li>Example from talkpython.fm: <a href="https://gist.github.com/mikeckennedy/4e1378477a6d174aa8d59921f8db89c3?featured_on=pythonbytes"><strong>asset_bundler_watcher.py</strong></a></li> <li><a href="https://livereload.readthedocs.io/en/latest/index.html?featured_on=pythonbytes">The docs are sparse</a>, so see the gist above</li> </ul> <p><strong>Extras</strong> </p> <p>Brian:</p> <ul> <li><a href="https://kjaymiller.com/blog/personal-blogs-are-no-longer-personal-when-ai-gets-too-involved.html?featured_on=pythonbytes">Personal Blogs are no longer personal when AI gets too involved</a> - KJayMiller</li> <li><a href="https://stefaniemolin.com/articles/devx/pre-commit/exif-stripper/?featured_on=pythonbytes">Mind Your Image Metadata</a> - Stefanie Molin</li> </ul> <p>Michael:</p> <ul> <li>14% of our listeners are in Germany, thanks Germany! <ul> <li>Prost!</li> </ul></li> <li><a href="https://www.hetzner.com/cloud/?featured_on=pythonbytes">Hetzner comes to the US</a></li> </ul> <p><strong>Joke:</strong> </p> <ul> <li>A programmer’s partner asks them: “Would you go get a loaf of bread from the store? And if they have eggs, get a dozen.” </li> <li>A while later, the programmer returns with 12 loaves of bread and says “They had eggs.” <ul> <li>From https://savvyprogrammer.io/software-jokes/</li> </ul></li> </ul>

HL7 FHIR Security with Basic Auth, OAuth and SSL/TLS 2024-10-21, by Dariusz Suchojad HL7 FHIR Security

Preliminary reading: HL7 FHIR Integrations in Python

FHIR servers offer their APIs using REST, which in turn means that they are HTTP servers under the hood. As a result, a few common security mechanisms can be employed to secure access to the servers when you invoke them.

• Basic Auth
• OAuth
• SSL/TLS encryption

When you integrate with a FHIR server, consult with its maintainers what of these, if any, should be used.

Note that Basic Auth and OAuth, when used over HTTP, are mutually exclusive. The HTTP protocol reserves only one header, called Authorization, for the authorization credentials and there is no standard way in the protocol to use more than one authorization mechanism.

On the other hand, the SSL/TLS encryption is independent of the credentials used and can be employed with Basic Auth, OAuth, or in scenarios when no authorization header is sent at all.

This is why, when you create or update a FHIR connection, you can set Basic Auth or OAuth and SSL/TLS options independently, as in the screenshot below:

Basic Auth

Basic Auth is a combination of username and password credentials. They are allocated in advance by the maintainers of a FHIR server.

To use them in your integrations, create a new Basic Auth definition in the Dashboard, set its password, and assign it to an existing or new outgoing FHIR connection. No restarts nor reloads are required.

OAuth

Authentication with OAuth is built around a notion of short-lived tokens, which are simple strings. Your Zato server has credentials, much like a username and password although they are called a client ID and secret, based on which an authentication server issues tokens that let Zato prove that in fact it does have the correct credentials and that it is allowed to invoke a particular API as it is defined through scopes.

Scopes can be likened to permissions to access a subset of resources from a FHIR server. When Zato receives a token from an authentication server the token is inherently linked to specific scopes within which it can interact with the FHIR server.

That tokens are short-lived means that they need to be refreshed periodically, e.g. at least once per hour Zato needs to ask the authentication server for a new token based on its credentials. This process takes place under the hood and requires no configuration on your part.

To use OAuth in your integrations, create a new OAuth definition in the Dashboard, set its secret, and assign it to an existing or new outgoing FHIR connection. No restarts nor reloads are required.

SSL/TLS encryption

If the FHIR server that a connection points to uses SSL/TLS then a question arises of how to validate the certificate of the Certificate Authority (CA) that signed the certificate that the FHIR server uses. There are several options:

• Skip validation - the certificate will not be validated at all and it will be always accepted.
• Default bundle - the certificate will have to belong to one of the publicly recognized CAs. The bundle contains the same certificates that common web browsers do so this option is good if the FHIR server is available in the public Internet, e.g. https://fhir.example.com
• Custom bundle - if the FHIR server's certificate was signed by an internal, non-public CA then the CA's certificate bundle, in the PEM format, can be uploaded through the Dashboard to make it possible to choose it when the FHIR connection is being created or updated.

Next steps:

➤ Read about how to use Python to build and integrate enterprise APIs that your tests will cover
➤ Python API integration tutorial
➤ Python Integration platform as a Service (iPaaS)
➤ What is an Enterprise Service Bus (ESB)? What is SOA?

More blog posts➤

In part I we explored the pre-requisite in order to code print : having a grasp on the framebuffer.

Here, we are gonna deep inside one of the most overlooked object oriented abastraction : a file and actually print what we can of hello world in 100 lines of code.

The file handler and the file descriptor

These two abstractions are the low level and high level abstractions of the same thing : a view on something more complex which access has been encapsulated in generic methods. Actually when you code a framebuffer driver you provide function pointers that are specialized to your device and you may omit those common to the class. This is done with a double lookup on the major node, minor node number. Of those « generic » methods you have : seek, write, tell, truncate, open, read, close ...
The file handler in python also handles extra bytes (pun) of facilities : like character encoding, stats, and buffering.

Here, we work with the low level abstraction : the file which we access with fileno through its file descriptor. And thanks to this abstraction, you don't care if the underlying implementation fragments the file itself (ex: on a hard drive), you can magically always ask without caring for the gory details to read any arbitrary block or chararacters at any given position.

Two of the most used methods on files here are seek and write.

The file descriptor method write is sensitive to the positionning set by seek. Hence, we can write a line, and position ourselves one line below to write the next line in a character.

matrices as a view of a row based array

When I speak of rows and columns I evocate abstractions that are not related to the framebuffer.

The coordinates are an abstraction we build for convenience to say I want to write from this line at this column.
And since human beings bug after 2 dimensions we split the last dimnension in a vector of dimension 4 called a pixel.
get_at function illustrates our use of this trick to position the (invisible) cursor at any given position on the screen expressed for clarity in size of glyphes.
We could actually code all this exercice through a 3D view of the framebuffer. I just wouldn't be able to pack the code in less than 100 lines of code and would introduce useless abstractions.

But if you have doubt on the numerous seek I do and why I mutiply lines and columns value the way I do check the preceding link for an understanding of raw based array nth matricial dimensionnal views.

fonts, chars glyphs...

Here we are gonna take matrices defining the glyphes (what you actually see on screen) by 8x8 = 64 1D array and map them on the screen with put_char. Put char does a little bit of magic by relying on python to do the chararcter to glyph conversion through the dict lookup that expecting strings does a de factor codepoint to glyph conversion without having to pass the codepoint conversion.

The set of characters to glyphs conversion with their common property is a font.

The hidden console

The console is an abstraction that keeps track of the global states such as : what is the current line we print at. Thus, here, being lazy I use the global variables instead of a singleton named « console » or « term » to keep track of them. But first and foremost, these « abstractions » are just expectations we share in a common mental mode. Like we expect « print » to add a newline at the end of the string and begin the printing at the next line.

The to be finished example

I limited the code to 100 lines so that it's fairly readable. I let as an exercise the following points :

• encoding the missing glyphes in the font to actually be able to write "hello world!",
• handling the edge case of reaching the bottom of the screen,

I want to point out the true python print function is WAY MORE COMPLEX than this terse example and also handle magic conversion from memory objects to their string counterpart (like integers that are converted to their decimal representation), it handles buffering, encoding, and so much more. This is merely a toy to dive into the complexity of the mission at hand.
This example is a part of a project to write « hello world » on the framebuffer in numerous languages, bash included.

Annexe : the code

#!/usr/bin/env python3 from struct import pack from os import SEEK_CUR, lseek as seek, write w,h =map(int, open("/sys/class/graphics/fb0/virtual_size").read().split(",")) so_pixel = 4 stride = w * so_pixel encode = lambda b,g,r,a : pack("4B",b,g,r,a) font = { "height":8, "width":8, 'void' : [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, ], "l":[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, ], "o": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0], "h": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0], } def go_at(fh, x, y): global stride seek(fh.fileno(),x*so_pixel + y *stride, 0) def next_line(fh, reminder): seek(fh.fileno(), stride - reminder, SEEK_CUR) def put_char(fh, x,y, letter): go_at(fh, x, y) black = encode(0,0,0,255) white = encode(255,255,255,255) char = font.get(letter, None) or font["void"] line = "" for col,pixel in enumerate(char): write(fh.fileno(), white if pixel else black) if (col%font["width"]==font["width"]-1): next_line(fh, so_pixel * font["width"]) COL=0 LIN=0 OUT = open("/dev/fb0", "bw") FD = OUT.fileno() def newline(): global OUT,LIN,COL LIN+=1 go_at(OUT, 0, LIN * font["height"]) def print_(line): global OUT, COL, LIN COL=0 for c in line: if c == "\n": newline() put_char(OUT,COL * font["width"] , LIN * font['height'], c) COL+=1 newline() for i in range(30): print_("hello lol")

Python and Sigstore About • Blog • Cool URLs Python and Sigstore

Published 2024-10-21 by Seth Larson
Reading time: minutes

This critical role would not be possible without funding from the Alpha-Omega project. Massive thank-you to Alpha-Omega for investing in the security of the Python ecosystem!

I was a guest on the Open Source Security podcast this week talking about Sigstore and Python among other things I'm working on at the Python Software Foundation.

Sigstore is a digital signing method that has been used by CPython since 3.11.0 which focuses on ergonomics and uses short-lived keys with strongly-bound human-readable identities via OpenID Connect.

CPython also provides digital signatures using PGP and has been doing so for much longer. Below is a diagram showing the current state of affairs:

Distro (Offline?)    Distro (Offline?)    python.orgpython.orgGitHubGitHubSource Code (tar.gz)Source Code (tar.gz)PGP SignaturePGP SignatureSigstore Bundle
...@python.orgSigstore Bundle...CPython
ReleaseInfraCPython...Distro
Package
(deb, rpm)Distro...Distro
Build
InfraDistro...PGP KeysPGP KeysAccounts (GitHub, Gmail)Accounts (Git...


ReleaseManagerReleaseMan...Rebuild from sourceRebuild from...Not used for verification
Not used for...Text is not SVG - cannot display
CPython offers two verification methods: PGP and Sigstore. Verifiers choose which method to use.

From this diagram you can see the two "sources" of identity provided by PGP and Sigstore both link back to release managers. PGP relies on private keys which are maintained and protected by individual release managers to create signatures of artifacts. Sigstore uses third-parties which support OpenID Connect such as GitHub and Google to bind a human-readable identity like "thomas@python.org" to a signing certificate and signature that can be later verified.

Why Sigstore?

The problem is that securely maintaining PGP private keys is not an easy task and the burden rests on volunteers for a minimum of 7 years (1 year of pre-releases then 5 years of bug and security fixes across at least two consecutive releases). Contrast that experience to Sigstore where release managers only need to click a button to OAuth sign-in during the release process.

Sigstore externalizes the operational burden of maintaining long-lived signing keys from individual volunteers to teams of people who run an identity platform professionally and the public-good code-signing certificate authority Fulcio. This seems like a pretty good trade-off, considering we already trust these platforms' identity management teams for things like GitHub accounts.

For this reason, I've authored PEP 761 providing a deprecation and discontinuance plan of PGP signatures for CPython artifacts. You can join the discussion of this PEP on discuss.python.org. Big thank-you to Hugo van Kemenade, the release manager for Python 3.14 for sponsoring my first PEP and helping me with the process and thanks to William Woodruff for reviewing the PEP draft and explaining the nitty-gritty details of Sigstore.

This PEP deprecates the expectation that future CPython releases will provide PGP signatures and sets a timeline for discontinuance (Python 3.14) and a mechanism for that timeline to be extended by a vote of the Steering Council, if necessary.

What do verifiers need?

Signatures are only useful if they are verified, so we must weigh the needs of verifiers!

CPython's expected downstream verifiers are primarily "distributions" of CPython, such as through a Linux distro (Debian, Fedora, Gentoo, etc) or in a container image. There are other distributions of Python such as pyenv and python-build-standalone.

These users of CPython's source code are great places to verify signatures, as they're likely to be high value targets themselves and can provide a consistent stream of verifications. If any one signature verification were to fail, it would signal that something is wrong with upstream CPython artifacts or python.org and would likely be investigated and remediated quickly.

This further constrains attackers looking to affect CPython downstream users, as compromising python.org would no longer be enough. Instead, attackers would need to compromise the build infrastructure or CPython source code.

From discussions, the requirements that I've gathered from verifiers are:

• Need a tool for verification that can be packaged by distros. The recommended tool for verifying with a CLI is either Cosign or sigstore-python, both of which have challenges for Linux distro packagers.
• OS packages would require Cosign to be packaged in those OS package managers. This isn't trivial as it requires the Go toolchain to build.
• Docker and other container images want verification tools to be available at the OS level. Needing to pull these externally (from Cosign's GitHub releases) would require multi-stage builds which they want to avoid if possible. Today Cosign is available in Alpine, but not yet in Debian (but there is the beginnings of support), Gentoo, or Fedora.
• Offline verification is important. Many package ecosystems ship the signatures inside the packages to enable "build from source" (such as Gentoo), and there's no guarantee that the package is being built online. It's okay that revocation or root of trust updates can't happen when offline. Cosign and other Sigstore verification tools support offline verification if the root of trust is "bundled" as a file locally.
• Online periodic updates to revocations and root of trust. Cosign and other Sigstore verifiers supports this use-case as the default behavior.

Overall, the availability of Cosign in OS package managers appears to be the biggest blocker to see adoption for verifying CPython's Sigstore signatures.

Why adopt a new signature method?

So why have CPython's Sigstore signatures not seen adoption despite being available for multiple years? Here's a short list of self-reinforcing reasons:

• Verifiers don't need to adopt the new signature method (Sigstore), because the existing one (PGP) works and there's no expectation for discontinuance.
• Signers can't migrate away from the old signature method because there's apparent demand from verifiers for the old signature method.
• Verifiers don't try or test the new signature method, so the maintainers of signature tooling can't learn about or improve verifier use-cases.
• Concurrently supporting multiple signature methods is more work for both signers and verifiers.
• There are fewer available signatures using the new signing method, so the value of adopting the method as a verifier is less (but maybe this will change soon?).

Keep in mind that almost everyone involved in the above scenarios are volunteers. Doing work to adopt a new process when existing processes are working can feel like "busy-work", but I don't think that's the case for Sigstore.

Sigstore's benefits for ergonomics paired with its ability to use workload identity are two stand-out features compared to PGP. Workload identity being extra important now that many projects are moving to hosted build infrastructure for releases.

Workload Identity

Sigstore supporting workload identity means that release manager accounts can no longer be hijacked to produce bad signatures. Artifacts get signed by the build infrastructure provider directly:

Distro (Offline?)Distro (Offline?)python.orgpython.orgGitHubGitHubSource Code (tar.gz)Source Code (tar.gz)Sigstore Bundle
python/releaseSigstore Bundle...CPython
ReleaseInfraCPython...Distro
Package
(deb, rpm)Distro...Distro
Build
InfraDistro...Accounts (GitHub, Gmail)Accounts (Git...


ReleaseManagerReleaseMan...Rebuild from sourceRebuild from...
Workload identity: verify artifacts came from CPython release infra

Switching to workload identity also means downstream verifiers no longer need to make changes when new release managers join the project, the expected identity would always be gh/python/release-tools/....

We still have a ways to go to adopt workload identity for CPython because our macOS and Windows release processes don't use hosted build platforms that support OpenID Connect and Sigstore. That means that for now we'll keep using release manager identities.

But this future may not be far off for Python packages hosted on PyPI...

Many more signatures are coming!

William Woodruff and the team at Trail of Bits have authored PEP 740 which is provisionally accepted. The PEP specifies how attestations that can be verified by PyPI (like Sigstore) using workload identities specified with the secure publishing feature "Trusted Publishers" and then served alongside artifacts on PyPI.

There's a lot more to this story (but it's not for me to tell). Given Trusted Publishers' success, there clearly are exciting times are ahead. Subscribe to the PyPI blog to learn more once the project is complete.

That's all for this post! 👋 If you're interested in more you can read the last report.

Have thoughts or questions? Let's chat over email or social:

• sethmichaellarson@gmail.com
• @sethmlarson@fosstodon.org

Want more articles like this one? Get notified of new posts by subscribing to the RSS feed or the email newsletter. I won't share your email or send spam, only whatever this is!

Want more content now? This blog's archive has ready-to-read articles. I also curate a list of cool URLs I find on the internet.

Find a typo? This blog is open source, pull requests are appreciated.

Thanks for reading! ♡ This work is licensed under CC BY-SA 4.0

︎

In this quiz, you’ll test your understanding of Pydantic. Pydantic is a powerful data validation library for Python. You can also use a related library, pydantic-settings, for settings management.

By working through this quiz, you’ll revisit how to work with data schemas with Pydantic’s BaseModel, write custom validators for complex use cases, validate function arguments with Pydantic’s @validate_call, and manage settings and configure applications with pydantic-settings.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The « hello world » example is about standing on the shoulders of the giant and learn how to use function as tools.

Most coders will use print during their whole life without actually coding it. However, it is a fun exercise.

The framebuffer

Given you are on linux you probably have a device named /dev/fb0 if you don't, you can't do this. The framebuffer is a view of the linear graphical memory used by your video card where what you see on the screen is stored ... at the condition you are in console mode and you have the rights.

On my debian centric distribution, to give the_user permissions to write in the framebuffer I must add the_user to group_video. This can be done with sudo adduser the_user video or sudo vigr.

Then, you have to be in console mode. To switch from xorg/wayland to the console back and forth I use the Ctrl + Alt + Fn combination to switch off from X and Alt + Fn to switch back to X (it's called switching the virtual console).

Once this is done you check you have rights by doing cat /dev/urandom > /dev/fb0 which should fill your screen with random colors and insult you stating there is no more room left on the device. SNAFU : everyhting works as intended.


The pixel

Framebuffer don't know about pixels made of Red, Green, Blue and alpha (given you have a video card that is less than 20 years old), they are just made of memory. We will have to slowly build up our understanding of what this is all about.

The in memory layout may differ according to the hardware, some are having a RGBA layout, mine, the i915 is having a BGRA layout. The following example may need to be rewritten with different hardware if the output is not consistent with your assumption.

Determining the memory layout and coordinates

We will do a test and validate code session : first we make assumption on where the colours are by writting 3 squares of Red, Blue and Green on the screen, then, we will snapshot the screen.

$ cat fb.py #!/usr/bin/env python3 from struct import pack w,h =map(int, open("/sys/class/graphics/fb0/virtual_size").read().split(",")) midx = w//2 midy = h//2 encode = lambda b,g,r,a : pack("4B",b,g,r,a) with open("/dev/fb0", "wb") as f: for y in range(0,h): for x in range(0,w): f.write(encode( not x%100 and 0xA0 or x<midx and 0xFF or 0, #blue y<midy and 0xFF or 0, #green x>midx and y>midy and 0xFF or 0, #red 0, )) The only « trick » is the use of pack to encode the four colour bytes in a byte array that is written to the framebuffer filehandler. If the code works correctly we should validate the following assumptions:

• coordinates are such as 0 is top left of the screen where green and blue should superpose
• my 1920x1080 screen should have 19 weired stripes (hence validating the geometry)
• each colours should be in its square, red bottom right, green top right, blue bottom left.
• RAM as a char device is accessing a low level file

And if the world is consistent we can read from the framebuffer and snapshot it in the most trivial picture encoding witch is Portable PixMap format. A portable Pixmap is made of

• a magic number P3 followed by
• width
• height
• the maximum colour value (here 255)
• the 3 colour bytes Red, Blue, Green without Alpha value per pixel

The code for this is straigh forward : $ cat snap.py #!/usr/bin/env python from struct import unpack w,h = map( int,open("/sys/class/graphics/fb0/virtual_size").read().split(",")) # returns b g r a decode = lambda pixel : unpack("4B", pixel) def pr(b,g,r,a): print("%d %d %d" % (r,g,b)) print(f"""P3 {w} {h} 255 """) with open("/dev/fb0", "rb") as fin: while pixel := fin.read(4): pr(*decode(pixel)) Here the only trick is we use the symetrical function of pack, unpack to decode the pixel in the four colour bytes.

wrapping up part one

Asumming you can install fim the framebuffer image wiewer and you installed imagemagick : you can now do ./fb.py && snap.py > this.ppm && convert this.ppm this.jpg && fim this.jpg Doing so, you should have the same picture showing twice without an error like this : As an exercise, you can vary the fb.py to make funny output, or code a PPM viewer that print back your ppm to the screen.

It's simple and common to allocate a marker object to represent missing or null data.
MISSING = object() There's a slightly more verbose construct with some advantages:
import enum class MissingType(enum.Enum): MISSING = "MISSING" MISSING = MissingEnum.MISSING Type checkers understand that MISSING is the only possible value of MissingType; so you can use is checks:
def or_1(val: float | MissingType = MISSING) -> float: if scale is not MISSING: return 1.0 return scale mypy understands this is type correct.
More broadly, the semantics of a single-value enum are the same as a singleton. For example, neither singletons nor enums should have additional instances allocated. Instead of fixing bugs one by one with custom __init__ and __deepcopy__, the correct behaviors come for free.

Life begins. An empty page. A hollow echo on an barren stage. We wander through the quiet air, Unsure of what awaits us there. With work and study, I filled the days, But purpose wanes in fading haze. An emptiness remained inside, A restless longing I tried to hide. But then a hand, warm in the night, Turns the dark to softest light. Two souls meet and pledge their vow, To fill what life is missing now. By chance, our paths became entwined, Through joy and laughter, lives combined. Soon tiny hands, starry-eyed, Fill our home with love and pride. With every laugh, with every tear, You fill the space, bring kindness near. Two hearts become one place, Of love and hope, a shared embrace. My gratitude grows deep and wide, For every moment, side by side. Through joy, sorrow, pain, persistence, You bring meaning to my existence.

How does a Python tool support all types of DataFrames and their various features? Could a lightweight library be used to add compatibility for newer formats like Polars or PyArrow? This week on the show, we speak with Marco Gorelli about his project, Narwhals.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The match…case statement allows us to execute different actions based on the value of an expression. In this tutorial, you will learn how to use the Python match…case with the help of examples.

In this episode, I dug into an issue with sign up. What I thought was going to be a minor issue to fix turned into a massively intense debugging session. We ended up going very deep with the django-allauth package to understand what was going on.

In this episode, I dug into an issue with sign up. What I thought was going to be a minor issue to fix turned into a massively intense debugging session. We ended up going very deep with the django-allauth package to understand what was going on.

In this quiz, you’ll test your understanding of Single and Double Underscores in Python Names.

By working through this quiz, you’ll revisit Python naming conventions that rely on using underscores (_), how to differentiate public and non-public names by using a single leading underscore, how to use double leading underscores to leverage name mangling in Python classes, and other common uses of underscores in Python names.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]