FLOSS Project Planets

In this episode, we welcome Will Vincent to the Python Show Podcast. Will has written several books on Django.

We chatted about all things Django and Python. Specifically, we covered the following topics:

• Favorite Python packages

• Podcasts

• Content creation

• Book writing

• Why Django versus another web framework

• AI and writing

• and more!

Links

• Follow Will on Mastodon

• Check out Will’s work on GitHub

• Learn Django with Will

• Will Vincent’s personal website

• Django News newsletter

• Django Chat podcast

• django-microframework

As promised in the last post, I have now orphaned the Pagure Debian package. Here’s the full text I posted on the BTS:

After several years, I finally decided to orphan pagure :-(.

I haven’t been using it as my personal forge anymore, and unfortunately upstream development slowed down quite a bit after the main author and maintainer stopped contributing regularly to the project. But that is not to say that upstream is dead; they are still working towards preparing the next release.

Pagure is a big package with several components and an extensive list of build dependencies (and an even bigger testsuite which I never managed to make fully work on Debian). It is not for the faint of heart, and most of the time is usually spent fixing its (build) dependencies so that it doesn’t get removed from testing.

If I may, I would like to leave some suggestions for a future maintainer.

• I never had the time to write dep8 tests, mainly because setting up the software is not trivial. It would be great if the package had more Debian-centric testing.

• Speaking of the hurdles to setting up Pagure, I believe the package installation could be made a bit more automated using debconf. I don’t see a way to fully automate it (look at d/README.Debian), but there is certainly room for improvement.

I also left a brief TODO list inside d/README.source; feel free to tackle any item there!

I wish the next maintainer can have as much fun with the package as I did when I first made it for Debian!

Thank you,

That’s it. It was while it lasted, but I needed to feel myself unburdened so that I don’t have that constant feeling of “I should be properly maintaining this package…”.

If you feel like you’d like to give it a try at maintaining Pagure, now is the time!

n this session, we will explore how to use Python to enhance your AI projects with:

PyCon US 2024 as Security Developer-in-Residence About • Blog • Newsletter • Links PyCon US 2024 as Security Developer-in-Residence

Published 2024-06-13 by Seth Larson
Reading time: minutes

This critical role would not be possible without funding from the Alpha-Omega project. Massive thank-you to Alpha-Omega for investing in the security of the Python ecosystem!

This was my first PyCon US as the Security Developer-in-Residence. I accepted the offer shortly after PyCon US 2023, where Deb Nicholson provided an exciting cliffhanger that the PSF was close to making a hiring decision for the role during the PSF update.

The timing of my hiring to coincide with PyCon US is quite the unique opportunity, it means I get to share all the things we've been able to accomplish together in the past year all-together in person. This also means I get to share the road ahead and learn from many people I don't get to see on a day-to-day, like maintainers of projects large and small, software users at organizations and companies, and users whose work extends beyond software like scientists and industrial control systems operators.

Overall I think this yearly cadence couldn't be better, I'm already looking forward to what I'll be able to share at PyCon US 2025 back in Pittsburgh. As I've said in the past, this role wouldn't be effective without the support of the community, so I want to thank you all for the parts you've played in me being successful.

State of Python Supply Chain Security with Alpha Omega

Alpha Omega is a sponsor of the Python Software Foundation, specifically for my role! Alpha Omega's goal is to "improve global software supply chain security by partnering with open source". I presented alongside Alpha Omega's co-founder Michael Winser. The slides for our presentation are available online.

Alpha Omega uses a two-pronged approach, and it's right there in the name: Alpha and Omega:

• Alpha: Improve the security posture of the most critical projects through staffing.
• Omega: Automated security analysis, metrics, and remediation for wider range of projects.

My role is full-time staffing, so is in the "Alpha" bucket of funding.

Speaking for Alpha Omega on the state of Python supply chain security

For readers of my blog, lots of the content of the presentation is review, but I went over the following accomplishments for the first year of the role:

• Python Software Foundation as a CVE Numbering Authority (and how we helped Linux, curl, and others with a guide).
• Joining Python Security Response Team and working on process
• Working with Python release managers to move the release process to GitHub Actions as an isolated build environment.
• Build reproducibility for the Python release process.
• Software Bill-of-Materials for Python release artifacts.
• Coordinated cross-ecosystem response to libwebp and xz-utils vulnerabilities.
• Tons of community work: PEP reviewer, talks, blogs, and more.

The more interesting part for existing readers is my plans for next year. My plan is to partially shift focus, continue working with Python core developer team of course, but to start making improvements to the wider community of projects using Python as well:

• Enabling Build Provenance and Software Bill-of-Materials for Python packages.
• Adoption of security features, hardening, and best practices for Python packages.
• Special focus on Python packaging tools and workflows.

I finished the talk by describing the unique nature of this role as being flexible and how that's a boon for the rapidly changing space of software supply chain security. This role has a fairly wide scoping, which means that when things come up (like xz-utils and novel social-engineering techniques) it's in this role's scope to think about how to respond for the Python ecosystem.

“Vuln Together”, an open space on vulnerability management

“Got vulns? Let's talk!” -- I co-hosted this open space with GitHub Security champion and CVE board member Madison Oliver. This open space focused on the soft-side of vulnerability management for open source projects: people!

Managing vulnerabilities for open source projects is a non-trivial and effort intensive process, because maintainers need to create and publish a security policy, accept private vulnerability reports, and then know what reporters need, request a CVE ID, know how to estimate severity and write an advisory text, and then how to publish an advisory alongside fixed versions. Phew!

Madison and I pictured holding the open space card for “Vuln Together”

The open space was a forum to discuss difficulties with managing vulnerabilities, answering questions, and pointing folks at resources. We shared recommendations like how maintainers can easily request CVEs from GitHub, enable Private Vulnerability Reporting on GitHub and Confidential Issues on GitLab to make reporting easier, and showed resources like the Guide for Vulnerability Disclosure Process from the OpenSSF Vulnerability Disclosures working group.

The open space was well attended!

Language Summit discussion on CPython security

My bingo space for "xz" was almost immediately filled at PyCon US when I attended the Python Language Summit as the blogger. Release manager and core developer Pablo Galindo Salgado spoke about the Python contribution security model in "the wake of xz-utils backdoor".

The complete language summit blog posts are coming soon to the Python Software Foundation blog (I should know, I authored them!) so if you're interested in this topic you can stay tuned over there. I won't spoil any of the contents of the blog post here, but it was a great discussion all around. Lots of interesting ideas and thoughts from core developers that'll help with figuring process improvements that work for CPython.

Meeting all the people

Mike Fiedler and I were on the stage briefly after Kate Chapman's keynote for a "Meet the Python Software Foundation Security Engineers" segment where we went over our plans for the upcoming year and recommended folks to follow the PSF and PyPI blogs for future updates.

Of course, I got to talk to so many people, too many to name individually. I chatted with the upcoming Release Manager for CPython, Hugo van Kemenade, on some ideas to further improve the CPython release process, specifically around signatures. I chatted with folks from specific sub-ecosystems like Jannis Leidel from Conda and Jazzband, David Lord from Flask, and Jarek Potiuk from Airflow.

Also I handed out stickers! I went through over a hundred, it brought me a lot of joy seeing how much folks liked the derpy snake knight design.

... and beyond!

PyCon US 2024 is the start of conferences this year for me. Shortly after PyCon US it was announced that I will be keynoting PyCon Taiwan in September and also speaking at All Things Open 2024 in Raleigh, North Carolina. If you're attending either of these events get in contact with me (and I promise to bring stickers).

That's all for this post! 👋 If you're interested in more you can read the previous report.

Thanks for reading! ♡ Did you find this article helpful and want more content like it? Get notified of new posts by subscribing to the RSS feed or the email newsletter.

This work is licensed under CC BY-SA 4.0

Over 180 individual programs plus dozens of programmer libraries and feature plugins are released simultaneously as part of KDE Gear.

Today they all get new bugfix source releases with updated translations, including:

• elisa: Fix DBus service activation (Commit, fixes bug #487905)
• kcalc: Handle certain expressions correctly (Commit, fixes bug #487614)
• kcolorpicker: Allow picking the screen color with Wayland (Commit, fixes bug #479406)

Distro and app store packagers should update their application packages.

• 24.05 release notes for information on tarballs and known issues.
• Package download wiki page
• 24.05.1 source info page
• 24.05.1 full changelog

For FreeBSD packaging (writing “ports”, as it is called), I usually just build on my host system – either in the actual host, or with poudriere – and call it a day. If it compiles on amd64 then it should work everywhere, right? (After all, “all the world’s a VAX”)

No exciting software engineering here, BTW, just a “here’s how I set up some infrastructure to help me build ports locally in various environments.”

I have recently finally fixed the build of Sayonara, an audio player that I quite like, on amd64. It still fails on i386, though, because of a mismatch between GStreamer integer sizes and host integer sizes, where the code on the Sayonara size assumes they are the same. Gladly, 32-bit code is rarely built and tested these days, but it keeps spamming me with build failures. So I set out to fix it with an i386 VM running FreeBSD 13.2 i386.

What I wanted to do was use the ports tree from my host – it is a git checkout of the upstream FreeBSD ports tree – and clone it into my VM. Then I can do packaging fixes on the host, commit them locally, update the checkout in the VM and test the packaging fixes there, too. I want to keep those commits local so that I don’t pollute the global upstream repo with a ton of “maybe this works” kind of commits. Once I’m done, I can squash it down to one commit and push, fixing the build for once and for all (ha!).

So, serve up a git checkout to some remote. It’s remarkably simple (as long as StackOverflow answers still show up in web search; to guard against future enshittification, these notes are my own copy):

ports$ git daemon --reuseaddr --base-path=. --export-all --verbose

In my ports/ directory, run the simple and insecure git server that comes with it. Export the ports tree. On the receiving (that is, in the VM) side, clone with git:// protocol.

# git clone git://host/ /usr/ports

And voila! That enables a quick turn-around for port fifxes that need to be done on a different machine from where I have my usual ports tree checkout (and my GPG keys, and …).

During his bachelor thesis, Joel Häberli designed and implemented a framework allowing for cashless withdrawals in GNU Taler.

It's been a number of years since we made the switch to DDEV, and have regretted it not once on any given day.

# # Debian / libvirt # # By default, qemu guests network interfaces are created with the # trustGuestRxFilters attribute set to no. This setting blocks # neighbor solicitation and other ipv6 multicast traffic. To enable # neighbor solicitations, modify your interface definition in your # guest definition XML. # For example if you have a guest named bird0 with an interface # definition that looks like this: $ sudo virsh dumpxml bird0 | grep -A7 "type='direct'" <interface type='direct' trustGuestRxFilters='no'> <mac address='52:54:00:AA:BB:CC'/> <source dev='ens2f0np0' mode='bridge'/> <target dev='macvtap3'/> <model type='virtio'/> <alias name='net4'/> <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/> </interface> # Then use 'virsh edit bird0' change the attribute in the <interface> # element to have the value 'yes' instead: <interface type='direct' trustGuestRxFilters='yes'> <mac address='52:54:00:AA:BB:CC'/> <source dev='ens2f0np0' mode='bridge'/> <target dev='macvtap3'/> <model type='virtio'/> <alias name='net4'/> <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/> </interface> # If the trustGuestRxFilters attribute is missing, it defaults to no, # so be sure that the attribute exists and that the attribute has a # value of 'yes' Tweet

My Monday started with a pleasant surprise that is only possible in cross-timezone collaborations: Ted “tedbow” 1 had made a huge leap forward on #3450957: Prevent modules from being uninstalled if they provide field types used in an Experience Builder (XB) field, where he’s working on the first aspect where XB’s JSON blobs in the database must be queried.
Drupal 11 requires versions of databases that support JSON querying. Ted rather quickly discovered that JSON querying support is not consistent across different databases … but fortunately Brad “bradjones1 is working on #3343634 to add explicit JSON support to Drupal’s database abstraction layer, which is one of the endeavors sponsored by Pitch-Burgh! So: database nerds, unite!

(That means reviewing Brad’s work in that issue is yet another way of contributing to XB — in addition to helping Single Directory Components (SDC) move forward.)

To ensure we are aware of what works on which database from day one, I updated XB’s CI pipeline to run the test suite against MariaDB+MySQL+PostgreSQL+SQLite.

Missed a prior week? See all posts tagged Experience Builder.

Goal: make it possible to follow high-level progress by reading ~5 minutes/week. I hope this empowers more people to contribute when their unique skills can best be put to use!

For more detail, join the #experience-builder Slack channel. Check out the pinned items at the top!

Later on Monday, Ben “bnjmnm” and Jesse landed an MR that starts to connect the UI that you saw two weeks ago to the back end, and brought it alive: there’s now a tree view to provide a sense of place on the canvas, and much more:

Try it yourself locally if you like, but there’s not much you can do yet.
Install the 0.x branch — the “Experience Builder PoC” toolbar item takes you there!

A day later, the first set of unit tests landed: those for StructuredDataPropExpressions (these express where to retrieve values mapped for SDC props).

On Thursday, Harumi “hooromoo”’s MR to provide the foundations for undo/redo support landed, after the most enthusiastic approval I think I’ve ever seen Lee “larowlan” give! :D

That same day, his colleague Griffyn “griffynh” ran the second asynchronous XB meeting in Drupal Slack (archive on d.o: #3453097), this time with 21 participants of the ~250 people in the #experience-builder channel (~100 more compared to 2 weeks ago).
The most heavily debated topics were the data model/persistence layer and Cypress vs vitest for tests.

By Friday, I managed to finish up the work that I already shared a video of a week prior (see week 3), where a PropSource (which itself uses a PropExpression) can be chosen for each component prop and edited using Drupal’s existing Field Widgets. Crucially, it now includes a fully annotated end-to-end test that verifies the demo in detail:

• an article using an XB field can be created
• expected data is stored for the XB field, with a handful of placed components, and all prop source types: static, dynamic and adapter
• each of these prop sources can be evaluated/resolved to a concrete value to pass into the corresponding SDC prop
• each of the placed components renders into the expected HTML

In other words: it’s making the concrete implementation of #3440578: JSON-based data storage proposal for component-based page building tangible and debuggable in a way that required reading dozens of files previously! 2

Thanks to Ted & Lauri for reviewing this!

1. Forever infamous for his shining sunglasses moment on stage during DrupalCon Pittsburgh — captured it in exquisite detail by Mike “mherchel”! ↩︎

2. Hence it’ll serve as a canary for the data model/storage back-end infrastructure in the short–medium term. ↩︎

• Drupal
• Acquia
• Experience Builder

• The XB UI now gained a tree view in the left sidebar that provides a sense of place, and is kept in sync with the canvas.

One of the main data structures you learn about early in your Python learning journey is the dictionary. Dictionaries are the most common and well-known of Python’s mappings. However, there are other mappings in Python’s standard library and third-party modules. Mappings share common characteristics, and understanding these shared traits will help you use them more effectively.

In this tutorial, you’ll learn about:

• Basic characteristics of a mapping
• Operations that are common to most mappings
• Abstract base classes Mapping and MutableMapping
• User-defined mutable and immutable mappings and how to create them

This tutorial assumes that you’re familiar with Python’s built-in data types, especially dictionaries, and with the basics of object-oriented programming.

Get Your Code: Click here to download the free sample code that you’ll use to learn about mappings in Python.

Take the Quiz: Test your knowledge with our interactive “Python Mappings” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

Python Mappings

In this quiz, you'll test your understanding of the basic characteristics and operations of Python mappings. By working through this quiz, you'll revisit the key concepts and techniques of creating a custom mapping.

Understanding the Main Characteristics of Python Mappings

A mapping is a collection that allows you to look up a key and retrieve its value. The keys in mappings can be objects of a broad range of types. However, in most mappings, there are object types that can’t be used as keys, as you’ll learn later in this tutorial.

The previous paragraph described mappings as collections. A collection is an iterable container that has a defined size. However, mappings also have additional features. You’ll explore each of these mapping characteristics with examples from Python’s main mapping types.

The feature that’s most characteristic of mappings is the ability to retrieve a value using a key. You can use a dictionary to demonstrate this operation:

Python >>> points = { ... "Denise": 3, ... "Igor": 2, ... "Sarah": 3, ... "Trevor": 1, ... } >>> points["Sarah"] 3 >>> points["Matt"] Traceback (most recent call last): ... KeyError: 'Matt' Copied!

The dictionary points contains four items, each with a key and a value. You can use the key within the square brackets to fetch the value associated with that key. However, if the key doesn’t exist in the dictionary, the code raises a KeyError.

You can use one of the mappings in the standard-library collections module to assign a default value for keys that aren’t present in the collection. The defaultdict type includes a callable that’s called each time you try to access a key that doesn’t exist. If you want the default value to be zero, you can use a lambda function that returns 0 as the first argument in defaultdict:

Python >>> from collections import defaultdict >>> points_default = defaultdict( ... lambda: 0, ... points, ... ) >>> points_default defaultdict(<function <lambda> at 0x104a95da0>, {'Denise': 3, 'Igor': 2, 'Sarah': 3, 'Trevor': 1}) >>> points_default["Sarah"] 3 >>> points_default["Matt"] 0 >>> points_default defaultdict(<function <lambda> at 0x103e6c700>, {'Denise': 3, 'Igor': 2, 'Sarah': 3, 'Trevor': 1, 'Matt': 0}) Copied!

The defaultdict constructor has two arguments in this example. The first argument is the callable that’s used when a default value is needed. The second argument is the dictionary you created earlier. You can use any valid argument when you call dict() as the second argument in defaultdict() or omit this argument to create an empty defaultdict.

When you access a key that’s missing from the dictionary, the key is added, and the default value is assigned to it. You can also create the same points_default object using the callable int as the first argument since calling int() with no arguments returns 0.

All mappings are also collections, which means they’re iterable containers with a defined length. You can explore these characteristics with another mapping in Python’s standard library, collections.Counter:

Python >>> from collections import Counter >>> letters = Counter("learning python") >>> letters Counter({'n': 3, 'l': 1, 'e': 1, 'a': 1, 'r': 1, 'i': 1, 'g': 1, ' ': 1, 'p': 1, 'y': 1, 't': 1, 'h': 1, 'o': 1}) Copied!

The letters in the string "learning python" are converted into keys in Counter, and the number of occurrences of each letter is used as the value corresponding to each key.

You can confirm that this mapping is iterable, has a defined length, and is a container:

Python >>> for letter in letters: ... print(letter) ... l e a r n i g p y t h o >>> len(letters) 13 >>> "n" in letters True >>> "x" in letters False Copied!

You can use the Counter object letters in a for loop, which confirms it’s iterable. All mappings are iterable. However, the iteration loops through the keys and not the values. You’ll see how to iterate through the values or through both keys and values later in this tutorial.

The built-in len() function returns the number of items in the mapping. This is equal to the number of unique characters in the original string, including the space character. The object is sized since len() returns a value.

You can use the in keyword to confirm which elements are in the mapping. This check alone isn’t sufficient to confirm that the mapping is a container. However, you can also access the object’s .__contains__() special method directly:

Python >>> letters.__contains__("n") True Copied! Read the full article at https://realpython.com/python-mappings/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

2024 Trends: What's New for Drupal Editor Wed, 06/12/2024 - 16:00

People always want to predict the future. They say that is not good, but this doesn't apply to the Drupal 2024 trends. Undoubtedly, Generative Artificial Intelligence will shape tech trends and stimulate further progress in the website development industry.
However, there are other matters worth paying attention to. Mintz, World Wildlife Fund, Chupa Chups, Mattel, and other prominent Drupal websites won’t sit idly by viewing this chaos of innovations and advancements.
Now is the time for unique insights with Golems web development agency about what Drupal 2024 will be like. The more aware you are of prospective game-changing rules, the more up in arms you will act in SEO, marketing, and business growth strategies. Stay tuned to take a sneak peek into the future!

Taking a dive into the deep, dark quagmire of life

Some people have career aspirations from a young age, but for the rest of us, discovering a career-worthy passion, or even learning how to distinguish a job from a career, can take some time. I was nearly ten years into a job in finance (managing insurance/retirement/wealth-building products) when I finally admitted to myself that I couldn’t do it anymore. I felt like I should have a career, not just a job. I wanted to feel proud of the way I spent my days.

Diving in

After some soul-searching, I signed up for a Full-Stack Web Development bootcamp. I was never the kid who tinkered with code or tried to see how my AOL Instant Messenger worked (yes, I was a kid of the 90s) - but I suddenly found myself fascinated with the combination of language, math, and puzzle that was hiding in my browser’s Inspect tool. The bootcamp’s cost made this a risky prospect, but it also made me more likely to commit fully. I can’t say I loved every moment of the grind; working full-time, raising three kids (whom I had adopted just two days into the bootcamp), attending evening classes, keeping up with homework, and maintaining sanity was not always the easiest, but I tend to thrive in chaos. I learned the MERN Stack (MongoDB, Express.js, React, and Node), and after six months I left the bootcamp with one of the highest scores in my class and even higher hopes.

Cue the mass tech layoffs of late 2022, pitting my emerging knowledge against developers with years of experience at tech giants such as the site formerly known as Twitter. My understanding of code was evident, but my ability to explain code was underdeveloped, leaving me flailing in interviews and unable to find anyone willing to take a chance on me. Anyone, that is, until my local Women Who Code chapter posted about Palantir.net’s fellowship program. I applied, spent three months learning Drupal through DrupalEasy's Drupal Career Online program, and started an internship with Palantir.net. That internship turned into a full-time engineering role, and now I’m creating innovative tech and building professional websites from scratch.

Going deep

How could I transition from finance to React/JavaScript to Drupal/PHP in such a short time? Same way I managed to learn French — its proximity to English based on its Latin roots gave me a leg up. Both JavaScript and PHP are Object-Oriented languages, meaning they derive from making real-world objects into code. Object-Oriented Programming, or OOP, is a paradigm that encompasses several coding languages, and since I already had a base in OOP via JavaScript, I mainly needed to learn the syntax of PHP as the concepts were mostly the same. Not only are the languages similar, but the overall folder structure and architecture of both Drupal and React can be set up using the same patterns. I quickly noticed my new Palantiri teammates using the MVC (Model-View-Controller) architecture, which I was already comfortable with.

Another benefit of that MVC architecture is its collaborative nature. By separating sections of the code based on functionality, different people could work simultaneously on building one piece of a site without risking overwriting one another. React most prominently offers collaboration through its separation of interests via Components, delivering an end product to users, while Drupal is set up to not only be collaboratively built, but to allow site builders to delegate content creation, and even site administration, to users.

Surfacing from murky depths

Perhaps the coolest thing I discovered through this journey is the ability to create a site harnessing the powers of both Drupal and React! Drupal’s Content Management System structure creates a powerful backend for React’s simple and extensive front-end library. This just leaves me with the accessibility question - how can we make Drupal a more popular and attainable technology for entry-level developers? According to Statista’s data analytic survey, Drupal ranks 29th among the most-used frameworks among developers worldwide as of 2023. In the same survey, React ranked second only to Node, which is another JavaScript framework.

I think we can find a path for learners of all types, we just need to harness the power of our community to get there. We are not in competition with React or Wordpress or any other framework, Drupal is just another tool that people should have an opportunity to explore at any level. Who knows? Maybe they’ll fall in love like so many others have.

Calm waters

I love the glimmer of awe in people’s eyes now when I tell them I’m a software engineer. I can’t say the path was easy, but I’ve never regretted making the career change, and I never tire of continuing to suck up every ounce of knowledge I can find. I love the curiosity and creativity that I can bring to work with me every day, bouncing from front-end to back-end work, developing brand new technologies, and exploring every little thing that interests me. I’m happy to say I have found my career. 

Culture Drupal People

In this quiz, you’ll test your understanding of the basic characteristics and operations of Python mappings. By working through this quiz, you’ll revisit the key concepts and techniques of creating a custom mapping.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Discover why 1xINTERNET swiftly supported the Drupal Starshot Initiative. In this interview by Kazima Abbas, Baddy Sonja Breidert, CEO and Co-Founder, shares the motivations, goals, and impact behind their involvement. Don't miss these valuable insights from a key player in the Drupal community!

A while back, I wrote about using the SSH agent protocol to satisfy WebAuthn requests. The main problem with this approach is that it required starting the SSH agent with a special argument and also involved being a little too friendly with the implementation - things worked because I could provide an arbitrary public key and the implementation never validated that, but it would be legitimate for it to start doing so and then break everything. And it also only worked for keys stored on tokens that ssh supports - there was no way to extend this to other keystores on the client (such as the Secure Enclave on Macs, or TPM-backed keys on PCs). I wanted a better solution.

It turns out that it was far easier than I expected. The ssh agent protocol is documented here, and the interesting part is the extension support extension mechanism. Basically, you can declare an extension and then just tunnel whatever you want over it. As before, my goto was the go ssh agent package which conveniently implements both the client and server side of this. Implementing the local agent is trivial - look up SSH_AUTH_SOCK, connect to it, create a new agent client that can communicate with that by calling NewClient, and then implement the ExtendedAgent interface, create a new socket, and call ServeAgent against that. Most of the ExtendedAgent functions should simply call through to the original agent, with the exception of Extension(). Just add a case statement against extensionType, define some reasonably namespaced extension, and you're done.

Now you need to use this agent. You probably don't want to use this for arbitrary hosts (agent forwarding should only be enabled for remote systems you trust, not arbitrary machines you connect to - if you enabled agent forwarding for github and github got compromised, github would be able to use any private keys loaded into your agent, and you probably don't want that). So the right approach is to add a Host entry to the ssh config with a ForwardAgent stanza pointing at the socket you created in your new agent. This way the configured subset of remote hosts will automatically talk to this new custom agent, while forwarding for anything else will still be at the user's discretion.

For the remote end things are even easier. Look up SSH_AUTH_SOCK and call NewClient as before, and then simply call client.Extension(). Whatever you stick in the contents argument will simply end up being received at the client end. You now have a communication channel between a the remote system and the local client, and what you do with that is up to you. I'm using it to allow a remote system to obtain auth tokens from Okta and forward WebAuthn challenges that can either be satisfied via a local WebAuthn token or by passing the query off to Mac TouchID, but there's fundamentally no constraints whatsoever on what can be done here.

(If you want to do this on Windows and still have everything work with existing clients you'll need to take this into account - Windows didn't really do Unix sockets until recently so everything there is awful)

comments

Like each month, have a look at the work funded by Freexian’s Debian LTS offering.

Debian LTS contributors

In May, 17 contributors have been paid to work on Debian LTS, their reports are available:

• Adrian Bunk did 34.25h (out of 24.0h assigned and 22.0h from previous period), thus carrying over 11.75h to the next month.
• Bastien Roucariès did 20.0h (out of 20.0h assigned).
• Ben Hutchings did 16.0h (out of 24.0h assigned), thus carrying over 8.0h to the next month.
• Chris Lamb did 18.0h (out of 18.0h assigned).
• Daniel Leidert did 8.0h (out of 10.0h assigned), thus carrying over 2.0h to the next month.
• Emilio Pozuelo Monfort did 35.5h (out of 46.0h assigned), thus carrying over 10.5h to the next month.
• Guilhem Moulin did 13.0h (out of 14.75h assigned and 5.25h from previous period), thus carrying over 7.0h to the next month.
• Lee Garrett did 11.0h (out of 37.25h assigned and 8.75h from previous period), thus carrying over 35.0h to the next month.
• Lucas Kanashiro did 10.0h (out of 20.0h assigned), thus carrying over 10.0h to the next month.
• Markus Koschany did 40.0h (out of 40.0h assigned).
• Ola Lundqvist did 6.5h (out of 22.5h assigned and 1.5h from previous period), thus carrying over 17.5h to the next month.
• Roberto C. Sánchez did 7.75h (out of 11.0h assigned and 1.0h from previous period), thus carrying over 4.25h to the next month.
• Santiago Ruano Rincón did 8.0h (out of 16.0h assigned), thus carrying over 8.0h to the next month.
• Sean Whitton did 5.5h (out of 5.5h assigned and 0.5h from previous period), thus carrying over 0.5h to the next month.
• Sylvain Beucler did 10.5h (out of 0.75h assigned and 45.25h from previous period), thus carrying over 35.5h to the next month.
• Thorsten Alteholz did 14.0h (out of 14.0h assigned).
• Tobias Frost did 7.75h (out of 10.0h assigned and 2.0h from previous period), thus carrying over 4.25h to the next month.

Evolution of the situation

In May, we have released 20 DLAs.

Notable security updates in May included:

• apache2: multiple vulnerabilities which may result in HTTP response splitting, denial of service, or authorization bypass (by Bastien Roucariès, in collaboration with apache2 maintainer Yadd)
• bind9: two vulnerabilities, called KeyTrap and NSEC3, which may result in denial of service (by Santiago Ruano Rincón)
• python-pymysql: potential SQL injection attack (by Chris Lamb)

The aforementioned apache2 was prepared by its Debian maintainer Yadd. This update also involved work on the package test suite in buster, which contributor Bastien Roucariès then forwarded to the apache2 package in unstable. More importantly, a regression in fossil was reported, and Bastien prepared a fix for it. Bastien coordinated the upload of both packages to minimize the introduction of regressions.

Contributor Daniel Leidert also prepared an upload of runc to Debian 11 in order fix a number of CVEs still affecting that package. Finally, contributor Thorsten Alteholz prepared uploads for qtbase-opensource-src, libjwt, and libmicrohttpd in Debian 11. Note that Debian 11 will pass into the LTS phase of support in August and these updates will improve the state and long-term supportability of Debian 11.

Debian 10 is presently in its final month of LTS support (as announced on the debian-lts-announce mailing list, support will end on June 30th), after which no new security updates will be made available on security.debian.org.

However, Freexian and its team of paid Debian contributors will continue to maintain Debian 10 going forward for the customers of the Extended LTS offer. Subscribe right away if you sill have Debian 10 which must be kept secure (and which cannot yet be upgraded).

Thanks to our sponsors

Sponsors that joined recently are in bold.

• Platinum sponsors:
  • TOSHIBA (for 105 months)
  • Civil Infrastructure Platform (CIP) (for 73 months)
  • VyOS Inc (for 37 months)
• Gold sponsors:
  • Roche Diagnostics International AG (for 116 months)
  • Linode (for 110 months)
  • Babiel GmbH (for 99 months)
  • Plat’Home (for 99 months)
  • CINECA (for 73 months)
  • University of Oxford (for 55 months)
  • Deveryware (for 42 months)
  • EDF SA (for 26 months)
  • CERN
• Silver sponsors:
  • Domeneshop AS (for 120 months)
  • Nantes Métropole (for 115 months)
  • Univention GmbH (for 106 months)
  • Université Jean Monnet de St Etienne (for 106 months)
  • Ribbon Communications, Inc. (for 100 months)
  • Exonet B.V. (for 90 months)
  • Leibniz Rechenzentrum (for 84 months)
  • Ministère de l’Europe et des Affaires Étrangères (for 67 months)
  • Cloudways by DigitalOcean (for 57 months)
  • Dinahosting SL (for 55 months)
  • Bauer Xcel Media Deutschland KG (for 49 months)
  • Platform.sh SAS (for 49 months)
  • Moxa Inc. (for 43 months)
  • sipgate GmbH (for 40 months)
  • OVH US LLC (for 38 months)
  • Tilburg University (for 38 months)
  • GSI Helmholtzzentrum für Schwerionenforschung GmbH (for 30 months)
  • Soliton Systems K.K. (for 27 months)
  • THINline s.r.o.
• Bronze sponsors:
  • Evolix (for 121 months)
  • Seznam.cz, a.s. (for 121 months)
  • Intevation GmbH (for 118 months)
  • Linuxhotel GmbH (for 118 months)
  • Daevel SARL (for 116 months)
  • Bitfolk LTD (for 115 months)
  • Megaspace Internet Services GmbH (for 115 months)
  • NUMLOG (for 115 months)
  • Greenbone AG (for 114 months)
  • WinGo AG (for 114 months)
  • Ecole Centrale de Nantes - LHEEA (for 110 months)
  • Entr’ouvert (for 105 months)
  • Adfinis AG (for 102 months)
  • GNI MEDIA (for 97 months)
  • Laboratoire LEGI - UMR 5519 / CNRS (for 97 months)
  • Tesorion (for 97 months)
  • Bearstech (for 88 months)
  • LiHAS (for 88 months)
  • Catalyst IT Ltd (for 83 months)
  • Supagro (for 78 months)
  • Demarcq SAS (for 77 months)
  • Université Grenoble Alpes (for 63 months)
  • TouchWeb SAS (for 55 months)
  • SPiN AG (for 52 months)
  • CoreFiling (for 47 months)
  • Institut des sciences cognitives Marc Jeannerod (for 42 months)
  • Observatoire des Sciences de l’Univers de Grenoble (for 39 months)
  • Tem Innovations GmbH (for 34 months)
  • WordFinder.pro (for 33 months)
  • CNRS DT INSU Résif (for 32 months)
  • Alter Way (for 25 months)
  • Institut Camille Jordan (for 14 months)

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler, “download now”.

This release bumps the long-awaited 3.12 support to a complete level. Now, Nuitka behaves identically to CPython 3.12 for the most part.

In terms of bug fixes, it’s also huge. Especially for Unicode paths and software with Unicode extension module names and Unicode program names, and even non-UTF8 code names, there have been massive amounts of improvements.

Table of Contents

• Bug Fixes

• New Features

• Optimization

• Organizational

• Tests

• Cleanups

• Summary

Bug Fixes

• Standalone: Added support for python-magic-bin package. Fixed in 2.2.1 already.

• Fix: The cache directory creation could fail when multiple compilations started simultaneously. Fixed in 2.2.1 already.

• macOS: For arm64 builds, DLLs can also have an architecture dependent suffix; check that as well. Makes the soundfile dependency scan work. Fixed in 2.2.1 already.

• Fix: Modules where lazy loaders handling adds hard imports when a module is first processed did not affect the current module, potentially causing it not to resolve hidden imports. Fixed in 2.2.1 already.

• macOS: The use of libomp in numba needs to cause the extension module not to be included and not to look elsewhere. Fixed in 2.2.1 already.

• Python3.6+: Fix, added support for keyword arguments of ModuleNotFoundError. Fixed in 2.2.1 already.

• macOS: Detect more versioned DLLs and arm64 specific filenames. Fixed in 2.2.1 already.

• Fix, was not annotating exception exit when converting an import to a hard submodule import. Fixed in 2.2.2 already.

• Fix, branches that became empty can still have traces that need to be merged.

  Otherwise, usages outside the branch will not see propagated assignment statements. As a result, these falsely became unassigned instead. Fixed in 2.2.2 already.

• Windows: Fix, uninstalled self-compiled Python didn’t have proper installation prefix added for DLL scan, resulting in runtime DLLs not picked up from there. Fixed in 2.2.2 already.

• Standalone: Added support for newer PySide6 version 6.7. It needed correction on macOS and has a new data file type. Fixed in 2.2.3 already.

• Standalone: Complete support for pyocd package. Fixed in 2.2.3 already.

• Module: Fix, the created .pyi files were incomplete.

  The list of imported modules created in the finalization step was incomplete, we now go over the actual done modules and mark all non-included modules as dependencies.

• Scons: Fix, need to avoid using Unicode paths towards the linker on Windows. Instead, use a temporary output filename and rename it to the actual filename after Scons has completed.

• Windows: Avoid passing Unicode paths to the dependency walker on Windows, as it cannot handle those. Also, the temporary filenames in the build folder must be in short paths, as it cannot handle them in case that is a Unicode path.

• Scons: For ccache on Windows, the log filename must be a short path too, if the build folder is a Unicode path.

• Windows: Make sure the Scons build executes inside a short path as well, so that a potential Unicode path is visible to the C compiler when resolving the current directory.

• Windows: The encoding of Unicode paths for accelerated mode values of __file__ was not making sure that hex sequences were correctly terminated, so in some cases, it produced ambiguous C literals.

• Windows: Execute binaries created with --windows-uac-admin with and --run options with proper UAC prompt.

• Fix, need to allow for non-UTF8 Unicode in variable names, function names, class names, and method names.

• Python3.10+: Fix, match statements that captured the rest of mapping checks were not working yet.

  match value: case {"key1": 5, **rest}: ... # rest was not assigned here

• Windows: When deleting build folders, make sure the retries leading to a complete deletion always.

• Python2: Fix, could crash with non-unicode program paths on Windows.

• Avoid giving SyntaxWarning from reading source code

  For example, the standard site module of Python 3.12 gives warnings about illegal escape sequences that nobody cares about apparently.

• Fix, the matplotlib warnings by options-nanny were still given even if the no-qt plugin was used, since the variable name referenced there was not actually set yet by that plugin.

• Windows: Fix, when using the uninstalled self-compiled Python, we need python.exe to find DLL dependencies. Otherwise it doesn’t locate the MSVC runtime and Python DLL properly.

• Standalone: Added support for freetype package.

New Features

• Support for Python 3.12 is finally there. We focused on scalability first and because we did things the correct way immediately, rather than rushing to get it working and improving only later.

  As a result, the correctness and performance of Nuitka with previous Python releases are improved as well.

  Some things got delayed, though. We need to do more work to take advantage of other core changes. Concerning exceptions normalized at creation time, the created module code doesn’t yet take advantage. Also, more efficient two-digit long handling is possible with Python 3.12, but not implemented. It will take more time before we have these changes completed.

• Experimental support for Python 3.13 beta 1 is also there, and potentially surprising, but we will try and follow its release cycle closely and aim to support it at the time of release.

  Nuitka has followed all of its core changes so far, and basic tests are passing; the accelerated, module, standalone, and onefile modes all work as expected. The only thing delayed is the uncompiled generator integration, where we need to replicate the exact CPython behavior. We need to have perfect integration only for working with the asyncio loop, so we wait with it until release candidates appear.

• Plugins: Added support to include directories entirely unchanged by adding raw_dir values for data-files section, see Nuitka Package Configuration.

• UI: The new command line option --include-raw-dir was added to allow including directories entirely unchanged.

• Module: Added support for creating modules with Unicode names. Needs a different DLL entry function name and to make use of two-phase initialization for the created extension module.

• Added support for OpenBSD standalone mode.

Optimization

• Python3: Avoid API calls for allocators

  Most effective with Python 3.11 or higher but also many other types like bytes, dict keys, float, and list objects are faster to create with all Python3 versions.

• Python3.5+: Directly use the Python allocator functions for object creation, avoiding the DLL API calls. The coverage is complete with Python3.11 or higher, but many object types like float, dict, list, bytes benefit even before that version.

• Python3: Faster creation of StopIteration objects.

  With Python 3.12, the object is created directly and set as the current exception without normalization checks.

  We also added a new specialized function to create the exception object and populate it directly, avoiding the overhead of calling of the StopIteration type.

• Python3.10+: When accessing freelists, we were not passing for tstate but locally getting the interpreter object, which can be slower by a few percent in some configurations. We now use the free lists more efficient with tuple, list, and dict objects.

• Python3.8+: Call uncompiled functions via vector calls.

  We avoid an API call that ends up being slower than using the same function via the vector call directly.

• Python3.4+: Avoid using _PyObject_LengthHint API calls in list.extend and have our variant that is faster to call.

• Added specialization for os.path.normpath. We might benefit from compile time analysis of it once we want to detect file accesses.

• Avoid using module constants accessor for global constant values

  For example, with (), we used the module-level accessor for no reason, as it is already available as a global value. As a result, constant blobs shrink, and the compiled code becomes slightly smaller , too.

• Anti-Bloat: Avoid using dask from the sparse module. Added in 2.2.2 already.

Organizational

• UI: Major change in console handling.

  Compiled programs on Windows now have a third mode, besides console or not. You can now create GUI applications that attach to an available console and output there.

  The new option --console controls this and allows to enforce console with the force value and disable using it with the disable value, the attach value activates the new behavior.

  Note

  Redirection of outputs to a file in attach mode only works if it is launched correctly, for example, interactively in a shell, but some forms of invocation will not work; prominently, subprocess.call without inheritable outputs will still output to a terminal.

  On macOS, the distinction doesn’t exist anymore; technically it wasn’t valid for a while already; you need to use bundles for non-console applications, though, by default otherwise a console is forced by macOS itself.

• Detect patchelf usage in buggy version 0.18.0 and ask the user to upgrade or downgrade it, as this specific version is known to be broken.

• UI: Make clear that the --nofollow-import-to option accepts patters.

• UI: Added warning for module mode and usage of the options to force outputs as they don’t have any effect.

• UI: Check the success of Scons in creating the expected binary immediately after running it and not only once we reach post-processing.

• UI: Detect empty user package configuration files

• UI: Do not output module ast when a plugin reports an error for the module, for example, a forbidden import.

• Actions: Update from deprecated action versions to the latest versions.

Tests

• Use Nuitka Project Options for the user plugin test rather than passing by environment variables to the test runner.

• Added a new search mode, skip, `` to complement ``resume which resumes right

  after the last test resume stopped on. We can use that while support for a Python version is not complete.

Cleanups

• Solved a TODO about using unified code for setting the StopIteration, coroutines, generators, and asyncgen used to be different.

• Unified how the binary result filename is passed to Scons for modules and executables to use the same result_exe key.

Summary

This release marks a huge step in catching up with compatibility of Python. After being late with 3.12 support, we will now be early with 3.13 support if all goes well.

The many Unicode support related changes also enhanced Nuitka to generate 2 phase loading extension modules, which also will be needed for sub-interpreter support later on.

From here on, we need to re-visit compatibility. A few more obscured 3.10 features are missing, the 3.11 compatibility is not yet complete, and we need to take advantage of the new caching possibilities to enhance performance for example with attribute lookups to where it can be with the core changes there.

For the coming releases until 3.13 is released, we hope to focus on scalability a lot more and get a much needed big improvement there, and complete these other tasks on the side.

After signing up for GitHub Sponsors, I had a nagging feeling that somehow asking for money from other people to support my open source work was inappropriate. But after much reflection, I realized that phrasing the use of GitHub Sponsors as a way to express patronage/support and appreciation for my work instead of sponsorship stopped me feeling bad about it. It also led me to reflect on to what degree people can express thanks to open source maintainers.

⚠️This blog post is entirely from my personal perspective and thus will not necessarily apply to every open source developer out there.Be nice

The absolutely easiest way to show thanks is to simply not be mean. It sounds simple, but plenty of people fail at even this basic level of civility. This isn&apost to say you can&apost say that a project didn&apost work for you or you disagree with something, but there&aposs a massive difference between saying "I tried the project and it didn&apost meet my needs" and "this project is trash".

People failing to support this basic level of civility is what leads to burnout.

Be an advocate

It&aposs rather indirect, but saying nice things about a project is a way of showing thanks. As an example, I have seen various people talk positively about pyproject.toml online, but not directly at me. That still feels nice due to how much effort I put into helping make that file exist and creating the [project] table.

Or put another way, you never know who is reading your public communications.

Produce your own open source

Another indirect way to show thanks is by sharing your own open source code. By maintaining your own code, you&aposll increase the likelihood I myself will become a user of your project. That then becomes a circuitous cycle of open source support between us.

Say thanks

Directly saying "thank you" actually goes a really long way. It takes a lot of positive interactions to counteract a single negative interaction. You might be surprised how much it might brighten someone&aposs day when someone takes the time and effort to reach out and say "thank you", whether that&aposs by DM, email, in-person at a conference, etc.

Fiscal support

As I said in the opening of this post, I set up GitHub Sponsors for myself as a way for people to show fiscal support for my open source work if that&aposs how they prefer to express their thanks (including businesses). Now I&aposm purposefully not saying "sponsor" as to me that implies that giving money leads to some benefit (e.g. getting a shout-out somewhere) which is totally reasonable for people to do. But for me, since every commit is a gift, I&aposm financially secure, and I&aposm not trying to make a living from my volunteer open source work or put in the effort to make sponsorship worth it, I have chosen to treat fiscal support as a way of showing reciprocity for the gift of sharing my code that you&aposve already received. This means I fully support all open source maintainers setting up fiscal support at a minimum, and if they want to put in the effort to go the sponsorship route then they definitely should.

Producing open source also isn&apost financially free. For instance, I pay for:

1. The hosting of this blog via Ghost(Pro)
2. Obsidian Sync to keep my open source notes available on all my devices so when I have an idea I can write it down
3. Obsidian Publish to share my open source notes
4. Computer upgrades (including ergonomic upgrades like keyboards)
5. My personal time away from my wife and child, family and friends (which my open source journal exists to try and point out for those who don&apost realize how much time I put into my volunteer work)

So while open source is "free" for you as the consumer, the producer very likely has concrete financial costs in producing that open source on top of the intangible costs like volunteering their personal time.

But as I listed earlier, there are plenty of other ways to show thanks without having to spend money that can be equally valuable to a maintainer.

I also specifically didn&apost mention contributing. I have said before that contributions are like giving someone a puppy: it seems like a lovely gift at the time, but the recipient is now being "gifted" daily walks involving scooping &#x1F4A9; and vet bills. As such, contributions from others can be a blessing and a curse all at the same time depending on the contribution itself, the attitude of the person making the contribution, etc. So I wouldn&apost always assume my contribution is as welcomed and desired as much as a "thank you" note.

Authored by Nadiia Nykolaichuk, João Paulo Constantino, Ana Carolina, and Gabriel Passarelli.