Feeds

Today I released Eve 2.2. It is a maintenance release that drops old Pythons and adds support for the latest versions of the language. Long overdue, it also gets rid of some annoying deprecation warnings. As always, see the changelog for details. Many thanks to Bret Curtis and Guillaume Le Pape for their contributions to this release.

Designed by Freepik What is an “online” system?

Networking is a complex topic, and there is lots of confusion around the definition of an “online” system. Sometimes the boot process gets delayed up to two minutes, because the system still waits for one or more network interfaces to be ready. Systemd provides the network-online.target that other service units can rely on, if they are deemed to require network connectivity. But what does “online” actually mean in this context, is a link-local IP address enough, do we need a routable gateway and how about DNS name resolution?

The requirements for an “online” network interface depend very much on the services using an interface. For some services it might be good enough to reach their local network segment (e.g. to announce Zeroconf services), while others need to reach domain names (e.g. to mount a NFS share) or reach the global internet to run a web server. On the other hand, the implementation of network-online.target varies, depending on which networking daemon is in use, e.g. systemd-networkd-wait-online.service or NetworkManager-wait-online.service. For Ubuntu, we created a specification that describes what we as a distro expect an “online” system to be. Having a definition in place, we are able to tackle the network-online-ordering issues that got reported over the years and can work out solutions to avoid delayed boot times on Ubuntu systems.

In essence, we want systems to reach the following networking state to be considered online:

1. Do not wait for “optional” interfaces to receive network configuration
2. Have IPv6 and/or IPv4 “link-local” addresses on every network interface
3. Have at least one interface with a globally routable connection
4. Have functional domain name resolution on any routable interface

A common implementation

NetworkManager and systemd-networkd are two very common networking daemons used on modern Linux systems. But they originate from different contexts and therefore show different behaviours in certain scenarios, such as wait-online. Luckily, on Ubuntu we already have Netplan as a unification layer on top of those networking daemons, that allows for common network configuration, and can also be used to tweak the wait-online logic.

With the recent release of Netplan v1.1 we introduced initial functionality to tweak the behaviour of the systemd-networkd-wait-online.service, as used on Ubuntu Server systems. When Netplan is used to drive the systemd-networkd backend, it will emit an override configuration file in /run/systemd/system/systemd-networkd-wait-online.service.d/10-netplan.conf, listing the specific non-optional interfaces that should receive link-local IP configuration. In parallel to that, it defines a list of network interfaces that Netplan detected to be potential global connections, and waits for any of those interfaces to reach a globally routable state.

Such override config file might look like this:

[Unit]
ConditionPathIsSymbolicLink=/run/systemd/generator/network-online.target.wants/systemd-networkd-wait-online.service

[Service]
ExecStart=
ExecStart=/lib/systemd/systemd-networkd-wait-online -i eth99.43:carrier -i lo:carrier -i eth99.42:carrier -i eth99.44:degraded -i bond0:degraded
ExecStart=/lib/systemd/systemd-networkd-wait-online --any -o routable -i eth99.43 -i eth99.45 -i bond0

In addition to the new features implemented in Netplan, we reached out to upstream systemd, proposing an enhancement to the systemd-networkd-wait-online service, integrating it with systemd-resolved to check for the availability of DNS name resolution. Once this is implemented upstream, we’re able to fully control the systemd-networkd backend on Ubuntu Server systems, to behave consistently and according to the definition of an “online” system that was lined out above.

Future work

The story doesn’t end there, because Ubuntu Desktop systems are using NetworkManager as their networking backend. This daemon provides its very own nm-online utility, utilized by the NetworkManager-wait-online systemd service. It implements a much higher-level approach, looking at the networking daemon in general instead of the individual network interfaces. By default, it considers a system to be online once every “autoconnect” profile got activated (or failed to activate), meaning that either a IPv4 or IPv6 address got assigned.

There are considerable enhancements to be implemented to this tool, for it to be controllable in a fine-granular way similar to systemd-networkd-wait-online, so that it can be instructed to wait for specific networking states on selected interfaces.

A note of caution

Making a service depend on network-online.target is considered an antipattern in most cases. This is because networking on Linux systems is very dynamic and the systemd target can only ever reflect the networking state at a single point in time. It cannot guarantee this state to be remained over the uptime of your system and has the potentially to delay the boot process considerably. Cables can be unplugged, wireless connectivity can drop, or remote routers can go down at any time, affecting the connectivity state of your local system. Therefore, “instead of wondering what to do about network.target, please just fix your program to be friendly to dynamically changing network configuration.” [source].

Way back (more than 10 years ago) when I was doing DVD-based backups, I knew that normal DVDs/Blu-Rays are no long-term archival solutions, and that if I was real about doing optical media backups, I need to switch to M-Disc. I actually bought a (small stack) of M-Disc Blu-Rays, but never used them.

I then switched to other backups solutions, and forgot about the whole topic. Until, this week, while sorting stuff, I happened upon a set of DVD backups from a range of years, and was very curious whether they are still readable after many years.

And, to my surprise, there were no surprises! Went backward in time, and:

• 2014, TDK DVD+R, fully readable
• 2012, JVC DVD+R and TDK DVD+R, fully readable
• 2010, Verbatim DVD+R, fully readable
• 2009/2008/2007, Verbatim DVD+R, 4 DVDs, fully readable

I also found stack of dual-layer DVD+R from 2012-2014, some for sure Verbatim, and some unmarked (they were intended to be printed on), but likely Verbatim as well. All worked just fine. Just that, even at ~8GiB per disk, backing up raw photo files took way too many disks, even in 2014 😅.

At this point I was happy that all 12+ DVDs I found, ranging from 10 to 14 years, are all good. Then I found a batch of 3 CDs! Here the results were mixed:

• 2003: two TDK “CD-R80”, “Mettalic”, 700MB: fully readable, after 21 years!
• unknown year, likely around 1999-2003, but no later, “Creation” CD-R, 700MB: read errors to the extent I can’t even read the disk signature (isoinfo -d).

I think the takeaway is that for all explicitly selected media - TDK, JVC and Verbatim - they hold for 10-20 years. Valid reads from summer 2003 is mind boggling for me, for (IIRC) organic media - not sure about the “TDK metallic” substrate. And when you just pick whatever (“Creation”), well, the results are mixed.

Note that in all this, it was about CDs and DVDs. I have no idea how Blu-Rays behave, since I don’t think I ever wrote a Blu-Ray. In any case, surprising to me, and makes me rethink a bit my backup options. Sizes from 25 to 100GB Blu-Rays are reasonable for most critical data. And they’re WORM, as opposed to most LTO media, which is re-writable (and to some small extent, prone to accidental wiping).

Now, I should check those M-Disks to see if they can still be written to, after 10 years 😀

Tuesday, 15 October 2024. Today KDE releases a bugfix update to KDE Plasma 6, versioned 6.2.1.

Plasma 6.2 was released in October 2024 with many feature refinements and new modules to complete the desktop experience.

This release adds a week's worth of new translations and fixes from KDE's contributors. The bugfixes are typically small but important and include:

• Discover Snap: Don't crash when a null channel is returned. Commit. Fixes bug #492657
• Kcms/wallpaper: fix crash when wallpaper config has invalid values. Commit.
• Spacebar: Fixup SMS sending. Commit.

View full changelog

The uv Python packaging tool provides fast replacements for tools like pip, and a variety of developer experience improvements. Unlike most Python packaging tools, uv doesn’t require Python to be installed to use it. Building on that ease of installation, one of its interesting and useful features is the ability to install Python for you.

As a developer, this is great: if you need a version of Python you don’t have installed, uv can install it for you (transparently, by default!). Imagine for some reason you need the long-defunct Python 3.7. You can easily install it like so:

$ python3.7 python3.7: command not found $ uv run --python=3.7 python ... uv downloaded 3.7 for me ... Python 3.7.9 (default, Aug 23 2020, 00:57:53) [Clang 10.0.1 ] on linux Type "help", "copyright", "credits" or "license" for more information. >>>

The next time you use uv to run Python 3.7 it will use the cached download.

The ability to install Python with uv adds interesting possibilities for production packaging. For example, you can use an Ubuntu 24.04 base Docker image, download uv, and rely on uv to trivially install any Python version. Which is to say, you won’t be limited to the versions Ubuntu packages for you.

But do you want to use this particular version of Python in production? In this article we’ll look into the implications of using uv’s Python, and in particular:

• Where this version of Python comes from.
• Portability and compatibility: Can you use this Python on various versions of Linux? Will it run your software correctly?
• Performance: Are you losing speed by using this version?
• Security: Will you get security updates if you use this approach?

Read more...

RcppDate wraps the featureful date library written by Howard Hinnant for use with R. This header-only modern C++ library has been in pretty wide-spread use for a while now, and adds to C++11/C++14/C++17 what will be (with minor modifications) the ‘date’ library in C++20.

This release, the first in 3 1/2 years, syncs the code with the recent date 3.0.2 release from a few days ago. It also updates a few packaging details such as URLs, badges or continuous integration.

Changes in version 0.0.4 (2024-10-14)

• Updated to upstream version 3.0.2 (and adjusting one pragma)

• Several small updates to overall packaging and testing

Courtesy of my CRANberries, there is also a diffstat report for the most recent release. More information is available at the repository or the package page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Happy 28th Birthday KDE!

Sorry my blog updates have been MIA. Let me tell you a story…

As some of you know, 3 months ago I was in a no fault car accident. Thankfully, the only injury was I ended up with a broken arm. ER sends me home in a sling and tells me it was a clean break and it will mend itself in no time. After a week of excruciating pain I went to my follow up doctor appointment, and with my x-rays in hand, the doc tells me it was far from a clean break and needs surgery. So after a week of my shattered bone scraping my nerves and causing pain I have never felt before, I finally go in for surgery! They put in a metal plate with screws to hold the bone in place so it can properly heal. The nerve pain was gone, so I thought I was on the mend. Some time goes by and the swelling still has not subsided, the doctors are not as concerned about this as I am, so I carry on until it becomes really inflamed and developed fever blisters. After no success in reaching the doctors office my husband borrows the neighbors car and rushes me to the ER. Good thing too, I had an infection. So after a 5 day stay in the hospital, they sent us home loaded with antibiotics and trained my husband in wound packing. We did everything right, kept the place immaculate, followed orders with the wound care, took my antibiotics, yet when they ran out there was still no sign of relief, or healing. Went to doctors and they gave me another month supply of antibiotics. Two days after my final dose my arm becomes inflamed again and with extra spectacular levels of pain to go with it. I call the doctor office… They said to come in on my appointment day ( 4 days away ). I asked, “You aren’t concerned with this inflammation?”, to which they replied, “No.”. Ok, maybe I am over reacting and it’s all in my head, I can power through 4 more days. The following morning my husband observed fever blisters and the wound site was clearly not right, so once again off we go to the ER. Well… thankfully we did. I was in Sepsis and could have died… After deliberating with the doctor on the course of action for treatment, the doctor accepted our plea to remove the plate, rather than tighten screws and have me drive 100 miles to hospital everyday for iv antibiotics (Umm I don’t have a car!?) So after another 4 day stay I am released into the world, alive and well. I am happy to report, the swelling is almost gone, the pain is minimal, and I am finally healing nicely. I am still in a sling and I have to be super careful and my arm was not fully knitted. So with that I am bummed to say, no traveling for me, no Ubuntu Summit

I still need help with that car, if it weren’t for our neighbor, this story would have ended much differently.

https://gofund.me/00942f47

Despite my tragic few months for my right arm, my left arm has been quite busy. Thankfully I am a lefty! On to my work progress report.

Kubuntu:

Kubuntu 24.10 Oracular Oriole Released With Plasma 6! A big thank you to the Debian KDE/QT team and Rik Mills, could not have done it without you!

KDE Snaps:

All release service snaps are done! Save a few problematic ones still WIP.. I have released 24.08.2 which you can find here:

https://snapcraft.io/publisher/kde

I completed the qt6 and KDE frameworks 6 content packs for core24

Snapcraft:

I have a PR in for kde-neon-6 extension core24 support.

That’s all for now. Thanks for stopping by!

Happy 28th Birthday KDE!

Sorry my blog updates have been MIA. Let me tell you a story…

As some of you know, 3 months ago I was in a no fault car accident. Thankfully, the only injury was I ended up with a broken arm. ER sends me home in a sling and tells me it was a clean break and it will mend itself in no time. After a week of excruciating pain I went to my follow up doctor appointment, and with my x-rays in hand, the doc tells me it was far from a clean break and needs surgery. So after a week of my shattered bone scraping my nerves and causing pain I have never felt before, I finally go in for surgery! They put in a metal plate with screws to hold the bone in place so it can properly heal. The nerve pain was gone, so I thought I was on the mend. Some time goes by and the swelling still has not subsided, the doctors are not as concerned about this as I am, so I carry on until it becomes really inflamed and developed fever blisters. After no success in reaching the doctors office my husband borrows the neighbors car and rushes me to the ER. Good thing too, I had an infection. So after a 5 day stay in the hospital, they sent us home loaded with antibiotics and trained my husband in wound packing. We did everything right, kept the place immaculate, followed orders with the wound care, took my antibiotics, yet when they ran out there was still no sign of relief, or healing. Went to doctors and they gave me another month supply of antibiotics. Two days after my final dose my arm becomes inflamed again and with extra spectacular levels of pain to go with it. I call the doctor office… They said to come in on my appointment day ( 4 days away ). I asked, “You aren’t concerned with this inflammation?”, to which they replied, “No.”. Ok, maybe I am over reacting and it’s all in my head, I can power through 4 more days. The following morning my husband observed fever blisters and the wound site was clearly not right, so once again off we go to the ER. Well… thankfully we did. I was in Sepsis and could have died… After deliberating with the doctor on the course of action for treatment, the doctor accepted our plea to remove the plate, rather than tighten screws and have me drive 100 miles to hospital everyday for iv antibiotics (Umm I don’t have a car!?) So after another 4 day stay I am released into the world, alive and well. I am happy to report, the swelling is almost gone, the pain is minimal, and I am finally healing nicely. I am still in a sling and I have to be super careful and my arm was not fully knitted. So with that I am bummed to say, no traveling for me, no Ubuntu Summit

I still need help with that car, if it weren’t for our neighbor, this story would have ended much differently.

https://gofund.me/00942f47

Despite my tragic few months for my right arm, my left arm has been quite busy. Thankfully I am a lefty! On to my work progress report.

Kubuntu:

Kubuntu 24.10 Oracular Oriole Released With Plasma 6! A big thank you to the Debian KDE/QT team and Rik Mills, could not have done it without you!

KDE Snaps:

All release service snaps are done! Save a few problematic ones still WIP.. I have released 24.08.2 which you can find here:

https://snapcraft.io/publisher/kde

I completed the qt6 and KDE frameworks 6 content packs for core24

Snapcraft:

I have a PR in for kde-neon-6 extension core24 support.

That’s all for now. Thanks for stopping by!

Today we are talking about Freemium Drupal Modules, The WordPress hub-bub, and Drupal, Now with AI with our hosts. We’ll also cover FullCalendar as our module of the week.

For show notes visit: https://www.talkingDrupal.com/471

Topics

• Freemium Drupal
• Wordpress controversy
• Drupal CMS and AI

Resources

• Dries Wordpress Blog Post
• Non-Code Contribution: Using your passion and skills to power open source.
• DrupalCon Barcelona Driesnote
• Drupal AI

Guests Hosts

Nic Laflin - nLighteneddevelopment.com nicxvan John Picozzi - epam.com johnpicozzi Aubrey Sambor - star-shaped.org starshaped Martin Anderson-Clutz - mandclu.com mandclu

MOTW Correspondent

Martin Anderson-Clutz - mandclu.com mandclu

• Brief description:
  • Have you ever wanted an interactive calendar to display your Drupal events with drag-and-drop rescheduling, and without using jQuery? There’s a module for that.
• Module name/project name:
  • FullCalendar
• Brief history
  • How old: created in Sep 2010 by ablondeau, though I’ve been behind the most recent releases
  • Versions available: 7.x-2.0 and 3.0.0-beta2 versions available, the latter of which supports Drupal 10 and 11
• Maintainership
  • Actively maintained, latest release was this morning
  • Security coverage, though technically the 3.0.x branch will have it once it’s stable
  • Test coverage, minimal but on the roadmap
  • Documentation - does have a user guide, but created for the D7 version, so newer documentation is needed
  • Number of open issues: 337 open issues, none of which are bugs against the 3.0.x branch
• Usage stats:
  • 3,388 sites, though the vast majority of those are for the D7 version, since the 3.0.x branch is very new
• Module features and usage
  • No jQuery!
  • Lots of configurability plus some extras specifically for Drupal
    • Drag-and-drop to alter events
    • Option to require confirmation
    • Can display toast-style notifications when updates are save
    • Double-click on a day or time to create an event at that time
    • Can display events from different content types, even if they use different fields to store dates, and yes, even different kinds of fields, so a mixture of core and Smart Date fields will work
    • You can set default colors and output type (block or the newer, list-item display), and the ability to override color based on content type or a taxonomy reference
  • This module had been essentially dormant for over 4 years, but I decided to work with Jürgen Haas on reviving it after a similar and popular project called Fullcalendar View was not only marked as “Minimally maintained” and “Maintenance fixes only”, but the project page directed users to contact the maintainer to pay for a premium version, in order to use the current version of the Fullcalendar JS library, or to load events via AJAX, which as been an often-requested feature because Fullcalendar View has had common reports of performance problems on sites with lots of event data.
  • Worse, the maintainer has closed as “won’t fix” issues that had community-provided patches, because he only wanted to provide said improvements in the paid, premium version
  • In my work on the Events recipe for Drupal CMS, I knew that having a solid calendar would be important, and I didn’t feel good about relying on a module that seemed to be pushing users more and more towards a paid model. I’m grateful to Jurgen and everyone who worked on FullCalendar before us for creating such a robust and extensible code base

The Drupal Association has published this guest blog on behalf of HeroDevs.

The official end-of-life (EOL) date for Drupal 7 is January 5, 2025. However, as outlined in the PSA issued on June 7, 2023, the Drupal Security Team announced a change in their support strategy that would take effect before the official EOL date. Starting August 1, 2023, they implemented a reduced support structure for moderately critical Drupal 7 issues.

Why the Reduced Support Structure Matters

When a Drupal 7 module is marked as unsupported, it means that no further updates or security patches will be provided. This situation can leave your site vulnerable to potential security risks and negatively impact performance. As highlighted in the PSA, the Drupal Security Team follows a structured process: they first notify maintainers and provide them with a two-week window to respond and address the issue. If maintainers do not act within this timeframe, the module may be marked as unsupported, and support will cease altogether. This can create significant challenges, particularly for sites reliant on these modules. 

HeroDevs’ Proactive and Comprehensive Approach  

At HeroDevs, we understand the urgency of this issue. We’ve proactively stepped in to bridge the gap by forking and maintaining modules that are no longer supported by their original maintainers, with Drupal 7 Never-Ending Support. Our approach means that even as official support dwindles, your modules continue to receive the necessary updates and fixes. By taking over security maintenance, we help keep your site secure and operational, allowing you to focus on what matters most without worrying about vulnerabilities or disruptions.

Forking and Maintaining Unsupported Modules: HeroDevs has already begun forking and maintaining modules that have been dropped by their original maintainers in the last year. This approach makes sure that essential functionality remains intact and that any emerging security vulnerabilities are promptly addressed. By doing so, we help maintain the security and integrity of your site even as support from the Drupal community wanes.

Guidance for Custom and Legacy Modules: While custom modules and unique code are not covered under the standard SLA, HeroDevs provides guidance and support to help you integrate and maintain these solutions. We collaborate with you to ensure that your custom developments are compatible and functional with our Drupal 7 NES offering.

Maintaining Compatibility and Functionality: Beyond just security patches, HeroDevs works to test the compatibility of your modules with the evolving web landscape. We address compatibility issues and provide a seamless experience for developers working with legacy systems. This comprehensive support approach helps you avoid disruptions and maintain smooth operations.

Conclusion

As the Drupal 7 ecosystem transitions into its extended support phase, HeroDevs is committed to delivering robust and proactive support for unsupported modules. Our dedication to maintaining security, functionality, and compatibility means you can rely on us to safeguard your Drupal 7 site and navigate the end-of-life transition with confidence. With HeroDevs by your side, you can focus on planning your migration or upgrades while we handle the challenges of unsupported modules. Contact us to learn more about Drupal 7 NES.

Python has several pieces of syntax that are syntactic sugar. This sugar is syntax that isn’t strictly necessary but gives Python some of its flavor as a readable, beginner-friendly, and powerful language. In this tutorial, you’ll explore some of Python’s most used pieces of syntactic sugar.

In practice, you already use most of these pieces of syntax, as they include many well-known Pythonic constructs. As you read on, you’ll see how Python works under the hood and learn how to use the language efficiently and securely.

In this tutorial, you’ll learn:

• What syntactic sugar is
• How syntactic sugar applies to operators
• How assignment expressions are syntactic sugar
• How for loops and comprehensions are syntactic sugar
• How other Python constructs are also syntactic sugar

To get the most out of this tutorial, you should be familiar with the basics of Python, including operators, expressions, loops, decorators, classes, context managers, and more.

Get Your Code: Click here to download the free sample code that shows you how to use syntactic sugar in Python.

Take the Quiz: Test your knowledge with our interactive “Syntactic Sugar: Why Python Is Sweet and Pythonic” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

Syntactic Sugar: Why Python Is Sweet and Pythonic

You can take this quiz to test your understanding of Python's most common pieces of syntactic sugar and how they make your code more Pythonic and readable.

Syntactic Sugar

In programming, syntactic sugar refers to pieces of syntax that simplify the code and make it more readable or concise. Syntactic sugar lets you express things in a clearer and more readable way.

It makes the language sweeter for human use: things can be expressed more clearly, more concisely, or in an alternative style that some may prefer. (Source)

However, syntactic sugar is something that you may not need in practice because you can get the same result using a different, and often more involved, construct.

Note: This tutorial is slightly inspired by Brett Cannon’s series of posts about unraveling syntactic sugar in Python. In that series, Brett goes deep into each piece of syntactic sugar. You can check out the series if you’d like a detailed discussion of the syntax constructs covered in this tutorial and others.

Python has many pieces of syntactic sugar that you’ll regularly use in your code. These syntax constructs make Python more readable, quicker to write, and user-friendly. Understanding these syntactic sugar pieces and their significance will help you better understand the inner workings of Python.

In rare situations, you’ll find that desugared versions of a given piece of syntactic sugar can better fulfill your needs. So, knowing about the alternative code to a given sugar can be a good skill to have.

Operators in Python

As with most programming languages, Python makes extensive use of operators. You’ll find several categories of operators, including arithmetic, assignment, augmented assignment, comparison, Boolean, and membership operators. All these operators are part of Python’s syntactic sugar constructs because they let you write expressions in a quick and readable way.

Note: To dive deeper into Python operators, check out the Operators and Expressions in Python tutorial.

For example, arithmetic operators allow you to create math expressions that are quick to write and read because they look pretty similar to what you learned in math class:

Python >>> 5 + 7 12 >>> 10 - 4 6 >>> 2 * 4 8 >>> 20 / 2 10 Copied!

In the first example, you use the plus operator (+) to add two numbers. In the second example, you use the subtraction operator (-) to subtract two numbers. The final two examples perform multiplication and division.

Python supports its arithmetic operators through special methods. Here’s a quick summary:

Operator Operation Method + Addition .__add__() - Subtraction .__sub__() * Multiplication .__mul__() / Division .__truediv__() // Integer division .__floordiv__() ** Exponentiation .__pow__()

What does it mean to say Python supports its operators through special methods? It means that every time you use an operator, Python calls the corresponding special method under the hood.

Note: To learn more about special methods, also known as magic or dunder methods, check out Python’s Magic Methods: Leverage Their Power in Your Classes.

To illustrate, here’s how you can express the arithmetic operations you wrote earlier using the appropriate special methods:

Read the full article at https://realpython.com/syntactic-sugar-python/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

I wrote my last post on fpdf2 18 months ago. We released 7 more versions of fpdf2 since then!

This article will present some of the major features introduced since v2.7.3 to v2.8.1 of fpdf2: click on the buttons below to reveal the various changes brought …

— Permalink

When setting up your YubiKey you have the option to require the user to touch the device to authorize an operation (be it signing, decrypting, or authenticating). While web browsers often provide clear prompts for this, other applications like SSH or GPG will not. Instead the operation will just hang without any visual indication that user input is required. The YubiKey itself will blink, but depending on where it is plugged in that is not very visible.

yubikey-touch-detector (fresh in unstable) solves this issue by providing a way for your desktop environment to signal the user that the device is waiting for a touch. It provides an event feed on a socket that other components can consume. It comes with libnotify support and there are some custom integrations for other environments.

For GNOME and KDE libnotify support should be sufficient, however you still need to turn it on:

$ mkdir -p ~/.config/yubikey-touch-detector $ sed -e 's/^YUBIKEY_TOUCH_DETECTOR_LIBNOTIFY=.*/YUBIKEY_TOUCH_DETECTOR_LIBNOTIFY=true/' \ < /usr/share/doc/yubikey-touch-detector/examples/service.conf.example \ > ~/.config/yubikey-touch-detector/service.conf $ systemctl --user restart yubikey-touch-detector

I would still have preferred a more visible, more modal prompt. I guess that would be an exercise for another time, listening to the socket and presenting a window. But for now, desktop notifications will do for me.

PS: I have not managed to get SSH's no-touch-required to work with YubiKey 4, while it works just fine with a YubiKey 5.

Managing Enterprise-Level Drupal Projects Editor Mon, 10/14/2024 - 14:16

In this era where websites are like storefronts for many businesses, there is an increasing need for strong and flexible CMS. More than 2.3 million websites worldwide, including some of the biggest and most complex digital platforms, use Drupal!
But, handling large Drupal projects for enterprises is a twist. As the need for complex digital experiences grows, managing such setups comes with many difficulties. You have to make certain that everything works smoothly together, from integrating well with current systems to keeping high-level safety measures intact while also optimizing performance to handle heavy traffic.

Community Driven Development For many years in the Drupal community, I’ve found that my greatest pleasure is not in actually creating the next awesome feature (and, wow, we have some awesome features coming…) but in helping to mentor those who will in the future. Recently, an opportunity came up that interested me a lot. Cambridge University Press & Assessment were looking for a seasoned Drupal developer to help grow the in-house team as they deliver projects built on the platform. I could hardly say no! I very much wanted to look at how better active participation in the community can be a key driver for development of the team’s capabilities.. Rachel Mon, 10/14/2024 - 09:01 Tags

• Drupal
• OpenSource

• Read more about Community Driven Development
• 1 comment
• Log in with GitHub or Google to post comments. Or, if you came across this post via Mastodon, just reply there!

<strong>Topics covered in this episode:</strong><br> <ul> <li><strong><a href="https://github.com/briefercloud/briefer?featured_on=pythonbytes">Briefer: Dashboards and notebooks in a single place</a></strong></li> <li><strong><a href="https://bioinf.comav.upv.es/courses/intro_programming/?featured_on=pythonbytes">Introduction to programming with Python</a></strong></li> <li><strong><a href="https://github.com/astral-sh/setup-uv?featured_on=pythonbytes">setup-uv</a></strong></li> <li><a href="https://htmlforpeople.com?featured_on=pythonbytes"><strong>HTML for people</strong></a></li> <li><strong>Extras</strong></li> <li><strong>Joke</strong></li> </ul><a href='https://www.youtube.com/watch?v=EB2emah5d5g' style='font-weight: bold;'data-umami-event="Livestream-Past" data-umami-event-episode="405">Watch on YouTube</a><br> <p><strong>About the show</strong></p> <p>Sponsored by ScoutAPM: <a href="https://pythonbytes.fm/scout"><strong>pythonbytes.fm/scout</strong></a></p> <p><strong>Connect with the hosts</strong></p> <ul> <li>Michael: <a href="https://fosstodon.org/@mkennedy"><strong>@mkennedy@fosstodon.org</strong></a></li> <li>Brian: <a href="https://fosstodon.org/@brianokken"><strong>@brianokken@fosstodon.org</strong></a></li> <li>Show: <a href="https://fosstodon.org/@pythonbytes"><strong>@pythonbytes@fosstodon.org</strong></a></li> </ul> <p>Join us on YouTube at <a href="https://pythonbytes.fm/stream/live"><strong>pythonbytes.fm/live</strong></a> to be part of the audience. Usually <strong>Monday</strong> at 10am PT. Older video versions available there too.</p> <p>Finally, if you want an artisanal, hand-crafted digest of every week of the show notes in email form? Add your name and email to <a href="https://pythonbytes.fm/friends-of-the-show">our friends of the show list</a>, we'll never share it.</p> <p><strong>Michael #1:</strong> <a href="https://github.com/briefercloud/briefer?featured_on=pythonbytes">Briefer: Dashboards and notebooks in a single place</a></p> <ul> <li>Notebooks and dashboards with Python, SQL, scheduling, native visualizations, code generation, and more.</li> <li>In Briefer, you can: <ul> <li><img src="https://paper.dropboxstatic.com/static/img/ace/emoji/1f4da.png?version=8.0.0" alt="books" /><img src="https://paper.dropboxstatic.com/static/img/ace/emoji/1f4ca.png?version=8.0.0" alt="bar chart" /> <strong>Create notebooks and dashboards</strong> using Markdown, Python, SQL, and native visualizations.</li> <li><img src="https://paper.dropboxstatic.com/static/img/ace/emoji/1f933.png?version=8.0.0" alt="selfie" /> <strong>Build interactive data apps</strong> using inputs, dropdowns, and date pickers.</li> <li><img src="https://paper.dropboxstatic.com/static/img/ace/emoji/1f916.png?version=8.0.0" alt="robot" /> <strong>Generate code and queries</strong> using an AI that understands your database schema and your notebook's context.</li> <li><img src="https://paper.dropboxstatic.com/static/img/ace/emoji/1f570.png?version=8.0.0" alt="mantelpiece clock" /> <strong>Schedule notebooks and dashboards</strong> to run and update periodically.</li> <li><img src="https://paper.dropboxstatic.com/static/img/ace/emoji/2699.png?version=8.0.0" alt="gear" /> <strong>Create and test ad-hoc pipelines</strong> using writebacks.</li> </ul></li> <li><strong>Briefer vs. Traditional BI Tools:</strong> Briefer is better than traditional BI tools because it's faster and more flexible, thanks to Python.</li> <li><strong>Briefer vs. Traditional Notebooks:</strong> In Briefer, you can run SQL queries against connected data sources directly in your notebook. Then, <strong>Briefer will automatically turn your query into a data frame and store it in a variable that you can use in your Python blocks</strong>.</li> </ul> <p><strong>Brian #2:</strong> <a href="https://bioinf.comav.upv.es/courses/intro_programming/?featured_on=pythonbytes">Introduction to programming with Python</a></p> <ul> <li>Jose Blanca <ul> <li>“Python intro aimed at students with no prior programming experience.”</li> <li>“Relies mainly on examples and exercises.”</li> <li>“Does not try to cover every detail of the Python language, but just what a beginner might need to start the journey.”</li> <li>Tech: “… built with the <a href="https://quarto.org/?featured_on=pythonbytes">quarto</a> publishing system complemented by the <a href="https://github.com/r-wasm/quarto-live?featured_on=pythonbytes">quarto live</a> extension that allows Python to run in the web browser by using <a href="https://pyodide.org/?featured_on=pythonbytes">pyodide</a>.”</li> </ul></li> <li>Runs on anything, since it doesn’t require a local install of Python</li> <li>Running 3.12.1, looks like. Although that’s a bit hidden. Seems like it should be more visible.</li> </ul> <p><strong>Michael #3:</strong> <a href="https://github.com/astral-sh/setup-uv?featured_on=pythonbytes">setup-uv</a></p> <ul> <li>Set up your GitHub Actions workflow with a specific version of uv</li> <li>Install a version of uv and add it to PATH</li> <li>Cache the installed version of uv to speed up consecutive runs on self-hosted runners</li> <li>Register problem matchers for error output</li> <li>(Optional) Persist the uv's cache in the GitHub Actions Cache</li> <li>(Optional) Verify the checksum of the downloaded uv executable</li> </ul> <p><strong>Brian #4:</strong> <a href="https://htmlforpeople.com?featured_on=pythonbytes"><strong>HTML for people</strong></a></p> <ul> <li>Teaching HTML in a rather fun way.</li> <li>Includes basic CSS</li> </ul> <p><strong>Extras</strong> </p> <p>Michael:</p> <ul> <li>A new article: <a href="https://mkennedy.codes/posts/we-must-replace-uwsgi-with-something-else-but-with-what/?featured_on=pythonbytes">We Must Replace uWSGI With Something Else</a></li> <li><a href="https://github.com/carltongibson/django-unique-user-email?featured_on=pythonbytes">Django unique email</a> login</li> </ul> <p><strong>Joke:</strong> <a href="https://github.com/thepracticaldev/orly-full-res?featured_on=pythonbytes">So much O’Really</a></p>

What is an API gateway? 2024-10-14, by Dariusz Suchojad

In this article, we are going to use Zato in its capacity as a multi-protocol Python API gateway - we will integrate a few popular technologies, accepting requests sent over protocols commonly used in frontend systems, enriching and passing them to backend systems and returning responses to the API clients using their preferred data formats. But first, let's define what an API gateway is.

Clearing up the terminology

Although we will be focusing on complex API integrations later on today, to understand the term API gateway we first need to give proper consideration to the very term gateway.

What comes to mind when we hear the word "gateway", and what is correct etymologically indeed, is an opening in an otherwise impermissible barrier. We use a gateway to access that which is in other circumstances inaccessible for various reasons. We use it to leave such a place too.

In fact, both "gate" and the verb "to go" stem from the same basic root and that, again, brings to mind a notion of passing through space specifically set aside for the purpose of granting access to what normally would be unavailable. And once more, when we depart from such an area, we use a gateway too.

From the perspective of its true intended purpose, a gateway letting everyone in and out as they are would amount to little more than a hole in a wall. In other words, a gateway without a gate is not the whole story.

Yes, there is undoubtedly an immense aesthetic gratification to be drawn from being close to marvels of architecture that virtually all medieval or Renaissance gates and gateways represent, but we know that, nowadays, they do not function to the fullest of their capacities as originally intended.

Rather, we can intuitively say that a gateway is in service as a means of entry and departure if it lets its operators achieve the following, though not necessarily all at the same time, depending on one's particular needs:

• Telling arrivals where they are, including projection of might and self-confidence
• Confirming that arrivals are who they say they are
• Checking if their port of origin is friendly or not
• Checking if they are allowed to enter that which is protected
• Directing them to specific areas behind the gateway
• Keeping a long term and short term log of arrivals
• Answering potential questions right by the gate, if answers are known to gatekeepers
• Cooperating with translators and coordinators that let arrivals make use of what is required during their stay

We can now recognize that a gateway operates on the border of what is internal and external and in itself, it is a relatively narrow, though possibly deep, piece of an architecture. It is narrow because it is only through the gateway that entry is possible but it may be deeper or not, depending on how much it should offer to arrivals.

We also keep in mind that there may very well be more than a single gateway in existence at a time, each potentially dedicated to different purposes, some overlapping, some not.

Finally, it is crucial to remember that gateways are structural, architectural elements - what a gateway should do and how it should do it is a decision left to architects.

With all of that in mind, it is easy to transfer our understanding of what a physical gateway is into what an API one should be.

• API clients should be presented with clear information that they are entering a restricted area
• Source IP addresses or their equivalents should be checked and requests rejected if an IP address or equivalent information is not among the allowed ones
• Usernames, passwords, API keys and similar representations of what they are should be checked by the gateway
• Permissions to access backend systems should be checked seeing as not every API client should have access to everything
• Requests should be dispatched to relevant backend systems
• Requests and responses should be logged in various formats, some meant to be read by programs and applications, some by human operators
• If applicable, responses can be served from the gateway's cache, taking the burden off the shoulders of the backend systems
• Requests and responses can be transformed or enriched which potentially means contacting multiple backend systems before an API caller receives a response

We can now define an API gateway as an element of a systems architecture that is certainly related to security, permissions and granting or rejecting access to backend systems, applications and data sources. On top of it, it may provide audit, data transformation and caching services. The definition will be always fluid to a degree, depending on an architect's vision, but this is what can be expected from it nevertheless.

Having defined what an API gateway is, let's create one in Zato and Python.

Clients and backend systems

In this article, we will integrate two frontend systems and one backend application. Frontend ones will use REST and WebSockets whereas the backend one will use AMQP. Zato will act as an API gateway between them all.

Not granting frontend API clients direct access to backend systems is usually a good idea because the dynamics involved in creation of systems on either side are typically very different. But they still need to communicate and hence the usage of Zato as an API gateway.

Python code

First, let's show the Python code that is needed to integrate the systems in our architecture:

# -*- coding: utf-8 -*- # Zato from zato.server.service import Service class APIGateway(Service): """ Dispatches requests to backend systems, enriching them along the way. """ name = 'api.gateway' def handle(self): # Enrich incoming request with metadata .. self.request.payload['_receiver'] = self.name self.request.payload['_correlation_id'] = self.cid self.request.payload['_date_received'] = self.time.utcnow() # .. AMQP configuration .. outconn = 'My Backend' exchange = '/incoming' routing_key = 'api' # .. publish the message to an AMQP broker .. self.out.amqp.send(data, outconn, exchange, routing_key) # .. and return a response to our API client. self.response.payload = {'result': 'OK, data accepted'}

There are a couple of points of interest:

• The gateway service enriches incoming requests with metadata but it could very well enrich it with business data too, e.g. it could communicate with yet another system to obtain required information and only then pass the request to the final backend system(s)

• In its current form we send all the information to AMQP brokers only but we could just as well send it to other systems, possibly modifying the requests along the way

• The code is very abstract and all of its current configuration could be moved to a config file, Redis or another data source to make it even more high-level

• Security configuration and other details are not declared directly in the body of the gateway service but they need to exist somewhere - we will describe it in the next section

Configuration

In Zato, API clients access the platform's services using channels - let's create a channel for REST and WebSockets then.

First REST:

Now WebSockets:

We create a new outgoing AMQP connection in the same way:

Using the API gateway

At this point, the gateway is ready - you can invoke it from REST or WebSockets and any JSON data it receives will be processed by the gateway service, the AMQP broker will receive it, and API clients will have replies from the gateway as JSON responses.

Let's use curl to invoke the REST channel with JSON payload on input:

$ curl http://api:<password-here>@localhost:11223/api/v1/user ; echo curl --data-binary @request.json http://localhost:11223/api/v1/user ; echo {"result": "OK, data accepted"} $

Taken together, the channels and the service allowed us to achieve this:

• Multiple API clients can access the backend AMQP systems, each client using its own preferred technology
• Client credentials are checked on input, before the service starts to process requests (authentication)
• It is possible to assign RBAC roles to clients, in this way ensuring they have access only to selected parts of the backend API (authorization)
• Message logs keep track of data incoming and outgoing
• Responses from channels can be cached which lessens the burden put on the shoulders of backend systems
• Services accepting requests are free to modify, enrich and transform the data in any way required by business logic. E.g., in the code above we only add metadata but we could as well reach out to other applications before requests are sent to the intended recipients.

We can take it further. For instance, the gateway service is currently completely oblivious to the actual content of the requests.

But, since we just have a regular Python dict in self.request.payload, we can with no effort modify the service to dispatch requests to different backend systems, depending on what the request contains or possibly what other backend systems decide the destination should be.

Such additional logic is specific to each environment or project which is why it is not shown here, and this is also why we end the article at this point, but the central part of it all is already done, the rest is only a matter of customization and plugging in more channels for API clients or outgoing connections for backend systems.

Finally, it is perfectly fine to split access to systems among multiple gateways - each may handle requests from selected technologies on the one hand but on the other hand, each may use different caching or rate-limiting policies. If there is more than one, it may be easier to configure such details on a per-gateway basis.

Next steps:

➤ Read about how to use Python to build and integrate enterprise APIs that your tests will cover
➤ Python API integration tutorial
➤ Python Integration platform as a Service (iPaaS)
➤ What is an Enterprise Service Bus (ESB)? What is SOA?

More blog posts➤

Your new Django project uses Python 3.13.

You’re really looking forward to using the new REPL… but python manage.py shell just shows the same old Python REPL. What gives?

Well, Django’s management shell uses Python’s code module to launch a custom REPL, but the code module doesn’t (yet) use the new Python REPL.

So you’re out of luck… or are you?

How stable do you need your shell command to be?

The new Python REPL’s code lives in a _pyrepl package. Surely there must be some way to launch the new REPL using that _pyrepl package!

First, note the _ before that package name. It’s _pyrepl, not pyrepl.

Any solution that relies on this module may break in future Python releases.

So… should we give up on looking for a solution, if we can’t get a “stable” one?

I don’t think so.

My shell command doesn’t usually need to be stable in more than one version of Python at a time. So I’m fine with a solution that attempts to use the new REPL and then falls back to the old REPL if it fails.

A working solution

So, let’s look at a working solution.

Stick this code in a management/commands/shell.py file within one of your Django apps:

1 2 3 4 5 6 7 8 9 10 """Python 3.13 REPL support using the unsupported _pyrepl module.""" from django.core.management.commands.shell import Command as BaseShellCommand class Command(BaseShellCommand): shells = ["ipython", "bpython", "pyrepl", "python"] def pyrepl(self, options): from _pyrepl.main import interactive_console interactive_console() How it works

Django’s shell command has made it very simple to add support for your favorite REPL of choice.

The code for the shell command loops through the shells list and attempts to run a method with that name on its own class. If an ImportError is raised then it attempts the next command, stopping once no exception occurs.

Our new command will try to use IPython and bpython if they’re installed and then it will try the new Python 3.13 REPL followed by the old Python REPL.

If Python 3.14 breaks our import by moving the interactive_console function, then an ImportError will be raised, causing us to fall back to the old REPL after we upgrade to Python 3.14 one day. If instead, the interactive_console function’s usage changes (maybe it will require arguments) then our shell command will completely break and we’ll need to manually fix it when we upgrade to Python 3.14.

What’s so great about the new REPL?

If you’re already using IPython or BPython as your REPL and you’re enjoying them, I would stick with them.

Third-party libraries move faster than Python itself and they’re often more feature-rich. IPython has about 20 years worth of feature development and it has features that the built-in Python REPL will likely never have.

If you’re using the default Python REPL though, this new REPL is a huge upgrade. I’ve been using it as my default REPL since May and I love it. See my screencast on Python 3.13 for my favorite features in the new REPL.

This year, one of the projects I was involved in at Sentry was the launch of The Open Source Pledge. The idea behind it is simple: companies pledge an amount proportional to the number of developers they employ to fund the Open Source projects they depend on. I have written about this before.

Since then, I've had the chance to engage in many insightful discussions about Open Source funding and licensing. In the meantime we have officially launched the pledge, and almost simultaneously WordPress entered a crisis. At the heart of that crisis is a clash between Open Source ideals and financial interests by people other than the original creators.

You might have a lot of opinions on David Heinemeier Hansson, but I encourage you to read two of his recent posts on that very topic. In Automattic is doing open source dirty David is laying out the case that Automattic has no right to impose moral obligations on beyond the scope of the license. This has been followed by Open source royalty and mad kings in which he goes deeper into the fallout that Matt Mullenweg (the creator of WordPress) is causing with his fight.

I'm largely in agreement with the posts. However I want to talk a bit about some pretty significant difference between David's opinions on Open Source funding (on which these posts appear to be based): the money element. In 2013 David wrote the following about money and Open Source:

[…] it's tempting to cash in on goodwill earned. […] It's a cliché, but once you've sold out, the goodwill might well be spent for good.

[…] part of the reason much of open source is so good, and often so superior to closed-source commercial projects, is the natural boundary of constraints. If you are not being paid or otherwise compensated directly for your work, you're less likely to needlessly embellish it. […]

—David Heinemeier Hansson, The perils of mixing open source and money

At face value, this suggests that Open Source and money shouldn’t mix, and that the absence of monetary rewards fosters a unique creative process. There's certainly truth to this, but in reality, Open Source and money often mix quickly.

If you look under the cover of many successful Open Source projects you will find companies with their own commercial interests supporting them (eg: Linux via contributors), companies outright leading projects they are also commercializing (eg: MariaDB, redis) or companies funding Open Source projects primarily for marketing / up-sell purposes (uv, next.js, pydantic, …). Even when money doesn't directly fund an Open Source project, others may still profit from it, yet often those are not the original creators. These dynamics create stresses and moral dilemmas.

I’ve said this before, but it’s no coincidence that Rails has a foundation, large conferences, a strong core team, and a trademark, while Flask has none of it. There are barriers and it takes a lot of energy and determination to push a project to a level where it can sustain itself.

Rails pushed through this barrier. I never did with any of my projects and I'm at peace with that. I got to learn a lot through my Open Source work, I achieved a certain level of notoriety that I benefit from. I built a meaningful career by leveraging my work and I even met my wonderful wife that way. All are consequences of my Open Source contributions. There were clear and indisputable benefits to it and by all accounts I'm a happy and grateful person.

But every now and then doubts creep in and I wonder if I should have done something more commercial with Flask, or if I should have pushed Rye further. As much as I love listening to Charlie talking about uv, there is also an unavoidable doubt lingering there what could have been if I dared to build out Rye with funding on my own.

Over the years, I have seen too many of my colleagues and acquaintances struggle one way or another. Psychological, mentally and professionally. Midlife crises, burnout, health, and dealing with a strong feeling of dread and disappointment. Many of this as a indirect or even direct result of their Open Source work. While projects like Rails and Laravel are great examples of successful open source stewardship, they are also outliers. Many others don't survive or grow to that level.

And yet even some of those lighthouse projects can become fallen stars and face challenges. WordPress by all accounts is a massive success. WordPress is in the top 1% of open source projects in terms of impact, success, and financial return for its creator. Yet despite that — and it finding an actual business model to commercialize it — its creator suffers from the same fate as many small Open Source libraries: a feeling of being wronged.

This is where the lines between law and morality blur. Matt feels mistreated, especially by a private equity firm, but neither trademarks nor license terms can resolve the issue for him. It’s a moral question, and sadly, Matt’s actions have alienated many who would otherwise support him. He's turning into a “mad king” and behaving immoral in his own ways.

The reality is that we humans are messy and unpredictable. We don't quite know how we will behave until we have been throw into a particular situation. Open Source walks a very fine line, and anyone claiming to have all the answers probably doesn't. I certainly don't.

Is it a wise to mix Open Source and money? Maybe not. Yet I also believe it's something that is just a reality we need to navigate. Today there are some projects too small to get any funding (xz) and there are projects large enough to find some way to sustain by funneling money to it (Rails, WordPress).

We target with the Pledge small projects in particular. It's our suggestion of how to give to projects for which the barrier to attract funding is too high. At the same time I recognize all the open questions it leaves. There are questions about tax treatments, there are questions about sustainabilty and incentives, questions about distribution and governance.

I firmly believe that the current state of Open Source and money is inadequate, and we should strive for a better one. Will the Pledge help? I hope for some projects, but WordPress has shown that we need to drive forward that conversation of money and Open Source regardless of thes size of the project.

This is the fifth article in a series of articles about the Batch API in Drupal. The Batch API is a system in Drupal that allows data to be processed in small chunks in order to prevent timeout errors or memory problems.

So far in this series we have looked at creating a batch process using a form, followed by creating a batch class so that batches can be run through Drush, using the finished state to control batch processing and then processing CSV files through a batch process. All of these articles give a good grounding of how to use the Drupal Batch API.

In this article we will take a closer look at how the batch system processes items by creating a batch run inside an already running batch process. This will show how batch systems run and what happens when you try to add additional operations to a running batch.

Let's setup the initial batch operation.

Setting Up The Batch

The setup for this batch process is similar to the batch processes on the other articles. This will kick off a batch process that will process 1,000 items in chunks of 100 each.

Read more