Feeds

Plasma 6 has come into its own over the last two releases. The wrinkles that always come with a major migration have been ironed out, and it’s time to start delivering on the promises of the new Qt 6 and Wayland technology platforms that Plasma is built on top of.

One of the outstanding issues has been to make Plasma a more artist-friendly environment by providing full support for the hardware that creative people need to get their work done.

So let’s start there…

What’s New For Digital Artists

Plasma 6.2 includes a smorgasbord of new features for users of drawing tablets. Open System Settings and look for Drawing Tablet to see various tools for configuring drawing tablets.

New in Plasma 6.2: a tablet calibration wizard and test mode; a feature to define the area of the screen that your tablet covers (the whole screen or a section); and the option to re-bind pen buttons to different kinds of mouse clicks.

All this is built into Plasma; there’s no need to install new drivers or software from device manufacturers.

And if your tablet is not yet supported, “We care about your Input” is a community-wide project that aims to provide support for unusual input devices. Let us know about your device so we can add it to the list!

Color Management

Related to the above — and to ensure consistent colors across monitors — we’ve implemented more complete support for the Wayland color management protocol, and enabled it by default.

We have also improved brightness handling for HDR and ICC profiles, as well as HDR performance. This will improve your experience when designing graphics, playing games, and watching videos.

A new tone mapping feature built into Plasma’s KWin compositor will help improve the look of images with a brightness or set of colors greater than what the screen can display, thus reducing the “blown out” look such images can otherwise exhibit.

Before After Power Management

Managing how much energy your system consumes and when are not only important for preserving its resources for when you need them, but also for using it in an environmentally responsible way.

You can now override misbehaving applications that block the system from going to sleep or locking the screen (and thus prevent saving power), and you can also adjust the brightness of each connected monitor machine separately.

As for the Power and Battery widget, it not only shows how much power is remaining, but also allows you to adjust power profiles for different scenarios. New in Plasma 6.2: hold down the Meta (Windows) key and press B to cycle through the different options one at a time. A little badge of a leaf will show up on the battery icon to indicate when the system is in power save mode, and a rocket for performance mode.

Discover and System Updating

Another thing we put you in complete control of is your software.

Plasma’s built-in app store and software management tool, Discover, now supports PostmarketOS packages for your mobile devices, helps you write better reviews of apps, and presents apps’ license information more accurately.

You can also now choose to shut down the system after applying an offline system update, in addition to the existing option to restart afterwards.

Accessibility

Since we made improving accessibility a community-wide project, we have increased the ways in which Plasma is easy to use for everyone.

In Plasma 6.2, we overhauled System Settings’ Accessibility page and added colorblindness filters. We also added support for the full “sticky keys” feature on Wayland.

UI/Visual Design

And of course, improving the look and feel of Plasma is always a high priority from one release to the next.

In Plasma 6.2, we tweaked accent colors and the System Tray, reworked the Widget Explorer, and unified the look of dialogs and pop-ups. Finally, we improved the Welcome Center, sound effects, and actions.

Many of these changes are subtle, but will provide a smoother and more enjoyable experience.

And All This Too…

• The Weather Report widget now shows “feels like” temperatures, adds more information for BBC weather forecasts, and more.
• You can turn off window borders in the Pager widget.
• The Minimize All widget now minimizes only windows on the current virtual desktop and activity.
• You can now give custom names to your custom shortcuts.
• There's now an integrated cropping tool when setting a new user avatar.
• We’ve added a once-a-year donation request notification — please consider using it to show your love for Plasma by donating!

…and there’s much more going on as well: numerous stability and performance improvements, smaller visual changes, more Wayland protocols, and more! Check out the complete changelog for Plasma 6.2.

I just installed a new FreeBSD desktop machine. For this one, I wanted to have KDE Plasma6, since that’s already running on my Linux laptop and gaming machine – and it’s time for me to dogfood on FreeBSD. Here’s some notes.

Base Install

FreeBSD has a Live ISO and a graphical installer. For licensing reasons, it isn’t Calamares (which is GPLv3). So I use the text-based installer. I downloaded the FreeBSD 14.1 memstick image. From boot to reboot into an installed system takes less than five minutes, but then you have an old-school UNIX system: a login: prompt.

From there:

• log in as root
• switch the package repository to latest
• pkg install pkg, to get the package manager
• pkg install git cmake, because you’re a developer
• pkg install plasma6-plasma, which is an 800MiB download
• pkg install sddm, because you’ll want a login-manager
• pkg install drm-kmod, because you need a graphics driver

I have an AMD RX550 video card (cheapest I could get this year) in the machine, so then to set that graphics driver:

• sysrc kld_list=amdgpu

Make sure DBus will run as a system service:

• sysrc dbus_enable=yes

Create a regular user and add them to the video group.

Then reboot.

KDE Plasma6 X11 by Hand

At this point, the system has KDE Plasma6 installed, but it won’t come up (the login manager isn’t enabled, for instance) so we need some convenience things from The Before Times to do a little testing. (Installed as “automatic” so they are easy to remove later).

• pkg install -A xinit xterm

As a normal user, create ~/.xinitrc and put this in it:

#! /bin/sh exec /usr/local/bin/startplasma-x11

Then make it executable and start X11 (like it’s 1994):

• chmod 755 .xinitrc
• startx

Voila. KDE Plasma6. Note that, in this state, there are no applications besides xterm and KInfoCenter (and a handful of other KDE Plasma internal things). There’s no configuration applied to the system. The window manager does not honor alt-tab or alt-F4. Use ctrl-Q to quit KInfoCenter.

But X11 is soooo passé. Let’s move on to the Future!

Any Wayland by Hand

First, let’s install some convenience things that will help in debugging.

• pkg install -A river foot

Like I wrote 3 years ago about river, it just works. You’ll need the example configuration file so that super-shift-E exits the compositor and window manager. super-shift-enter gets you a terminal window.

I started river as a regular user with

• ck-launch-session river

KDE Plasma6 Wayland by Hand

Yeah, right.

So there was a brief time in 2021 that it worked. Since then, not so much – certainly not for me, not from the regular ports tree. So this post is a start of “ok, let’s give it another shot”.

I know that Wayland can work on FreeBSD with hardware rendering – that’s why that river section is there.

Here is a very short script that can launch KDE Plasma6 with software rendering. The resulting desktop experience is rather slow.

#! /bin/sh export KWIN_COMPOSE=Q exec /usr/local/bin/ck-launch-session \ /usr/local/lib/libexec/plasma-dbus-run-session-if-needed \ /usr/local/bin/startplasma-wayland

So the TODO part of this post is: figure out why opening dri/card0 fails when kwin_wayland is not using the software renderer.

KDE Applications5 and KDE Plasma6

Yeah, right.

Most (maybe even all) of the KDE Applications – for instance, konsole, or kmail – are still KDE Frameworks 5 based. Unfortunately, there are KDE Frameworks that have file-collisions between versions 5 and 6. As an example, package kf6-baloo and kf5-baloo both want to install a libbalooplugin.so. I mentioned the co-installability problem a half-year ago, but we haven’t fixed it since.

Edit 2024-10-08: this is entirely a packaging problem, where we could install the things to separate prefixes. That hasn’t happened, because of a lack of person-time to actually do it (and test it).

On this front I think we’re at a chicken-and-egg place: we would like to switch wholesale to newer applications releases and just drop the existing KDE Applications 5, but are so bogged down with Other Stuff that it’s not happening. On the Linux side of things KDE Applications 6 are doing fine.

All that said, there’s the KDE-FreeBSD ports development fork with a branch where newer KDE-FreeBSD packages are prepared. That already has a KF6-based KMail. So the TODO part of this section is: I need to double-check what’s holding that up.

Takeaways

• for a nice KDE Plasma6 experience on a BSD, why not try OpenBSD?
• KDE-FreeBSD has a nasty amount of hardware-testing to do.
• The future is here, just not in the ports tree.

At work-work the system uses, for historical reasons, a lot of SystemV shared memory. The SysV shared memory API has C functions like shmat(2). There is also a different shared memory API, POSIX shared memory, which has functions like shm_open(3). For reasons, on some work-work systems we’re constrained to Python 3.7 and no additional libraries. I wanted to mess with the shared memory on such a system, from Python for convenience, so I wrote some very simple wrappers. Here’s a recap.

As usual, corrections are welcome, or tips (by email). I write these notes as much for future me as anyone else.

Here is the core of the story (I have also added this to my personal GitHub repository, which I won’t link because it’s not future-proof storage).

import ctypes lib = ctypes.cdll.LoadLibrary(None) shmget = lib.shmget shmget.argtypes = [ctypes.c_int, ctypes.c_size_t, ctypes.c_int] shmget.restype = ctypes.c_int shmget.__doc__ = """See shmget(2)"""

This works on FreeBSD, where SysV shmem is in the core libraries. On Linux, I think you need to call LoadLibrary("librt"). Anyway, wrapping the library-loading to be safe isn’t the point here.

Once ctypes has loaded a library, you can extract function pointers from the library. By adding annotations, you can give the Python function the same prototype as the C manpage for shmget.

Note that the manpage points to some special flag values. For those, you need to dig into the C headers. On FreeBSD, the special value IPC_PRIVATE is equal to 0, so that’s easy enough to write in Python. The following snippet is then sufficient to create a shared memory segment (one that is 1024 bytes large and world-readable) and print out its ID. The returned value is -1 on error.

print(shmget(0, 1024, 0o644))

The ID can be cross-checked with command ipcs -m (it’s installed by default on FreeBSD and in my KDE Neon machine, so seems like a common tool). To get rid of the segment, ipcrm -m <id> does the trick.

Similar wrappers are there for shmat, shmdt and shmctl – but those wrangle void * in C, and how does that work with Python?

The void pointer

CTypes has a c_void_p type, which can be created from None (a null pointer, seems reasonable) and returned from C functions. It can cast to-and-fro (in classic C style, the thing in memory is what I say is in memory) to other pointer types, and without a typed-pointer type at the machine level that just works (but don’t ask me how).

So the C function int shmctl(int shmid, int cmd, struct shmid_ds *buf) gets these types in Python: shmctl.argtypes = [ctypes.c_int, ctypes.c_int, ctypes.c_void_p], which presents the struct-pointer as a void-pointer.

The function void *shmat(int shmid, const void *addr, int flag) works similarly. When calling it, unless you have specific address needs, parameter addr can be nullptr (er .. ok, this is C, so NULL and in Python None). The pointer it returns is where the shared memory is attached.

Actually doing something with a void * takes work in C, it also takes work in Python with ctypes. You can cast to an int * for instance, with iaddr = ctypes.cast(addr, ctypes.POINTER(ctypes.c_int)) (a cast to char * is also readily available).

A special case is when you need to provide a void * to some C function. Where do they come from? In C you would just declare a (character) buffer of some size and pass it in. In Python, ctypes.create_string_buffer() does the job. Give it a size and get a memory-managed buffer.

Wrangling shared-memory segment destruction

There’s shmget() to create a segment, shmat() to attach (map it into memory of a process) to it, shmdt() to detach from a segment, but destroying a shared-memory segment does not have a simple C call to do it. There is shmctl() which does special-control-actions on a shared-memory segement, and destruction is one of them.

I ended up writing this little wrapper.

def shmrm(shmid : int) -> int: return shmctl(shmid, 0, None) Sending messages

As an experiment, I wrote a program that can create, read, write and destroy a shared-memory segment. By writing (from one invocation) and then reading (from another invocation) I can “send” messages from the past! Or to the future! It is nearly as convenient as writing the messages to a file.

Here’s the write function. It attaches the shared-memory segment and then writes a Pascal-style string to that memory (Pascal-style in the sense of “starts with a length, followed by the actual data, no NUL-termination”). For bloggy purposes I have removed error-handling.

def write(shmid : int, v : str): addr = shmat(shmid, ctypes.c_void_p(None), 0) iaddr = ctypes.cast(addr, ctypes.POINTER(ctypes.c_int)) caddr = ctypes.cast(addr, ctypes.POINTER(ctypes.c_char)) ustring = v.encode("utf-8") iaddr[0] = len(ustring) for i in range(len(ustring)): caddr[4+i] = ustring[i] return shmdt(addr)

Here, shmat() returns a void * and I cast that to two typed pointers to the segment. I haven’t figured out how to do pointer arithmetic, so on the assumption there are 32-bit integers, the integer goes first and then the message goes starting at byte (char) number 4.

Takeaways

CTypes is really cool! It makes wrangling C APIs in Python .. well, let’s call it “acceptable”.

Starting with Python 3.8, everything I’ve written above is unnecessary because there is a good shared-memory abstraction in the standard Python library, but for my work-work purposes in a very restricted environment, this particular tool has turned out to be really useful.

Welcome to the September 2024 report from the Reproducible Builds project!

Our reports attempt to outline what we’ve been up to over the past month, highlighting news items from elsewhere in tech where they are related. As ever, if you are interested in contributing to the project, please visit our Contribute page on our website.

Table of contents:

1. New binsider tool to analyse ELF binaries
2. Unreproducibility of GHC Haskell compiler “95% fixed”
3. Mailing list summary
4. Towards a 100% bit-for-bit reproducible OS…
5. Two new reproducibility-related academic papers
6. Distribution work
7. diffoscope
8. Other software development
9. Android toolchain core count issue reported
10. New Gradle plugin for reproducibility
11. Website updates
12. Upstream patches
13. Reproducibility testing framework

New binsider tool to analyse ELF binaries

Reproducible Builds developer Orhun Parmaksız has announced a fantastic new tool to analyse the contents of ELF binaries. According to the project’s README page:

Binsider can perform static and dynamic analysis, inspect strings, examine linked libraries, and perform hexdumps, all within a user-friendly terminal user interface!

More information about Binsider’s features and how it works can be found within Binsider’s documentation pages.

Unreproducibility of GHC Haskell compiler “95% fixed”

A seven-year-old bug about the nondeterminism of object code generated by the Glasgow Haskell Compiler (GHC) received a recent update, consisting of Rodrigo Mesquita noting that the issue is:

95% fixed by [merge request] !12680 when -fobject-determinism is enabled. […]

The linked merge request has since been merged, and Rodrigo goes on to say that:

After that patch is merged, there are some rarer bugs in both interface file determinism (eg. #25170) and in object determinism (eg. #25269) that need to be taken care of, but the great majority of the work needed to get there should have been merged already. When merged, I think we should close this one in favour of the more specific determinism issues like the two linked above.

Mailing list summary

On our mailing list this month:

• Fay Stegerman let everyone know that she started a thread on the Fediverse about the problems caused by unreproducible zlib/deflate compression in .zip and .apk files and later followed up with the results of her subsequent investigation.

• Long-time developer kpcyrd wrote that “there has been a recent public discussion on the Arch Linux GitLab [instance] about the challenges and possible opportunities for making the Linux kernel package reproducible”, all relating to the CONFIG_MODULE_SIG flag. […]

• Bernhard M. Wiedemann followed-up to an in-person conversation at our recent Hamburg 2024 summit on the potential presence for Reproducible Builds in recognised standards. […]

• Fay Stegerman also wrote about her worry about the “possible repercussions for RB tooling of Debian migrating from zlib to zlib-ng” as reproducibility requires identical compressed data streams. […]

• Martin Monperrus wrote the list announcing the latest release of maven-lockfile that is designed aid “building Maven projects with integrity”. […]

• Lastly, Bernhard M. Wiedemann wrote about potential role of reproducible builds in combatting silent data corruption, as detailed in a recent Tweet and scholarly paper on faulty CPU cores. […]

Towards a 100% bit-for-bit reproducible OS…

Bernhard M. Wiedemann began writing on journey towards a 100% bit-for-bit reproducible operating system on the openSUSE wiki:

This is a report of Part 1 of my journey: building 100% bit-reproducible packages for every package that makes up [openSUSE’s] minimalVM image. This target was chosen as the smallest useful result/artifact. The larger package-sets get, the more disk-space and build-power is required to build/verify all of them.

This work was sponsored by NLnet’s NGI Zero fund.

Two new reproducibility-related academic papers

Marvin Strangfeld published his bachelor thesis, “Reproducibility of Computational Environments for Software Development” from RWTH Aachen University. The author offers a more precise theoretical definition of computational environments compared to previous definitions, which can be applied to describe real-world computational environments. Additionally, Marvin provide a definition of reproducibility in computational environments, enabling discussions about the extent to which an environment can be made reproducible. The thesis is available to browse or download in PDF format.

In addition, Shenyu Zheng, Bram Adams and Ahmed E. Hassan of Queen’s University, ON, Canada have published an article on “hermeticity” in Bazel-based build systems:

A hermetic build system manages its own build dependencies, isolated from the host file system, thereby securing the build process. Although, in recent years, new artifact-based build technologies like Bazel offer build hermeticity as a core functionality, no empirical study has evaluated how effectively these new build technologies achieve build hermeticity. This paper studies 2,439 non-hermetic build dependency packages of 70 Bazel-using open-source projects by analyzing 150 million Linux system file calls collected in their build processes. We found that none of the studied projects has a completely hermetic build process, largely due to the use of non-hermetic top-level toolchains. […]

Distribution work

In Debian this month, 14 reviews of Debian packages were added, 12 were updated and 20 were removed, all adding to our knowledge about identified issues. A number of issue types were updated as well. […][…]

In addition, Holger opened 4 bugs against the debrebuild component of the devscripts suite of tools. In particular:

• #1081047: Fails to download .dsc file.
• #1081048: Does not work with a proxy.
• #1081050: Fails to create a debrebuild.tar.
• #1081839: Fails with E: mmdebstrap failed to run error.

Last month, an issue was filed to update the Salsa CI pipeline (used by 1,000s of Debian packages) to no longer test for reproducibility with reprotest’s build_path variation. Holger Levsen provided a rationale for this change in the issue, which has already been made to the tests being performed by tests.reproducible-builds.org. This month, this issue was closed by Santiago R. R., nicely explaining that build path variation is no longer the default, and, if desired, how developers may enable it again.

In openSUSE news, Bernhard M. Wiedemann published another report for that distribution.

diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made the following changes, including preparing and uploading version 278 to Debian:

• New features:

  • Add a helpful contextual message to the output if comparing Debian .orig tarballs within .dsc files without the ability to “fuzzy-match” away the leading directory.  […]

• Bug fixes:

  • Drop removal of calculated os.path.basename from GNU readelf output. […]
  • Correctly invert “X% similar” value and do not emit “100% similar”. […]

• Misc:

  • Temporarily remove procyon-decompiler from Build-Depends as it was removed from testing (via #1057532). (#1082636)
  • Update copyright years. […]

For trydiffoscope, the command-line client for the web-based version of diffoscope, Chris Lamb also:

• Added an explicit python3-setuptools dependency. (#1080825)
• Bumped the Standards-Version to 4.7.0. […]

Other software development

disorderfs is our FUSE-based filesystem that deliberately introduces non-determinism into system calls to reliably flush out reproducibility issues. This month, version 0.5.11-4 was uploaded to Debian unstable by Holger Levsen making the following changes:

• Replace build-dependency on the obsolete pkg-config package with one on pkgconf, following a Lintian check. […]
• Bump Standards-Version field to 4.7.0, with no related changes needed. […]

In addition, reprotest is our tool for building the same source code twice in different environments and then checking the binaries produced by each build for any differences. This month, version 0.7.28 was uploaded to Debian unstable by Holger Levsen including a change by Jelle van der Waa to move away from the pipes Python module to shlex, as the former will be removed in Python version 3.13 […].

Android toolchain core count issue reported

Fay Stegerman reported an issue with the Android toolchain where a part of the build system generates a different classes.dex file (and thus a different .apk) depending on the number of cores available during the build, thereby breaking Reproducible Builds:

We’ve rebuilt [tag v3.6.1] multiple times (each time in a fresh container): with 2, 4, 6, 8, and 16 cores available, respectively:

• With 2 and 4 cores we always get an unsigned APK with SHA-256 14763d682c9286ef….
• With 6, 8, and 16 cores we get an unsigned APK with SHA-256 35324ba4c492760… instead.

New Gradle plugin for reproducibility

A new plugin for the Gradle build tool for Java has been released. This easily-enabled plugin results in:

reproducibility settings [being] applied to some of Gradle’s built-in tasks that should really be the default. Compatible with Java 8 and Gradle 8.3 or later.

Website updates

There were a rather substantial number of improvements made to our website this month, including:

• Chris Lamb:

  • Attempt to use GitLab CI to ‘artifact’ the website; hopefully useful for testing branches. […]
  • Correct the linting rule whilst building the website. […]
  • Make a number of small changes to Kees’ post written by Vagrant. […][…][…]
  • Add the Civil Infrastructure Platform to the Projects page. […]
  • Miscellaneous administration of misfiled images. […][…]

• Evangelos Tzaras made a huge number of changes related to the recent Hamburg 2024 summit […][…][…][…][…] as well as proposed an infographic about which question Reproducible Builds is trying to answer.

• Holger Levsen added his two presentations (Reproducible Builds: The First Eleven Years and Preserving *other* build artifacts) to the website. […]

• Jelle van der Waa completely modernised the System Images documentation, noting that “a lot has changed since 2017(!); ext4, erofs and FAT filesystems can now be made reproducible”. […]

• Developer RyanSquared replaced the continuous integration test link for Arch Linux on our Projects page with an external instance […][…] as well as updated the documentation to reflect the dependencies required to build the website […].

• Vagrant Cascadian pushed a lengthy interview with Linux developer Kees Cook. […][…][…][…]

Upstream patches

The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

• Bernhard M. Wiedemann:

  • agama-integration-tests (random)
  • contrast (FTBFS-nocheck)
  • cpython (FTBFS-2038)
  • crash (parallelism, race)
  • ghostscript (toolchain date)
  • glycin-loaders (FTBFS -j1)
  • gstreamer-plugins-rs (date, other)
  • kernel-doc/Sphinx (toolchain bug, parallelism/race)
  • kernel (parallelism in BTF)
  • libcamera (random key)
  • libgtop (uname -r)
  • libsamplerate (random temporary directory)
  • lua-luarepl (FTBFS)
  • meson (toolchain)
  • netty (modification time in .a)
  • nvidia-persistenced (date)
  • nvidia-xconfig (date-related issue)
  • obs-build (build-tooling corruption)
  • perl (Perl records kernel version)
  • pinentry (make efl droppable)
  • python-PyGithub (FTBFS 2024-11-25)
  • python-Sphinx (parallelism/race)
  • python-chroma-hnswlib (CPU)
  • python-libcst
  • python-pygraphviz (random timing)
  • python312 (.pyc embeds modification time)
  • python312 (drop .pyc from documentation time)
  • scap-security-guide (date)
  • seahorse (parallelism)
  • subversion (minor Java .jar modification times)
  • xen/acpica (date-related issue in toolchain)
  • xmvn (random)
• Fridrich Strba:
  • ant (jar mtime)
  • xmvn (various fixes)

• Chris Lamb:

  • #1082702 filed against magic-wormhole-transit-relay.
  • #1082706 filed against python-sphobjinv.
  • #1082707 filed against lomiri-content-hub.
  • #1082796 filed against python-mt-940.
  • #1082806 filed against tree-puzzle.
  • #1083053 filed against muon-meson.

• James Addison:

  • guake (fix a parallelism/timing-race build bug)
  • sphobjinv (duplicates fix from Debian bug #1082706)

Reproducibility testing framework

The Reproducible Builds project operates a comprehensive testing framework running primarily at tests.reproducible-builds.org in order to check packages and other artifacts for reproducibility. In September, a number of changes were made by Holger Levsen, including:

• Debian-related changes:

  • Upgrade the osuosl4 node to Debian trixie in anticipation of running debrebuild and rebuilderd there. […][…][…]
  • Temporarily mark the osuosl4 node as offline due to ongoing xfs_repair filesystem maintenance. […][…]
  • Do not warn about (very old) broken nodes. […]
  • Add the risc64 architecture to the multiarch version skew tests for Debian trixie and sid. […][…][…]
  • Mark the virt{32,64}b nodes as down. […]

• Misc changes:

  • Add support for powercycling OpenStack instances. […]
  • Update the fail2ban to ban hosts for 4 weeks in total […][…] and take care to never ban our own Jenkins instance. […]

In addition, Vagrant Cascadian recorded a disk failure for the virt32b and virt64b nodes […], performed some maintenance of the cbxi4a node […][…] and marked most armhf architecture systems as being back online.

Finally, If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

• IRC: #reproducible-builds on irc.oftc.net.

• Mastodon: @reproducible_builds@fosstodon.org

• Mailing list: rb-general@lists.reproducible-builds.org

• Twitter: @ReproBuilds

Today we are talking about Creating Recipes, What Recipes already exist, and helpful tips and tricks with guest Jim Birch. We’ll also cover Features as our module of the week.

For show notes visit: https://www.talkingDrupal.com/470

Topics

• What are recipes
• How do you recommend someone get started writing recipes
• Where can people find recipes
• Can you include sub recipes
• How should you test recipes
• Any tools that make writing recipes easier
• What recipes are needed that do not exist
• How can people move recipes forward

Resources

• Recipe Author Guide
• Drupal Core Recipes
• Preconditions for recipes
• Drupal Recipes Cookbook
• Recipes Packagist
• Recipe type
• Phase 2 roadmap
• Umami profile recipes
• Minimal profile recipes

Guests

Jim Birch - linkedin.com/in/jimbirch thejimbirch

Hosts

Nic Laflin - nLighteneddevelopment.com nicxvan John Picozzi - epam.com johnpicozzi Aubrey Sambor - star-shaped.org starshaped

MOTW Correspondent

Martin Anderson-Clutz - mandclu.com mandclu

• Brief description:
  • Have you ever wanted an admin UI to manage sets of configuration, to version and share across Drupal sites? There’s a module for that.
• Module name/project name:
  • Features
• Brief history
  • How old: created in Mar 2009 by yhahn, though recent releases are by Dave Reid
  • Versions available: 7.x-2.15 and 8.x-3.14, the latter of which works with Drupal 9.4 and 10
• Maintainership
  • Minimally maintained
  • Security coverage
  • Test coverage
  • Documentation: Has a documentation guide and probably hundreds if not thousands of of tutorials available
  • Number of open issues: 610 open issues, 54 of which are bugs against the 8.x branch
• Usage stats:
  • Almost 117,000 sites, though the majority are using the D7 version
• Module features and usage
  • Many listeners will remember Features as the de facto solution for configuration management in Drupal 7 and earlier
  • As the name implies, it was really intended to share common capabilities across different Drupal sites
  • Unlike recipes, Features can have version numbers, because there is a path to sync configuration updates across sites using a Feature, though this is where a lot of teams found Features could be complex to use
  • We did previously cover Features as MOTW all the way back in episode #147, but I thought it was relevant to today’s discussion because of the way it provides a UI for organizing and exporting specific sets of configuration
  • There is an open issue for Features to directly export recipes, because it already does a lot of the time-consuming work of collecting together necessary config files, including dependencies
  • Even its current state, it could be a time saver for anyone wanting to start creating their own recipes

Python 3.13 comes with a brand new REPL and improvements to virtual environments and the Python debugger.

Table of contents

1. Important but not my favorite
2. The New Python REPL
3. Git-friendly virtual environments
4. Python Debugger improvements
5. Try out Python 3.13

Important but not my favorite

First, I'd like to note that I'm not going to talk about the experimental free-threaded mode, the experimental just-in-time compilation option, or many other features that don't affect most Python developers today.

Instead, let's focus on some of the more fun things.

The New Python REPL

My favorite feature by far …

Read the full article: https://www.pythonmorsels.com/python-313-whats-new/

Maybe, you want to experiment small stuffs in wish (the tcl/tk) interpreter because of a post claiming that direct python tcl/tk is simpler in some simple cases than tkinter.

As a convinced tkinter/FreeSimpleGUI user, I see this as an extreme claim that requires solid evidences.

When all is said and done, wish interpreter is not interactive, and for testing simple stuff it can get annoying very fast. Thus, it would be nice to add readline to the interface.

So here is a less than 100 line of code exercice of soing exactly so while having fun with : readline and multiprocessing (I would have taken multithreading if threads were easy to terminate).

Readline I quote The readline module defines a number of functions to facilitate completion and reading/writing of history files from the Python interpreter. Basically, it adds arrow navigation in history, back search with Ctrl+R, Ctrl+K for cuting on the right, Ctrl+Y for yanking ... all the facilities of interaction you have in bash or ipython for instance.

We are gonna use multiprocessing because tcl/tl is event oriented, hence, asynchronuous hence, we may have string coming from the tcl stdout while we do nothing and we would like to print them.

We also introduce like in ipython some magic prefixed with # (comment in tcl) like #? for the help. A session should look like this : # pack [ button .c -text that -command { puts "hello" } ] # tcl output> hello # here we pressed the button "that" tcl output> hello # here we pressed the button "that" # set name 32 # puts $name tcl output> 32 # #? #l print current recorded session #? print current help #! calls python code like #!save(name="temp") which saves the current session in current dir in "temp" file bye exit quit quit the current session # #l pack [ button .c -text that -command { puts "hello" } ] set name 32 puts $name # #!save("my_test.tcl") # quit The code in itself is fairly easy to read the only catch is that wish accepts multiline input. I can't because I don't know how to parse tcl. As a result I « eval in tcl » every line to know if there is an error and ask politely tcl to do the job of signaling the error with a « catch/error » (the equivalent of python try + raise an exception). #!/usr/bin/env python3 # -*- coding: utf8 -*- from subprocess import Popen, PIPE, STDOUT from multiprocessing import Process import sys, os import atexit import os import readline from select import select from time import sleep ### interactive session with history with readline histfile = os.path.join(os.path.expanduser("~"), ".wish_history") try: readline.read_history_file(histfile) # default history len is -1 (infinite), which may grow unruly readline.set_history_length(-1) except FileNotFoundError: pass ### saving history at the end of the session atexit.register(readline.write_history_file, histfile) ### opening wish wish = Popen(['wish'], stdin=PIPE, stdout=PIPE, stderr=PIPE, bufsize=-1, ) os.set_blocking(wish.stdout.fileno(), False) os.set_blocking(wish.stderr.fileno(), False) os.set_blocking(wish.stdin.fileno(), False) def puts(s): out = f"""set code [ catch {{ {s} }} p ] if {{$code}} {{ error $p }} """ select([], [wish.stdin], []) wish.stdin.write(out.encode()) def gets(): while True: wish.stdout.flush() tin = wish.stdout.read() if tin: print("\ntcl output> " + tin.decode()) sleep(.1) def save(fn="temp"): with open(fn,"wt") as f: f.write(session) session=s="" def load(fn="temp"): global session with open(fn, "rt") as f: while l:= f.readline(): session+=l + "\n" puts(l) # async io in tcl requires a background process to read the input t =Process(target=gets, arwish=()) t.start() while True: s = input("# ") if s in { "bye", "quit", "exit" }: t.terminate() wish.stdin.write("destroy .".encode()) break elif s == "#l": print(session) elif s == "#?": print(""" #l print current recorded session #? print current help #! calls python code like #!save(name="temp") which saves the current session in current dir in "temp" file #!load(name="temp") which load the session stored in current dir in "temp" file bye exit quit quit the current session """ ) continue elif s.startswith("#!"): print(eval(s[2:])) continue else: puts(s) if err:=wish.stderr.readline(): sys.stderr.write(err.decode()) else: if s and not s.startswith("#"): session += s + "\n" This code is available on pypi as iwish (interactive wish) and the git link is in the README.

After Camp: Stay connected with our Mid-Day Meetup Published Date Monday, October 7th, 2024 - 12:52 pm cosmicdreams Mon, 10/07/2024 - 12:52

We're restarting our Mid-Day Meetup:  a remote-only meetup we have over Zoom.  You can RSVP to attend.

https://groups.drupal.org/node/537103

We'll be talking about the knowledge shared at the recent Twin Cities Drupal Camp and Drupalcon Barcelona.  We're also trying to expand the reach of our Mid-Day meetup beyond our regional meetup and welcome everybody into our remote hour of Drupal chatter.

Agenda

• 15m - Meet and Greet
• 30m - Discuss new stuff going on in Drupal / Drupalcon Barcelona's "Mother of all Demos"
• 15m - cosmicdreams is hoping to do a demo of adapting an existing component library system to Single Directory Components (SDC).

Help us spread the word of these Mid-Day meetups by sharing this with your coworkers.

 

 

Posted In Drupal Planet

 

Python 3.13.0 is now available
https://www.python.org/downloads/release/python-3130/
This is the stable release of Python 3.13.0

Python 3.13.0 is the newest major release of the Python programming language, and it contains many new features and optimizations compared to Python 3.12. (Compared to the last release candidate, 3.13.0rc3, 3.13.0 contains two small bug fixes and some documentation and testing changes.)

Major new features of the 3.13 series, compared to 3.12

Some of the new major new features and changes in Python 3.13 are:

New features

• A new and improved interactive interpreter, based on PyPy’s, featuring multi-line editing and color support, as well as colorized exception tracebacks.
• An experimental free-threaded build mode, which disables the Global Interpreter Lock, allowing threads to run more concurrently. The build mode is available as an experimental feature in the Windows and macOS installers as well.
• A preliminary, experimental JIT, providing the ground work for significant performance improvements.
• The locals() builtin function (and its C equivalent) now has well-defined semantics when mutating the returned mapping, which allows debuggers to operate more consistently.
• A modified version of mimalloc is now included, optional but enabled by default if supported by the platform, and required for the free-threaded build mode.
• Docstrings now have their leading indentation stripped, reducing memory use and the size of .pyc files. (Most tools handling docstrings already strip leading indentation.)
• The dbm module has a new dbm.sqlite3 backend that is used by default when creating new files.
• The minimum supported macOS version was changed from 10.9 to 10.13 (High Sierra). Older macOS versions will not be supported going forward.
• WASI is now a Tier 2 supported platform. Emscripten is no longer an officially supported platform (but Pyodide continues to support Emscripten).
• iOS is now a Tier 3 supported platform.
• Android is now a Tier 3 supported platform.

Typing

• Support for type defaults in type parameters.
• A new type narrowing annotation, typing.TypeIs.
• A new annotation for read-only items in TypeDicts.
• A new annotation for marking deprecations in the type system.

Removals and new deprecations

• PEP 594 (Removing dead batteries from the standard library) scheduled removals of many deprecated modules: aifc, audioop, chunk, cgi, cgitb, crypt, imghdr, mailcap, msilib, nis, nntplib, ossaudiodev, pipes, sndhdr, spwd, sunau, telnetlib, uu, xdrlib, lib2to3.
• Many other removals of deprecated classes, functions and methods in various standard library modules.
• C API removals and deprecations. (Some removals present in alpha 1 were reverted in alpha 2, as the removals were deemed too disruptive at this time.)
• New deprecations, most of which are scheduled for removal from Python 3.15 or 3.16.

For more details on the changes to Python 3.13, see What’s new in Python 3.13.

More resources

• Online Documentation
• PEP 719, 3.13 Release Schedule
• Report bugs via GitHub Issues.
• Help fund Python directly (or via GitHub Sponsors), and support the Python community.

We hope you enjoy the new releases!

Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation.

Choo-choo from the release train,

Your release team,
Thomas Wouters 
Ned Deily 
Steve Dower 
Łukasz Langa 

October is always an important month for Python, as this is when a new major version is released. Python 3.13 is the new version this year, and it brings several new features that lay the groundwork for other changes in the future. As one version of Python comes to life, another is put to rest. Python 3.8 is already five years old, which means that this version won’t be supported any longer.

There are also exciting developments happening in the wider Python community. In this newsletter, you can read about Polars’ improved support for plotting, as well as how Django developers gathered for the annual DjangoCon US conference.

Time to jump in and read about what’s happening in the world of Python!

Python 3.13 Release Slightly Delayed

The release of Python 3.13, the newest version of Python, was originally scheduled for October 1, 2024. However, a few days before that date, release manager Thomas Wouters decided to postpone the release until October 7, 2024:

I’m a little concerned with the impact of the incremental GC change in 3.13, which recently showed up. It’s not clear that the incremental GC provides significant improvements (although the smaller pauses are probably desirable), it clearly has slightly more overhead in common cases, and we’re still discovering pathological cases.

I don’t think we should release 3.13.0 with the incremental GC. (Source)

The incremental garbage collector was a small improvement slated for Python 3.13. In many cases, the new garbage collection algorithm improves performance. Unfortunately, it was found to slow down Python significantly in some rare cases.

As a result, the core developers decided to revert the implementation and use the traditional garbage collector in Python 3.13. At the same time, the new implementation is being scrutinized and currently the goal is to include incremental garbage collection in Python 3.14.

Delaying a major Python release is never an easy choice. However, erring on the side of caution is a good approach, and it’s great to see that the Python 3.13 release is being handled responsibly.

Python 3.13 Highlights

As always, a new Python release brings many improvements and new features. You can explore these in-depth in Python 3.13: Cool New Features for You to Try. In particular, the new release includes:

• A brand new interactive interpreter (REPL)
• Colored tracebacks and improved error messages
• A separate, free-threaded version of Python that runs without the global interpreter lock (GIL)
• An experimental just-in-time (JIT) compiler
• Several improvements to Python’s static type system

For free threading and the JIT compiler, you need to compile Python with special build flags. Read Python 3.13 Preview: Free Threading and a JIT Compiler to learn more about how to explore these two new features. Additionally, Python 3.13 Preview: A Modern REPL provides more detail on the new REPL.

Read the full article at https://realpython.com/python-news-october-2024/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

DrupalCon week! On Monday, we landed the last issue to achieve the 0.1 milestone: The XB annotations and labels should not change size when zooming — thanks Utkarsh “utkarsh_33”, Atul “soaratul” Dubey and Bálint “balintbrews” Kléri for guiding it across the finish line!

That was the only noteworthy commit of the week, because of Acquia’s team working full-time on Experience Builder (XB), Ben “bnjmnm” Mullins, Jesse “jessebaker” Baker, Lauri “lauriii” Timmanee and Bálint (he helped us achieve 0.1 on Monday and traveled on Tuesday!) were attending DrupalCon. Also at DrupalCon was Dave “longwave” Long, who we’re sponsoring part time.

So it was with a lot of satisfaction that I tagged the 0.1.0-alpha1 release on the morning of the DriesNote :)

Not at DrupalCon: research mode

With roughly half of the team at DrupalCon this week, and with 0.1.0 done, the rest of us pivoted to preparing for the next milestone: 0.2.0. Many technical details need to be figured out for the next batch of product requirements that Lauri prioritized (together with Alex “effulgentsia” Bronstein).

We started research on:

• #3475672: auto-saving, drafts, and all possible ways to achieve that, led by Ted “tedbow” Bowman and with the much-needed help of both Travis “traviscarden” Carden and Dave
• #3475363: in-UI (JS) component creation Proof-of-Concept using StackBlitz by Harumi “hooroomoo” Jang and (newly joined!) Xinran “xinran” Cao, led by Alex
• #3475584: blocks-as-XB-components, led by Feliksas “f.mazeikis” Mazeikis and with the help of Dave and I, while reusing and building on top of the work that Lee “larowlan” Rowlands did ~2 months ago
• #3446434: Document “Semi-Coupled Theme Engine” and “Redux-integrated Field Widgets” components, kickstarted by me, because we proved this can work but now it’s time to mature this; this will be led by Ben, with again input from Dave 1

XB data model meeting — 50/50 remote & in-person

During DrupalCon, Lauri, Ted, Alex, I met with with core committers Alex Pott, catch and Dave met to discuss XB’s JSON-based data storage model that XB currently implements. We’re not yet fully aligned (catch pointed out the search index aspect is important to support — the question is how to support that without compromising the UX Lauri envisions), but the discussion is much clearer today than it was in June, because there’s now concrete code to point to. That removed a lot of confusion on both “sides” (we’re all on the same side: we want the brightest future for Drupal!).

The meeting we had during DrupalCon led to:

1. Alex Bronstein identifying a possible alternative implementation that would meet both the original goals, and address most concerns: #3477428: Refactor the XB field type to be multi-valued, to de-jsonify the tree, and to reference the field_union type of the prop values.
2. Me unpostponing the #3467870: Support {type: array, …} prop shapes issue and pushing it forward. First making this work would help prevent #3477428 (see prior point) going in a direction that would make it impossible to support type: array Single Directory Component (SDC) props, which should be represented by multi-value fields (fields configured for multiple cardinality). I made the back-end pieces work during DrupalCon, but to make it work end-to-end additional infrastructure on the client side is needed first. For that: see the last “research” bullet above.

Missed a prior week? See all posts tagged Experience Builder.

Goal: make it possible to follow high-level progress by reading ~5 minutes/week. I hope this empowers more people to contribute when their unique skills can best be put to use!

For more detail, join the #experience-builder Slack channel. Check out the pinned items at the top!

Presentations at DrupalCon

Of course, Dries included and demonstrated Experience Builder 0.1.0 during the DriesNote:

The XB section of the DriesNote starts at 50:44.

Lauri talked about what’s been happening with XB and what will happen next:

Many of the things Lauri shared with all of you had only been seen by Lauri, not by anybody else! :D

After his session, Lauri had many hallway conversations that increased our conviction that we’re on the right track with XB! :)

And in my humble opinion the most inspiring — Ben’s session about how XB uses parts of the JSX theme engine and Redux:

Ben walks you through how XB leverages React and Redux to achieve the UX we need, while using existing Drupal field widgets. This will become even more important once we integrate the content entity form, with field widgets for base and bundle fields.

You have to watch the 20 seconds starting at 1:37 — pure genius: not the predictable AI-generated images to illustrate his talk, instead … his son’s drawings! :D

I hope to follow in his footsteps at a future DrupalCon, because I too am becoming a dad, very soon! :D I’ll be working at a very reduced rate during my paternity leave, but will be keeping these weekly blog posts going — it’s my way of keeping myself in the loop as well as all of you. That is also why I’ve shifted attention to meta things, to ensure the right expertise is present in areas that need to keep moving during my upcoming paternity leave :)

Week 20 was September 23–29, 2024.

1. We’re asking Dave to weigh in on a number of areas, to point his critical, independent core committer eye to key decisions early on. ↩︎

• Drupal
• Acquia
• Experience Builder

<strong>Topics covered in this episode:</strong><br> <ul> <li><strong><a href="https://www.python.org/downloads/release/python-3130/?featured_on=pythonbytes">Python 3.13.0 released Oct 7</a></strong></li> <li><strong><a href="https://peps.python.org/pep-0759/?featured_on=pythonbytes">PEP 759 – External Wheel Hosting</a></strong></li> <li><strong><a href="https://pypi.org/project/pytest-freethreaded/?featured_on=pythonbytes">pytest-freethreaded</a></strong></li> <li><strong><a href="https://github.com/MrMino/pytest-edit?featured_on=pythonbytes">pytest-edit</a></strong></li> <li><strong>Extras</strong></li> <li><strong>Joke</strong></li> </ul><a href='https://www.youtube.com/watch?v=6w3-YkKMJ0E' style='font-weight: bold;'data-umami-event="Livestream-Past" data-umami-event-episode="404">Watch on YouTube</a><br> <p><strong>About the show</strong></p> <p>Sponsored by ScoutAPM: <a href="https://pythonbytes.fm/scout"><strong>pythonbytes.fm/scout</strong></a></p> <p><strong>Connect with the hosts</strong></p> <ul> <li>Michael: <a href="https://fosstodon.org/@mkennedy"><strong>@mkennedy@fosstodon.org</strong></a></li> <li>Brian: <a href="https://fosstodon.org/@brianokken"><strong>@brianokken@fosstodon.org</strong></a></li> <li>Show: <a href="https://fosstodon.org/@pythonbytes"><strong>@pythonbytes@fosstodon.org</strong></a></li> </ul> <p>Join us on YouTube at <a href="https://pythonbytes.fm/stream/live"><strong>pythonbytes.fm/live</strong></a> to be part of the audience. Usually <strong>Monday</strong> at 10am PT. Older video versions available there too.</p> <p>Finally, if you want an artisanal, hand-crafted digest of every week of the show notes in email form? Add your name and email to <a href="https://pythonbytes.fm/friends-of-the-show">our friends of the show list</a>, we'll never share it. </p> <p><strong>Brian #1:</strong> <a href="https://www.python.org/downloads/release/python-3130/?featured_on=pythonbytes">Python 3.13.0 released Oct 7</a></p> <ul> <li>That’s today!</li> <li><a href="https://docs.python.org/3.13/whatsnew/3.13.html?featured_on=pythonbytes">What’s New In Python 3.13</a> <ul> <li>Interpreter (REPL) improvements <ul> <li>exit works (really, this is worth the release right here)</li> <li>Multiline editing with history preservation. <ul> <li>history sticks around between sessions</li> </ul></li> <li>Direct support for REPL-specific commands like help, exit, and quit, without the need to call them as functions.</li> <li>Prompts and tracebacks with color enabled by default.</li> <li>Interactive help browsing using F1 with a separate command history.</li> <li>History browsing using F2 that skips output as well as the &gt;&gt;&gt; and … prompts.</li> <li>“Paste mode” with F3 that makes pasting larger blocks of code easier (press F3 again to return to the regular prompt).</li> <li>exit now works without parens</li> </ul></li> <li>Improved error messages <ul> <li>Colorful tracebacks</li> <li>Better messages for <ul> <li>naming a script/module the same name as a stdlib module.</li> <li>naming a script/module the same name as an installed third party module.</li> <li>misspelling a keyword argument</li> </ul></li> </ul></li> <li>Free threaded CPython <ul> <li>Included in official installers on <a href="https://docs.python.org/3.13/using/windows.html#install-freethreaded-windows">Windows</a> and <a href="https://docs.python.org/3.13/using/mac.html#install-freethreaded-macos">macOS</a> <ul> <li>Read these links to figure out how - it’s not turned on by default</li> </ul></li> </ul></li> <li>Lot’s more. see the What’s new page</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href="https://peps.python.org/pep-0759/?featured_on=pythonbytes">PEP 759 – External Wheel Hosting</a></p> <ul> <li>pypi.org ships <a href="https://www.fastly.com/customers/python-software-foundation/?featured_on=pythonbytes">over 66 petabytes / month</a> backed by Fastly</li> <li>There are hard project size limits for publishers to PyPI</li> <li>We can host the essence of a <strong>.whl</strong> as a <strong>.rim</strong> file, then allow an external download URL</li> <li>Security: Several factors as described in this proposal should mitigate security concerns with externally hosted wheels, such as: <ul> <li>Wheel file checksums <strong>MUST</strong> be included in .rim files, and once uploaded cannot be changed. Since the checksum stored on PyPI is immutable and required, it is not possible to spoof an external wheel file, even if the owning organization lost control of their hosting domain.</li> <li>Externally hosted wheels <strong>MUST</strong> be served over HTTPS.</li> <li>In order to serve externally hosted wheels, organizations <strong>MUST</strong> be approved by the PyPI admins.</li> </ul></li> </ul> <p><strong>Brian #3:</strong> <a href="https://pypi.org/project/pytest-freethreaded/?featured_on=pythonbytes">pytest-freethreaded</a></p> <ul> <li>PyCon JP 2024 Team: <ul> <li>This extension was created at PyCon JP sprints with <a href="https://github.com/tonybaloney/pytest-freethreaded/blob/main/README.md#credits">Anthony Shaw and 7 other folks listed in credits.</a></li> </ul></li> <li>“A pytest plugin for helping verify that your tests and libraries are thread-safe with the Python 3.13 experimental freethreaded mode.”</li> <li>Testing your project for compatibility with freethreaded Python. <ul> <li>Testing in single thread doesn’t test that.</li> <li>Neither does testing with pytest-xdist, because it uses multiprocessing to parallelize tests.</li> <li>So, Ant and others “made this plugin to help you run your tests in a thread-pool with the GIL disabled, to help you identify if your tests are thread-safe.”</li> <li>“And the first library we tested it on (which was marked as compatible) caused a segmentation fault in CPython! So you should give this a go if you're a package maintainer.”</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href="https://github.com/MrMino/pytest-edit?featured_on=pythonbytes">pytest-edit</a></p> <ul> <li>A simple Pytest plugin for opening editor on the failed tests.</li> <li>Type <code>pytest --edit</code> to open the failing test code <img src="https://paper.dropboxstatic.com/static/img/ace/emoji/2728.png?version=8.0.0" alt="sparkles" /></li> <li>Be sure to set your favorite editor in the ENV variables</li> </ul> <p><strong>Extras</strong> </p> <p>Michael:</p> <ul> <li><a href="https://training.talkpython.fm/courses/all?featured_on=pythonbytes">New way to explore</a> Talk Python courses via topics <ul> <li>This has been in our <a href="https://training.talkpython.fm/apps?featured_on=pythonbytes">mobile apps</a> since their rewrite but finally comes to <a href="https://training.talkpython.fm/courses/all?featured_on=pythonbytes">the web</a></li> </ul></li> <li><a href="https://mkennedy.codes/posts/lets-go-easy-on-pypi-ok/?featured_on=pythonbytes">Let's go easy on PyPI, OK? essay</a></li> <li>Hynek’s video: <a href="https://www.youtube.com/watch?v=8UuW8o4bHbw">uv IS the Future of Python Packaging </a><a href="https://www.youtube.com/watch?v=8UuW8o4bHbw"><img src="https://paper.dropboxstatic.com/static/img/ace/emoji/1f40d.png?version=8.0.0" alt="snake" /></a><a href="https://www.youtube.com/watch?v=8UuW8o4bHbw"><img src="https://paper.dropboxstatic.com/static/img/ace/emoji/1f4e6.png?version=8.0.0" alt="package" /></a></li> <li><a href="https://github.com/adamchainz/djade-pre-commit?featured_on=pythonbytes">djade-pre-commit</a></li> <li>Polyfill.io, BootCDN, Bootcss, Staticfile attack <a href="https://www.bleepingcomputer.com/news/security/polyfillio-bootcdn-bootcss-staticfile-attack-traced-to-1-operator/?featured_on=pythonbytes">traced to 1 operator</a></li> <li><a href="https://purgecss.com/CLI.html?featured_on=pythonbytes">PurgeCSS CLI</a> </li> <li><a href="https://blog.python.org/2024/10/python-3127-released.html?featured_on=pythonbytes">Python 3.12.7 released</a></li> <li><a href="https://discuss.python.org/t/incremental-gc-and-pushing-back-the-3-13-0-release/65285/1?featured_on=pythonbytes">Incremental GC and pushing back the 3.13.0 release</a></li> <li><a href="https://fosstodon.org/@midnitte/113239548093653991">uv making the rounds</a></li> <li><a href="https://fosstodon.org/@mkennedy/113247714541010618">LLM fatigue</a>, is it real?</li> <li>Take the <a href="https://survey.alchemer.com/s3/8009809/l-python-developers-survey-2024/?utm_source=pythonbytespodcast">Python Developers Survey 2024</a></li> </ul> <p><strong>Joke:</strong> <a href="https://inzonedesign.com/blog/28-cleverly-funny-creative-404-error-pages/?featured_on=pythonbytes">Funny 404 pages</a></p> <ul> <li>We have <a href="https://pythonbytes.fm/not-found-page-or-something">something at least interesting at pythonbytes.fm</a></li> </ul>

API Testing in Pure English 2024-10-07, by Dariusz Suchojad How to test APIs in pure English

Do you have 20 minutes to learn how to test APIs in pure English, without any programming needed?

Great, the API testing tutorial is here.

Right after you complete it, you'll be able to write API tests as the one below.

Next steps:

➤ Read about how to use Python to build and integrate enterprise APIs that your tests will cover
➤ Python API integration tutorial
➤ Python Integration platform as a Service (iPaaS)
➤ What is an Enterprise Service Bus (ESB)? What is SOA?

More blog posts➤

This weekend "The KDE Alberts"[1] attended Google Summer of Code Mentor Summit 2024 in Sunnyvale, California.

The Google Summer of Code Mentor Summit is an annual unconference that every project participating in Google Summer of Code 2024 is invited to attend. This year it was the 20th year celebration of the program!

I was too late to take a picture of the full cake!

We attended many sessions ranging from how to try to avoid falling into the "xz problem" to collecting donations or shaping the governance of open source projects.

 

We met lots of people that knew what KDE was and were happy to congratulate us on the job done and also a few that did not know KDE and were happy to learn about what we do.

 

We also did a quick lightning talk about the GSOC projects KDE mentored this year and led two sessions: one centered around the problems some open source application developers are having publishing to the Google Play Store and another session about Desktop Linux together with our Gnome friends.

 

All in all a very productive unconference. We encourage KDE mentors to take the opportunity to attend the Google Summer of Code Mentor Summit next year, it's a great experience! 

 

[1] me and Albert Vaca, people were moderately amused that both of us had the same name, contribute to the same community and are from the same city.

The diffoscope maintainers are pleased to announce the release of diffoscope version 279. This version includes the following changes:

[ Chris Lamb ] * Drop removal of calculated basename from readelf output. (Closes: reproducible-builds/diffoscope#394)

You find out more by visiting the project homepage.

Dear Debian community,

this are my bits from DPL for September.

New lintian maintainer

I'm pleased to welcome Louis-Philippe Véronneau as a new Lintian maintainer. He humorously acknowledged his new role, stating, "Apparently I'm a Lintian maintainer now". I remain confident that we can, and should, continue modernizing our policy checker, and I see this as one important step toward that goal.

SPDX name / license tools

There was a discussion about deprecating the unique names for DEP-5 and migrating to fully compliant SPDX names.

Simon McVittie wrote: "Perhaps our Debian-specific names are better, but the relevant question is whether they are sufficiently better to outweigh the benefit of sharing effort and specifications with the rest of the world (and I don't think they are)." Also Charles Plessy sees the value of deprecating the Debian ones and align on SPDX.

The thread on debian-devel list contains several practical hints for writing debian/copyright files.

proposal: Hybrid network stack for Trixie

There was a very long discussion on debian-devel list about the network stack on Trixie that started in July and was continued in end of August / beginning of September. The discussion was also covered on LWN. It continued in a "proposal: Hybrid network stack for Trixie" by Lukas Märdian.

Contacting teams

I continued reaching out to teams in September. One common pattern I've noticed is that most teams lack a clear strategy for attracting new contributors. Here's an example snippet from one of my outreach emails, which is representative of the typical approach:

Q: Do you have some strategy to gather new contributors for your team? A: No. Q: Can I do anything for you? A: Everything that can help to have more than 3 guys :-D

Well, only the first answer, "No," is typical. To help the JavaScript team, I'd like to invite anyone with JavaScript experience to join the team's mailing list and offer to learn and contribute. While I've only built a JavaScript package once, I know this team has developed excellent tools that are widely adopted by others. It's an active and efficient team, making it a great starting point for those looking to get involved in Debian. You might also want to check out the "Little tutorial for JS-Team beginners".

Given the lack of a strategy to actively recruit new contributors--a common theme in the responses I've received--I recommend reviewing my talk from DebConf23 about teams. The Debian Med team would have struggled significantly in my absence (I've paused almost all work with the team since becoming DPL) if I hadn't consistently focused on bringing in new members. I'm genuinely proud of how the team has managed to keep up with the workload (thank you, Debian Med team!). Of course, onboarding newcomers takes time, and there's no guarantee of long-term success, but if you don't make the effort, you'll never find out.

OS underpaid

The Register, in its article titled "Open Source Maintainers Underpaid, Swamped by Security, Going Gray", summarizes the 2024 State of the Open Source Maintainer Report. I find this to be an interesting read, both in general and in connection with the challenges mentioned in the previous paragraph about finding new team members.

Kind regards Andreas.

Last time I exposed an old way learned in physical labs to do C or python/tk like in the old days: by summoning a tcl/tk interpreter and piping commands to it.

But what fun is it?

It's funnier if the tcl/tk interperpreter talks back to python :D as an hommage to the 25 years awaited TK9 versions that solves a lot of unicode trouble.

Beforehand, to make sense to the code a little warning is required : this code targets only POSIX environment and loses portability because I chose to use a way that is not the « one best way » for enabling bidirectionnal talks. By using os.set_blocking(p.stdout.fileno(), False) we can have portable non blocking IO, which means this trick has been tested on linux, freeBSD and windows successfully.

First and foremost, the Popen now use p.stdout=PIPE enabling the channel on which tcl will talk. As a joke puts/gets are named from tcl/tk functions and are used in python to push/get strings from tcl.

Instead of using multithreading having one thread listen to the output and putting the events in a local queue that the main thread will consume I chose the funniest technique of setting tcl/tk output non blocking which does not work on windows. This is the fnctl part of the code.

Then, I chose not to parse the output of tcl/tk but exec it, making tcl/tk actually push python commands back to python. That's the exec part of the code.

For this I needed an excuse : so I added buttons to change minutes/hours back and forth.

That's the moment we all are gonna agree that tcl/tk that tcl/tk biggest sin is its default look. Don't worry, next part is about using themes.

Compared to the first post, changes are minimal :D This is how it should look : And here is the code, largely still below 100 sloc (by 3 lines). #!/usr/bin/env python from subprocess import Popen, PIPE from time import sleep, time, localtime # import fcntl import os # let's talk to tk/tcl directly through p.stdin p = Popen(['wish'], stdin=PIPE, stdout=PIPE) # best non portable answer on stackoverflow #fd = p.stdout.fileno() #flag = fcntl.fcntl(fd, fcntl.F_GETFL) #fcntl.fcntl(fd, fcntl.F_SETFL, flag | os.O_NONBLOCK) # ^-- this 3 lines can be replaced with this one liner --v # portable non blocking IO os.set_blocking(p.stdout.fileno(), False) def puts(s): for l in s.split("\n"): p.stdin.write((l + "\n").encode()) p.stdin.flush() def gets(): ret=p.stdout.read() p.stdout.flush() return ret WIDTH=HEIGHT=400 puts(f""" canvas .c -width {WIDTH} -height {HEIGHT} -bg white pack .c . configure -background white ttk::button .ba -command {{ puts ch-=1 }} -text << pack .ba -side left -anchor w ttk::button .bb -command {{ puts cm-=1 }} -text < pack .bb -side left -anchor w ttk::button .bc -command {{ puts ch+=1 }} -text >> pack .bc -side right -anchor e ttk::button .bd -command {{ puts cm+=1 }} -text > pack .bd -side right -anchor e """) # Constant are CAPitalized in python by convention from cmath import pi as PI, e as E ORIG=complex(WIDTH/2, HEIGHT/2) # correcting python notations j => I I = complex("j") rad_per_sec = 2.0 * PI /60.0 rad_per_min = rad_per_sec / 60 rad_per_hour = rad_per_min / 12 origin_vector_hand = WIDTH/2 * I size_of_sec_hand = .9 size_of_min_hand = .8 size_of_hour_hand = .65 rot_sec = lambda sec : -E ** (I * sec * rad_per_sec ) rot_min = lambda min : -E ** (I * min * rad_per_min ) rot_hour = lambda hour : -E ** (I * hour * rad_per_hour ) to_real = lambda c1,c2 : "%f %f %f %f" % (c1.real,c1.imag,c2.real, c2.imag) for n in range(60): direction= origin_vector_hand * rot_sec(n) start=.9 if n%5 else .85 puts(f".c create line {to_real(ORIG+start*direction,ORIG+.95*direction)}") sleep(.01) diff_offset_in_sec = (time() % (24*3600)) - \ localtime()[3]*3600 -localtime()[4] * 60.0 \ - localtime()[5] ch=cm=0 while True: # eventually parsing tcl output back = gets() # trying is more concise than checking try: back = back.decode() exec(back) except Exception as e: pass t = time() s= t%60 m = m_in_sec = t%(60 * 60) + cm * 60 h = h_in_sec = (t- diff_offset_in_sec)%(24*60*60) + ch * 3600 + cm * 60 puts(".c delete second") puts(".c delete minute") puts(".c delete hour") c0=ORIG+ -.1 * origin_vector_hand * rot_sec(s) c1=ORIG+ size_of_sec_hand * origin_vector_hand * rot_sec(s) puts( f".c create line {to_real(c0,c1)} -tag second -fill blue -smooth true") c1=ORIG+size_of_min_hand * origin_vector_hand * rot_min(m) puts(f".c create line {to_real(ORIG, c1)} -tag minute -fill green -smooth true") c1=ORIG+size_of_hour_hand * origin_vector_hand * rot_hour(h) puts(f".c create line {to_real(ORIG,c1)} -tag hour -fill red -smooth true") puts("flush stdout") sleep(.1)
Some history about this code.

I have been mentored in a physical lab where we where doing the pipe, fork, dup2 dance to tcl/tk from C to give a nice output to our simulations so we could control our instuition was right and could extract pictures for the publications. This is a trick that is almost as new as my arteries.
My mentor used to say : we are not coders, we need stuff to work fast and neither get drowned in computer complexity or endless quest for « the one best way » nor being drowned in bugs, we aim for the Keep It Simple Stupid Ways.

Hence, this is a Keep It Simple Stupid approach that I revived for the sake of seeing if it was still robust after 35 years without using it.

Well, if it's robust and it's working: it ain't stupid even if it isn't the « one best idiomatic way ». :P

Marble gets an update, KDE Connect gets a speed boost, and Kate gets all fluttery

Welcome to a new issue of “This Week in KDE Apps”! In case you missed it, we announced this series a few weeks ago, and our goal is to cover as much as possible of what's happening in the world of KDE apps and supplement Nate's This Week in Plasma published yesterday.

This week we had new releases of Tellico and Krita. We are also covering news regarding KDE Connect, the link between all your devices; Kate, the KDE advanced text editor; Itinerary, the travel assistant that helps you plan all your trips; Marble, KDE's map application; and more.

Let's get started!

Dolphin Manage your files

Dolphin now uses ripgrep-all or ripgrep for content search when Baloo indexing is disabled. Detailed information (Jin Liu, 24.12.0. Link)

The checksum and permissions tab in the property dialog used by Dolphin and other KIO-enabled applications is now more consitent with the other tabs. (Thomas Duckworth, Frameworks 6.8. Link)

Kaidan User-friendly and modern chat app for every device

Kaidan, KDE's XMPP instant messaging app, improves support for group chats. (Melvin Keskin, Link)

Kate Advanced Text Editor

Kate adds out of the box support for debugging Flutter projects. (Waqar Ahmed, 24.12.0. Link 1, link 2)

The option to 'Reopen latest closed documents' has been added to the tab context menu. (Waqar Ahmed, 24.12.0. Link)

Kdenlive Video editor

KDE e.V. and Kdenlive have posted two job offers for contractors to work on Kdenlive. Will this be your opportunity to contribute to KDE and get paid too?

KDE Connect Seamless connection of your devices

KDE Connect starts up much faster on macOS — startup time has gone from 3s to 100ms! (Albert Vaca Cintora, 24.12.0. Link)

Kleopatra Certificate Manager and Unified Crypto GUI

Kleopatra makes its decryption errors easier to understand when content was encrypted with a certificate you don't have. (Tobias Fella, 24.12.0. Link)

Krita Digital Painting, Creative Freedom

Krita 5.2.6 is out and fixes a critical issue that popped up in last week's release. More information.

Krusader File Manager

Krusader has been migrated to Qt6 and KF6. (Alex Bikadorov, 3.0.0. Link)

KStars Desktop Planetarium

KStars 3.7.3 is out with exciting features for astrophotography buffs. You're going to want to update if you're using multiple cameras with per-camera targeting and scheduling, leader-and-follower jobs, and focus synchronization. Read more here!

Weather View real-time weather forecasts

KWeather removes the "Add current location" button, as it doesn't work anymore since the shutdown of Mozilla's location service. (Devin Lin, 24.08.2. Link)

The setup wizard has been overhauled. (Devin Lin, 24.08.2. Link)

KDE Itinerary Digital travel assistant

A new bi-monthly blog post about Itinerary and the infrastructure behind it is out: August/September in KDE Itinerary

Itinerary now extracts membership ids in German-language Eurostar (Thalys) tickets (Luca Weiss, 24.08.2. Link)

It can extract seat reservation data from SBB QR codes (Volker Krause, 24.08.2. Link)

If you are arranging accommodations, Itinerary can handle German language variants of NH Hotels booking confirmations (Volker Krause, 24.08.2. Link)

LabPlot Interactive Data Visualization and Analysis

The Color Maps Browser now has multiple view modes. Including one that shows detailed information about the used colors in the color map and that also allows to copy those values. (Alexander Semke, Link)

Added a new visualization type: Run Chart, (Alexander Semke, Link)

NeoChat Chat with your friends on matrix

NeoChat has a fix for a frequent and random crash on Android caused by receiving a notification. (James Graham, 24.12.0. Link)

The hover actions for the messages are now more reliable. (Carl Schwan, 24.12.0. Link)

Marble Virtual Globe

Marble Behaim got a new logo, similar to the Marble Maps logo. (Mathis Brüchert, 24.12.0. Link)

Marble Maps routing functionality was ported to Qt6 and redesigned. (Carl Schwan, 24.12.0. Link)

Spectacle Screenshot Capture Utility

Spectacle fixed a crash when saving while the system's timezone is misconfigured (Noah Davis, 24.08.2. Link)

Tellico Collection Manager

Tellico, the KDE app that helps you manage all your collecions, is out with version 4.0.1. This version includes fixes for Qt6. More information.

Tokodon Browse the Fediverse

Tokodon fetches public servers and displays them in a list for registration. The list is fetched from joinmastodon.org and more filtering options will be added later. (Joshua Goins, 24.12.0. Link)

Instead of wrapping all the tags for a post, they are now made scrollable. (Joshua Goins, 24.12.0. Link)

Apps on Windows

KDE Apps on windows now have better looking tooltips and menus without black corners. (Carl Schwan, Breeze 6.2.1. Link)

Third Party Applications

To get your application mentioned here. Please contact us on invent or in Matrix.

Kraft Quotes and invoices for small business

Kraft is a desktop app making it easy to create offers and invoices quickly and beautifully in small companies. Version 1.2.2 was just released and contains some small bug fixes. This is the last release before Kraft 2.0. More information.

…And Everything Else

This blog only covers the tip of the iceberg! If you’re hungry for more, check out Nate's blog about Plasma and KDE's Planet, where you can find more news from other KDE contributors.

Get Involved

The KDE organization has become important in the world, and your time and contributions have helped us get there. As we grow, we're going to need your support for KDE to become sustainable.

You can help KDE by becoming an active community member and getting involved. Each contributor makes a huge difference in KDE; you are not a number or a cog in a machine! You don’t have to be a programmer, either. There are many things you can do: you can help hunt and confirm bugs, even maybe solve them; contribute designs for wallpapers, web pages, icons and app interfaces; translate messages and menu items into your own language; promote KDE in your local community; and a ton more things.

You can also help us by donating. Any monetary contribution, however small, will help us cover operational costs, salaries, travel expenses for contributors and in general help KDE continue bringing Free Software to the world.

To get your application mentioned here. Please let us know in invent or in Matrix.

You're about to launch your new app or API, or even just a big refactor of your current project. Will it stand up and deliver when you put it into production or when that big promotion goes live? Or will it wither and collapse? How would you know? Well you would test that of course. We have Anthony Shaw back on the podcast to dive into a wide range of tools and techniques for performance and loading testing of web apps.<br/> <br/> <strong>Episode sponsors</strong><br/> <br/> <a href='https://talkpython.fm/sentry'>Sentry Error Monitoring, Code TALKPYTHON</a><br> <a href='https://talkpython.fm/workos'>WorkOS</a><br> <a href='https://talkpython.fm/training'>Talk Python Courses</a><br/> <br/> <strong>Links from the show</strong><br/> <br/> <div><b>Anthony on Twitter</b>: <a href="https://twitter.com/anthonypjshaw?featured_on=talkpython" target="_blank" >@anthonypjshaw</a><br/> <b>Anthony's PyCon Au Talk</b>: <a href="https://www.youtube.com/watch?v=or3PbMGMz4o" target="_blank" >youtube.com</a><br/> <b>locust load testing tool</b>: <a href="https://locust.io?featured_on=talkpython" target="_blank" >locust.io</a><br/> <b>playwright</b>: <a href="https://playwright.dev?featured_on=talkpython" target="_blank" >playwright.dev</a><br/> <b>mimesis</b>: <a href="https://github.com/lk-geimfari/mimesis?featured_on=talkpython" target="_blank" >github.com</a><br/> <b>mimesis providers</b>: <a href="https://mimesis.name/en/master/providers.html?featured_on=talkpython" target="_blank" >mimesis.name</a><br/> <b>vscode pets</b>: <a href="https://marketplace.visualstudio.com/items?itemName=tonybaloney.vscode-pets&featured_on=talkpython" target="_blank" >marketplace.visualstudio.com</a><br/> <b>vscode power-mode</b>: <a href="https://marketplace.visualstudio.com/items?itemName=hoovercj.vscode-power-mode&featured_on=talkpython" target="_blank" >marketplace.visualstudio.com</a><br/> <b>opentelemetry</b>: <a href="https://opentelemetry.io?featured_on=talkpython" target="_blank" >opentelemetry.io</a><br/> <b>uptime-kuma</b>: <a href="https://github.com/louislam/uptime-kuma?featured_on=talkpython" target="_blank" >github.com</a><br/> <b>Talk Python uptime / status</b>: <a href="https://talkpython.fm/status" target="_blank" >talkpython.fm/status</a><br/> <b>when your serverless computing bill goes parabolic...</b>: <a href="https://www.youtube.com/watch?v=SCIfWhAheVw" target="_blank" >youtube.com</a><br/> <b>Watch this episode on YouTube</b>: <a href="https://www.youtube.com/watch?v=W6UVq8zVtxU" target="_blank" >youtube.com</a><br/> <b>Episode transcripts</b>: <a href="https://talkpython.fm/episodes/transcript/479/designing-effective-load-tests-for-your-python-app" target="_blank" >talkpython.fm</a><br/> <br/> <b>--- Stay in touch with us ---</b><br/> <b>Subscribe to us on YouTube</b>: <a href="https://talkpython.fm/youtube" target="_blank" >youtube.com</a><br/> <b>Follow Talk Python on Mastodon</b>: <a href="https://fosstodon.org/web/@talkpython" target="_blank" ><i class="fa-brands fa-mastodon"></i>talkpython</a><br/> <b>Follow Michael on Mastodon</b>: <a href="https://fosstodon.org/web/@mkennedy" target="_blank" ><i class="fa-brands fa-mastodon"></i>mkennedy</a><br/></div>

SVG cursor themes is a new feature in Plasma 6.2, which we are really excited about. In this blog post, I would like to provide more background behind what motivated us to add support for them, what they are, and how to build them.

(Classic) cursor theme format

A cursor theme is a collection of images defining the contents of various cursor shapes and additional metadata (for example, the human readable name of the theme, whether the cursor theme inherits/extends another cursor theme, etc). On disk, it looks as follows

The cursors/ directory contains a list of Xcursor files and symbolic links to represent cursor shape aliases, e.g. the arrow being an alias for default. The XCursor format has been in use for a very long time now and it has a pretty simple structure

The layout of an XCursor file

An XCursor file consists of a header that includes a magic number to determine whether particular file is actually an XCursor file, the size of the header in bytes, the file version, and the number of ToC entries. Every ToC entry provides the information about the corresponding chunk, for example the chunk type and where the chunk can be found in the file. Lastly, a chunk contains some useful data. A chunk may contain image data or text data, etc.

For example, here’s the image data that can be found in the “default” cursor shape file in the Adwaita cursor theme

As you can see, the Adwaita cursor theme provides the following sizes: 24, 32, 48, 64, and 96.

The index.theme files looks as follows

[Icon Theme] Inherits=breeze_cursors Name=Cool Cursor Comment=That is a cool cursor theme

Cursor themes can be found in $DATADIR/icons directories. For example, /usr/share/icons or ~/.local/share/icons.

X11 vs Wayland cursors

Xcursor cursors are used both on X11 and Wayland, but the way how the cursor size is interpreted is different on the two platforms. X11 assumes that the cursor size is specified in the device pixels, while Wayland assumes that it’s in the logical pixels. Logical pixels have the same visual size across various devices, while physical pixels are specific to particular device. For example, 24 logical pixels on an output with a scale factor of 2 corresponds to 48 physical pixels.

Cursor sizes in Xcursor files are specified in the device pixels.

Another very important detail is that the XCURSOR_SIZE environment variable is treated differently by X11 and Wayland native applications. For example, if XCURSOR_SIZE is set to 24 and the output scale is 2, an X11 application would load a cursor with the size 24, but a Wayland application would effectively load a cursor with the size 48 (24 * 2) because it would see that the output is scaled so the provided cursor needs to be scaled accordingly as well.

“XCURSOR_SIZE=24 dolphin -platform xcb” (left) vs “XCURSOR_SIZE=24 dolphin -platform wayland” (right). Note that “Apply scaling themselves” has been selected in the display settings in Plasma Wayland Limitations of Xcursor

The most painful thing about Xcursor is its lack of the proper HiDPI support. As it was said in the previous chapter, the cursor size in Xcursor files is specified in the device pixels. On X, it’s not a problem because all geometries are specified in the device pixels. It also means that if you change the scaling factor on X, you need to change the cursor size manually so the cursor is not too small. On Wayland, the cursor size is specified in the logical pixels so the compositor and the clients have to scale the cursor size in order to match the output scale. For example, if the configured cursor size is 24 and the window is on an output with a scale factor of 2, the application needs to load an Xcursor cursor with the size 48. If the cursor theme provides cursors with such a size, perfect! But what if it doesn’t? At the moment, every compositor and client applies its own policies. Some find the cursor with the closest size and use that, some find the cursor with the closest size and then scale it to match the requested size at the cost of adding some blurriness, and so on. It’s a mess. Because neither compositors nor clients can agree how to handle such a case, you could easily observe the cursor changing its size when moving between windows owned by different applications or when moving the cursor between the window and its decoration, e.g.

A script element has been removed to ensure Planet works properly. Please find it in the original post.

It’s worth noting though that this issue can be worked around by using the cursor-shape protocol because with it, the application can delegate the compositor the task of loading and displaying cursors. But the bottom line is that the Xcursor format is unsuitable for the HiDPI model that we have present on Wayland.

Another issue with the Xcursor format is that the image data is stored in an uncompressed format. It is okay if you need to provide cursors with small sizes, for example up to 72, but there are cases when you need to display a cursor at a very large size. For example, one such a case is the shake cursor accessibility feature in the Plasma Wayland session.

With the shake cursor feature enabled, the cursor will be inflated when it’s shaken. In order to operate, it needs to load the default cursor shape with a size around 250. If cursor themes provided images for such sizes, their package sizes would easily blow up beyond the 100MiB mark. That’s not good. And as a workaround, in Plasma 6.1, the shake cursor uses its own high resolution images of the Breeze cursor themes.

A script element has been removed to ensure Planet works properly. Please find it in the original post. Shake cursor without any workarounds in 6.1 A script element has been removed to ensure Planet works properly. Please find it in the original post. Shake cursor with workarounds in 6.1

The XCursor format was perfectly suitable for the use cases that existed back in the late 90s and early 2000s, but things have changed over the years and its current raster nature can’t keep up with the use cases that we have now (2024). We’ve got fractional scaling, we’ve got accent colors, we’ve got features that enlarge the cursor, and so on.

SVG cursor format

First of all, let’s build a list of requirements that the svg cursor format must satisfy:

• obviously, it must support the ability to define cursor contents using svg files so we can fix HiDPI issues, etc
• easy porting process for existing clients and compositors
• it should be easy to develop and analyze svg cursor themes. Xcursor is a binary file format, which requires a special tool to create Xcursor files, we would like to avoid that with svg cursors
• last and the most important requirement is that there must be some compatibility with the existing cursor theme format. We must not be required to write a new system settings module to handle the new cursor format, and the apps that don’t support svg cursors should easily fallback to the Xcursor format.

Here’s how a cursor theme providing svg cursors would look like

index.theme has the same format both for XCursor and SVG cursors. cursors/ directory contains the XCursor cursors, and cursors_scalable/ contains the SVG cursors.

In cursors_scalable/, every cursor shape must have its own directory, or if it’s an alias, then it must be a symlink. Every cursor shape directory must contain the cursor image and a metadata.json file providing the information about the cursor.

For a static cursor, the metadata.json file looks as follows

[    {        "filename": "default.svg",        "hotspot_x": 4,        "hotspot_y": 4,        "nominal_size": 24    } ]

The filename property specifies the filename of the svg file. The hotspot_x and the hotspot_y properties specify the coordinates of the hot spot. The hot spot in the cursor determines the point where interaction with other elements on the screen occurs, e.g. clicks. The nominal_size property specifies the cursor size that the svg file represents. The nominal size is used to decide how much the svg image and the hotspot coordinates need to be scaled in order to get a cursor with the requested size. Note that the nominal size can’t be determined based on the <svg>‘s width and height attributes because there exist themes such as Breeze whose canvas is bigger than the represented cursor size. As an example, in the Breeze cursor theme, the canvas size is 32x32 even though the represented cursor size is 24 in order to accommodate for additional elements that can be attached to the arrow cursor, e.g. a little circle with a plus sign or a question mark.

For an animated cursor, the metadata.json file looks as follows

[ { "filename": "wait-01.svg", "delay": 30, "hotspot_x": 16, "hotspot_y": 15, "nominal_size": 24 }, { "filename": "wait-02.svg", "delay": 30, "hotspot_x": 16, "hotspot_y": 15, "nominal_size": 24 }, ... { "filename": "wait-42.svg", "delay": 30, "hotspot_x": 16, "hotspot_y": 15, "nominal_size": 24 } ]

The only new thing is the delay property. The delay property indicates the animation delay to the next frame.

A cursor theme that ships SVG cursors is required to have XCursor cursors too. This is needed to provide fallback for legacy applications that are unaware of the cursor-shape-v1 protocol or simply too old applications that are unlikely to be changed anymore. This restriction might be lifted in the future.

It is worth mentioning that SVG supports animations natively. However, that approach was not chosen for cursor animations for two reasons: to allow caching svg render results more easily and require fewer changes in the compositors and the apps to adapt the svg cursor format.

You can find the json schema for metadata.json over here.

Accent colors

Since the cursor contents is specified using the SVG format, it should be possible to re-color the cursor based on the currently configured accent color. As of now, it is not implemented, but, in general, this is doable and perhaps such a feature will be added to Plasma some day.

Standardization

This cursor format is not officially standardized. We are looking forward to making it upstream, but for now, the main focus is on confirming that the new format lives up to our and cursor creator needs.

To cursor theme creators

Breeze and Breeze Light are the only two cursor themes that support SVG cursors at the moment, but we would love to see custom themes adapting them too so users experience fewer issues with fractional scaling or other features in Plasma when using their favorite 3rdparty cursor themes. We would also like to hear feedback from the cursor theme creators regarding whether it’s easy to adapt this cursor format or whether some additional features are needed. You can reach out to us at Matrix in the #kwin room https://webchat.kde.org/#/room/#kwin:kde.org or in the kwin mailing list.

Examples

If you need an example of a cursor theme that supports SVG cursors, please check the Breeze cursor theme.

Closing words

The new SVG cursor format is amazing. Please try it!