Planet Debian

We did it again™! Just in time, we’re excited to announce the release of Grml stable version 2024.12, code-named ‘Adventgrenze’! (If you’re not familiar with Grml, it’s a Debian-based live system tailored for system administrators.)

This new release is built on Debian trixie, and for the first time, we’re introducing support for 64-bit ARM CPUs (arm64 architecture)!

I’m incredibly proud of the hard work that went into this release. A significant amount of behind-the-scenes effort went into reworking our infrastructure and redesigning the build process. Special thanks to Chris and Darsha – our Grml developer days in November and December were a blast!

For a detailed overview of the changes between releases 2024.02 and 2024.12, check out our official release announcement. And, as always, after a release comes the next one – exciting improvements are already in the works!

BTW: recently we also celebrated 20(!) years of Grml Releases. If you’re a Grml and or grml-zsh user, please join us in celebrating and send us a postcard!

A coworker asked recently about how people use VMs locally for dev work, so I figured I’d take a few minutes to write up a bit about what I do. There are many use cases for local virtual machines in software development and testing. They’re self-contained, meaning you can make a mess of them without impacting your day-to-day computing environment. They can run different distributions, kernels, and even entirely different operating systems from the one you use regularly. Etc. They’re also cheaper than cloud services and provide finer grained control over the resources.

I figured I’d share a little bit about how I manage different virtual machines in case anybody finds this useful. This is what works for me, but it won’t necessarily work for you, or maybe you’ve already got something better. I’ve found it to be easy to work with, light weight, and is easy to evolve my needs change.

Use short-lived VMs

Rather than keep a long-lived “development” VM around that you customize over time, I recommend automating the common customizations and provisioning new VMs regularly. If I’m working on reproducing a bug or testing a change prior to submitting it upstream, I’ll do this work in a VM and delete the VM when when I’m done. When provisioning VMs this frequently, though, walking through the installation process for every new VM is tedious and a waste of time. Since most of my work is done in Debian, so I start with images generated daily by the cloud team. These images are available for multiple releases and architectures. The ‘nocloud’ variant boots to a root prompt and can be useful directly, or the ‘generic’ images can be used for cloud-init based customization.

Automating image preparation

This makefile lets me do something like make image and get a new qcow2 image with the latest build of a given Debian release (sid by default, with others available by specifying DIST).

DATESTAMP=$(shell date +"%Y-%m-%d") FLAVOR?=generic ARCH?=$(shell dpkg --print-architecture) DIST?=sid RELEASE=$(DIST) URL_PATH=https://cloud.debian.org/images/cloud/$(DIST)/daily/latest/ ifeq ($(DIST),trixie) RELEASE=13 endif ifeq ($(DIST),bookworm) RELEASE=12 endif ifeq ($(DIST),bullseye) RELEASE=11 endif debian-$(DIST)-$(FLAVOR)-$(ARCH)-daily.tar.xz: curl --fail --connect-timeout 20 -LO \ $(URL_PATH)/debian-$(RELEASE)-$(FLAVOR)-$(ARCH)-daily.tar.xz $(DIST)-$(FLAVOR)-$(DATESTAMP).qcow2: debian-$(RELEASE)-$(FLAVOR)-$(ARCH)-daily.tar.xz tar xvf debian-$(RELEASE)-$(FLAVOR)-$(ARCH)-daily.tar.xz qemu-img convert -O qcow2 disk.raw $@ rm -f disk.raw qemu-img resize $@ 20g qemu-img snapshot -c untouched $@ image: $(DIST)-$(FLAVOR)-$(DATESTAMP).qcow2 .PHONY: image Customize the VM environment with cloud-init

While the ‘nocloud’ images can be useful, I typically find that I want to apply the same modifications to each new VM I launch, and they don’t provide facilities for automating this. The ‘generic’ images, on the other hand, run cloud-init by default. Using cloud-init, I can create my user account, point apt at local mirrors, install my preferred tools, ensure the root filesystem is resized to make full use of the backing storage, etc.

The cloud-init configuration on the generic images will read from a local config drive, which can contain an ISO9660 (cdrom) filesystem image. This image can be generated from a subdirectory containing the various cloud-init input files using the following make syntax:

IMDS_FILES=$(shell find seedconfig -path '*/.git/*' \ -prune -o -type f -name '*.in.json' -print) \ seedconfig/openstack/latest/user_data seed.iso: $(IMDS_FILES) genisoimage -V config-2 -o $@ -J -R -m '*~' -m '.git' seedconfig

With the image in place, the VM can be created with

qemu-system-x86_64 -machine q35,accel=kvm -cpu host -m 4g -drive file=${img},index=0,if=virtio,media=disk -drive file=seed.iso,media=cdrom,format=raw,index=2,if=virtio -nic user -nographic

This invokes qemu with the root volume and ISO image attached as disks, uses an emulated “q35” machine with the host’s CPU and KVM acceleration, the userspace network stack, and a serial console. The first time the VM boots, cloud-init will apply the configuration from the cloud-config available in the ISO9660 filesystem.

Alternatives to cloud-init

virt-customize is another tool accomplishing the same type of customization. I use cloud-init because it works directly with cloud providers in addition to local VM images. You could also use something like ansible.

Variations

I have a variant of this that uses a bridged network, which I’ll write more about later. The bridge is nice because it’s more featureful, with full support for IPv6, etc, but it needs a bit more infrastructure in place.

It also can be helpful to use 9p or virtfs to share filesystem state between the host the VM. I don’t tend to rely on these, and will instead use rsync or TRAMP for moving files around.

Containers are also useful, of course, and there are plenty of times when the full isolation of a VM is not worth the overhead.

After the MiniDebConf Marseille 2019, COVID-19 made it impossible or difficult to organize new MiniDebConfs for a few years. With the gradual resumption of in-person events (like FOSDEM, DebConf, etc.), the idea emerged to host another MiniDebConf in France, but with a lighter organizational load. In 2023, we decided to reach out to the organizers of Capitole du Libre to repeat the experience of 2017: hosting a MiniDebConf alongside their annual event in Toulouse in November. However, our request came too late for 2023. After discussions with Capitole du Libre in November 2023 in Toulouse and again in February 2024 in Brussels, we confirmed that a MiniDebConf Toulouse would take place in November 2024!

We then assembled a small organizing team and got to work: a Call for Papers in May 2024, adding a two-day MiniDebCamp, coordinating with the DebConf video team, securing sponsors, creating a logo, ordering T-shirts and stickers, planning the schedule, and managing registrations. Even with lighter logistics (conference rooms, badges, and catering during the weekend were handled by Capitole du Libre), there was still quite a bit of preparation to do.

On Thursday, November 14, and Friday, November 15, 2024, about forty developers arrived from around the world (France, Spain, Italy, Switzerland, Germany, England, Brazil, Uruguay, India, Brest, Marseille…) to spend two days at the MiniDebCamp in the beautiful collaborative spaces of Artilect in Toulouse city center.

Then, on Saturday, November 16, and Sunday, November 17, 2024, the MiniDebConf took place at ENSEEIHT as part of the Capitole du Libre event. The conference kicked off on Saturday morning with an opening session by Jérémy Lecour, which included a tribute to Lunar (Nicolas Dandrimont). This was followed by Reproducible Builds – Rebuilding What is Distributed from ftp.debian.org (Holger Levsen) and Discussion on My Research Work on Sustainability of Debian OS (Eda). After lunch at the Capitole du Libre food trucks, the intense afternoon schedule began: What’s New in the Linux Kernel (and What’s Missing in Debian) (Ben Hutchings), Linux Live Patching in Debian (Santiago Ruano Rincón), Trixie on Mobile: Are We There Yet? (Arnaud Ferraris), PostgreSQL Container Groups, aka cgroups Down the Road (Cédric Villemain), Upgrading a Thousand Debian Hosts in Less Than an Hour (Jérémy Lecour and myself), and Using Debusine to Automate Your QA (Stefano Rivera & co).

Sunday marked the second day, starting with a presentation on DebConf 25 (Benjamin Somers), which will be held in Brest in July 2025. The morning continued with talks: How LTS Goes Beyond LTS (Santiago Ruano Rincón & Roberto C. Sánchez), Cross-Building (Helmut Grohne), and State of JavaScript (Bastien Roucariès). In the afternoon, there were Lightning Talks, PyPI Security: Past, Present & Future (Salvo “LtWorf” Tomaselli), and the classic Bits from DPL (Andreas Tille), before closing with the final session led by Pierre-Elliott Bécue.

All talks are available on video (a huge thanks to the amazing DebConf video team), and many thanks to our sponsors (Viridien, Freexian, Evolix, Collabora, and Data Bene). A big thank-you as well to the entire Capitole du Libre team for hosting and supporting us… see you in Brest in July 2025!

Articles about (or mentioning) MiniDebConf Toulouse:

• A Mini Adventure at MiniDebConf Toulouse
• Bits from MiniDebConf Toulouse 2024
• Reproducible Builds: November 2024 Updates
• Bits from the DPL: December 2024
• Chris Watson’s Blog: November 2024 Activity
• Freexian Blog: Debian Contributions November 2024

A concerned nutritional epidemiologist in Tokyo realizes that if you are what you eat, that means…

It’s a similar situation in Seoul, albeit with less oil and more confidence.

I am using GitLab CI/CD pipelines for several upstream projects (libidn, libidn2, gsasl, inetutils, libtasn1, libntlm, …) and a long-time concern for these have been that there is too little testing on GNU Guix. Several attempts have been made, and earlier this year Ludo’ came really close to finish this. My earlier effort to idempotently rebuild Debian recently led me to think about re-bootstrapping Debian. Since Debian is a binary distribution, it re-use earlier binary packages when building new packages. The prospect of re-bootstrapping Debian in a reproducible way by rebuilding all of those packages going back to the beginning of time does not appeal to me. Instead, wouldn’t it be easier to build Debian trixie (or some future release of Debian) from Guix, by creating a small bootstrap sandbox that can start to build Debian packages, and then make sure that the particular Debian release can idempotently rebuild itself in a reproducible way? Then you will eventually end up with a reproducible and re-bootstrapped Debian, which pave the way for a trustworthy release of Trisquel. Fortunately, such an endeavour appears to offer many rabbit holes. Preparing Guix container images for use in GitLab pipelines is one that I jumped into in the last few days, and just came out of.

Let’s go directly to the point of this article: here is a GitLab pipeline job that runs in a native Guix container image that builds libksba after installing the libgpg-error dependency from Guix using the pre-built substitutes.

test-amd64-latest-wget-configure-make-libksba: image: registry.gitlab.com/debdistutils/guix/container:latest before_script: - lndir /gnu/store/*profile/etc/ /etc - rm -f /etc/group - groupadd --system guixbuild - for i in $(seq -w 1 10); do useradd -g guixbuild -G guixbuild -d /var/empty -s $(command -v nologin) -c "Guix build user $i" --system guixbuilder$i; done - export HOME=/ - export LANG=C.UTF-8 - guix-daemon --disable-chroot --build-users-group=guixbuild & - guix archive --authorize < /share/guix/ci.guix.gnu.org.pub - guix archive --authorize < /share/guix/bordeaux.guix.gnu.org.pub - guix describe - guix package -i libgpg-error - GUIX_PROFILE="//.guix-profile" - . "$GUIX_PROFILE/etc/profile" script: - wget https://www.gnupg.org/ftp/gcrypt/libksba/libksba-1.6.7.tar.bz2 - tar xfa libksba-1.6.7.tar.bz2 - cd libksba-1.6.7 - ./configure - make V=1 - make check VERBOSE=t V=1

You can put that in a .gitlab-ci.yml and push it to GitLab and you will end up with a nice pipeline job output.

As you may imagine, there are several things that are sub-optimal in the before_script above that ought to be taken care of by the Guix container image, and I hope to be able to remove as much of the ugliness as possible. However that doesn’t change that these images are useful now, and I wanted to announce this work to allow others to start testing them and possibly offer help. I have started to make use of these images in some projects, see for example the libntlm commit for that.

You are welcome to join me in the Guix container images for GitLab CI/CD project! Issues and merge requests are welcome – happy hacking folks!

Boost is a very large and comprehensive set of (peer-reviewed) libraries for the C++ programming language, containing well over one hundred individual libraries. The BH package provides a sizeable subset of header-only libraries for (easier, no linking required) use by R. It is fairly widely used: the (partial) CRAN mirror logs (aggregated from the cloud mirrors) show over 38.5 million package downloads.

Version 1.87.0 of Boost was released last week following the regular Boost release schedule of April, August and December releases. As before, we packaged it almost immediately and started testing following our annual update cycle which strives to balance being close enough to upstream and not stressing CRAN and the user base too much. The reverse depends check revealed six packages requiring changes or adjustments. We opened issue #103 to coordinate the issue (just as we did in previous years). Our sincere thanks to Matt Fidler who fixed two packages pretty much immediately.

As I had not heard back from the other maintainers since filing the issue, I uploaded the package to CRAN suggesting that the coming winter break may be a good opportunity for the four other packages to catch up. CRAN concurred, and 1.87.0-1 is now available there.

There are no other changes apart from cosmetics in the DESCRIPTION file. For once, we did not add any new Boost libraries. The short NEWS entry follows.

Changes in version 1.87.0-1 (2024-12-17)

• Upgrade to Boost 1.87.0, patched as usual to comment-out diagnostic suppression messages per the request of CRAN

• Switched to Authors@R

Via my CRANberries, there is a diffstat report relative to the previous release. Comments and suggestions about BH are welcome via the issue tracker at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post is a review for Computing Reviews for The science of detecting LLM-generated text , a article published in Communications of the ACM

While artificial intelligence (AI) applications for natural language processing (NLP) are no longer something new or unexpected, nobody can deny the revolution and hype that started, in late 2022, with the announcement of the first public version of ChatGPT. By then, synthetic translation was well established and regularly used, many chatbots had started attending users’ requests on different websites, voice recognition personal assistants such as Alexa and Siri had been widely deployed, and complaints of news sites filling their space with AI-generated articles were already commonplace. However, the ease of prompting ChatGPT or other large language models (LLMs) and getting extensive answers–its text generation quality is so high that it is often hard to discern whether a given text was written by an LLM or by a human–has sparked significant concern in many different fields. This article was written to present and compare the current approaches to detecting human- or LLM-authorship in texts.

The article presents several different ways LLM-generated text can be detected. The first, and main, taxonomy followed by the authors is whether the detection can be done aided by the LLM’s own functions (“white-box detection”) or only by evaluating the generated text via a public application programming interface (API) (“black-box detection”).

For black-box detection, the authors suggest training a classifier to discern the origin of a given text. Although this works at first, this task is doomed from its onset to be highly vulnerable to new LLMs generating text that will not follow the same patterns, and thus will probably evade recognition. The authors report that human evaluators find human-authored text to be more emotional and less objective, and use grammar to indicate the tone of the sentiment that should be used when reading the text–a trait that has not been picked up by LLMs yet. Human-authored text also tends to have higher sentence-level coherence, with less term repetition in a given paragraph. The frequency distribution for more and less common words is much more homogeneous in LLM-generated texts than in human-written ones.

White-box detection includes strategies whereby the LLMs will cooperate in identifying themselves in ways that are not obvious to the casual reader. This can include watermarking, be it rule based or neural based; in this case, both processes become a case of steganography, as the involvement of a LLM is explicitly hidden and spread through the full generated text, aiming at having a low detectability and high recoverability even when parts of the text are edited.

The article closes by listing the authors’ concerns about all of the above-mentioned technologies. Detecting an LLM, be it with or without the collaboration of the LLM’s designers, is more of an art than a science, and methods deemed as robust today will not last forever. We also cannot assume that LLMs will continue to be dominated by the same core players; LLM technology has been deeply studied, and good LLM engines are available as free/open-source software, so users needing to do so can readily modify their behavior. This article presents itself as merely a survey of methods available today, while also acknowledging the rapid progress in the field. It is timely and interesting, and easy to follow for the informed reader coming from a different subfield.

Review: Iris Kelly Doesn't Date, by Ashley Herring Blake

Series: Bright Falls #3 Publisher: Berkley Romance Copyright: October 2023 ISBN: 0-593-55058-7 Format: Kindle Pages: 381

Iris Kelly Doesn't Date is a sapphic romance novel (probably a romantic comedy, although I'm bad at romance subgenres). It is the third book in the Bright Falls series. In the romance style, it has a new set of protagonists, but the protagonists of the previous books appear as supporting characters and reading this will spoil the previous books.

Among the friend group we were introduced to in Delilah Green Doesn't Care, Iris was the irrepressible loudmouth. She's bad at secrets, good at saying whatever is on her mind, and has zero desire to either get married or have children. After one of the side plots of Astrid Parker Doesn't Fail, she has sworn off dating entirely.

Iris is also now a romance novelist. Her paper store didn't get enough foot traffic to justify staying open, so she switched her planner business to online only and wrote a romance novel that was good enough to get a two-book deal. Now she needs to write a second book and she has absolutely nothing. Her own avoidance of romantic situations is not helping, but neither is her meddling family who are convinced her choices about marriage and family can be overturned with sufficient pestering. She desperately needs to shake up her life, get out of her creative rut, and do something new. Failing that, she'll settle for meeting someone in a bar and having some fun.

Stevie is a barista and actress living in Portland. Six months ago, she broke up with Adri, her creative partner, girlfriend of six years, and the first person with whom she had a serious relationship. More precisely, Adri broke up with her. They're still friends, truly, even though that friendship is being seriously strained by Adri dating Vanessa, another member of their small and close-knit friend group. Stevie has occasionally-crippling anxiety, not much luck in finding real acting roles in Portland, and a desperate desire to not make waves. Ren, the fourth member of their friend group, thinks Stevie needs a new relationship, or at least a fling. That's how Stevie, with Ren as backup and encouragement, ends up at the same bar with Iris.

The resulting dance and conversation was rather fun for both Stevie and Iris. The attempted one-night stand afterwards was a disaster due to Stevie's anxiety, and neither of them expected to see the other again. Stevie therefore felt safe pretending they'd hit it off to get her friends off her back. When Iris's continued restlessness lands her in an audition for Adri's fundraiser play that she also talked Stevie into performing in, this turns into a full-blown fake dating trope.

These books continue to be impossible to put down. I'm not sure what Blake is doing to make the pacing so perfect, but as with the previous books of the series I found this utterly compulsive reading. I started it in the afternoon, took a break in the evening for a few hours, and then finished it at 2am.

I wasn't sure if a book focused on Iris would work as well, but I need not have worried. Iris Kelly Doesn't Date is both more dramatic and more trope-centered than the earlier books, but Blake handles that in a way that fits Iris's personality and wasn't annoying even to a reader like me, who has an aversion to many types of relationship drama. The secret is Stevie, and specifically having the other protagonist be someone with severe anxiety.

No was never a very easy word for Stevie when it came to Adri, when it came to anyone, really. She could handle the little stuff — do you want a soda, have you seen this movie, do you like onions on your pizza — but the big stuff, the stuff that caused disappointed expressions and down-turned mouths... yeah, she sucked at that part. Her anxiety would flare, and she'd spend the next week convinced her friends hated her, she'd die alone and miserable, and wasn't worth a damn to anyone. Then, when said friend or family member eventually got ahold of her to tell her that, no, of course they didn't hate her, why in the world would she think that, her anxiety would crest once again, convincing her that she was terrible at understanding people and could never trust her own brain to make heads or tails of any social situation.

This is a spot-on description of a particular type of anxiety, but also this is the perfect protagonist to pair with Iris. Throughout the series, Iris has always been the ride-or-die friend, the person who may have no idea how to help but who will show up anyway and at least try to distract you. Stevie's anxiety makes Iris feel protective, which reveals one of the best sides of Iris's personality, and then the protectiveness plays off against Iris's own relationship issues and tendency to avoid taking anything too seriously. It's one of those relationships that starts a bit one-sided and then becomes mutually supporting once Stevie gets her feet under her. That's a relationship pattern I really enjoy reading about.

As with the rest of the series, the friendship dynamics are great. Here we get to see two friend groups at work: Iris's, which we've seen in the previous two volumes and which expanded interestingly in Astrid Parker Doesn't Fail, and Stevie's, which is new. I liked all of these people, even Adri in her own way (although she's the hardest to like). The previous happily-ever-afters do get a bit awkward here, but Blake tries to make that part of the plot and also avoids most of the problem of somewhat-boring romantic bliss by spreading the friendship connections a bit wider.

Stevie's friend group formed at orientation at Reed College, and that let me put my finger on another property of this series: essentially all of the characters are from a very specific social class. They're nearly all arts people (bookstore owner, photographer, interior decorator, actress, writer, director), they've mostly gone to college, and while most of them don't have lots of money, there's always at least one person in each friend group with significant wealth. Jordan, from the previous book, is a bit of an exception since she works in a trade (a carpenter), but she still acts like someone from that same social class. It's a bit like reading Jane Austen novels and realizing that the protagonists are drawn from a very specific and very narrow portion of society.

This is not a complaint, to be clear; I have no objections to reading about a very specific social class. But if one has already read lots of books about this class of people, I could see that diminishing the appeal of this series a bit. There are a lot of assumptions baked into the story that aren't really questioned, such as the ubiquity of therapists. (I don't know how Stevie affords one on a barista salary.) There are also some small things in the terminology (therapy speak, for example) and in the specific type of earnestness with which the books attempt to be diverse on most axes other than social class that I suspect may grate a bit for some readers. If that's you, this is your warning.

There is a third-act breakup here, just like the previous volumes. There is also a defense of the emotional punch of third-act breakups in romance novels in the book itself, put into Iris's internal monologue, so I suspect that's the author's answer to critics like myself who don't like the trope. I was less frustrated by this one because it fit the drama level of the protagonists, but I'll also know to expect a third-act breakup in any Blake novel I read in the future.

But, all that said, the summary once again is that I loved this book and could not put it down. Iris is dramatic and occasionally self-destructive but has a core of earnest empathy that makes her easy to like. She's exactly the sort of extrovert who is soothing to introverts rather than draining because she carries the extrovert load of social situations. Stevie is adorably earnest and thoughtful beneath her anxiety. They two of them are wildly different and yet remarkably good together, and I loved reading their story.

Highly recommended, along with the whole series. Start with Delilah Green Doesn't Care; if you like that, you're in for a treat.

Content note: This book is also rather sex-forward and pretty explicit in the sex scenes, maybe a touch more than Astrid Parker Doesn't Fail. If that is or is not your thing in romance novels, be aware going in.

Rating: 9 out of 10

Welcome to post 45 in the $R^4 series!

We introduced r-ci here in post #32 here nearly four years ago. It has found pretty widespread use and adoption, and we received a few kind words then (in the linked issue) and also more recently (in a follow-up comment) from which we merrily quote:

[…] almost 3 years later on and I have had zero problems with this CI setup. For people who want reliable R software, resources like these are invaluable.

And while we followed up with post #41 about r2u for simple continuous integration, we may not have posted when we based r-ci on r2u (for the obvious Linux usage case). So let’s make time now for a (comparitively smaller) update, and an update usage examples.

We made two changes in the last few days. One is a (obvious in hindsight) simplification. Given that the bootstrap step was always executed, and needed no parameters, we pulled it into a new aggregated setup simply called r-ci that includes it so that it can be omitted as a step in the yaml file. Second, we recently needed Fortran on macOS too, and realized it was not installed by default so we just added that too.

With that a real and used example is now as simple as the screenshot to the left (and hence one ‘paragraph’ shorter). The trained eye will no doubt observe that there is nothing specific to a given repo. And that is basically the key feature: we can simply copy this file around and get fast and easy and reliable CI by taking advantage of the underlying robustness of r2u solving all dependency automagically and reliably. The option to enable macOS is also solid and compelling as the GitHub runners are fast (but more ‘expensive’ in how the count against the limit of minutes—so again a tradeoff to make), as is the option to run coverage if one so desires. Some of my repos do too.

Take a look at the r-ci website which has more examples for the other supported CI servics it can used with, and feel free to ask questions as issue in the repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can now sponsor me at GitHub. Please report excessive re-aggregation in third-party for-profit settings.

Review: Finders, by Melissa Scott

Series: Firstborn, Lastborn #1 Publisher: Candlemark & Gleam Copyright: 2018 ISBN: 1-936460-87-4 Format: Kindle Pages: 409

Finders is a far future science fiction novel with cyberpunk vibes. It is the first of a series, but the second (and, so far, only other) book of the series is a prequel. It stands alone reasonably well (more on that later).

Cassilde Sam is a salvor. That means she specializes in exploring ancient wrecks and ruins left behind by the Ancients and salvaging materials that can be reused. The most important of those are what are called Ancestral elements: BLUE, which can hold programming; GOLD, which which reacts to BLUE instructions; RED, which produces actions or output; and GREEN, the rarest and most valuable, which powers everything else. Cassilde and her partner Dai Winter file claims on newly-discovered or incompletely salvaged Ancestor sites and then extract elemental material and anything else of value in their small salvage ship.

Cassilde is also dying. She has Lightman's, an incurable degenerative disease that can only be treated with ever-increasing quantities of GREEN. It's hard to sleep, hard to get warm, hard to breathe, and eventually she'll run out of money to pay for the GREEN and she'll die.

To push that day off into the future, she and Dai need work. The good news is that the wreckage of a new Ancestor sky palace was discovered in a long orbit and will create enough salvage work for every experienced salvor in the system. The bad news is that they're not qualified to bid on it. They need a scholar with a class-one license to bid on the best sections, and they haven't had a reliable scholar since their former partner and lover Summerland Ashe picked the opposite side in the Troubles and left the Fringe for the Entente, the more densely settled and connected portion of human space.

But, unexpectedly and suspiciously, Ashe may be back and offering to work with them again.

So, first, I love this setting. This is far from the first SF novel that is set in the aftermath of a general collapse of human civilization and revolving around discovering lost mysteries. Most examples of that genre are post-apocalyptic novels limited to Earth or the local solar system, but Kate Elliott's Unconquerable Sun comes immediately to mind. It's also not the first space archaeology series I've read; Kristine Kathyrn Rusch's story series starting with "Diving into the Wreck" also came to mind. But I don't recall the last time I've seen the author sell the setting so effectively.

This is a world with starships and spaceports and clearly advanced technology, but it feels like a post-collapse society that's built on ruins. It's not just that technology runs on half-understood Ancestral elements and states fight over control of debris fields. It's also that the society repurposes Ancestral remnants in ways that both they and the reader know weren't originally intended, and that sometimes are more ingenious or efficient than how the Ancestors probably used them. There's a creative grittiness here that reminds me of good cyberpunk.

It's not just good atmospheric writing, though. Scott makes a world-building decision that is going to sound trivial when I say it, but that has brilliant implications for the rest of the setting. There was not just one collapse; there were two.

The Ancestor civilization, presumed to be the first human civilization, has passed into myth, quite literally when it comes to the stories around its downfall in the aftermath of a war against AIs. After the Ancestors came the Successors, who followed a similar salvage and rebuild approach and got as far as inventing their own warp drive technology that was based on but different than the Ancestor technology. Then they also collapsed, leaving their adapted technology and salvage operations layered over Ancestor sites. Cassilde's civilization is the third human starfaring civilization, and it is very specifically the third, neither the second nor one of dozens.

This has so many small but effective implications that improve this story. A fall happened twice, so it feels like a pattern that makes Cassilde's civilization paranoid, but it happened for two very different reasons, so there is room to argue against it being a pattern. Salvage is harder because of the layering of Ancestor and Successor activity. Successors had their own way of controlling technology that is not accessible to Cassilde and her crew but is also not how the technology was intended to be used, which sends small ripples of interesting complexity through the background. And salvors are competing not only against each other but also against Successor salvage operations for which they have fragmentary records. It's a beautifully effective touch.

Melissa Scott has been publishing science fiction for forty years, and it shows in this book. The protagonists are older characters: established professionals with resource problems but also social connections and an earned reputation, people who are trying to do a job and live their lives, not change the world. The writing is competent, deft, and atmospheric, with the confidence of long practice, but it also has the feel of an earlier era of science fiction. I mentioned the cyberpunk influence, which shows in the grittiness of the descriptions, the marginality of the characters in society, and the background theme of repurposing and reusing technology in unintended ways. This is the sort of book that feels solidly in the center of science fiction, without the genre mixing into either fantasy or romance that has become somewhat more common, and also without the dramatics of space opera (although the reader discovers that the stakes of this novel may be higher than anyone realized).

And yet, so much of this book is about navigating a complicated romantic relationship, and that's where the story structure felt a bit odd. Cassilde, Dai, and Ashe were a polyamorous triad (polyamory also shows up in Scott's excellent Roads of Heaven series), and much of the first third of the book deals with the fracturing of trust with Ashe and their renegotiation of that relationship given his return. This is refreshingly written as the thoughtful interaction of three adults who take issues of trust seriously, but that also means it's much less dramatic than it sounds, and that means this book starts exceptionally slow. Scott is going somewhere, and the slow build became engrossing around the midpoint of the book, but I had to fight to stick with it at the start.

About 80% of the way through this book, I had no idea how Scott was going to wrap things up in the pages remaining and was bracing myself for some sort of series cliffhanger. This is not what happens; the plot is not fully resolved in every detail, but it reaches a conclusion of sorts that does not mandate a sequel. I did think the end was a little bit unsatisfying, though, and I want another book that explores the implications of the ending. I think it would have to be a much different book, and the tonal shift might be stark.

I've had this book on my to-read list for a while and kept putting it off because I wasn't sure I was in the mood for something precarious and gritty. This turned out to be an accurate worry: this is literally a book about salvaging the pieces of something full of wonders inextricably connected to dangers. You have to be in a cyberpunk sort of mood. But I've never read a bad Melissa Scott book, and this is no exception. The simplicity and ALL-CAPSNESS of the Ancestral elements grated a bit, but apart from that, the world-building is exceptional and well worth the trip. Recommended, although be warned that, if you're like me, it may not grab you from the first page.

Followed by Fallen, but that book is a prequel that does not share any protagonists.

Content notes: disability and degenerative illness in a universe where magical cures are possible, so be warned if that specific thematic combination is not what you're looking for.

Rating: 7 out of 10

The Aim

I just bought a Hisense 65U80G 65″ Inch 8K ULED Android TV (2021 model) for $1,568 including delivery. I got that deal by googling refurbished 8K TVs and finding the cheapest one I could buy. Amazon and eBay didn’t have any good prices on second hand 8K TVs and new ones start at $3,000 on special. I didn’t assess how Hisense compares to other TVs, as far as I could determine there was only one model of 8K TV on sale in Australia in the price range I was prepared to pay. So I won’t review how this TV compares to other models but how refurbished TVs compare to other display options.

I bought this because the highest resolution monitor in my price range is 5120*2160 [1]. While I could get a 5128*2880 monitor for around $1,500 paying 3* the money for 33% more pixels is bad value for money. Getting 4* the pixels for under 3* the price is good value even when it’s a TV with the lower display quality that involves.

Before buying this TV I read this blog post by Daniel Lawrence about using an 8K TV as a primary monitor [2]. While he has an interesting setup with a 65″ TV on a large desk it’s not what I plan to do at this time.

My Plans for Use

I don’t plan to make it a main monitor. While 5120*2160 isn’t as good as I like on my desk it’s bearable and the quality of the display is high. High resolution isn’t needed for all tasks, for example I’m writing this blog post on my laptop while watching a movie on the 8K TV.

One thing I’d like to do with the 8K TV when I get it working as a monitor is to share the screen for team programming projects. I don’t have any specific plans other than team coding projects at the moment. But it will be interesting to experiment with it when I get it working.

Technical Issues with High Resolution Monitors Hardware Needed

A lot of the graphic hardware out there don’t support resolutions higher than 5120*2880. It seems that most laptops don’t support resolutions higher than that and higher resolutions than 4K are difficult. Only quite recent and high end video cards will do 8K. Apparently the RTX 2080 is one of the oldest ones that does and that’s $400 on ebay. Strangely the GPU chipset spec pages don’t list the maximum resolution and there’s the additional complication that the other chips might not support the resolutions that the GPU itself can support.

As an aside I don’t use NVidia cards for regular workstations due to reliability problems. But they are good for ML work and for special purpose systems.

Interface Versions

To do 8K video it seems that you need HDMI 2.1 (or maybe 2.0 with 4:2:0 chroma subsampling) or DisplayPort 1.3 for 30Hz with 24bit color and 2.0 for higher refresh rates. But using a particular version of the interface doesn’t require supporting all the resolutions that it might support. This TV has HDMI 2.1 inputs, I’ve bought an adaptor cable that does DisplayPort 1.4 to HDMI 2.1 at 8K resolution. So I need a video card that does DisplayPort 1.4 or HDMI 2.1 output. That doesn’t mean that the card will work, but it could work.

It’s a pity that no-one has made a USB-C video controller that has a basic frame-buffer supporting 8K and the minimal GPU capabilities. The consensus of opinion is that no games will run well at 8K at this time so anyone using 8K resolution doesn’t need GPU power unless it’s for ML stuff.

I’m thinking of making a system that can be used as a ML server and X/Wayland server so a GPU with a decent amount of RAM and compute power would be good. I’m not particularly interested in spending $1,500+ to get a GPU that can drive a $1,568 TV. I’m looking into getting a RTX A2000 with 12G of RAM which should be adequate for ML experiments and can handle 8K@60Hz output.

I’ve ordered a DisplayPort to HDMI converter cable so if I get a DisplayPort card it will work.

Software Support

When I first got started with 4K monitors I had significant problems in adjusting the UI to be usable. The support for scaling software is much better now than it was then and 8K 65″ has a lower DPI than 4K 32″. So I hope this won’t be an issue.

Progress So Far

My first Hisense 8K TV stopped working properly. It would change to a mostly white screen after being used for some time. The screen would change in ways that correlate to changes in what should appear, but not in a way that was usable. It was just a different pattern of white blobs when I changed to a menu view not anything that allowed using it. I presume that this was the problem that drove a need for refurbishment as when I first got the TV it was still signed in to Google accounts for YouTube and to NetFlix.

Best Buy Electrical was good about providing a quick replacement, they took away the old TV and delivered a new one on the same visit and it’s now working well.

I’ve obtained a NVidia card that can allegedly do 8K output and a combination of cables that might be able to carry an 8K signal. Now I just need to get the NVidia drivers to not cause a kernel panic to get things to work.

• [1] https://etbe.coker.com.au/2024/07/23/more-5120×2160-monitor/
• [2] https://daniel.lawrence.lu/blog/y2023m12d15/

Related posts:

1. DisplayPort and 4K The Problem Video playback looks better with a higher scan...
2. More About Kogan 5120*2160 Monitor On the 18th of May I blogged about my new...
3. Dell 32″ 4K Monitor and DisplayPort Switch After determining that the Philips 43″ monitor was too large...

I recently got a OnePlus 6 for the purpose of running Debian, here’s the Debian wiki page about it [1]. It runs Debian nicely and the basic functions all work, but the problem I’m having now is that AldiMobile (Telstra) and KoganMobile (Vodafone) don’t enable VoLTE for that and all the Australian telcos have turned off 3G. The OnePlus 6 does VoLTE with Chinese SIMs so the phone itself can do it.

The OnePlus 6 was never sold in Australia by the telcos, so they are all gray-market imports which aren’t designed by OnePlus to work in Australia. Until recently that wasn’t a problem, but now that the 3G network has been turned off we need VoLTE and OnePlus didn’t include that in the OS. Reddit has documentation on how to fix this but it has to be done on Android [2]. So I had to go back from Mobian to Android to get VoLTE (and VoWifi) working and then install Mobian again.

For people with similar issues Telstra has a page for checking which phones are supported [3], it’s the only way to determine if it’s the phone or the network that makes VoLTE not work – Android isn’t informative about such things. Telstra lists the OP6 as a suitable phone.

Now after doing this I still can’t get the OP6 working for phone calls on Phosh or PlasmaMobile and I’m not sure why. I’m going to give the PinePhone Pro another go and see if it now works better. In the past I had problems with the PinePhonePro battery discharging too fast, charging too slowly, and having poor call quality [4]. The battery discharge issue should be at least alleviated by some of the changes in the Plasma 6 code that’s now in Debian/Unstable.

I’ve also been lent a PinePhone (non-pro) and been told that it will have better battery life in many situations. I’ll do some tests of that. The PinePhonePro isn’t capable of doing the convergence things I was hoping to do so the greater RAM and CPU power that it has aren’t as relevant as they otherwise would be.

I have a vision for how phones should work. I am not discouraged by the Librem 5, PinePhonePro, Note 9, and OnePlus 6 failing in various ways to do what I hoped for. I will eventually find a phone that I can get working well enough.

• [1] https://wiki.debian.org/InstallingDebianOn/OnePlus/OnePlus6
• [2] https://tinyurl.com/29lo6srp
• [3] https://tinyurl.com/23ta7pjf
• [4] https://etbe.coker.com.au/2023/10/11/pinephone-status/

Related posts:

1. PineTime Status Since my last blog post about the PineTime [1] I...
2. Samsung Galaxy Note 9 Review After the VoLTE saga [1] and the problems with battery...
3. PinePhonePro First Impression Hardware I received my PinePhone Pro [1] on Thursday, it...

This is the story of an investigation conducted by Jochen Sprickerhof, Helmut Grohne, and myself. It was true teamwork, and we would have not reached the bottom of the issue working individually. We think you will find it as interesting and fun as we did, so here is a brief writeup. A few of the steps mentioned here took several days, others just a few minutes. What is described as a natural progression of events did not always look very obvious at the moment at all.

Let us go through the Six Stages of Debugging together.

Stage 1: That cannot happen

Official Debian GCC builds start failing on multiple architectures in late November.

The build error happens on the build servers when running the testuite, but we know this cannot happen. GCC builds are not meant to fail in case of testsuite failures! Return codes are not making the build fail, make is begin called with -k, it just cannot happen.

A lot of the GCC tests are always failing in fact, and an extensive log of the results is posted to the debian-gcc mailing list, but the packages always build fine regardless.

On the build daemons, build failures take several hours.

Stage 2: That does not happen on my machine

Building on my machine running Bookworm is just fine. The Build Daemons run Bookworm and use a Sid chroot for the build environment, just like I am. Same kernel.

Debian packages are built by a network of autobuilding machines using a program called sbuild. In my last blog post I mentioned the transition from the schroot backend to a new one based on unshare.

The only obvious difference between my setup and the Debian buildds is that I am using sbuild 0.85.0 from bookworm, and the buildds have 0.86.3~bpo12+1 from bookworm-backports. Trying again with 0.86.3~bpo12+1, the build fails on my system too. The build daemons were updated to the bookworm-backports version of sbuild at some point in late November. Ha.

Stage 3: That should not happen

There are quite a few sbuild versions in between 0.85.0 and 0.86.3~bpo12+1, but looking at recent sbuild bugs shows that sbuild 0.86.0 was breaking "quite a number of packages". Indeed, with 0.86.0 the build still fails. Trying the version immediately before, 0.85.11, the build finishes correctly. This took more time than it sounds, one run including the tests takes several hours. We need a way to shorten this somehow.

The Debian packaging of GCC allows to specify which languages you may want to skip, and by default it builds Ada, Go, C, C++, D, Fortran, Objective C, Objective C++, M2, and Rust. When running the tests sequentially, the build logs stop roughly around the tests of a runtime library for D, libphobos. So can we still reproduce the failure by skipping everything except for D? With DEB_BUILD_OPTIONS=nolang=ada,go,c,c++,fortran,objc,obj-c++,m2,rust the build still fails, and it fails faster than before. Several minutes, not hours. This is progress, and time to file a bug. The report contains massive spoilers, so no link. :-)

Stage 4: Why does that happen?

Something is causing the build to end prematurely. It’s not the OOM killer, and the kernel does not have anything useful to say in the logs. Can it be that the D language tests are sending signals to some process, and that is what’s killing make ? We start tracing signals sent with bpftrace by writing the following script, signals.bt:

tracepoint:signal:signal_generate { printf("%s PID %d (%s) sent signal %d to PID %d\n", comm, pid, args->sig, args->pid); }

And executing it with sudo bpftrace signals.bt.

The build takes its sweet time, and it fails. Looking at the trace output there’s a suspicious process.exe terminating stuff.

process.exe (PID: 2868133) sent signal 15 to PID 711826

That looks interesting, but we have no clue what PID 711826 may be. Let’s change the script a bit, and trace signals received as well.

tracepoint:signal:signal_generate { printf("PID %d (%s) sent signal %d to %d\n", pid, comm, args->sig, args->pid); } tracepoint:signal:signal_deliver { printf("PID %d (%s) received signal %d\n", pid, comm, args->sig); }

The working version of sbuild was using dumb-init, whereas the new one features a little init in perl. We patch the current version of sbuild by making it use dumb-init instead, and trace two builds: one with the perl init, one with dumb-init.

Here are the signals observed when building with dumb-init.

PID 3590011 (process.exe) sent signal 2 to 3590014 PID 3590014 (sleep) received signal 9 PID 3590011 (process.exe) sent signal 15 to 3590063 PID 3590063 (std.process tem) received signal 9 PID 3590011 (process.exe) sent signal 9 to 3590065 PID 3590065 (std.process tem) received signal 9

And this is what happens with the new init in perl:

PID 3589274 (process.exe) sent signal 2 to 3589291 PID 3589291 (sleep) received signal 9 PID 3589274 (process.exe) sent signal 15 to 3589338 PID 3589338 (std.process tem) received signal 9 PID 3589274 (process.exe) sent signal 9 to 3589340 PID 3589340 (std.process tem) received signal 9 PID 3589274 (process.exe) sent signal 15 to 3589341 PID 3589274 (process.exe) sent signal 15 to 3589323 PID 3589274 (process.exe) sent signal 15 to 3589320 PID 3589274 (process.exe) sent signal 15 to 3589274 PID 3589274 (process.exe) received signal 9 PID 3589341 (sleep) received signal 9 PID 3589273 (sbuild-usernsex) sent signal 9 to 3589320 PID 3589273 (sbuild-usernsex) sent signal 9 to 3589323

There are a few additional SIGTERM being sent when using the perl init, that’s helpful. At this point we are fairly convinced that process.exe is worth additional inspection. The source code of process.d shows something interesting:

1221 @system unittest 1222 { [...] 1247 auto pid = spawnProcess(["sleep", "10000"], [...] 1260 // kill the spawned process with SIGINT 1261 // and send its return code 1262 spawn((shared Pid pid) { 1263 auto p = cast() pid; 1264 kill(p, SIGINT);

So yes, there’s our sleep and the SIGINT (signal 2) right in the unit tests of process.d, just like we have observed in the bpftrace output.

Can we study the behavior of process.exe in isolation, separatedly from the build? Indeed we can. Let’s take the executable from a failed build, and try running it under /usr/libexec/sbuild-usernsexec.

First, we prepare a chroot inside a suitable user namespace:

unshare --map-auto --setuid 0 --setgid 0 mkdir /tmp/rootfs cd /tmp/rootfs cat /home/ema/.cache/sbuild/unstable-arm64.tar | unshare --map-auto --setuid 0 --setgid 0 tar xf - unshare --map-auto --setuid 0 --setgid 0 mkdir /tmp/rootfs/whatever unshare --map-auto --setuid 0 --setgid 0 cp process.exe /tmp/rootfs/

Now we can run process.exe on its own using the perl init, and trace signals at will:

/usr/libexec/sbuild-usernsexec --pivotroot --nonet u:0:100000:65536 g:0:100000:65536 /tmp/rootfs ema /whatever -- /process.exe

We can compare the behavior of the perl init vis-a-vis the one using dumb-init in milliseconds instead of minutes.

Stage 5: Oh, I see.

Why does process.exe send more SIGTERMs when using the perl init is now the big question. We have a simple reproducer, so this is where using strace becomes possible.

sudo strace --user ema --follow-forks -o sbuild-dumb-init.strace ./sbuild-usernsexec-dumb-init --pivotroot --nonet u:0:100000:65536 g:0:100000:65536 /tmp/dumbroot ema /whatever -- /process.exe

We start comparing the strace output of dumb-init with that of perl-init, looking in particular for different calls to kill.

Here is what process.exe does under dumb-init:

3593883 kill(-2, SIGTERM) = -1 ESRCH (No such process)

No such process. Under perl-init instead:

3593777 kill(-2, SIGTERM <unfinished ...>

The process is there under perl-init!

That is a kill with negative pid. From the kill(2) man page:

If pid is less than -1, then sig is sent to every process in the process group whose ID is -pid.

It would have been very useful to see this kill with negative pid in the output of bpftrace, why didn’t we? The tracepoint used, tracepoint:signal:signal_generate, shows when signals are actually being sent, and not the syscall being called. To confirm, one can trace tracepoint:syscalls:sys_enter_kill and see the negative PIDs, for example:

PID 312719 (bash) sent signal 2 to -312728

The obvious question at this point is: why is there no process group 2 when using dumb-init?

Stage 6: How did that ever work?

We know that process.exe sends a SIGTERM to every process in the process group with ID 2. To find out what this process group may be, we spawn a shell with dumb-init and observe under /proc PIDs 1, 16, and 17. With perl-init we have 1, 2, and 17. When running dumb-init, there are a few forks before launching the program, explaining the difference. Looking at /proc/2/cmdline we see that it’s bash, ie. the program we are running under perl-init. When building a package, that is dpkg-buildpackage itself.

The test is accidentally killing its own process group.

Now where does this -2 come from in the test?

2363 // Special values for _processID. 2364 enum invalid = -1, terminated = -2;

Oh. -2 is used as a special value for PID, meaning "terminated". And there’s a call to kill() later on:

2694 do { s = tryWait(pid); } while (!s.terminated); [...] 2697 assertThrown!ProcessException(kill(pid));

What sets pid to terminated you ask?

Here is tryWait:

2568 auto tryWait(Pid pid) @safe 2569 { 2570 import std.typecons : Tuple; 2571 assert(pid !is null, "Called tryWait on a null Pid."); 2572 auto code = pid.performWait(false);

And performWait:

2306 _processID = terminated;

The solution, dear reader, is not to kill.

Like each month, have a look at the work funded by Freexian’s Debian LTS offering.

Debian LTS contributors

In November, 20 contributors have been paid to work on Debian LTS, their reports are available:

• Abhijith PA did 14.0h (out of 6.0h assigned and 8.0h from previous period).
• Adrian Bunk did 53.0h (out of 15.0h assigned and 85.0h from previous period), thus carrying over 47.0h to the next month.
• Andrej Shadura did 7.0h (out of 7.0h assigned).
• Arturo Borrero Gonzalez did 1.0h (out of 10.0h assigned), thus carrying over 9.0h to the next month.
• Bastien Roucariès did 20.0h (out of 20.0h assigned).
• Ben Hutchings did 0.0h (out of 24.0h assigned), thus carrying over 24.0h to the next month.
• Chris Lamb did 18.0h (out of 18.0h assigned).
• Daniel Leidert did 17.0h (out of 26.0h assigned), thus carrying over 9.0h to the next month.
• Emilio Pozuelo Monfort did 40.5h (out of 60.0h assigned), thus carrying over 19.5h to the next month.
• Guilhem Moulin did 7.25h (out of 7.5h assigned and 12.5h from previous period), thus carrying over 12.75h to the next month.
• Jochen Sprickerhof did 3.5h (out of 10.0h assigned), thus carrying over 6.5h to the next month.
• Lee Garrett did 14.75h (out of 15.25h assigned and 44.75h from previous period), thus carrying over 45.25h to the next month.
• Lucas Kanashiro did 10.0h (out of 54.0h assigned and 10.0h from previous period), thus carrying over 54.0h to the next month.
• Markus Koschany did 20.0h (out of 40.0h assigned), thus carrying over 20.0h to the next month.
• Roberto C. Sánchez did 6.75h (out of 9.75h assigned and 14.25h from previous period), thus carrying over 17.25h to the next month.
• Santiago Ruano Rincón did 24.75h (out of 23.5h assigned and 1.5h from previous period), thus carrying over 0.25h to the next month.
• Sean Whitton did 2.0h (out of 6.0h assigned), thus carrying over 4.0h to the next month.
• Sylvain Beucler did 21.5h (out of 9.5h assigned and 50.5h from previous period), thus carrying over 38.5h to the next month.
• Thorsten Alteholz did 11.0h (out of 11.0h assigned).
• Tobias Frost did 12.0h (out of 10.5h assigned and 1.5h from previous period).

Evolution of the situation

In November, we have released 38 DLAs.

The LTS coordinators, Roberto and Santiago, delivered a talk at the Mini-DebConf event in Toulouse, France. The title of the talk was “How LTS goes beyond LTS”. The talk covered work done by the LTS Team during the past year. This included contributions related to individual packages in Debian (such as tomcat, jetty, radius, samba, apache2, ruby, and many others); improvements to tooling and documentation useful to the Debian project as a whole; and contributions to upstream work (apache2, freeimage, node-dompurify, samba, and more). Additionally, several contributors external to the LTS Team were highlighted for their contributions to LTS. Readers are encouraged to watch the video of the presentation for a more detailed review of various ways in which the LTS team has contributed more broadly to the Debian project and to the free software community during the past year.

We wish to specifically thank Salvatore (of the Debian Security Team) for swiftly handling during November the updates of needrestart and libmodule-scandeps-perl, both of which involved arbitrary code execution vulnerabilities. We are happy to see increased involvement in LTS work by contributors from outside the formal LTS Team.

The work of the LTS Team in November was otherwise unremarkable, encompassing the customary triage, development, testing, and release of numerous DLAs, along with some associated contributions to related packages in stable and unstable.

Thanks to our sponsors

Sponsors that joined recently are in bold.

• Platinum sponsors:
  • Toshiba Corporation (for 110 months)
  • Civil Infrastructure Platform (CIP) (for 78 months)
  • VyOS Inc (for 42 months)
• Gold sponsors:
  • Roche Diagnostics International AG (for 120 months)
  • Akamai - Linode (for 114 months)
  • Babiel GmbH (for 104 months)
  • Plat’Home (for 103 months)
  • CINECA (for 78 months)
  • University of Oxford (for 60 months)
  • Deveryware (for 47 months)
  • EDF SA (for 32 months)
  • Dataport AöR (for 7 months)
  • CERN (for 5 months)
• Silver sponsors:
  • Domeneshop AS (for 125 months)
  • Nantes Métropole (for 119 months)
  • Univention GmbH (for 111 months)
  • Université Jean Monnet de St Etienne (for 111 months)
  • Ribbon Communications, Inc. (for 105 months)
  • Exonet B.V. (for 95 months)
  • Leibniz Rechenzentrum (for 89 months)
  • Ministère de l’Europe et des Affaires Étrangères (for 73 months)
  • Cloudways by DigitalOcean (for 62 months)
  • Dinahosting SL (for 60 months)
  • Bauer Xcel Media Deutschland KG (for 54 months)
  • Platform.sh SAS (for 54 months)
  • Moxa Inc. (for 48 months)
  • sipgate GmbH (for 46 months)
  • OVH US LLC (for 44 months)
  • Tilburg University (for 44 months)
  • GSI Helmholtzzentrum für Schwerionenforschung GmbH (for 35 months)
  • Soliton Systems K.K. (for 32 months)
  • THINline s.r.o. (for 8 months)
  • Copenhagen Airports A/S
• Bronze sponsors:
  • Evolix (for 125 months)
  • Seznam.cz, a.s. (for 125 months)
  • Intevation GmbH (for 122 months)
  • Linuxhotel GmbH (for 122 months)
  • Daevel SARL (for 121 months)
  • Bitfolk LTD (for 120 months)
  • Megaspace Internet Services GmbH (for 120 months)
  • Greenbone AG (for 119 months)
  • NUMLOG (for 119 months)
  • WinGo AG (for 118 months)
  • Entr’ouvert (for 110 months)
  • Adfinis AG (for 107 months)
  • Laboratoire LEGI - UMR 5519 / CNRS (for 102 months)
  • Tesorion (for 102 months)
  • GNI MEDIA (for 101 months)
  • Bearstech (for 93 months)
  • LiHAS (for 93 months)
  • Catalyst IT Ltd (for 88 months)
  • Supagro (for 83 months)
  • Demarcq SAS (for 82 months)
  • Université Grenoble Alpes (for 68 months)
  • TouchWeb SAS (for 60 months)
  • SPiN AG (for 57 months)
  • CoreFiling (for 53 months)
  • Institut des sciences cognitives Marc Jeannerod (for 48 months)
  • Observatoire des Sciences de l’Univers de Grenoble (for 44 months)
  • Tem Innovations GmbH (for 39 months)
  • WordFinder.pro (for 38 months)
  • CNRS DT INSU Résif (for 37 months)
  • Alter Way (for 30 months)
  • Institut Camille Jordan (for 20 months)
  • SOBIS Software GmbH (for 5 months)

Welcome to the 44th post in the $R^4 series.

A few weeks ago, and following an informal ‘call for talks’ by James Lamb, I had an opportunity to talk about r2u to the Chicago ML and MLops meetup groups. You can find the slides here.

Over the last 2 1/2 years, r2u has become a widely-deployed mechanism in a number of settings, including (but not limited to) software testing via continuous integration, deployment on cloud servers—besides of course to more standard use on local laptops or workstation. 30 million downloads illustrate this. My thesis for the talk was that this extends equally to ML(ops) where no surprises, no hickups automated deployments are key for large-scale model training, evaluation and of course production deployments.

In this context, I introduce r2u while giving credit both to what came before it, the existing alternatives (or ‘competitors’ for mindshare if one prefers that phrasing), and of course what lies underneath it.

The central takeaway, I argue, is that r2u can and does take advantage of a unique situation in that we can ‘join’ the package manager task for the underlying (operating) system and and the application domain, here R and its unique CRAN repository network. Other approaches can, and of course do, provide binaries, but by doing this outside the realm of the system package manager can only arrive at a lesser integration (and I show a common error arising in that case). So where r2u is feasible, it dominates the alternatives (while the alternatives may well provide deployment on more platforms which, even when less integrated, may be of greater importance for some). As always, it all depends.

But the talk, and its slides, motivate and illustrate why we keep calling r2u by its slogan of r2u: Fast. Easy. Reliable. Pick All Three.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can now sponsor me at GitHub. Please report excessive re-aggregation in third-party for-profit settings.

The distinction between hardware and software has historically been relatively easy to understand - hardware is the physical object that software runs on. This is made more complicated by the existence of programmable logic like FPGAs, but by and large things tend to fall into fairly neat categories if we're drawing that distinction.

Conversations usually become more complicated when we introduce firmware, but should they? According to Wikipedia, Firmware is software that provides low-level control of computing device hardware, and basically anything that's generally described as firmware certainly fits into the "software" side of the above hardware/software binary. From a software freedom perspective, this seems like something where the obvious answer to "Should this be free" is "yes", but it's worth thinking about why the answer is yes - the goal of free software isn't freedom for freedom's sake, but because the freedoms embodied in the Free Software Definition (and by proxy the DFSG) are grounded in real world practicalities.

How do these line up for firmware? Firmware can fit into two main classes - it can be something that's responsible for initialisation of the hardware (such as, historically, BIOS, which is involved in initialisation and boot and then largely irrelevant for runtime[1]) or it can be something that makes the hardware work at runtime (wifi card firmware being an obvious example). The role of free software in the latter case feels fairly intuitive, since the interface and functionality the hardware offers to the operating system is frequently largely defined by the firmware running on it. Your wifi chipset is, these days, largely a software defined radio, and what you can do with it is determined by what the firmware it's running allows you to do. Sometimes those restrictions may be required by law, but other times they're simply because the people writing the firmware aren't interested in supporting a feature - they may see no reason to allow raw radio packets to be provided to the OS, for instance. We also shouldn't ignore the fact that sufficiently complicated firmware exposed to untrusted input (as is the case in most wifi scenarios) may contain exploitable vulnerabilities allowing attackers to gain arbitrary code execution on the wifi chipset - and potentially use that as a way to gain control of the host OS (see this writeup for an example). Vendors being in a unique position to update that firmware means users may never receive security updates, leaving them with a choice between discarding hardware that otherwise works perfectly or leaving themselves vulnerable to known security issues.

But even the cases where firmware does nothing other than initialise the hardware cause problems. A lot of hardware has functionality controlled by registers that can be locked during the boot process. Vendor firmware may choose to disable (or, rather, never to enable) functionality that may be beneficial to a user, and then lock out the ability to reconfigure the hardware later. Without any ability to modify that firmware, the user lacks the freedom to choose what functionality their hardware makes available to them. Again, the ability to inspect this firmware and modify it has a distinct benefit to the user.

So, from a practical perspective, I think there's a strong argument that users would benefit from most (if not all) firmware being free software, and I don't think that's an especially controversial argument. So I think this is less of a philosophical discussion, and more of a strategic one - is spending time focused on ensuring firmware is free worthwhile, and if so what's an appropriate way of achieving this?

I think there's two consistent ways to view this. One is to view free firmware as desirable but not necessary. This approach basically argues that code that's running on hardware that isn't the main CPU would benefit from being free, in the same way that code running on a remote network service would benefit from being free, but that this is much less important than ensuring that all the code running in the context of the OS on the primary CPU is free. The maximalist position is not to compromise at all - all software on a system, whether it's running at boot or during runtime, and whether it's running on the primary CPU or any other component on the board, should be free.

Personally, I lean towards the former and think there's a reasonably coherent argument here. I think users would benefit from the ability to modify the code running on hardware that their OS talks to, in the same way that I think users would benefit from the ability to modify the code running on hardware the other side of a network link that their browser talks to. I also think that there's enough that remains to be done in terms of what's running on the host CPU that it's not worth having that fight yet. But I think the latter is absolutely intellectually consistent, and while I don't agree with it from a pragmatic perspective I think things would undeniably be better if we lived in that world.

This feels like a thing you'd expect the Free Software Foundation to have opinions on, and it does! There are two primarily relevant things - the Respects your Freedoms campaign focused on ensuring that certified hardware meets certain requirements (including around firmware), and the Free System Distribution Guidelines, which define a baseline for an OS to be considered free by the FSF (including requirements around firmware).

RYF requires that all software on a piece of hardware be free other than under one specific set of circumstances. If software runs on (a) a secondary processor and (b) within which software installation is not intended after the user obtains the product, then the software does not need to be free. (b) effectively means that the firmware has to be in ROM, since any runtime interface that allows the firmware to be loaded or updated is intended to allow software installation after the user obtains the product.

The Free System Distribution Guidelines require that all non-free firmware be removed from the OS before it can be considered free. The recommended mechanism to achieve this is via linux-libre, a project that produces tooling to remove anything that looks plausibly like a non-free firmware blob from the Linux source code, along with any incitement to the user to load firmware - including even removing suggestions to update CPU microcode in order to mitigate CPU vulnerabilities.

For hardware that requires non-free firmware to be loaded at runtime in order to work, linux-libre doesn't do anything to work around this - the hardware will simply not work. In this respect, linux-libre reduces the amount of non-free firmware running on a system in the same way that removing the hardware would. This presumably encourages users to purchase RYF compliant hardware.

But does that actually improve things? RYF doesn't require that a piece of hardware have no non-free firmware, it simply requires that any non-free firmware be hidden from the user. CPU microcode is an instructive example here. At the time of writing, every laptop listed here has an Intel CPU. Every Intel CPU has microcode in ROM, typically an early revision that is known to have many bugs. The expectation is that this microcode is updated in the field by either the firmware or the OS at boot time - the updated version is loaded into RAM on the CPU, and vanishes if power is cut. The combination of RYF and linux-libre doesn't reduce the amount of non-free code running inside the CPU, it just means that the user (a) is more likely to hit since-fixed bugs (including security ones!), and (b) has less guidance on how to avoid them.

As long as RYF permits hardware that makes use of non-free firmware I think it hurts more than it helps. In many cases users aren't guided away from non-free firmware - instead it's hidden away from them, leaving them less aware that their freedom is constrained. Linux-libre goes further, refusing to even inform the user that the non-free firmware that their hardware depends on can be upgraded to improve their security.

Out of sight shouldn't mean out of mind. If non-free firmware is a threat to user freedom then allowing it to exist in ROM doesn't do anything to solve that problem. And if it isn't a threat to user freedom, then what's the point of requiring linux-libre for a Linux distribution to be considered free by the FSF? We seem to have ended up in the worst case scenario, where nothing is being done to actually replace any of the non-free firmware running on people's systems and where users may even end up with a reduced awareness that the non-free firmware even exists.

[1] Yes yes SMM

comments

Sometimes you want to restrict access to something to a specific set of devices - for instance, you might want your corporate VPN to only be reachable from devices owned by your company. You can't really trust a device that self attests to its identity, for instance by reporting its MAC address or serial number, for a couple of reasons:

• These aren't fixed - MAC addresses are trivially reprogrammable, and serial numbers are typically stored in reprogrammable flash at their most protected
  
• A malicious device could simply lie about them
  

If we want a high degree of confidence that the device we're talking to really is the device it claims to be, we need something that's much harder to spoof. For devices with a TPM this is the TPM itself. Every TPM has an Endorsement Key (EK) that's associated with a certificate that chains back to the TPM manufacturer. By verifying that certificate path and having the TPM prove that it's in posession of the private half of the EK, we know that we're communicating with a genuine TPM[1].

Android has a broadly equivalent thing called ID Attestation. Android devices can generate a signed attestation that they have certain characteristics and identifiers, and this can be chained back to the manufacturer. Obviously providing signed proof of the device identifier is kind of problematic from a privacy perspective, so the short version[2] is that only apps installed using a corporate account rather than a normal user account are able to do this.

But that's still not ideal - the device identifiers involved included the IMEI and serial number of the device, and those could potentially be used to correlate devices across privacy boundaries since they're static[3] identifiers that are the same both inside a corporate work profile and in the normal user profile, and also remains static if you move between different employers and use the same phone[4]. So, since Android 12, ID Attestation includes an "Enterprise Specific ID" or ESID. The ESID is based on a hash of device-specific data plus the enterprise that the corporate work profile is associated with. If a device is enrolled with the same enterprise then this ID will remain static, if it's enrolled with a different enterprise it'll change, and it just doesn't exist outside the work profile at all. The other device identifiers are no longer exposed.

But device ID verification isn't enough to solve the underlying problem here. When we receive a device ID attestation we know that someone at the far end has posession of a device with that ID, but we don't know that that device is where the packets are originating. If our VPN simply has an API that asks for an attestation from a trusted device before routing packets, we could pass that on to said trusted device and then simply forward the attestation to the VPN server[5]. We need some way to prove that the the device trying to authenticate is actually that device.

The answer to this is key provenance attestation. If we can prove that an encryption key was generated on a trusted device, and that the private half of that key is stored in hardware and can't be exported, then using that key to establish a connection proves that we're actually communicating with a trusted device. TPMs are able to do this using the attestation keys generated in the Credential Activation process, giving us proof that a specific keypair was generated on a TPM that we've previously established is trusted.

Android again has an equivalent called Key Attestation. This doesn't quite work the same way as the TPM process - rather than being tied back to the same unique cryptographic identity, Android key attestation chains back through a separate cryptographic certificate chain but contains a statement about the device identity - including the IMEI and serial number. By comparing those to the values in the device ID attestation we know that the key is associated with a trusted device and we can now establish trust in that key.

"But Matthew", those of you who've been paying close attention may be saying, "Didn't Android 12 remove the IMEI and serial number from the device ID attestation?" And, well, congratulations, you were apparently paying more attention than Google. The key attestation no longer contains enough information to tie back to the device ID attestation, making it impossible to prove that a hardware-backed key is associated with a specific device ID attestation and its enterprise enrollment.

I don't think this was any sort of deliberate breakage, and it's probably more an example of shipping the org chart - my understanding is that device ID attestation and key attestation are implemented by different parts of the Android organisation and the impact of the ESID change (something that appears to be a legitimate improvement in privacy!) on key attestation was probably just not realised. But it's still a pain.

[1] Those of you paying attention may realise that what we're doing here is proving the identity of the TPM, not the identity of device it's associated with. Typically the TPM identity won't vary over the lifetime of the device, so having a one-time binding of those two identities (such as when a device is initially being provisioned) is sufficient. There's actually a spec for distributing Platform Certificates that allows device manufacturers to bind these together during manufacturing, but I last worked on those a few years back and don't know what the current state of the art there is

[2] Android has a bewildering array of different profile mechanisms, some of which are apparently deprecated, and I can never remember how any of this works, so you're not getting the long version

[3] Nominally, anyway. Cough.

[4] I wholeheartedly encourage people not to put work accounts on their personal phones, but I am a filthy hypocrite here

[5] Obviously if we have the ability to ask for attestation from a trusted device, we have access to a trusted device. Why not simply use the trusted device? The answer there may be that we've compromised one and want to do as little as possible on it in order to reduce the probability of triggering any sort of endpoint detection agent, or it may be because we want to run on a device with different security properties than those enforced on the trusted device.

comments

A new release 0.2.13 of RcppCCTZ is now on CRAN.

RcppCCTZ uses Rcpp to bring CCTZ to R. CCTZ is a C++ library for translating between absolute and civil times using the rules of a time zone. In fact, it is two libraries. One for dealing with civil time: human-readable dates and times, and one for converting between between absolute and civil times via time zones. And while CCTZ is made by Google(rs), it is not an official Google product. The RcppCCTZ page has a few usage examples and details. This package was the first CRAN package to use CCTZ; by now several others packages (four the last time we counted) include its sources too. Not ideal, but beyond our control.

This version include most routine package maintenance as well as one small contributed code improvement. The changes since the last CRAN release are summarised below.

Changes in version 0.2.13 (2024-12-11)

• No longer set compilation standard as recent R version set a sufficiently high minimum

• Qualify a call to cctz::format (Michael Quinn in #44)

• Routine updates to continuous integration and badges

• Switch to Authors@R in DESCRIPTION

Courtesy of my CRANberries, there is a diffstat report relative to to the previous version. More details are at the RcppCCTZ page; code, issue tickets etc at the GitHub repository. If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Discovering Open Source: How I Got Introduced
Hey there! I’m Divine Attah-Ohiemi, a sophomore studying Computer Science. My journey into the world of open source was anything but grand. It all started with a simple question to my sister: “How do people get jobs without experience?” Her answer? Open source! I dove into this vibrant community, and it felt like discovering a hidden treasure chest filled with knowledge and opportunities.

Choosing Debian: Why This Community?
Why Debian, you ask? Well, I applied to Outreachy twice, and both times, I chose Debian. It’s not just my first operating system; it feels like home. The Debian community is incredibly welcoming, like a big family gathering where everyone supports each other. Whether I was updating my distro or poring over documentation, the care and consideration in this community were palpable. It reminded me of the warmth of homeschooling with relatives. Plus, knowing that Debian's name comes from its creator Ian and his wife Debra adds a personal touch that makes me feel even more honored to contribute to making the website better!

Why I Applied to Outreachy: What Inspired Me
Outreachy is my golden ticket to the open source world! As a 19-year-old, I see this internship as a unique opportunity to gain invaluable experience while contributing to something meaningful. It’s the perfect platform for me to learn, grow, and connect with like-minded individuals who share my passion for technology and community.

I’m excited for this journey and can’t wait to see where it takes me! 🌟

As someone who daily drives rolling release operating systems, having a bootable USB stick with a live install of Debian is essential. It has saved reverting broken packages and making my device bootable again at least a couple of times. However, creating a bootable USB stick usually uses the entire storage of the USB stick, which seems unnecessary given that USB sticks easily have 64GiB or more these days while live ISOs still don’t use more than 8GiB.

In this how-to, I will explain how to create a bootable USB stick, that also holds a FAT32 partition, which allows the USB stick used as usual.

Creating the partition partble

The first step is to create the partition table on the new drive. There are several tools to do this, I recommend the ArchWiki page on this topic for details. For best compatibility, MBR should be used instead of the more modern GPT. For simplicity I just went with the GParted since it has an easy GUI, but feel free to use any other tool. The layout should look like this:

Type │ Partition │ Suggested size ──────┼───────────┼─────────────── EFI │ /dev/sda1 │ 8GiB FAT32 │ /dev/sda2 │ remainder

Notes:

• The disk names are just an example and have to be adjusted for your system.
• Don’t set disk labels, they don’t appear on the new install anyway and some UEFIs might not like it on your boot partition.
• If you used GParted, create the EFI partition as FAT32 and set the esp flag.

Mounting the ISO and USB stick

The next step is to download your ISO and mount it. I personally use a Debian Live ISO for this. To mount the ISO, use the loop mounting option:

mkdir -p /tmp/mnt/iso sudo mount -o loop /path/to/image.iso /tmp/mnt/iso

Similarily, mount your USB stick:

mkdir -p /tmp/mnt/usb sudo mount /dev/sda1 /tmp/mnt/usb Copy the ISO contents

To copy the contents from the ISO to the USB stick, use this command:

sudo cp -r -L /tmp/mnt/iso/. /tmp/mnt/usb

The -L option is required to deference the symbolic links present in the ISO, since FAT32 does not support symbolic links. Note that you might get warning about cyclic symbolic links. Nothing can be done to fix those, except hoping that they won’t cause a problem. For me this never was a problem though.

Finishing touches

Umnount the ISO and USB stick:

sudo umount /tmp/mnt/usb sudo umount /tmp/mnt/iso

Now the USB stick should be bootable and have a usable data partition that can be used on essentially every operating system. I recommend testing that the stick is bootable to make sure it actually works in case of an emergency.