Planet Debian

Right now playing with minikube, to run a three nodes highly available Kubernetes control plane. I am using the docker driver of minikube, so each Kubernetes node component is running inside a docker container, instead of using full blown VMs.

In my experience this works better than Kind, as using Kind you cannot correctly restart a cluster deployed in highly available mode.

This is the topology of the cluster:

$ minikube node list minikube 192.168.49.2 minikube-m02 192.168.49.3 minikube-m03 192.168.49.4

Each kubernetes node is actually a docker container:

$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 977046487e5e gcr.io/k8s-minikube/kicbase:v0.0.45 "/usr/local/bin/entr…" 31 minutes ago Up 6 minutes 127.0.0.1:32812->22/tcp, 127.0.0.1:32811->2376/tcp, 127.0.0.1:32810->5000/tcp, 127.0.0.1:32809->8443/tcp, 127.0.0.1:32808->32443/tcp minikube-m03 8be3549f0c4c gcr.io/k8s-minikube/kicbase:v0.0.45 "/usr/local/bin/entr…" 31 minutes ago Up 6 minutes 127.0.0.1:32807->22/tcp, 127.0.0.1:32806->2376/tcp, 127.0.0.1:32805->5000/tcp, 127.0.0.1:32804->8443/tcp, 127.0.0.1:32803->32443/tcp minikube-m02 4b39f1c47c23 gcr.io/k8s-minikube/kicbase:v0.0.45 "/usr/local/bin/entr…" 31 minutes ago Up 6 minutes 127.0.0.1:32802->22/tcp, 127.0.0.1:32801->2376/tcp, 127.0.0.1:32800->5000/tcp, 127.0.0.1:32799->8443/tcp, 127.0.0.1:32798->32443/tcp minikube

The whole list of pods running in the cluster is as this:

$ minikube kubectl -- get pods --all-namespaces kube-system coredns-6f6b679f8f-85n9l 0/1 Running 1 (44s ago) 3m32s kube-system coredns-6f6b679f8f-pnhxv 0/1 Running 1 (44s ago) 3m32s kube-system etcd-minikube 1/1 Running 1 (44s ago) 3m37s kube-system etcd-minikube-m02 1/1 Running 1 (42s ago) 3m21s ... kube-system kube-proxy-84gm6 0/1 Error 1 (31s ago) 3m4s

There is this container in Error status, let us check the logs

$ minikube kubectl -- logs -n kube-system kube-proxy-84gm6 E1210 11:50:42.117036 1 run.go:72] "command failed" err="failed complete: too many open files"

This can be fixed by increasing the number of inotify watchers:

# cat > /etc/sysctl.d/minikube.conf <<EOF fs.inotify.max_user_watches = 524288 fs.inotify.max_user_instances = 512 EOF # sysctl --system ... * Applying /etc/sysctl.d/minikube.conf ... * Applying /etc/sysctl.conf ... ... fs.inotify.max_user_watches = 524288 fs.inotify.max_user_instances = 512

Restart the cluster

$ minikube stop $ minikube start $ minikube kubectl -- get pods --all-namespaces | grep Error $

I haven't made one of these posts since... last year? Good lord. I've therefore already read and reviewed a lot of these books.

Kemi Ashing-Giwa — The Splinter in the Sky (sff)
Moniquill Blackgoose — To Shape a Dragon's Breath (sff)
Ashley Herring Blake — Delilah Green Doesn't Care (romance)
Ashley Herring Blake — Astrid Parker Doesn't Fail (romance)
Ashley Herring Blake — Iris Kelly Doesn't Date (romance)
Molly J. Bragg — Scatter (sff)
Sarah Rees Breenan — Long Live Evil (sff)
Michelle Browne — And the Stars Will Sing (sff)
Steven Brust — Lyorn (sff)
Miles Cameron — Beyond the Fringe (sff)
Miles Cameron — Deep Black (sff)
Haley Cass — Those Who Wait (romance)
Sylvie Cathrall — A Letter to the Luminous Deep (sff)
Ta-Nehisi Coates — The Message (non-fiction)
Julie E. Czerneda — To Each This World (sff)
Brigid Delaney — Reasons Not to Worry (non-fiction)
Mar Delaney — Moose Madness (sff)
Jerusalem Demsas — On the Housing Crisis (non-fiction)
Michelle Diener — Dark Horse (sff)
Michelle Diener — Dark Deeds (sff)
Michelle Diener — Dark Minds (sff)
Michelle Diener — Dark Matters (sff)
Elaine Gallagher — Unexploded Remnants (sff)
Bethany Jacobs — These Burning Stars (sff)
Bethany Jacobs — On Vicious Worlds (sff)
Micaiah Johnson — Those Beyond the Wall (sff)
T. Kingfisher — Paladin's Faith (sff)
T.J. Klune — Somewhere Beyond the Sea (sff)
Mark Lawrence — The Book That Wouldn't Burn (sff)
Mark Lawrence — The Book That Broke the World (sff)
Mark Lawrence — Overdue (sff)
Mark Lawrence — Returns (sff collection)
Malinda Lo — Last Night at the Telegraph Club (historical)
Jessie Mihalik — Hunt the Stars (sff)
Samantha Mills — The Wings Upon Her Back (sff)
Lyda Morehouse — Welcome to Boy.net (sff)
Cal Newport — Slow Productivity (non-fiction)
Naomi Novik — Buried Deep and Other Stories (sff collection)
Claire O'Dell — The Hound of Justice (sff)
Keanu Reeves & China Miéville — The Book of Elsewhere (sff)
Kit Rocha — Beyond Temptation (sff)
Kit Rocha — Beyond Jealousy (sff)
Kit Rocha — Beyond Solitude (sff)
Kit Rocha — Beyond Addiction (sff)
Kit Rocha — Beyond Possession (sff)
Kit Rocha — Beyond Innocence (sff)
Kit Rocha — Beyond Ruin (sff)
Kit Rocha — Beyond Ecstasy (sff)
Kit Rocha — Beyond Surrender (sff)
Kit Rocha — Consort of Fire (sff)
Geoff Ryman — HIM (sff)
Melissa Scott — Finders (sff)
Rob Wilkins — Terry Pratchett: A Life with Footnotes (non-fiction)
Gabrielle Zevin — Tomorrow, and Tomorrow, and Tomorrow (mainstream)

That's a lot of books, although I think I've already read maybe a third of them? Which is better than I usually do.

A bit of history: Drupal at my workplace (and in Debian)

My main day-to-day responsability in my workplace is, and has been for 20 years, to take care of the network infrastructure for UNAM’s Economics Research Institute. One of the most visible parts of this responsability is to ensure we have a working Web presence, and that it caters for the needs of our academic community.

I joined the Institute in January 2005. Back then, our designer pushed static versions of our webpage, completely built in her computer. This was standard practice at the time, and lasted through some redesigns, but I soon started advocating for the adoption of a Content Management System. After evaluating some alternatives, I recommended adopting Drupal. It took us quite a bit to do the change: even though I clearly recall starting work toward adopting it as early as 2006, according to the Internet Archive, we switched to a Drupal-backed site around June 2010. We started using it somewhere in the version 6’s lifecycle.

As for my Debian work, by late 2012 I started getting involved in the maintenance of the drupal7 package, and by April 2013 I became its primary maintainer. I kept the drupal7 package up to date in Debian until ≈2018; the supported build methods for Drupal 8 are not compatible with Debian (mainly, bundling third-party libraries and updating them without coordination with the rest of the ecosystem), so towards the end of 2016, I announced I would not package Drupal 8 for Debian.

By March 2016, we migrated our main page to Drupal 7. By then, we already had several other sites for our academics’ projects, but my narrative follows our main Web site. I did manage to migrate several Drupal 6 (D6) sites to Drupal 7 (D7); it was quite involved process, never transparent to the user, and we did have the backlash of long downtimes (or partial downtimes, with sites half-available only) with many of our users. For our main site, we took the opportunity to do a complete redesign and deployed a fully new site.

You might note that March 2016 is after the release of D8 (November 2015). I don’t recall many of the specifics for this decision, but if I’m not mistaken, building the new site was a several months long process — not only for the technical work of setting it up, but for the legwork of getting all of the needed information from the different areas that need to be represented in the Institute. Not only that: Drupal sites often include tens of contributed themes and modules; the technological shift the project underwent between its 7 and 8 releases was too deep, and modules took a long time (if at all — many themes and modules were outright dumped) to become available for the new release.

Naturally, the Drupal Foundation wanted to evolve and deprecate the old codebase. But the pain to migrate from D7 to D8 is too big, and many sites have remained under version 7 — Eight years after D8’s release, almost 40% of Drupal installs are for version 7, and a similar proportion runs a currently-supported release (10 or 11). And while the Drupal Foundation made a great job at providing very-long-term support for D7, I understand the burden is becoming too much, so close to a year ago (and after pushing several times the D7, they finally announced support will finish this upcoming January 5.

Drupal 7 must go!

I found the following usage graphs quite interesting: the usage statistics for all Drupal versions follows a very positive slope, peaking around 2014 during the best years of D7, and somewhat stagnating afterwards, staying since 2015 at the 25000–28000 sites mark (I’m very tempted to copy the graphs, but builtwith’s terms of use are very clear in not allowing it). There is a sharp drop in the last year — I attribute it to the people that are leaving D7 for other technologies after its end-of-life announcement. This becomes clearer looking only at D7’s usage statistics: D7 peaks at ≈15000 installs in 2016 stays there for close to 5 years, and has a sharp drop to under 7500 sites in the span of one year.

D8 has a more “regular” rise, peak and fall peaking at ~8500 between 2020 and 2021, and down to close to 2500 for some months already; D9 has a very brief peak of almost 9000 sites in 2023 and is now close to half of it. Currently, the Drupal king appears to be D10, still on a positive slope and with over 9000 sites. Drupal 11 is still just a blip in builtwith’s radar, with… 3 registered sites as of September 2024 :-Þ

After writing this last paragraph, I came across the statistics found in the Drupal webpage; the methodology for acquiring its data is completely different: while builtwith’s methodology is their trade secret, you can read more about how Drupal’s data is gathered (and agree or disagree with it 😉, but at least you have a page detailing 12 years so far of reported data, producing the following graph (which can be shared under the CC BY-SA license 😃):

This graph is disgregated into minor versions, and I don’t want to come up with yet another graph for it 😉 but it supports (most of) the narrative I presented above… although I do miss the recent drop builtwith reported in D7’s numbers!

And what about Backdrop?

During the D8 release cycle, a group of Drupal developers were not happy with the depth of the architectural changes that were being adopted, particularly the transition to the Symfony PHP component framework, and forked the D7 codebase to create the Backdrop CMS, a modern version of Drupal, without dropping the known and tested architecture it had. The Backdrop developers keep working closely together with the Drupal community, and although its usage numbers are way smaller than Drupal’s, seems to be sustainable and lively. Of course, as I presented their numbers in the previous section, you can see Backdrop’s numbers in builtwith… are way, way lower.

I have found it to be a very warm and welcoming community, eager to receive new members. And, thanks to its contributed D2B Migrate module, I found it is quite easy to migrate a live site from Drupal 7 to Backdrop.

Migration by playbook!

So… Well, I’m an academic. And (if it’s not obvious to you after reading so far 😉), one of the things I must do in my job is to write. So I decided to write an article to invite my colleagues to consider Backdrop for their D7 sites in Cuadernos Técnicos Universitarios de la DGTIC, a young journal in our university for showcasing technical academical work. And now that my article got accepted and published, I’m happy to share it with you — of course, if you can read Spanish 😉 But anyway…

Given I have several sites to migrate, and that I’m trying to get my colleagues to follow suite, I decided to automatize the migration by writing an Ansible playbook to do the heavy lifting. Of course, the playbook’s users will probably need to tweak it a bit to their personal needs. I’m also far from an Ansible expert, so I’m sure there is ample room fo improvement in my style.

But it works. Quite well, I must add.

But with this size of database…

I did stumble across a big pebble, though. I am working on the migration of one of my users’ sites, and found that its database is… huge. I checked the mysqldump output, and it got me close to 3GB of data. And given the D2B_migrate is meant to work via a Web interface (my playbook works around it by using a client I wrote with Perl’s WWW::Mechanize), I repeatedly stumbled with PHP’s maximum POST size, maximum upload size, maximum memory size…

I asked for help in Backdrop’s Zulip chat site, and my attention was taken off fixing PHP to something more obvious: Why is the database so large? So I took a quick look at the database (or rather: my first look was at the database server’s filesystem usage). MariaDB stores each table as a separate file on disk, so I looked for the nine largest tables:

# ls -lhS|head total 3.8G -rw-rw---- 1 mysql mysql 2.4G Dec 10 12:09 accesslog.ibd -rw-rw---- 1 mysql mysql 224M Dec 2 16:43 search_index.ibd -rw-rw---- 1 mysql mysql 220M Dec 10 12:09 watchdog.ibd -rw-rw---- 1 mysql mysql 148M Dec 6 14:45 cache_field.ibd -rw-rw---- 1 mysql mysql 92M Dec 9 05:08 aggregator_item.ibd -rw-rw---- 1 mysql mysql 80M Dec 10 12:15 cache_path.ibd -rw-rw---- 1 mysql mysql 72M Dec 2 16:39 search_dataset.ibd -rw-rw---- 1 mysql mysql 68M Dec 2 13:16 field_revision_field_idea_principal_articulo.ibd -rw-rw---- 1 mysql mysql 60M Dec 9 13:19 cache_menu.ibd

A single table, the access log, is over 2.4GB long. The three following tables are, cache tables. I can perfectly live without their data in our new site! But I don’t want to touch the slightest bit of this site until I’m satisfied with the migration process, so I found a way to exclude those tables in a non-destructive way: given D2B_migrate works with a mysqldump output, and given that mysqldump locks each table before starting to modify it and unlocks it after its job is done, I can just do the following:

$ perl -e '$output = 1; while (<>) { $output=0 if /^LOCK TABLES `(accesslog|search_index|watchdog|cache_field|cache_path)`/; $output=1 if /^UNLOCK TABLES/; print if $output}' < /tmp/d7_backup.sql > /tmp/d7_backup.eviscerated.sql; ls -hl /tmp/d7_backup.sql /tmp/d7_backup.eviscerated.sql -rw-rw-r-- 1 gwolf gwolf 216M Dec 10 12:22 /tmp/d7_backup.eviscerated.sql -rw------- 1 gwolf gwolf 2.1G Dec 6 18:14 /tmp/d7_backup.sql

Five seconds later, I’m done! The database is now a tenth of its size, and D2B_migrate is happy to take it. And I’m a big step closer to finishing my reliance on (this bit of) legacy code for my highly-visible sites 😃

Debian LTS

This was my hundred-twenty-fifth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian. During my allocated time I uploaded or worked on:

• [DLA 3968-1] netatalk security update to fix four CVEs related to heap buffer overflow and writing arbitrary files. The patches have been prepared by the maintainer.
• [DLA 3976-1] tgt update to fix one CVE related to not using a propper seed for rand()
• [DLA 3977-1] xfpt update to fix one CVE related to a stack-based buffer overflow
• [DLA 3978-1] editorconfig-core update to fix two CVEs related to buffer overflows.

I also continued to work on a fix for glewlwyd, which is more difficult than expected. Besides I started to work on ffmpeg and haproxy.

Last but not least I did a week of FD this month and attended the monthly LTS/ELTS meeting.

Debian ELTS

This month was the seventy-sixth ELTS month. During my allocated time I uploaded or worked on:

• [ELA-1259-1]editorconfig-core security update for two CVEs in Buster to fix buffer overflows.

I also started to work on a fix for kmail-account-wizzard. Unfortunately preparing a testing environment takes some time and I did not finish testing this month. Besides I started to work on ffmpeg and haproxy.

Last but not least I did a week of FD this month and attended the monthly LTS/ELTS meeting.

Debian Printing

Unfortunately I didn’t found any time to work on this topic.

Debian Matomo

Unfortunately I didn’t found any time to work on this topic.

Debian Astro

This month I uploaded new packages or new upstream or bugfix versions of:

• … planetary-system-stacker

I also sponsored an upload of calceph.

Debian IoT

This month I uploaded new upstream or bugfix versions of:

• … libjwt
• … libmatthew-java

Debian Mobcom

This month I uploaded new packages or new upstream or bugfix versions of:

• … libosmo-cc (package prepared by Nathan)
• … libosmo-abis

misc

This month I uploaded new upstream or bugfix versions of:

• … setserial
• … rplay

I also did some NMU of opensta, kdrill, glosstex, irsim, pagetools, afnix, cpm, to fix some RC bugs.

FTP master

This month I accepted 266 and rejected 16 packages. The overall number of packages that got accepted was 269.

Focus

This month I didn't have any particular focus. I just worked on issues in my info bubble.

Changes

• swh-web: adjust admin contact wording
• zygolophodon: add option to output links
• ArchiveBot: ignore more URL sharing sites, add option to hide filtered jobs
• Debian wiki pages: PortsDocs/New, bugs.debian.org/usertags, Statistics, Teams/Debbugs/ArchitectureTags

Issues

• Feature in trurl
• Warnings in imediff

Review

• Patches: plasma-browser-integration webext

  Administration

• Debian IRC: set secret mode for an unused channel that traps people

Communication

• Respond to queries from Debian users and contributors on IRC

Sponsors

The SWH work was sponsored. All other work was done on a volunteer basis.

Debian Contributions: 2024-11

Contributing to Debian is part of Freexian’s mission. This article covers the latest achievements of Freexian and their collaborators. All of this is made possible by organizations subscribing to our Long Term Support contracts and consulting services.

Transition management, by Emilio Pozuelo Monfort

Emilio has been helping finish the mpi-defaults switch to mpich on 32-bit architectures, and the openmpi transitions. This involves filing bugs for the reverse dependencies, doing NMUs, and requesting removals for outdated (Not Built from Source) binaries on 32-bit architectures where openmpi is no longer available. Those transitions got entangled with a few others, such as the petsc stack, and were blocking many packages from migrating to testing. These transitions were completed in early December.

cPython 3.12.7+ update uploads, by Stefano Rivera

Python 3.12 had failed to build on mips64el, due to an obscure dh_strip failure. The mips64el porters never figured it out, but the missing build on mips64el was blocking migration to Debian testing. After waiting a month, enough changes had accumulated in the upstream 3.12 maintenance git branch that we could apply them in the hope of changing the output enough to avoid breaking dh_strip. This worked.

Of course there were other things to deal with too. A test started failing due to a Debian-specific patch we carry for python3.x-minimal, and it needed to be reworked. And Stefano forgot to strip the trailing + from PY_VERSION, which confuses some python libraries. This always requires another patch when applying git updates from the maintenance branch. Stefano added a build-time check to catch this mistake in the future. Python 3.12.7 migrated.

Python 3.13 Transition, by Stefano Rivera and Colin Watson

During November the Python 3.13-add transition started. This is the first stage of supporting a new version of Python in Debian archive (after preparatory work), adding it as a new supported but non-default version. All packages with compiled Python extensions need to be re-built to add support for the new version.

We have covered the lead-up to this transition in the past. Due to preparation, many of the failures we hit were expected and we had patches waiting in the bug tracker. These could be NMUed to get the transition moving. Others had been known about but hadn’t been worked on, yet.

Some other packages ran into new issues, as we got further into the transition than we’d been able to in preparation. The whole Debian Python team has been helping with this work.

The rebuild stage of the 3.13-add transition is now over, but many packages need work before britney will let python3-defaults migrate to testing.

Limiting build concurrency based on available RAM, by Helmut Grohne

In recent years, the concurrency of CPUs has been increasing as has the demand for RAM by linkers. What has not been increasing as quickly is the RAM supply in typical machines. As a result, we more frequently run into situations where the package builds exhaust memory when building at full concurrency. Helmut initiated a discussion about generalizing an approach to this in Debian packages. Researching existing code that limits concurrency as well as providing possible extensions to debhelper and dpkg to provide concurrency limits based on available system RAM. Thus far there is consensus on the need for a more general solution, but ideas are still being collected for the precise solution.

MiniDebConf Toulouse at Capitole du Libre

The whole Freexian Collaborator team attended MiniDebConf Toulouse, part of the Capitole du Libre event. Several members of the team gave talks:

• Santiago spoke on Linux Live Patching in Debian, presenting an update on the idea since DebConf 24. This includes the initial requirements for the livepatch package format, that would be used to distribute the livepatches.
• Stefano, Colin, Enrico, and Carles spoke on Using Debusine to Automate QA.
• Santiago and Roberto spoke on How LTS Goes Beyond LTS.
• Helmut spoke on Cross Building.
• Carles gave a lightning talk on po-debconf-manager.

Stefano and Anupa worked as part of the video team, streaming and recording the event’s talks.

Miscellaneous contributions

• Stefano looked into packaging the latest upstream python-falcon version in Debian, in support of the Python 3.13 transition. This appeared to break python-hug, which is sadly looking neglected upstream, and the best course of action is probably its removal from Debian.

• Stefano uploaded videos from various 2024 Debian events to PeerTube and YouTube.

• Stefano and Santiago visited the site for DebConf 2025 in Brest, after the MiniDebConf in Toulouse, to meet with the local team and scout out the venue.

  The on-going DebConf 25 organization work of last month also included handling the logo and artwork call for proposals.

• Stefano helped the press team to edit a post for bits.debian.org on OpenStreetMap’s migration to Debian.

• Carles implemented multiple language support on po-debconf-manager and tested it using Portuguese-Brazilian during MiniDebConf Toulouse. The system was also tested and improved by reviewing more than 20 translations to Catalan, creating merge requests for those packages, and providing user support to new users. Additionally, Carles implemented better status transitions, configuration keys management and other small improvements.

• Helmut sent 32 patches for cross build failures. The wireplumber one was an interactive collaboration with Dylan Aïssi.

• Helmut continued to monitor the /usr-move, sent a patch for lib64readline8 and continued several older patch conversations. lintian now reports some aliasing issues in unstable.

• Helmut initiated a discussion on the semantics of *-for-host packages. More feedback is welcome.

• Helmut improved the crossqa.debian.net infrastructure to fail running lintian less often in larger packages.

• Helmut continued maintaining rebootstrap mostly dropping applied patches and continuing discussions of submitted patches.

• Helmut prepared a non-maintainer upload of gzip for several long-standing bugs.

• Colin came up with a plan for resolving the multipart vs. python-multipart name conflict, and began work on converting reverse-dependencies.

• Colin upgraded 42 Python packages to new upstream versions. Some were complex: python-catalogue had some upstream version confusion, pydantic and rpds-py involved several Rust package upgrades as prerequisites, and python-urllib3 involved first packaging python-quart-trio and then vendoring an unpackaged test-dependency.

• Colin contributed Incus support to needrestart upstream.

• Lucas set up a machine to do a rebuild of all ruby reverse dependencies to check what will be broken by adding ruby 3.3 as an alternative interpreter. The tool used for this is mass-rebuild and the initial rebuilds have already started. The ruby interpreter maintainers are planning to experiment with debusine next time.

• Lucas is organizing a Debian Ruby sprint towards the end of January in Paris. The plan of the team is to finish any missing bits of Ruby 3.3 transition at the time, try to push Rails 7 transition and fix RC bugs affecting the ruby ecosystem in Debian.

• Anupa attended a Debian Publicity team meeting in-person during MiniDebCamp Toulouse.

• Anupa moderated and posted in the Debian Administrator group in LinkedIn.

A new version of our pinp package arrived on CRAN today, and is the first release in four years. The pinp package allows for snazzier one or two column Markdown-based pdf vignettes, and is now used by a few packages. A screenshot of the package vignette can be seen below. Additional screenshots are at the pinp page.

This release contains no new features or new user-facing changes but reflects the standard package and repository maintenance over the four-year window since the last release: updating of actions, updating of URLs and addressing small packaging changes spotted by ever-more-vigilant R checking code.

The NEWS entry for this release follows.

Changes in pinp version 0.0.11 (2024-12-08)

• Standard package maintenance for continuous integration, URL updates, and packaging conventions

• Correct two minor nags in the Rd file

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the ping page. For questions or comments use the issue tracker off the GitHub repo. If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Intro

I always find it amazing the opportunities I have thanks to my contributions to the Debian Project. I am happy to receive this recognition through the help I receive with travel to attend events in other countries.

This year, two MiniDebConfs were scheduled for the second half of the year in Europe: the traditional edition in Cambridge in UK and a new edition in Toulouse in France. After weighing the difficulties and advantages that I would have to attend one of them, I decided to choose Toulouse, mainly because it was cheaper and because it was in November, giving me more time to plan the trip. I contacted the current DPL Andreas Tille explaining my desire to attend the event and he kindly approved my request for Debian to pay for the tickets. Thanks again to Andreas!

MiniDebConf Toulouse 2024 was held in November 16th and 17th (Saturday and Sunday) and took place in one of the rooms of a traditional Free Software event in the city named Capitole du Libre. Before MiniDebConf, the team organized a MiniDebCamp in November 14th and 15th at a coworking space.

The whole experience promised to be incredible, and it was! From visiting a city in France for the first time, to attending a local Free Software event, and sharing four days with people from the Debian community from various countries.

Travel and the city

My plan was to leave Belo Horizonte on Monday, pass through São Paulo, and arrive in Toulouse on Tuesday night. I was going to spend the whole of Wednesday walking around the city and then participate in the MiniDebCamp on Thursday.

But the flight that was supposed to leave São Paulo in the early hours of Monday to Tuesday was cancelled due to a problem with the airplane and I had spent all Tuesday waiting. I was rebooked on another flight that left in the evening and arrived in Toulouse on Wednesday afternoon. Even though I was very tired from the trip, I still took advantage of the end of the day to walk around the city. But it was a shame to have lost an entire day of sightseeing.

On Thursday I left early in the morning to walk around a little more before going to the MiniDebCamp venue. I walked around a lot and saw several tourist attractions. The city is really very beautiful, as they say, especially the houses and buildings made of pink bricks. I was impressed by the narrow and winding streets; at one point it seemed like I was walking through a maze. I arrived to a corner and there would be 5 streets crossing in different directions.

The riverbank that runs through the city is very beautiful and people spend their evenings there just hanging out. There was a lot of history around there.

I stayed in an airbnb 25 minutes walking from the coworking space and only 10 minutes from the event venue. It was a very spacious apartment that was much cheaper than a hotel.

MiniDebCamp

I arrived at the coworking space where the MiniDebCamp was being held and met up with several friends. I also met some new people, talked about the translation work we do in Brazil, and other topics.

We already knew that the organization would pay for lunch for everyone during the two days of MiniDebCamp, and at a certain point they told us that we could go to the room (which was downstairs from the coworking space) to have lunch. They set up a table with quiches, breads, charcuterie and LOTS of cheese :-) There were several types of cheese and they were all very good. I just found it a little strange because I’m not used to having cheese for lunch :-)

In the evening, we went as a group to dinner at a restaurant in front of the Capitolium, the city’s main tourist attraction.

On the second day, in the morning, I walked around the city a bit more, then went to the coworking space and had another incredible cheese table for lunch.

Video Team

One of my ideas for going to Toulouse was to be able to help the video team in setting up the equipment for broadcasting and recording the talks. I wanted to follow this work from the beginning and learn some details, something I can’t do before the DebConfs because I always arrive after the people have already set up the infrastructure. And later reproduce this work in the MiniDebConfs in Brazil, such as the one in Maceió that is already scheduled for May 1-4, 2025.

As I had agreed with the people from the video team that I would help set up the equipment, on Friday night we went to the University and stayed in the room working. I asked several questions about what they were doing, about the equipment, and I was able to clear up several doubts. Over the next two days I was handling one of the cameras during the talks. And on Sunday night I helped put everything away.

Thanks to olasd, tumbleweed and ivodd for their guidance and patience.

The event in general

There was also a meeting with some members of the publicity team who were there with the DPL. We went to a cafeteria and talked mainly about areas that could be improved in the team.

The talks at MiniDebConf were very good and the recordings are also available here.

I ended up not watching any of the talks from the general schedule at Capitole du Libre because they were in French. It’s always great to see free software events abroad to learn how they are done there and to bring some of those experiences to our events.

I hope that MiniDebConf in Toulouse will continue to take place every year, or that the French community will hold the next edition in another city and I will be able to join again :-) If everything goes well, in July next year I will return to France to join DebConf25 in Brest.

More photos

Review: Why Buildings Fall Down, by Matthys Levy & Mario Salvadori

Illustrator: Kevin Woest Publisher: W.W. Norton Copyright: 1992 Printing: 1994 ISBN: 0-393-31152-X Format: Trade paperback Pages: 314

Why Buildings Fall Down is a non-fiction survey of the causes of structure collapses, along with some related topics. It is a sequel of sorts to Why Buildings Stand Up by Mario Salvadori, which I have not read. Salvadori was, at the time of writing, Professor Emeritus of Architecture at Columbia University (he died in 1997). Levy is an award-winning architectural engineer, and both authors were principals at the structural engineering firm Weidlinger Associates. There is a revised and updated 2002 edition, but this review is of the original 1992 edition.

This is one of those reviews that comes with a small snapshot of how my brain works. I got fascinated by the analysis of the collapse of Champlain Towers South in Surfside, Florida in 2021, thanks largely to a random YouTube series on the tiny channel of a structural engineer. Somewhere in there (I don't remember where, possibly from that channel, possibly not) I saw a recommendation for this book and grabbed a used copy in 2022 with the intent of reading it while my interest was piqued. The book arrived, I didn't read it right away, I got distracted by other things, and it migrated to my shelves and sat there until I picked it up on an "I haven't read nonfiction in a while" whim.

Two years is a pretty short time frame for a book to sit on my shelf waiting for me to notice it again. The number of books that have been doing that for several decades is, uh, not small.

Why Buildings Fall Down is a non-technical survey of structure failures. These are mostly buildings, but also include dams, bridges, and other structures. It's divided into 18 fairly short chapters, and the discussion of each disaster is brisk and to the point. Most of the structures discussed are relatively recent, but the authors talk about the Meidum Pyramid, the Parthenon (in the chapter on intentional destruction by humans), and the Pavia Civic Tower (in the chapter about building death from old age). If you are someone who has already been down the structural failure rabbit hole, you will find chapters on the expected disasters like the Tacoma Narrows Bridge collapse and the Hyatt Regency walkway collapse, but there are a lot of incidents here, including a short but interesting discussion of the Leaning Tower of Pisa in the chapter on problems caused by soil properties.

What you're going to get, in other words, is a tour of ways in which structures can fail, which is precisely what was promised by the title. This wasn't quite what I was expecting, but now I'm not sure why I was expecting something different. There is no real unifying theme here; sometimes the failure was an oversight, sometimes it was a bad design, sometimes it was a last-minute change, and sometimes it was something unanticipated. There are a lot of factors involved in structure design and any of them can fail. The closest there is to a common pattern is a lack of redundancy and sufficient safety factors, but that lack of redundancy was generally not deliberate and therefore this is not a guide to preventing a collapse. The result is a book that feels a bit like a grab-bag of structural trivia that is individually interesting but only occasionally memorable.

The writing style I suspect will be a matter of taste, but once I got used to it, I rather enjoyed it. In a co-written book, it's hard to separate the voices of the authors, but Salvadori wrote most of the chapter on the law in the first person and he's clearly a character. (That chapter is largely the story of two trials he testified in, which, from his account, involved him verbally fencing with lawyers who attempted to claim his degrees from the University of Rome didn't count as real degrees.) If this translates to his speaking style, I suspect he was a popular lecturer at Columbia.

The explanations of the structural failures are concise and relatively clear, although even with Kevin Woest's diagrams, it's hard to capture the stresses and movement in a written description. (I've found from watching YouTube videos that animations, or even annotations drawn while someone is talking, help a lot.) The framing discussion, well, sometimes that is bombastic in a way that I found amusing:

But we, children of a different era, do not want our lives to be enclosed, to be shielded from the mystery. We are eager to participate in it, to gather with our brothers and sisters in a community of thought that will lift us above the mundane. We need to be together in sorrow and in joy. Thus we rarely build monolithic monuments. Instead, we build domes.

It helps that passages like this are always short and thus don't wear out their welcome. My favorite line in the whole book is a throwaway sentence in a discussion of building failures due to explosions:

With a similar approach, it can be estimated that the chance of an explosion like that at Forty-fifth Street was at most one in thirty million, and probably much less. But this is why life is dangerous and always ends in death.

Going hard, structural engineering book!

It's often appealing to learn about things from their failures because the failures are inherently more dramatic and thus more interesting, but if you were hoping for an introduction to structural engineering, this is probably not the book you want. There is an excellent and surprisingly engaging appendix that covers the basics of structural analysis in 45 pages, but you would probably be better off with Why Buildings Stand Up or another architecture or structural engineering textbook (or maybe a video course). The problem with learning by failure case study is that all the case studies tend to blend together, despite the authors' engaging prose, and nearly every collapse introduces a new structural element with new properties and new failure modes and only the briefest of explanations. This book might make you a slightly more informed consumer of the news, but for most readers I suspect it will be a collection of forgettable trivia told in an occasionally entertaining style.

I think the book I wanted to read was something that went deeper into the process of forensic engineering, not just the outcomes. It's interesting to know what the cause of a failure was, but I'm more interested in how one goes about investigating a failure. What is the process, how do you organize the investigation, and how does the legal system around engineering failures work? There are tidbits and asides here, but this book is primarily focused on the structural analysis and elides most of the work done to arrive at those conclusions.

That said, I was entertained. Why Buildings Fall Down is a bit dated — the opening chapter on airplanes hitting buildings reads much differently now than when it was written in 1992, and I'm sure it was updated in the 2002 edition — but it succeeds in being clear without being soulless or sounding like a textbook. I appreciate an occasional rant about nuclear weapons in a book about architecture. I'm not sure I really recommend this, but I had a good time with it.

Also, I'm now looking for opportunities to say "this is why life is dangerous and always ends in death," so there is that.

Rating: 6 out of 10

Hi

While updating my Debian package, I often have to update a field from debian/control file.

This field is named Standards-Version and it declares which version of Debian policy the package complies to. When updating this field, one must follow the upgrading checklist.

That being said, I maintain a lot of similar package and I often have to update this Standards-Version field.

This field can be updated manually with cme fix dpkg (see Managing Debian packages with cme). But this command may make other changes and does not commit the result.

So I’ve created a new update-standards-version cme script that:

• udpate Standards-Version field
• commit the changed

For instance:

$ cme run update-standards-version Reading package lists... Done Building dependency tree... Done Reading state information... Done Connecting to api.ftp-master.debian.org to check 31 package versions. Please wait... Got info from api.ftp-master.debian.org for 31 packages. Warning in 'source Standards-Version': Current standards version is '4.7.0'. Please read https://www.debian.org/doc/debian-policy/upgrading-checklist.html for the changes that may be needed on your package to upgrade it from standard version '4.6.2' to '4.7.0'. Offending value: '4.6.2' Changes applied to dpkg-control configuration: - source Standards-Version: '4.6.2' -> '4.7.0' [master 552862c1] control: declare compliance with Debian policy 4.7.0 1 file changed, 1 insertion(+), 1 deletion(-)

Here’s the generated commit. Note that the generated log mentions the new policy version:

domi@ylum:~/private/debian-dev/perl-stuff/libconfig-model-perl$ git show commit 552862c1f24479b1c0c8c35a6289557f65e8ff3b (HEAD -> master) Author: Dominique Dumont <dod[at]debian.org> Date: Sat Dec 7 19:06:14 2024 +0100 control: declare compliance with Debian policy 4.7.0 diff --git a/debian/control b/debian/control index cdb41dc0..e888012e 100644 --- a/debian/control +++ b/debian/control @@ -48,7 +48,7 @@ Build-Depends-Indep: dh-sequence-bash-completion, libtext-levenshtein-damerau-perl, libyaml-tiny-perl, po-debconf -Standards-Version: 4.6.2 +Standards-Version: 4.7.0 Vcs-Browser: https://salsa.debian.org/perl-team/modules/packages/libconfig-model-perl Vcs-Git: https://salsa.debian.org/perl-team/modules/packages/libconfig-model-perl.git Homepage: https://github.com/dod38fr/config-model/wiki

Notes:

• this script can run only if there’s not pending change. Please commit or stash these changes before running this script.
• this script requires:
  • cme >= 1.041
  • libconfig-model-perl >= 2.155
  • libconfig-model-dpkg-perl >= 3.006

I hope this will be useful to all my fellow Debian developers to reduce the boring parts of packaging activities.

All the best

Review: Dark Minds, by Michelle Diener

Series: Class 5 #3 Publisher: Eclipse Copyright: 2016 ISBN: 0-6454658-5-2 Format: Kindle Pages: 328

Dark Minds is the third book in the self-published Class 5 science-fiction romance series. It is a sequel to Dark Deeds and should not be read out of order, but as with the other books of the series, it follows new protagonists.

Imogen, like another Earth woman before her, was kidnapped by a Class 5 for experimentation by the Tecran. She was subsequently transferred to a research facility and has been held in cages and secret military bases for over three months. As the story opens, she had been imprisoned in a Tecran transport for a couple of weeks when the transport is taken by the Krik who kill all of the Tecran crew. Next stop: another Class 5.

That same Class 5 is where the Krik are taking Camlar Kalor and his team. Cam is a Grih investigator from the United Council, sent to look into reports of a second Earth woman rescued from a Garmman ship. (The series reader will recognize this as the plot of Dark Deeds.) Now he's trapped with the crews of a bunch of other random ships in the cargo hold of a Class 5 with Krik instead of Tecran roaming the halls, apparently at the behest of the ship's banned AI, and a mysterious third Earth woman already appears to be befriending it.

Imogen is the woman Fiona saw signs of in Dark Deeds. She's had a rougher time than the protagonists of the previous two books in this series, and she's been dropped into a less stable situation. The Class 5 she's brought to at the start of the story is far more suspicious (with quite a lot of cause) and somewhat more hostile than the AIs we've encountered previously. The rest of the story formula is roughly the same, though: hunky Grih officer with a personality completely indistinguishable from the hunky Grih officers in the previous two books, an AI with a sketchy concept of morality that desperately needs an ally, the Grih obsession with singing, the eventual discovery of useful armor and weaponry that Imogen can use to surprise people, and more political maneuvering over the Sentient Beings Agreement.

This entry in the series mostly abandons the Grih shock and horror at how badly the Earth women have been treated. This makes sense, given how dangerous Earth women have proven over the course of this series, and I like that Diener is changing the political dynamics as the story develops. I do sometimes miss that appalled anger of Dark Horse, but Dark Minds focuses more on the politics and corridor fighting of a tense multi-sided stand-off.

I found the action more gripping in Dark Minds than in Dark Deeds, and I liked Imogen more as a character than Fiona. She doesn't have the delightfully calm competence of Rose from Dark Horse, but she's a bit more hardened, a bit more canny, and is better at taking control of situations. I also like that Diener avoids simplistically pairing Earth women off with Class 5s. The series plot is progressing faster than I had expected, and that gives this book a somewhat different shape than the previous ones.

Cam is probably the least interesting of the men in this series so far and appears to exist solely to take up the man-shaped hole in the plot. This is not a great series for gender roles; thankfully, the romance is a small part of the plot and largely ignorable. The story is about the women and the AIs, and all of the women and most of the AIs of the previous books make an appearance. It's clear they're forming an alliance whether the Grih like it or not, and that part of the story was very satisfying.

Up to this book, this series had been all feel-good happy endings. I will risk the small spoiler and warn that this is not true to the same degree here, so you may not want to read this one if you want something entirely fluffy, light, and positive (inasmuch as a series involving off-screen experimentation on humans can be fluffy, light, and positive). That caught me by surprise in a way I didn't entirely like, and I wish Diener had stuck with the entirely positive tone.

Other than that, though, this was fun, light, readable entertainment. It's not going to win any literary awards, it's formulaic, the male protagonist comes from central casting, and the emphasis by paragraph break is still a bit grating in places, but I will probably pick up the next book when I'm in the mood for something light. Dark Minds is an improvement over book two, which bodes well for the rest of the series.

Followed by Dark Matters.

Rating: 7 out of 10

After weeks – dare I say months – of work, it is finally done. lintian.debian.org is back online!

Many, many thanks to everyone who worked hard to make this possible:

• Thanks to Nicolas Peugnet, the author of lintian-ssg, who handed us this custom static site generator on a silver platter. I'm happy I didn't have to code this myself :)
• Thanks to Otto Kekäläinen, maintainer of the lintian-ssg package in Debian, who worked in tandem with Nicolas to iron out problems.
• Thanks to Philipp Kern, who did the work on the DSA side to put the website back online.

All in all, I did very little (mostly coordinating these fine folks) and they should get the credit for this very useful service being back.

The new Firebuild release contains plenty of small fixes and a few notable improvements.

Experimental macOS support

The most frequently asked question from people getting to know Firebuild was if it worked on their Mac and the answer sadly used to be that well, it did, but only in a Linux VM. This was far from what they were looking for. 🙁

Linux and macOS have common UNIX roots, but porting Firebuild to macOS included bigger challenges, like ensuring that dyld(1), macOS’s dynamic loader initializes the preloaded interceptor library early enough to catch all interesting calls, and avoid using anything that uses malloc() or thread local variables which are not yet set up then.

Preloading libraries on Linux is really easy, running LD_PRELOAD=my_lib.so ls just works if the library exports the symbols to be interposed, while macOS employs multiple lines of defense to prevent applications from using unknown libraries. Firebuild’s guide for making DYLD_INSERT_LIBRARIES honored on Macs can be helpful with other projects as well that rely on injecting libraries.

Since GitHub’s Arm64 macOS runners don’t allow intercepting binaries with arm64e ABI yet, Firebuild’s Apple Silicon tests are run at Bitrise, who are proud to be first to provide the latest Xcode stacks and were also quick to make the needed changes to their infrastructure to support Firebuild (thanks! ).

Firebuild on macOS can already accelerate simple projects and rebuild itself with Xcode. Since Xcode introduces a lot of nondeterminism to the build, Firebuild can’t shine in acceleration with Xcode yet, but can provide nice reports to show which part of the build is the most time consuming and how each sub-command is called.

If you would like to try Firebuild on macOS please compile it from the GitHub repository for now. Precompiled binaries will be distributed on the Mac App Store and via CI providers. Contact us to get notified when those channels become available.

Dealing with the ‘Epochalypse’

Glibc’s API provides many functions with time parameters and some of those functions are intercepted by Firebuild. Time parameters used to be passed as 32-bit values on 32-bit systems, preventing them to accurately represent timestamps after year 2038, which is known as the Y2038 problem or the Epochalypse.

To deal with the problem glibc 2.34 started providing new function symbol variants with 64-bit time parameters, e.g clock_gettime64() in addition to clock_gettime(). The new 64-bit variants are used when compiling consumers of the API with _TIME_BITS=64 defined.

Processes intercepted by Firebuild may have been compiled with or without _TIME_BITS=64, thus libfirebuild now provides both variants on affected systems running glibc >= 34 to work safely with binaries using 64-bit and 32-bit time representation.

Many Linux distributions already stopped supporting 32-bit architectures, but Debian and Ubuntu still supports armhf, for example, where the Y2038 problem still applies. Both Debian and Ubuntu performed a transition rebuilding every library (and their reverse dependencies) with -D_FILE_OFFSET_BITS=64 set where the libraries exported symbols that changed when switching to 64-bit time representation (thanks to Steve Langasek for driving this!) . Thanks to the transition most programs are ready for 2038, but interposer libraries are trickier to fix and if you maintain one it might be a good idea to check if it works well both 32-bit and 64-bit libraries. Faketime, for example is not fixed yet, see #1064555.

Select passed through environment variables with regular expressions

Firebuild filters out most of the environment variables set when starting a build to make the build more reproducible and achieve higher cache hit rate. Extra environment variables to pass through can be specified on the command line one by one, but with many similarly named variables this may become hard to maintain. With regular expressions this just became easier:

firebuild -o 'env_vars.pass_through += "MY_VARS_.*"' my_build_command

If you are not interested in acceleration just would like to explore what the build does by generating a report you can simply pass all variables:

firebuild -r -o 'env_vars.pass_through += ".*"' my_build_command Other highlights from the 0.8.3 release

• Fixed and nicer report in Chrome and other WebKit based browsers
• Support GLibc 2.39 by intercepting pidfd_spawn() and pidfd_spawnp()
• Even faster Rust build acceleration

For all the changes please check out the release page on GitHub!

(This post is also published on The Firebuild blog.)

Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research code into production environments. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 1197 other packages on CRAN, downloaded 37.5 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint / vignette) by Conrad and myself has been cited 605 times according to Google Scholar.

Conrad released a minor version 14.2.2 yesterday. This followed a bit of recent work a few of us did in the ensmallen and mlpack repositories following the [14.2.0 release]. Use of (member functions) .min(index) and .max(index) was deprecated in Armadillo in favor of .index_mix() and .index_max(). By now ensmallen and mlpack have been updated at CRAN. To add some spice, CRAN emailed that the (very much unreleased as of now, but coming likely next spring) gcc-15 was unhappy with RcppArmadillo due to some Armadillo code. This likely related to the listed gcc-15 C++ change about “Qualified name lookup failure into the current instantiation”. Anyway, Conrad fixed it within days and that change too is part of this new version (as is a small behaviour normalization between the two indexing methods that matters in case of ties, this was in 14.2.1).

The changes since the last CRAN release are summarised below.

Changes in RcppArmadillo version 14.2.2-1 (2024-12-05)

• Upgraded to Armadillo release 14.2.2 (Smooth Caffeine)

  • Workarounds for regressions in pre-release versions of GCC 15

  • More selective detection of symmetric/hermitian matrices by various functions

Changes in RcppArmadillo version 14.2.1-1 (2024-11-24) (GitHub Only)

• Upgraded to Armadillo release 14.2.1 (Smooth Caffeine)

  • Fix for index_min() and index_max() to ensure that the first index of equal extremum values is found

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the Rcpp R-Forge page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

The diffoscope maintainers are pleased to announce the release of diffoscope version 284. This version includes the following changes:

[ Chris Lamb ] * Simplify tests_quines.py::test_{differences,differences_deb} to use assert_diff and not mangle the expected test output. * Update some tests to support file(1) version 5.46. (Closes: reproducible-builds/diffoscope#395)

You find out more by visiting the project homepage.

Welcome to the November 2024 report from the Reproducible Builds project!

Our monthly reports outline what we’ve been up to over the past month and highlight items of news from elsewhere in the world of software supply-chain security where relevant. As ever, if you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website.

Table of contents:

1. Reproducible Builds mourns the passing of Lunar
2. Introducing reproduce.debian.net
3. New landing page design
4. SBOMs for Python packages
5. Debian updates
6. Reproducible builds by default in Maven 4
7. PyPI now supports digital attestations
8. “Dependency Challenges in OSS Package Registries”
9. Zig programming language demonstrated reproducible
10. Website updates
11. Upstream patches
12. Misc development news
13. Reproducibility testing framework

Reproducible Builds mourns the passing of Lunar

The Reproducible Builds community sadly announced it has lost its founding member, Lunar. Jérémy Bobbio aka ‘Lunar’ passed away on Friday November 8th in palliative care in Rennes, France.

Lunar was instrumental in starting the Reproducible Builds project in 2013 as a loose initiative within the Debian project. He was the author of our earliest status reports and many of our key tools in use today are based on his design. Lunar’s creativity, insight and kindness were often noted.

You can view our full tribute elsewhere on our website. He will be greatly missed.

Introducing reproduce.debian.net

In happier news, this month saw the introduction of reproduce.debian.net. Announced at the recent Debian MiniDebConf in Toulouse, reproduce.debian.net is an instance of rebuilderd operated by the Reproducible Builds project.

rebuilderd is our server designed monitor the official package repositories of Linux distributions and attempts to reproduce the observed results there.

In November, reproduce.debian.net began rebuilding Debian unstable on the amd64 architecture, but throughout the MiniDebConf, it had attempted to rebuild 66% of the official archive. From this, it could be determined that it is currently possible to bit-for-bit reproduce and corroborate approximately 78% of the actual binaries distributed by Debian — that is, using the .buildinfo files hosted by Debian itself.

reproduce.debian.net also contains instructions how to setup one’s own rebuilderd instance, and we very much invite everyone with a machine to spare to setup their own version and to share the results. Whilst rebuilderd is still in development, it has been used to reproduce Arch Linux since 2019. We are especially looking for installations targeting Debian architectures other than i386 and amd64.

New landing page design

As part of a very productive partnership with the Sovereign Tech Fund and Neighbourhoodie, we are pleased to unveil our new homepage/landing page.

We are very happy with our collaboration with both STF and Neighbourhoodie (including many changes not directly related to the website), and look forward to working with them in the future.

SBOMs for Python packages

The Python Software Foundation has announced a new “cross-functional project for SBOMs and Python packages”. Seth Michael Larson writes that the project is “specifically looking to solve these issues”:

• Enable Python users that require SBOM documents (likely due to regulations like CRA or SSDF) to self-serve using existing SBOM generation tools.
• Solve the “phantom dependency” problem, where non-Python software is bundled in Python packages but not recorded in any metadata. This makes the job of software composition analysis (SCA) tools difficult or impossible.
• Make the adoption work by relevant projects such as build backends, auditwheel-esque tools, as minimal as possible. Empower users who are interested in having better SBOM data for the Python projects they are using to be able to contribute engineering time towards that goal.

A GitHub repository for the initiative is available, and there are a number of queries, comments and remarks on Seth’s Discourse forum post.

Debian updates

There was significant development within Debian this month. Firstly, at the recent MiniDebConf in Toulouse, France, Holger Levsen gave a Debian-specific talk on rebuilding packages distributed from ftp.debian.org — that is to say, how to reproduce the results from the official Debian build servers:

Holger described the talk as follows:

For more than ten years, the Reproducible Builds project has worked towards reproducible builds of many projects, and for ten years now we have build Debian packages twice—with maximal variations applied—to see if they can be build reproducible still.

Since about a month, we’ve also been rebuilding trying to exactly match the builds being distributed via ftp.debian.org. This talk will describe the setup and the lessons learned so far, and why the results currently are what they are (spoiler: they are less than 30% reproducible), and what we can do to fix that.

The Debian Project Leader, Andreas Tille, was present at the talk and remarked later in his Bits from the DPL update that:

It might be unfair to single out a specific talk from Toulouse, but I’d like to highlight the one on reproducible builds. Beyond its technical focus, the talk also addressed the recent loss of Lunar, whom we mourn deeply. It served as a tribute to Lunar’s contributions and legacy. Personally, I’ve encountered packages maintained by Lunar and bugs he had filed. I believe that taking over his packages and addressing the bugs he reported is a meaningful way to honor his memory and acknowledge the value of his work.

Holger’s slides and video in .webm format are available.

Next, rebuilderd is the server to monitor package repositories of Linux distributions and attempt to reproduce the observed results. This month, version 0.21.0 released, most notably with improved support for binNMUs by Jochen Sprickerhof and updating the rebuilderd-debian.sh integration to the latest debrebuild version by Holger Levsen. There has also been significant work to get the rebuilderd package into the Debian archive, in particular, both rust-rebuilderd-common version 0.20.0-1 and rust-rust-lzma version 0.6.0-1 were packaged by kpcyrd and uploaded by Holger Levsen.

Related to this, Holger Levsen submitted three additional issues against rebuilderd as well:

• rebuildctl should be more verbose when encountering issues. […]
• Please add an option to used randomised queues. […]
• Scheduling and re-scheduling multiple packages at once. […]

… and lastly, Jochen Sprickerhof submitted one an issue requested that rebuilderd downloads the source package in addition to the .buildinfo file […] and kpcyrd also submitted and fixed an issue surrounding dependencies and clarifying the license […]

Separate to this, back in 2018, Chris Lamb filed a bug report against the sphinx-gallery package as it generates unreproducible content in various ways. This month, however, Dmitry Shachnev finally closed the bug, listing the multiple sub-issues that were part of the problem and how they were resolved.

Elsewhere, Roland Clobus posted to our mailing list this month, asking for input on a bug in Debian’s ca-certificates-java package. The issue is that the Java key management tools embed timestamps in its output, and this output ends up in the /etc/ssl/certs/java/cacerts file on the generated ISO images. A discussion resulted from Roland’s post suggesting some short- and medium-term solutions to the problem.

Holger Levsen uploaded some packages with reproducibility-related changes:

• devscripts versions 2.24.3, 2.24.4 and 2.24.5 were uploaded, including several fixes for the debrebuild and debootsnap and scripts.

• cdbs version 0.4.167 uploaded in order to drop dh_buildinfo support, as dpkg has generated .buildinfo files since 2016 and the results of dh_buildinfo are typically unreproducible. Related to this a mass bug filing by Helmut Grohne intended to remove the obsolete and deprecated dh-buildinfo package from the archive. At the time of writing, this still affects 311 packages in Debian unstable.

Lastly, 12 reviews of Debian packages were added, 5 were updated and 21 were removed this month adding to our knowledge about identified issues in Debian.

Reproducible builds by default in Maven 4

On our mailing list this month, Hervé Boutemy reported the latest release of Maven (4.0.0-beta-5) has reproducible builds enabled by default. In his mailing list post, Hervé mentions that “this story started during our Reproducible Builds summit in Hamburg”, where he created the upstream issue that builds on a “multi-year” effort to have Maven builds configured for reproducibility.

PyPI now supports digital attestations

Elsewhere in the Python ecosystem and as reported on LWN and elsewhere, the Python Package Index (PyPI) has announced that it has finalised support for PEP 740 (“Index support for digital attestations”).

Trail of Bits, who performed much of the development work, has an in-depth blog post about the work and its adoption, as well as what is left undone:

One thing is notably missing from all of this work: downstream verification. […]

This isn’t an acceptable end state (cryptographic attestations have defensive properties only insofar as they’re actually verified), so we’re looking into ways to bring verification to individual installing clients. In particular, we’re currently working on a plugin architecture for pip that will enable users to load verification logic directly into their pip install flows.

There was an in-depth discussion on LWN’s announcement page, as well as on Hacker News.

“Dependency Challenges in OSS Package Registries”

At BENEVOL, the Belgium-Netherlands Software Evolution workshop in Namur, Belgium, Tom Mens and Alexandre Decan presented their paper, “An Overview and Catalogue of Dependency Challenges in Open Source Software Package Registries”.

The abstract of their paper is as follows:

While open-source software has enabled significant levels of reuse to speed up software development, it has also given rise to the dreadful dependency hell that all software practitioners face on a regular basis. This article provides a catalogue of dependency-related challenges that come with relying on OSS packages or libraries. The catalogue is based on the scientific literature on empirical research that has been conducted to understand, quantify and overcome these challenges. […]

A PDF of the paper is available online.

Zig programming language demonstrated reproducible

Motiejus Jakšty posted an interesting and practical blog post on his successful attempt to reproduce the Zig programming language without using the pre-compiled binaries checked into the repository, and despite the circular dependency inherent in its bootstrapping process.

As a summary, Motiejus concludes that:

I can now confidently say (and you can also check, you don’t need to trust me) that there is nothing hiding in zig1.wasm [the checked-in binary] that hasn’t been checked-in as a source file.

The full post is full of practical details, and includes a few open questions.

Website updates

Notwithstanding the significant change to the landing page (screenshot above), there were an enormous number of changes made to our website this month. This included:

• Alex Feyerke and Mariano Giménez:

  • Dramatically overhaul the website’s landing page with new “benefit” cards tailored to the expected visitors to our website and a reworking of the visual hierarchy and design. […][…][…][…][…][…][…][…][…][…]

• Bernhard M. Wiedemann:

  • Update the “System images” page to document the e2fsprogs approach. […]

• Chris Lamb:

  • Cachebust every CSS file per-release. […]
  • Replace some inline markdown with HTML. […]
  • Use spaces on the “Publications” page. […]
  • Add a news article about the passing of Lunar. […][…][…][…]
  • Add a black memorial band to the top of the page. […]

• FC (Fay) Stegerman:

  • Replace more inline markdown with HTML on the “Success stories” page. […]
  • Add some links, fix some other links and correct some spelling errors on the “Tools” page. […]

• Holger Levsen:

  • Add a historical presentation (“Reproducible builds everywhere eg. in Debian, OpenWrt and LEDE”) from October 2016. […]
  • Add jochensp and Oejet to the list of known contributors. […][…]

• Julia Krüger:

  • Add a new “Stripping of unreproducible information page to the documentation. […]

• Ninette Adhikari & hulkoba:

  • Add/rework the list of success stories into a new page that clearly shows milestones in Reproducible Builds. […][…][…][…][…][…]

• Philip Rinn:

  • Import 47 historical weekly reports. […]

• hulkoba:

  • Add alt text to almost all images (!). […][…]
  • Fix a number of links on the “Talks”. […][…]
  • Avoid so-called ‘ghost’ buttons by not using <button> elements as links, as the affordance of a <button> implies an action with (potentially) a side effect. […][…]
  • Center the sponsor logos on the homepage. […]

  • Move publications and generate them instead from a data.yml file with an improved layout. […][…]

  • Make a large number of small but impactful stylisting changes. […][…][…][…]

  • Expand the “Tools” to include a number of missing tools, fix some styling issues and fix a number of stale/broken links. […][…][…][…][…][…]

Upstream patches

The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

• Bernhard M. Wiedemann:

  • clisp (fix contributed by Bruno Haible)
  • conky (date-related issue)
  • emacs-auctex (date-related gzip issue)
  • javadoc (filesystem ordering issue)
  • jboss-websocket-1.0-api (embeds uname -r)
  • lcms2 (CPU issue)
  • LiE (ASLR-related issue)
  • make_ext4fs (toolchain-related issue for for VM images)
  • obs-build (issue when running builds with certain CPU types or core numbers)
  • perl-Time-modules (fails to build far in the future)
  • python-bson (fails to build far in the future)
  • python-exiv2 (fails to build far in the future)
  • python-moto (date-related gzip issue)
  • python-pyhanko-certvalidator (fails to build far in the future)
  • python-python-gvm (concurrency-related issue)
  • python310 (fails to build far in the future)
  • python313 (fails to build far in the future)
  • reproducible-faketools (toolchain for emacs)
  • shadowsocks-rust (date-related issue)
  • swipl (fails to build far in the future)

• Chris Lamb:

  • #1087330 filed against python-pydash.
  • #1087485 filed against fritzconnection.
  • #1087486 filed against tracy.
  • #1088238 filed against rust-broot.
  • #1088353 filed against python-aiovlc.
  • #1088742 filed against python-aiohomekit.

• James Addison:

  • #1088144 filed against cdbs.

Misc development news

• Bernhard M. Wiedemann published another report for the openSUSE distribution.

• Martin Abente Lahaye updated diffoscope to fix a crash when objdump is missing. […]

• On our mailing list, Jan-Benedict Glaw announced the publication of the fifth NetBSD Reproducibility Report

Reproducibility testing framework

The Reproducible Builds project operates a comprehensive testing framework running primarily at tests.reproducible-builds.org in order to check packages and other artifacts for reproducibility. In November, a number of changes were made by Holger Levsen, including:

• reproduce.debian.net-related changes:

  • Create and introduce a new reproduce.debian.net service and subdomain […]
  • Make a large number of documentation changes relevant to rebuilderd. […][…][…][…][…]
  • Explain a temporary workaround for a specific issue in rebuilderd. […]
  • Setup another rebuilderd instance on the o4 node and update installation documentation to match. […][…]
  • Make a number of helpful/cosmetic changes to the interface, such as clarifying terms and adding links. […][…][…][…][…]
  • Deploy configuration to the /opt and /var directories. […][…]
  • Add an infancy (or ‘alpha’) disclaimer. […][…]
  • Add more notes to the temporary rebuilderd documentation. […]
  • Commit an nginx configuration file for reproduce.debian.net’s “Stats” page. […]
  • Commit a rebuilder-worker.conf configuration for the o5 node. […]

• Debian-related changes:

  • Grant jspricke and jochensp access to the o5 node. […][…]
  • Build the qemu package with the nocheck build flag. […]

• Misc changes:

  • Adapt the update_jdn.sh script for new Debian trixie systems. […]
  • Stop installing the PostgreSQL database engine on the o4 and o5 nodes. […]
  • Prevent accidental reboots of the o4 node because of a long-running job owned by josch. […][…]

In addition, Mattia Rizzolo addressed a number of issues with reproduce.debian.net […][…][…][…]. And lastly, both Holger Levsen […][…][…][…] and Vagrant Cascadian […][…][…][…] performed node maintenance.

If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

• IRC: #reproducible-builds on irc.oftc.net.

• Mastodon: @reproducible_builds@fosstodon.org

• Mailing list: rb-general@lists.reproducible-builds.org

• Twitter: @ReproBuilds

Review: Paladin's Hope, by T. Kingfisher

Series: The Saint of Steel #3 Publisher: Red Wombat Studio Copyright: 2021 ISBN: 1-61450-613-2 Format: Kindle Pages: 303

Paladin's Hope is a fantasy romance novel and the third book of The Saint of Steel series. Each book of that series features different protagonists in closer to the romance series style than the fantasy series style and stands alone reasonably well. There are a few spoilers for the previous books here, so you probably want to read the series in order.

Galen is one of the former paladins of the Saint of Steel, left bereft and then adopted by the Temple of the Rat after their god dies. Even more than the paladin protagonists of the previous two books, he reacted very badly to that death and has ongoing problems with nightmares and going into berserker rages when awakened. As the book opens, he's the escort for a lich-doctor named Piper who is examining a corpse found in the river.

The last of the five was the only one who did not share a certain martial quality. He was slim and well-groomed and would be considered handsome, but he was also extraordinarily pale, as if he lived his life underground.

It was this fifth man who nudged the corpse with the toe of his boot and said, "Well, if you want my professional opinion, this great goddamn hole in his chest is probably what killed him."

As it turns out, slim and well-groomed and exceedingly pale is Galen's type.

This is another paladin romance, this time between two men. It's almost all romance; the plot is barely worth mentioning. About half of the book is an exploration of a puzzle dungeon of the sort that might be fun in a video game or tabletop RPG, but that I found rather boring and monotonous in a novel. This creates a lot more room for the yearning and angst.

Kingfisher tends towards slow-burn romances. This romance is a somewhat faster burn than some of her other books, but instead implodes into one of the most egregiously stupid third-act break-ups that I've read in a romance plot. Of all the Kingfisher paladin books, I think this one was hurt the most by my basic difference in taste from the author. Kingfisher finds constant worrying and despair over being good enough for the romantic partner to be an enjoyable element, and I find it incredibly annoying. I think your enjoyment of this book will heavily depend on where you fall on that taste divide.

The saving grace of this book are the gnoles, who are by far the best part of this world. Earstripe, a gnole constable, is the one who found the body that the book opens with and he drives most of the plot, such that it is. He's also the source of the best banter in the book, which is full of pointed and amused gnole observations about humans and their various stupidities. Given that I was also grumbling about human stupidities for most of the book, the gnole viewpoint and I got along rather well.

"God's stripes." Earstripe shook his head in disbelief. "Bone-doctor would save some gnole, yes? If some gnole was hurt."

"Of course," said Piper. "If I could."

"And tomato-man would save some gnole?" He swung his muzzle toward Galen. "If some gnome needed big human with sword?"

"Yes, of course."

Earstripe spread his hands, claws gleaming. "A gnole saves some human. Same thing." He took a deep breath, clearly choosing his words carefully. "A gnole's compassion does not require fur."

We learn a great deal more about gnole culture, all of which I found fascinating, and we get a rather satisfying amount of gnole acerbic commentary. Kingfisher is very good at banter, and dialogue in general, which also smoothes over the paucity of detailed plot. There was no salvaging the romance, at least for me, but I did at least like Piper, and Galen wasn't too bad when he wasn't being annoyingly self-destructive.

I had been wondering a little if gay romance would, like sapphic romance, avoid my dislike of heterosexual gender roles. I think the jury is still out, but it did not work in this book because Galen is so committed to being the self-sacrificing protector who is unable to talk about his feelings that he single-handedly introduced a bunch of annoying pieces of the male gender role anyway. I will have to try that experiment with a book that doesn't involve hard-headed paladins.

I have yet to read a bad T. Kingfisher novel, but I thought this one was on the weaker side. The gnoles are great and kept me reading, but I wish there had been a more robust plot, a lot less of the romance, and no third-act break-up. As is, I recommend the other Saint of Steel books over this one. Ah well.

Followed by Paladin's Faith.

Rating: 6 out of 10

An updated version of the corels package is now on CRAN! The ‘Certifiably Optimal RulE ListS (Corels)’ learner provides interpretable decision rules with an optimality guarantee—a nice feature which sets it apart in machine learning. You can learn more about corels at its UBC site.

The changes concern mostly maintenance for both the repository (such as continunous integration setup, badges, documentation links, …) and the package level (such as removing the no-longer-requiring C++ compilation standard setter now emitting a NOTE at CRAN.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

I am still here. Sadly while I battle this insane infection from my broken arm I got back in July, the hackers got my blog. I am slowly building it back up. Further bad news is I have more surgeries, first one tomorrow. Furthering my current struggles I cannot start my job search due to hospitalization and recovery. Please consider a donation. https://gofund.me/6e99345d

On the open source work front, I am still working on stuff, mostly snaps ( Apps 24.08.3 released )

Thank you everyone that voted me into the Ubuntu Community Council!

I am trying to stay positive, but it seems I can’t catch a break. I will have my computer in the hospital and will work on what I can. Have a blessed day and see you soon.

Scarlett

Sometimes you've to look at the content of x509 certificate chains. Usually one finds them pem encoded and concatenated in a text file.

Since the openssl x509 subcommand only decodes the first certificate it will find in a file, I did something like this:

csplit -z -f 'cert' fullchain.pem '/-----BEGIN CERTIFICATE-----/' '{*}' for x in cert*; do openssl x509 -in $x -noout -text; done

Apparently that's the "wrong" way and the more appropriate way is using the openssl crl2pkcs7 subcommand albeit we do not try to parse a revocation list here.

openssl crl2pkcs7 -nocrl -certfile fullchain.pem | \ openssl pkcs7 -print_certs -noout

Learned that one in a webinar presented by Victor Dukhovni. If you're new to the topic worth watching.