Planet Debian

Subscribe to Planet Debian feed
Planet Debian - https://planet.debian.org/
Updated: 4 hours 21 min ago

Dirk Eddelbuettel: RcppArmadillo 0.12.8.0.0 on CRAN: New Upstream, Interface Polish

Thu, 2024-02-08 18:33

Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research code into production environments. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 1119 other packages on CRAN, downloaded 32.5 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint / vignette) by Conrad and myself has been cited 575 times according to Google Scholar.

This release brings a new (stable) upstream (minor) release Armadillo 12.8.0 prepared by Conrad two days ago. We, as usual, prepared a release candidate which we tested against the over 1100 CRAN packages using RcppArmadillo. This found no issues, which was confirmed by CRAN once we uploaded and so it arrived as a new release today in a fully automated fashion.

We also made a small change that had been prepared by GitHub issue #400: a few internal header files that were cluttering the top-level of the include directory have been moved to internal directories. The standard header is of course unaffected, and the set of ‘full / light / lighter / lightest’ headers (matching we did a while back in Rcpp) also continue to work as one expects. This change was also tested in a full reverse-dependency check in January but had not been released to CRAN yet.

The set of changes since the last CRAN release follows.

Changes in RcppArmadillo version 0.12.8.0.0 (2024-02-06)
  • Upgraded to Armadillo release 12.8.0 (Cortisol Injector)

    • Faster detection of symmetric expressions by pinv() and rank()

    • Expanded shift() to handle sparse matrices

    • Expanded conv_to for more flexible conversions between sparse and dense matrices

    • Added cbrt()

    • More compact representation of integers when saving matrices in CSV format

  • Five non-user facing top-level include files have been removed (#432 closing #400 and building on #395 and #396)

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the Rcpp R-Forge page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Categories: FLOSS Project Planets

Sven Hoexter: Use GitHub CLI to List all Repository Secrets

Thu, 2024-02-08 05:12

Write it down before I forget about it again:

for x in $(gh api graphql --paginate -f query='query($endCursor:String) { organization(login:"myorg") { repositories(first: 100, after: $endCursor, isArchived:false) { pageInfo { hasNextPage endCursor } nodes { name } } } }' --jq '.data.organization.repositories.nodes[].name'); do secrets=$(gh secret list --json name --jq '.[].name' -R "myorg/${x}" | tr '\n' ',') if ! [ -z "${secrets}" ]; then echo "${x},${secrets}" fi done

Requests a list of all not archived repositories in a GitHub org and queries repository secrets. If we find some we output the repo name and the secrets in a comma separated list. Not real CSV, but good enough for further processing. I've to admit it's kinda beautiful what you can do with the gh cli by now. Sadly it seems the secrets are not yet available via GraphQL (or I missed it in the docs), so I just use the gh cli to do the REST calls.

Categories: FLOSS Project Planets

Bits from Debian: DebConf24 Logo Contest Results

Thu, 2024-02-08 00:00

Earlier this month the DebConf team announced the DebConf24 Logo Contest asking aspiring artists, designers, and contributors to submit an image that would represent the host city of Busan, the host nation of South Korea, and promote the next Debian Developer Conference.

The logo contest for DebConf24 received 10 submissions and garnered 354 responses with 3 proposals in particular getting very close to first place. The winning logo received 88 votes, the 2nd favored logo received 87 votes, and the 3rd most favored received 86 votes.

Thank you to Woohee Yang and Junsang Moon for sharing their artistic visions.

A very special Thank You to everyone who took the time to vote for our beautiful new logo!

The DebConf24 Team is proud to share for preview only the winning logo for the 24th Debian Developer Conference:

'sun-seagull-sea' by Woohee Yang

This is a preview copy, other revisions will occur for sizing, print, and media... but we had to share it with you all now. :).

Looking forward to seeing you all at #debconf24 in #Busan, South Korea 2024!

Categories: FLOSS Project Planets

Reproducible Builds: Reproducible Builds at FOSDEM 2024

Wed, 2024-02-07 19:00

Core Reproducible Builds developer Holger Levsen presented at the main track at FOSDEM on Saturday 3rd February this year in Brussels, Belgium. Titled Reproducible Builds: The First Ten Years

In this talk Holger ‘h01ger’ Levsen will give an overview about Reproducible Builds: How it started with a small BoF at DebConf13 (and before), then grew from being a Debian effort to something many projects work on together, until in 2021 it was mentioned in an Executive Order of the President of the United States. And of course, the talk will not end there, but rather outline where we are today and where we still need to be going, until Debian stable (and other distros!) will be 100% reproducible, verified by many.

h01ger has been involved in reproducible builds since 2014 and so far has set up automated reproducibility testing for Debian, Fedora, Arch Linux, FreeBSD, NetBSD and coreboot.

More information can be found on FOSDEM’s own page for the talk.

Separate from Holger’s talk, however, there were a number of other talks about reproducible builds at FOSDEM this year:

… and there was even an entire track on Software Bill of Materials.

Categories: FLOSS Project Planets

Reproducible Builds: Reproducible Builds in January 2024

Wed, 2024-02-07 17:16

Welcome to the January 2024 report from the Reproducible Builds project. In these reports we outline the most important things that we have been up to over the past month. If you are interested in contributing to the project, please visit our Contribute page on our website.

“How we executed a critical supply chain attack on PyTorch”

John Stawinski and Adnan Khan published a lengthy blog post detailing how they executed a supply-chain attack against PyTorch, a popular machine learning platform “used by titans like Google, Meta, Boeing, and Lockheed Martin”:

Our exploit path resulted in the ability to upload malicious PyTorch releases to GitHub, upload releases to [Amazon Web Services], potentially add code to the main repository branch, backdoor PyTorch dependencies – the list goes on. In short, it was bad. Quite bad.

The attack pivoted on PyTorch’s use of “self-hosted runners” as well as submitting a pull request to address a trivial typo in the project’s README file to gain access to repository secrets and API keys that could subsequently be used for malicious purposes.


New Arch Linux forensic filesystem tool

On our mailing list this month, long-time Reproducible Builds developer kpcyrd announced a new tool designed to forensically analyse Arch Linux filesystem images.

Called archlinux-userland-fs-cmp, the tool is “supposed to be used from a rescue image (any Linux) with an Arch install mounted to, [for example], /mnt.” Crucially, however, “at no point is any file from the mounted filesystem eval’d or otherwise executed. Parsers are written in a memory safe language.”

More information about the tool can be found on their announcement message, as well as on the tool’s homepage. A GIF of the tool in action is also available.


Issues with our SOURCE_DATE_EPOCH code?

Chris Lamb started a thread on our mailing list summarising some potential problems with the source code snippet the Reproducible Builds project has been using to parse the SOURCE_DATE_EPOCH environment variable:

I’m not 100% sure who originally wrote this code, but it was probably sometime in the ~2015 era, and it must be in a huge number of codebases by now.

Anyway, Alejandro Colomar was working on the shadow security tool and pinged me regarding some potential issues with the code. You can see this conversation here.

Chris ends his message with a request that those with intimate or low-level knowledge of time_t, C types, overflows and the various parsing libraries in the C standard library (etc.) contribute with further info.


Distribution updates

In Debian this month, Roland Clobus posted another detailed update of the status of reproducible ISO images on our mailing list. In particular, Roland helpfully summarised that “all major desktops build reproducibly with bullseye, bookworm, trixie and sid provided they are built for a second time within the same DAK run (i.e. [within] 6 hours)”. Additionally 7 of the 8 bookworm images from the official download link build reproducibly at any later time.

In addition to this, three reviews of Debian packages were added, 17 were updated and 15 were removed this month adding to our knowledge about identified issues.

Elsewhere, Bernhard posted another monthly update for his work elsewhere in openSUSE.


Community updates

There were made a number of improvements to our website, including Bernhard M. Wiedemann fixing a number of typos of the term ‘nondeterministic’. [] and Jan Zerebecki adding a substantial and highly welcome section to our page about SOURCE_DATE_EPOCH to document its interaction with distribution rebuilds. [].


diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made a number of changes such as uploading versions 245 and 255 to Debian but focusing on triaging and/or merging code from other contributors. This included adding support for comparing eXtensible ARchive’ (.XAR/.PKG) files courtesy of Seth Michael Larson [][], as well considerable work from Vekhir in order to fix compatibility between various and subtle incompatible versions of the progressbar libraries in Python [][][][]. Thanks!


Reproducibility testing framework

The Reproducible Builds project operates a comprehensive testing framework (available at tests.reproducible-builds.org) in order to check packages and other artifacts for reproducibility. In January, a number of changes were made by Holger Levsen:

  • Debian-related changes:

    • Reduce the number of arm64 architecture workers from 24 to 16. []
    • Use diffoscope from the Debian release being tested again. []
    • Improve the handling when killing unwanted processes [][][] and be more verbose about it, too [].
    • Don’t mark a job as ‘failed’ if process marked as ‘to-be-killed’ is already gone. []
    • Display the architecture of builds that have been running for more than 48 hours. []
    • Reboot arm64 nodes when they hit an OOM (out of memory) state. []
  • Package rescheduling changes:

    • Reduce IRC notifications to ‘1’ when rescheduling due to package status changes. []
    • Correctly set SUDO_USER when rescheduling packages. []
    • Automatically reschedule packages regressing to FTBFS (build failure) or FTBR (build success, but unreproducible). []
  • OpenWrt-related changes:

    • Install the python3-dev and python3-pyelftools packages as they are now needed for the sunxi target. [][]
    • Also install the libpam0g-dev which is needed by some OpenWrt hardware targets. []
  • Misc:

    • As it’s January, set the real_year variable to 2024 [] and bump various copyright years as well [].
    • Fix a large (!) number of spelling mistakes in various scripts. [][][]
    • Prevent Squid and Systemd processes from being killed by the kernel’s OOM killer. []
    • Install the iptables tool everywhere, else our custom rc.local script fails. []
    • Cleanup the /srv/workspace/pbuilder directory on boot. []
    • Automatically restart Squid if it fails. []
    • Limit the execution of chroot-installation jobs to a maximum of 4 concurrent runs. [][]

Significant amounts of node maintenance was performed by Holger Levsen (eg. [][][][][][][] etc.) and Vagrant Cascadian (eg. [][][][][][][][]). Indeed, Vagrant Cascadian handled an extended power outage for the network running the Debian armhf architecture test infrastructure. This provided the incentive to replace the UPS batteries and consolidate infrastructure to reduce future UPS load. []

Elsewhere in our infrastructure, however, Holger Levsen also adjusted the email configuration for @reproducible-builds.org to deal with a new SMTP email attack. []


Upstream patches

The Reproducible Builds project tries to detects, dissects and fix as many (currently) unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

Separate to this, Vagrant Cascadian followed up with the relevant maintainers when reproducibility fixes were not included in newly-uploaded versions of the mm-common package in Debian — this was quickly fixed, however. []


If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

Categories: FLOSS Project Planets

Jonathan Dowland: carbon

Wed, 2024-02-07 14:24

I got a new work laptop this year: A Thinkpad X1 Carbon (Gen 11). It wasn't the one I wanted: I'd ordered an X1 Nano, which had a footprint very reminiscent to me of my beloved x40.

Never mind! The Carbon is lovely. Despite ostensibly the same size as the T470s it's replacing, it's significantly more portable, and more capable. The two USB-A ports, as well as the full-size HDMI port, are welcome and useful (over the Nano).

I used to keep notes on setting up Linux on different types of hardware, but I haven't really bothered now for years. Things Just Work. That's good!

My old machine naming schemes are stretched beyond breaking point (and I've re-used my favourite hostname, qusp, one too many times) so I went for something new this time: Riffing on Carbon, I settled (for now) on carbyne, a carbon allotrope which is of interest to nanotechnologists (Seems appropriate)

Categories: FLOSS Project Planets

Louis-Philippe Véronneau: Montreal's Debian & Stuff - February 2024

Tue, 2024-02-06 14:30

New Year, Same Great People! Our Debian User Group met for the first of our 2024 bi-monthly meetings on February 4th and it was loads of fun. Around twelve different people made it this time to Koumbit, where the meeting happened.

As a reminder, our meetings are called "Debian & Stuff" because we want to be as open as possible and welcome people that want to work on "other stuff" than Debian.

Here is what we did:

pollo:

  • tested a laptop that had a defective battery with a known good one (the problem was indeed with the battery :D)
  • renewed his expiring OpenPGP key
  • worked on removing trapperkeeper-scheduler-clojure from the DebCI reject_list
  • helped anarcat with packaging sigal
  • managed to feed most of the people present :)

LeLutin:

  • worked with lavamind to upload new upstream release of smokeping

mjeanson:

  • migrated from lxd to incus on his servers
  • helped anarcat with flashing his AirGradient

lavamind:

  • submitted a bug report on r-cran-rserve (promptly fixed/uploaded by maintainer!)
  • reviewed and uploaded smokeping
  • bug triaged the facter package
  • worked on puppet-agent new upstream version 8.4.0

viashimo:

  • updated puppet-strings from 2.9.0 to 4.2.1
  • reported upstream test failures on puppet-strings with recent versions of mdl

tvaz & tassia:

  • drafted a call & request for funding for the Vanier College FLOSS Club hardware marathon at Eastern Bloc
  • worked on an application to conduct research at Vanier College on Debian usability
  • babysitted :-)

joeDoe:

  • worked on his AirGradient
  • debugged the WiFi and VPN setup on his new laptop

anarcat:

Pictures

I was pretty busy this time around and ended up not taking a lot of pictures. Here's a bad one of the ceiling at Koumbit I took, and a picture by anarcat of the content of his boxes of loot:

Categories: FLOSS Project Planets

Robert McQueen: Flathub: Pros and Cons of Direct Uploads

Tue, 2024-02-06 05:57

I attended FOSDEM last weekend and had the pleasure to participate in the Flathub / Flatpak BOF on Saturday. A lot of the session was used up by an extensive discussion about the merits (or not) of allowing direct uploads versus building everything centrally on Flathub’s infrastructure, and related concerns such as automated security/dependency scanning.

My original motivation behind the idea was essentially two things. The first was to offer a simpler way forward for applications that use language-specific build tools that resolve and retrieve their own dependencies from the internet. Flathub doesn’t allow network access during builds, and so a lot of manual work and additional tooling is currently needed (see Python and Electron Flatpak guides). And the second was to offer a maybe more familiar flow to developers from other platforms who would just build something and then run another command to upload it to the store, without having to learn the syntax of a new build tool. There were many valid concerns raised in the room, and I think on reflection that this is still worth doing, but might not be as valuable a way forward for Flathub as I had initially hoped.

Of course, for a proprietary application where Flathub never sees the source or where it’s built, whether that binary is uploaded to us or downloaded by us doesn’t change much. But for an FLOSS application, a direct upload driven by the developer causes a regression on a number of fronts. We’re not getting too hung up on the “malicious developer inserts evil code in the binary” case because Flathub already works on the model of verifying the developer and the user makes a decision to trust that app – we don’t review the source after all. But we do lose other things such as our infrastructure building on multiple architectures, and visibility on whether the build environment or upload credentials have been compromised unbeknownst to the developer.

There is now a manual review process for when apps change their metadata such as name, icon, license and permissions – which would apply to any direct uploads as well. It was suggested that if only heavily sandboxed apps (eg no direct filesystem access without proper use of portals) were permitted to make direct uploads, the impact of such concerns might be somewhat mitigated by the sandboxing.

However, it was also pointed out that my go-to example of “Electron app developers can upload to Flathub with one command” was also a bit of a fiction. At present, none of them would pass that stricter sandboxing requirement. Almost all Electron apps run old versions of Chromium with less complete portal support, needing sandbox escapes to function correctly, and Electron (and Chromium’s) sandboxing still needs additional tooling/downstream patching to run inside a Flatpak. Buh-boh.

I think for established projects who already ship their own binaries from their own centralised/trusted infrastructure, and for developers who have understandable sensitivities about binary integrity such such as encryption, password or financial tools, it’s a definite improvement that we’re able to set up direct uploads with such projects with less manual work. There are already quite a few applications – including verified ones – where the build recipe simply fetches a binary built elsewhere and unpacks it, and if this already done centrally by the developer, repeating the exercise on Flathub’s server adds little value.

However for the individual developer experience, I think we need to zoom out a bit and think about how to improve this from a tools and infrastructure perspective as we grow Flathub, and as we seek to raise funds for different sources for these improvements. I took notes for everything that was mentioned as a tooling limitation during the BOF, along with a few ideas about how we could improve things, and hope to share these soon as part of an RFP/RFI (Request For Proposals/Request for Information) process. We don’t have funding yet but if we have some prospective collaborators to help refine the scope and estimate the cost/effort, we can use this to go and pursue funding opportunities.

Categories: FLOSS Project Planets

Pages