Feeds

Are you ready for 11?

Even though you may have just upgraded your current site to Drupal 10, there’s a new version that’s already here! Do you need to upgrade? If so, when?

Let's take a quick look at what 11.0 has to offer.

I started writing blog 20 years ago, not on this domain, but this blog still has all the old posts starting from 8th August 2004. Though I used to write mostly one line blog posts, which is equivalent of Mastodon posts these days.

I started writing another blog, but in Swedish. So that I can feel less scared with the language.

Tools used

• Started on blogspot in 2004
• Moved to Wordpress in 2007
• Moved to Nikola, first time for me in static blogging system in 2012.
• Moved to Shonku my Golang based static blogging tool in 2013.
• Moved to khata moved to my Rust based blogging tool in 2019.

Hopefully I will write more in the coming months. But, who knows :)

Here's what I've been working on for my LocalGov Drupal contributions this week. Thanks to Big Blue Door for sponsoring the time to work on these.

Welcome to the July 2024 report from the Reproducible Builds project!

In our reports, we outline what we’ve been up to over the past month and highlight news items in software supply-chain security more broadly. As always, if you are interested in contributing to the project, please visit our Contribute page on our website.

Table of contents:

1. Reproducible Builds Summit 2024
2. Pulling Linux up by its bootstraps
3. Towards Idempotent Rebuilds?
4. AROMA: Automatic Reproduction of Maven Artifacts
5. Community updates
6. Android Reproducible Builds at IzzyOnDroid with rbtlog
7. Extending the Scalability, Flexibility and Responsiveness of Secure Software Update Systems
8. Development news
9. Website updates
10. Upstream patches
11. Reproducibility testing framework

Reproducible Builds Summit 2024

Last month, we were very pleased to announce the upcoming Reproducible Builds Summit, set to take place from September 17th — 19th 2024 in Hamburg, Germany. We are thrilled to host the seventh edition of this exciting event, following the success of previous summits in various iconic locations around the world, including Venice, Marrakesh, Paris, Berlin and Athens. Our summits are a unique gathering that brings together attendees from diverse projects, united by a shared vision of advancing the Reproducible Builds effort. During this enriching event, participants will have the opportunity to engage in discussions, establish connections and exchange ideas to drive progress in this vital field. Our aim is to create an inclusive space that fosters collaboration, innovation and problem-solving.

If you’re interesting in joining us this year, please make sure to read the event page, which has more details about the event and location. We are very much looking forward to seeing many readers of these reports there.

”Pulling Linux up by its bootstraps” (LWN)

In a recent edition of Linux Weekly News, Daroc Alden has written an article on “bootstrappable” builds. Starting with a brief introduction that…

… a bootstrappable build is one that builds existing software from scratch — for example, building GCC without relying on an existing copy of GCC. In 2023, the Guix project announced that the project had reduced the size of the binary bootstrap seed needed to build its operating system to just 357-bytes — not counting the Linux kernel required to run the build process.

The article goes onto to describe that “now, the live-bootstrap project has gone a step further and removed the need for an existing kernel at all.” and concludes:

The real benefit of bootstrappable builds comes from a few things. Like reproducible builds, they can make users more confident that the binary packages downloaded from a package mirror really do correspond to the open-source project whose source code they can inspect. Bootstrappable builds have also had positive effects on the complexity of building a Linux distribution from scratch […]. But most of all, bootstrappable builds are a boon to the longevity of our software ecosystem. It’s easy for old software to become unbuildable. By having a well-known, self-contained chain of software that can build itself from a small seed, in a variety of environments, bootstrappable builds can help ensure that today’s software is not lost, no matter where the open-source community goes from here

Towards Idempotent Rebuilds?

Trisquel developer Simon Josefsson wrote an interesting blog post comparing the output of the .deb files from our tests.reproducible-builds.org testing framework and the ones in the official Debian archive. Following up from a previous post on the reproducibility of Trisquel, Simon notes that “typically [the] rebuilds do not match the official packages, even when they say the package is reproducible”, Simon correctly identifies that “the purpose of [these] rebuilds are not to say anything about the official binary build, instead the purpose is to offer a QA service to maintainers by performing two builds of a package and declaring success if both builds match.”

However, Simon’s post swiftly moves on to announce a new tool called debdistrebuild that performs rebuilds of the difference between two distributions in a GitLab pipeline and displays diffoscope output for further analysis.

AROMA: Automatic Reproduction of Maven Artifacts

Mehdi Keshani, Tudor-Gabriel Velican, Gideon Bot and Sebastian Proksch of the Delft University of Technology, Netherlands, have published a new paper in the ACM Software Engineering on a new tool to automatically reproduce Apache Maven artifacts:

Reproducible Central is an initiative that curates a list of reproducible Maven libraries, but the list is limited and challenging to maintain due to manual efforts. [We] investigate the feasibility of automatically finding the source code of a library from its Maven release and recovering information about the original release environment. Our tool, AROMA, can obtain this critical information from the artifact and the source repository through several heuristics and we use the results for reproduction attempts of Maven packages. Overall, our approach achieves an accuracy of up to 99.5% when compared field-by-field to the existing manual approach [and] we reveal that automatic reproducibility is feasible for 23.4% of the Maven packages using AROMA, and 8% of these packages are fully reproducible.

Community updates

On our mailing list this month:

• Nichita Morcotilo reached out to the community, first to share their efforts “to build reproducible packages cross-platform with a new build tool called rattler-build, noting that “as you can imagine, building packages reproducibly on Windows is the hardest challenge (so far!)”. Nichita goes onto mention that the Apple ecosystem appears to be using ZERO_AR_DATE over SOURCE_DATE_EPOCH. […]

• Roland Clobus announced that the Debian bookworm 12.6 live images are “nearly reproducible”, with more detail in the post itself and input in the thread from other contributors.

• As reported in last month’s report, Pol Dellaiera completed his master thesis on Reproducibility in Software Engineering at the University of Mons, Belgium. This month, Pol announced this on the list with more background info. Since the master thesis sources have been available, it has received some feedback and contributions. As a result, an updated version of the thesis has been published containing those community fixes.

• Daniel Gröber asked for help in getting the Yosys documentation to build reproducibly, citing issues in inter alia the PDF generation causing differing CreationDate metadata values.

• James Addison continued his long journey towards getting the Sphinx documentation generator to build reproducible documentation. In this thread, James concerns himself with the problem that even “when SOURCE_DATE_EPOCH is configured, Sphinx projects that have configured their copyright notices using dynamic elements can produce nonsensical output under some circumstances.” James’ query ended up generating a number of replies.

• Allen ‘gunner’ Gunner posted a brief update on the progress the core team is making towards introducing a Code of Conduct (CoC) such that it is “in place in time for the RB Summit in Hamburg in September”. In particular, gunner asks “if you are interested in helping with CoC design and development in the weeks ahead, simply email rb-core@lists.reproducible-builds.org and let us know”. […]

Android Reproducible Builds at IzzyOnDroid with rbtlog

On our mailing list, Fay Stegerman announced a new Reproducible Builds collaboration in the Android ecosystem:

We are pleased to announce “Reproducible Builds, special client support and more in our repo”: a collaboration between various independent interoperable projects: the IzzyOnDroid team, 3rd-party clients Droid-ify & Neo Store, and rbtlog (part of my collection of tools for Android Reproducible Builds) to bring Reproducible Builds to IzzyOnDroid and the wider Android ecosystem.

Extending the Scalability, Flexibility and Responsiveness of Secure Software Update Systems

Congratulations to Marina Moore of the New York Tandon School of Engineering who has submitted her PhD thesis on Extending the Scalability, Flexibility and Responsiveness of Secure Software Update Systems. The introduction outlines its contributions to the field:

[S]oftware repositories are a vital component of software development and release, with packages downloaded both for direct use and to use as dependencies for other software. Further, when software is updated due to patched vulnerabilities or new features, it is vital that users are able to see and install this patched version of the software. However, this process of updating software can also be the source of attack. To address these attacks, secure software update systems have been proposed. However, these secure software update systems have seen barriers to widespread adoption. The Update Framework (TUF) was introduced in 2010 to address several attacks on software update systems including repository compromise, rollback attacks, and arbitrary software installation. Despite this, compromises continue to occur, with millions of users impacted by such compromises. My work has addressed substantial challenges to adoption of secure software update systems grounded in an understanding of practical concerns. Work with industry and academic communities provided opportunities to discover challenges, expand adoption, and raise awareness about secure software updates. […]

Development news

In Debian this month, 12 reviews of Debian packages were added, 13 were updated and 6 were removed this month adding to our knowledge about identified issues. A new toolchain issue type was identified as well, specifically ordering_differences_in_pkg_info.

Colin Percival filed a bug against the LLVM compiler noting that building i386 binaries on the i386 architecture is different when building i386 binaries under amd64. The fix was narrowed down to “x87 excess precision, which can result in slightly different register choices when the compiler is hosted on x86_64 or i386” and a fix committed. […]

Fay Stegerman performed some in-depth research surrounding her apksigcopier tool, after some Android .apk files signed with the latest apksigner could no longer be verified as reproducible. Fay identified the issue as follows:

Since build-tools >= 35.0.0-rc1, backwards-incompatible changes to apksigner break apksigcopier as it now by default forcibly replaces existing alignment padding and changed the default page alignment from 4k to 16k (same as Android Gradle Plugin >= 8.3, so the latter is only an issue when using older AGP). […]

She documented multiple available workarounds and filed a bug in Google’s issue tracker.

Lastly, diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb uploaded version 272 and Mattia Rizzolo uploaded version 273 to Debian, and the following changes were made as well:

• Chris Lamb:

  • Ensure that the convert utility is from ImageMagick version 6.x. The command-line interface has seemingly changed with the 7.x series of ImageMagick. […]
  • Factor out version detection in test_jpeg_image. […]
  • Correct the import of the identify_version method after a refactoring change in a previous commit. […]
  • Move away from using DSA OpenSSH keys in tests as support has been deprecated and removed in OpenSSH version 9.8p1. […]
  • Move to assert_diff in the test_openssh_pub_key package. […]
  • Update copyright years. […]

• Mattia Rizzolo:

  • Add support for ffmpeg version 7.x which adds some extra context to the diff. […]
  • Rework the handling of OpenSSH testing of DSA keys if OpenSSH is strictly 9.7, and add an OpenSSH key test with a ed25519-format key […][…][…]
  • Temporarily disable a few packages that are not available in Debian testing. […][…]
  • Stop ignoring the results of Debian testing in the continuous integration system. […]
  • Adjust options in debian/source to make sure not to pack the Python sdist directory into the binary Debian package. […]
  • Adjust Lintian overrides. […]

Website updates

There were a number of improvements made to our website this month, including:

• Bernhard M. Wiedemann updated the SOURCE_DATE_EPOCH page to include instructions on how to create reproducible .zip files from within Python using the zipfile module. […]

• Chris Lamb fixed a potential duplicate heading on the Projects page. […]

• Fay Stegerman added rbtlog to the Tools page […] and IzzyOnDroid to the Projects page […], also ensuring that the latter page was always sorted regardless of the ordering within the input data files. […]

• Holger Levsen added Linus Nordberg to our global list of contributors […] as well as made a number of changes to the page for the upcoming Reproducible Builds summit later this year […][…][…][…].

• Mattia Rizzolo updated the Civil Infrastructure Platform logo […] and also updated the 2024 summit page […][…].

• Nichita Morcotilo added rattler-build to the Projects page. […][…][…]

• Pol Dellaiera updated the Academic Publications page, adding two publications. […][…]

Upstream patches

The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

• Bernhard M. Wiedemann:

  • armagetron (date)
  • blaspp (hostname)
  • cligen (GnuTLSs date)
  • cloudflared (date)
  • dpdk (Sphinx doctrees)
  • fonttosfnt/xorg-x11-fonts (toolchain, date)
  • gegl (build machine details)
  • gettext-runtime (jar mtime)
  • kf6-kirigami+kf6-qqc2-desktop-style (race-condition)
  • kubernetes1.26 (backport upstream fix for random path)
  • lapackpp (hostname)
  • latex2html (nochecks)
  • libdb-4_8 (.jar modification time)
  • librcc (already merged upstream)
  • libreoffice (strip .jar mtimes + clucene-core toolchain)
  • maliit-keyboard (nocheck)
  • nautilus (date)
  • openblas (CPU type, fixed)
  • openssl-3 (random-related issue)
  • python-ruff (ASLR)
  • python3 (date, parallelism/race)
  • reproducible-faketools (0.5.2)
  • sphinx (GZip modification time)
  • sphinxcontrib (gzip mtime)

• Chris Lamb:

  • #1076368 filed against nautilus.
  • #1076507 filed against mccode.
  • #1076806 filed against meson-python.
  • #1077479 filed against debcraft.
  • #1077485 filed against pytest.
  • #1077601 filed against setuptools.

• Fridrich Strba:

  • javapackages-tools
  • ant
  • java-21-openjdk

• Evangelos Ribeiro Tzaras:

  • buffybox

Reproducibility testing framework

The Reproducible Builds project operates a comprehensive testing framework running primarily at tests.reproducible-builds.org in order to check packages and other artifacts for reproducibility. In July, a number of changes were made by Holger Levsen, including:

• Grant bremner access to the ionos7 node. […][…]
• Perform a dummy change to force update of all jobs. […][…]

In addition, Vagrant Cascadian performed some necessary node maintenance of the underlying build hosts. […]

If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

• IRC: #reproducible-builds on irc.oftc.net.

• Mastodon: @reproducible_builds@fosstodon.org

• Mailing list: rb-general@lists.reproducible-builds.org

• Twitter: @ReproBuilds

There's no doubt that the ECA module is changing the way Drupal sites are built  - at least for those of us that use it regularly. I recently used it to pre-populate a form field on a node add page and it turned out to be easier than I figured it would be.

The task was to pass the value I wanted to pre-populate into the form via a query string variable. Something like:

https://mysite.ddev.site/node/add/article?myvalue=1

On the node add page, there exists an entity reference field whose value I'd like to pre populate with a value of 1 (myvalue). 

Modules required

Modules used in the following solution are:

• Token
• BPMN.iO for ECA
• ECA BPMN (part of ECA module)
• ECA Form (part of ECA module)
• ECA UI (part of ECA module)
• ECA Core (part of ECA module)

The Token module is required because the ECA model will access the myvalue query string value via a token. 

ECA components

The ECA Form module (a sub-module of ECA) provides the necessary components to achieve the goal. 

The Build Form event and the Form Field: Set Default Value task components are all that are needed. Configuration as follows:

Build Form event

• Restrict by form id: node-article-form
• Restrict by entity type ID: node
• Restrict by entity bundle: article

Form Field: Set Default Value task 

• Field name: field_linked_content
• Value: [current-page:query:myvalue]
   

Once the model is created and saved, enjoy the fruits of your labor!

Other potential solutions

Was ECA the only way to complete this task? Certainly not. A small custom module could be written to achieve the same goal. In addition, there's the Pre-populate and Entity prepopulate modules.

Why use an ECA-based solution then? If you're already using ECA on the project for something else, then I'd recommend this type of ECA solution in order to avoid adding an additional contrib module just for your pre-populating needs.
 

As we wrap up PyCon US 2024, we can’t express enough gratitude to everyone who joined us, whether in-person or online, and made our first time together in Pittsburgh, PA a special and unforgettable experience. Not to mention, a record-breaking year - for the first time since before 2020, PyCon US sold out in-person tickets with over 2,700 tickets sold!

We had an amazing and diverse group of community members join us for PyCon US 2024, attending from 95 different countries! By the numbers, we saw a total attendance of 2,991 – with 2,551 attendees joining us in person and 440 joining us online. We couldn’t be more grateful for all who supported the Python ecosystem and helped make PyCon US 2024 a huge success.

Check out a full comprehensive recap of this year’s PyCon US conference here:

Find more photos from PyCon US 2024, captured by PSF Board Member, Kushal Das, here. 

We are also excited to announce that all PyCon US 2024 recordings are now available on the PyCon US YouTube channel! Be sure to subscribe to our channel for notifications of any new content.

We send the biggest thank you to all the presenters and speakers for their time, energy, and efforts in providing the wonderful content presented at PyCon US 2024, as well as to our incredible AV team, Altitude C, for their hard work and attention to capture recordings and provide AV this year.

The attendees, volunteers, speakers, staff, and sponsors truly make PyCon US what it is! The work of the Python Software Foundation is only possible with you all. A huge heartfelt thank you to the whole community!

We can’t wait to see you all back in Pittsburgh, PA for PyCon US 2025! If you’d like to be notified when the CFP opens and when tickets go on sale, you can watch this blog or subscribe to PyCon US News. Until then, if you have any questions or feedback, please reach out to pycon-reg@python.org.

I’m finishing typing up this blog entry hours before my last 13 hour leg back home, after I spent 2 weeks in Busan, South Korea for DebCamp24 and DebCamp24. I had a rough year and decided to take it easy this DebConf. So this is the first DebConf in a long time where I didn’t give any talks. I mostly caught up on a bit of packaging, worked on DebConf video stuff, attended a few BoFs and talked to people. Overall it was a very good DebConf, which also turned out to be more productive than I expeced it would.

In the welcome session on the first day of DebConf, Nicolas Dandrimont mentioned that a benefit of DebConf is that it provides a sort of caffeine for your Debian motivation. I could certainly feel that affect swell as the days went past, and it’s nice to be excited about some ideas again that would otherwise be fading.

Recovering DPL

It’s a bit of a gear shift being DPL for 4 years, and DebConf Committee for nearly 5 years before that, and then being at DebConf while some issue arise (as it always does during a conference). At first I jump into high alert mode, but then I have to remind myself “it’s not your problem anymore” and let others deal with it.

It was nice spending a little in-person time with Andreas Tille, our new DPL, we did some more handover and discussed some current issues. I still have a few dozen emails in my DPL inbox that I need to collate and forward to Andreas, I hope to finish all that up by the end of August.

During the Bits from the DPL talk, the usual question came up whether Andreas will consider running for DPL again, to which he just responded in a slide “Maybe”. I think it’s a good idea for a DPL to do at least two terms if it all works out for everyone, since it takes a while to get up to speed on everything.

Also, having been DPL for four years, I have a lot to say about it, and I think there’s a lot we can fix in the role, or at least discuss it. If I had the bandwidth for it I would have scheduled a BoF for it, but I’ll very likely do that for the next DebConf instead!

Video team

I set up the standby loop for the video streaming setup. We call it loopy, it’s a bunch of OBS scenes that provide announcements, shows sponsors, the schedule and some social content. I wrote about it back in 2020, but it’s evolved quite a bit since then, so I’m probably due to write another blog post with a bunch of updates on it. I hope to organise a video team sprint in Cape Town in the first half of next year, so I’ll summarize everything before then.

It would’ve been great if we could have some displays in social areas that could show talks, the loop and other content, but we were just too pressed for time for that. This year’s DebConf had a very compressed timeline, and there was just too much that had to be done and that had to be figured out on the last minute. This put quite a lot of strain on the organisers, but I was glad to see how, for the most part, most attendees were very sympathetic to some rough edges (but I digress…).

I added more of the OBS machine setup to the videoteam’s ansible repository, so as of now it just needs an ansible setup and the OBS data and it’s good to go. The loopy data is already in the videoteam git repository, so I could probably just add a git pull and create some symlinks in ansible and then that machine can be installed from 0% to 100% by just installing via debian-installer with our ansible hooks.

This DebConf I volunteered quite a bit for actual video roles during the conference, something I didn’t have much time for in recent DebConfs, and it’s been fun, especially in a session or two where nearly none of the other volunteers showed up. Sometimes chaos is just fun :-)

Baekyongee is the university mascot, who’s visible throughout the university. So of course we included this four legged whale creature on the loop too!

Packaging

I was hoping to do more packaging during DebCamp, but at least it was a non-zero amount:

• Uploaded gdisk 1.0.10-2 to unstable (previously tested effects of adding dh-sequence-movetousr) (Closes: #1073679).
• Worked a bit on bcachefs-tools (updating git to 1.9.4), but has a build failure that I need to look into (we might need a newer bindgen) – update: I’m probably going to ROM this package soon, it doesn’t seem suitable for packaging in Debian.
• Calamares: Tested a fix for encrypted installs, and uploaded it.
• Calamares: Uploaded (3.3.8-1) to backports (at the time of writing it’s still in backports-NEW).
• Backport obs-gradient-source for bookworm.
• Did some initial packaging on Cambalache, I’ll upload to unstable once gtk-4-dev (4.14.0) is in unstable (currently in experimental).
• Pixelorama 1.0 – I did some initial packaging for Pixelorama back when we did the MiniDebConf Gaming Edition, but it had a few stoppers back then. Version 1.0 seems to fix all of that, but it depends on Godot 4.2 and we’re still on the 3 series in Debian, so I’ll upload this once Godot 4.2 hits at least experimental. Godot software/games is otherwise quite easy to run, it’s basically just source code / data that is installed and then run via godot-runner (godot3-runner package in Debian).

BoFs

Python Team BoF

Link to the etherpad / pad archive link and video can be found on the talk page: https://debconf24.debconf.org/talks/31-python-bof/

The session ended up being extended to a second part, since all the issues didn’t fit into the first session.

I was distracted by too many thing during the Python 3.12 transition (to the point where I thought that 3.11 was still new in Debian), so it was very useful listening to the retrospective of that transition.

There was a discussion whether Python 3.13 could still make it to testing in time for freeze, and it seems that there is consensus that it can, although, likely with new experimental features like disabling the global interpreter lock and the just in time compiler disabled.

I learned for the first time about the “dead batteries” project, PEP-0594, which removes ancient modules that have mostly been superseded, from the Python standard library.

There was some talk about the process for changing team policy, and a policy discussion on whether we should require autopkgtests as a SHOULD or a MUST for migration to testing. As with many things, the devil is in the details and in my opinion you could go either way and achieve a similar result (the original MUST proposal allowed exceptions which imho made it the same as the SHOULD proposal).

There’s an idea to do some ongoing remote sprints, like having co-ordinated days for bug squashing / working on stuff together. This is a nice idea and probably a good way to energise the team and also to gain some interest from potential newcomers.

Louis-Philipe Véronneau was added as a new team admin and there was some discussion on various Sphinx issues and which Lintian tags might be needed for Python 3.13. If you want to know more, you probably have to watch the videos / read the notes :)

Debian.net BoF

Link to the etherpad / pad archive link can be found on the talk page: https://debconf24.debconf.org/talks/37-debiannet-team-bof

Debian Developers can set up services on subdomains on debian.net, but a big problem we’ve had before was that developers were on their own for hosting those services. This meant that they either hosted it on their DSL/fiber connection at home, paid for the hosting themselves, or hosted it at different services which became an accounting nightmare to claim back the used funds. So, a few of us started the debian.net hosting project (sometimes we just call it debian.net, this is probably a bit of a bug) so that Debian has accounts with cloud providers, and as admins we can create instances there that gets billed directly to Debian.

We had an initial rush of services, but requests have slowed down since (not really a bad thing, we don’t want lots of spurious requests). Last year we did a census, to check which of the instances were still used, whether they received system updates and to ask whether they are performing backups. It went well and some issues were found along the way, so we’ll be doing that again.

We also gained two potential volunteers to help run things, which is great.

Debian Social BoF

Link to the etherpad / pad archive link can be found on the talk page: https://debconf24.debconf.org/talks/34-debiansocial-bof

We discussed the services we run, you can view the current state of things at: https://wiki.debian.org/Teams/DebianSocial

Pleroma has shown some cracks over the last year or so, and there are some forks that seem promising. At the same time, it might be worth while considering Mastodon too. So we’ll do some comparison of features and maintenance and find a way forward. At the time when Pleroma was installed, it was way ahead in terms of moderation features.

Pixelfed is doing well and chugging along nicely, we should probably promote it more.

Peertube is working well, although we learned that we still don’t have all the recent DebConf videos on there. A bunch of other issues should be fixed once we move it to a new machine that we plan to set up.

We’re removing writefreely and plume. Nice concepts, but it didn’t get much traction yet, and no one who signed up for these actually used it, which is fine, some experimentation with services is good and sometimes they prove to be very popular and other times not.

The WordPress multisite instance has some mild use, otherwise haven’t had any issues.

Matrix ended up to be much, much bigger than we thought, both in usage and in its requirements. It’s very stateful and remembers discussions for as long as you let it, so it’s Postgres database is continuously expanding, this will also be a lot easier to manage once we have this on the new host.

Jitsi is also quite popular, but it could probably be on jitsi.debian.net instead (we created this on debian.social during the initial height of COVID-19 where we didn’t have the debian.net hosting yet), although in practice it doesn’t really matter where it lives.

Most of our current challenges will be solved by moving everything to a new big machine that has a few public IPs available for some VMs, so we’ll be doing that shortly.

Debian Foundation Discussion BoF

This was some brainstorming about the future structure of Debian, and what steps might be needed to get there. It’s way too big a problem to take on in a BoF, but we made some progress in figuring out some smaller pieces of the larger puzzle. The DPL is going to get in touch with some legal advisors and our trusted organisations so that we can aim to formalise our relationships a bit more by the time it’s DebConf again.

I also introduced my intention to join the Debian Partners delegation. When I was DPL, I enjoyed talking with external organisations who wanted to help Debian, but helping external organisations help Debian turned out to be too much additional load on the usual DPL roles, so I’m pursuing this with the Debian Partners team, more on that some other time.

This session wasn’t recorded, but if you feel like you missed something, don’t worry, all intentions will be communicated and discussed with project members before anything moves forward. There was a strong agreement in the room though that we should push forward on this, and not reach another DebConf where we didn’t make progress on formalising Debian’s structure more.

Social

Conference Dinner

Conference Dinner Photo from Santiago

The conference dinner took place in the university gymnasium. I hope not many people do sports there in the summer, because it got HOT. There was also some interesting observations on the thermodynamics of the attempted cooling solutions, which was amusing. On the plus side, the food was great, the company was good, and the speeches were kept to a minimum, so it was a great conference dinner, even though it was probably cut a bit short due to the heat.

Cheese and Wine

Cheese and Wine happened on 1 August, which happens to be the date I became a DD at DebConf17 in Montréal seven years before, so this was a nice accidental celebration of my Debiversary :)

Since I’m running out of time, I’ll add some more photos to this post some time after publishing it :P

Group Photo

As per DebConf tradition, Aigars took the group photo. You can find the high resolution version on Debian’s GitLab instance.

Debian annual conference Debconf 24, Busan, South Korea
Photography: Aigars Mahinovs aigarius@debian.org
License: CC-BYv3+ or GPLv2+

Talking

Ah yes, talking to people is a big part of DebConf, but I didn’t keep track of it very well.

• I mostly listened to Alper a bit about his ideas for his talk about debian installer.
• I talked to Rhonda a bit about ActivityPub and MQTT and whether they could be useful for publicising Debian activity.
• Listened to Gunnar and Julian have a discussion about GPG and APT which was interesting.
• I learned that you can learn Hangul, the Korean alphabet, in about an hour or so (I wish I knew that in all my years of playing StarCraft II).
• We had the usual continuous keysigning party. Besides it’s intended function, this is always a good ice breaker and a way to for shy people to meet other shy people.
• … and many other fly-by discussions.

Stuff that didn’t happen this DebConf

• loo.py – A simple Python script that could eventually replace the obs-advanced-scene-switcher sequencer in OBS. It would also be extremely useful if we’d ever replace OBS for loopy. I was hoping to have some time to hack on this, and try to recreate the current loopy in loo.py, but didn’t have the time.
• toetally – This year videoteam had to scramble to get a bunch of resistors to assemble some tally light. Even when assembled, they were a bit troublesome. It would’ve been nice to hack on toetally and get something ready for testing, but it mostly relies on having something like a rasbperry pi zero with an attached screen in order to work on further. I’ll try to have something ready for the next mini conf though.
• extrepo on debian live – I think we should have extrepo installed by default on desktop systems, I meant to start a discussion on this, but perhaps it’s just time I go ahead and do it and announce it.
• Live stream to peertube server – It would’ve been nice to live stream DebConf to PeerTube, but the dependency tree to get this going got a bit too huge. Following our plans discussed in the Debian Social BoF, we should have this safely ready before the next MiniDebConf and should be able to test it there.
• Desktop Egg – there was this idea to get a stand-in theme for Debian testing/unstable until the artwork for the next release is finalized (Debian bug: #1038660), I have an idea that I meant to implement months ago, but too many things got in the way. It’s based on Juliette Taka’s Homeworld theme, and basically transforms the homeworld into an egg. Get it? Something that hasn’t hatched yet? I also only recently noticed that we never used the actual homeworld graphics (featuring the world image) in the final bullseye release. lol.

So, another DebConf and another new plush animal. Last but not least, thanks to PKNU for being such a generous and fantastic host to us! See you again at DebConf25 in Brest, France next year!

The Drupal website experienced a brief outage, but the issue has been resolved by the team. Fran Garcia Linares provided updates during the downtime, assuring users that the team was working on restoring services.

Presenting "Self-hosted DDEV on GitHub Actions" by Jon Pugh at DrupalGovCon 2024 Jon Pugh Thu, 08/08/2024 - 07:18

Next week I'm headed to DrupalGovCon 2024 to present on a brand new technique I created for hosting fast and reliable preview/test environments using DDEV and GitHub Actions.

DDEV supports what they call "Casual Hosting". With the right config tweaks, you can run multiple DDEV sites on a server for hosting sites on the internet.

JAKALA, the leading data and AI-driven portfolio company of Ardian Buyout has acquired FFW, a key player in digital experience solutions. This acquisition, the largest non-publicly traded digital agency deal in Europe for 2023, expands JAKALA's global footprint and service offerings, pushing its workforce to over 3,000 professionals and turnover beyond €500 million. The merger aims to enhance client experiences through combined expertise in data, technology, and AI.

Akademy needs you! Volunteering is a great way to make new friends and Akademy wouldn't be possible without us all pitching in to make it happen. Find a task or two that sounds fun and sign yourself up! All you need to do is add yourself to a timeslot on the wiki page

Drupal and AEM are two popular content management systems used by many companies. They offer advanced features for creating, editing, and publishing content but differ in many ways. In this article, we will compare Drupal to AEM to help you find the right CMS for your needs.

DebConf24 is now over! I'm very happy I was able to attend this year. If you haven't had time to look at the schedule yet, here is a selection of talks I liked.

What happens if I delete setup.py?: a live demo of upgrading to PEP-518 Python packaging

A great talk by Weezel showcasing how easy it is to migrate to PEP-518 for existing Python projects.

This is the kind of thing I've been doing a lot when packaging upstream projects that still use setup.py. I encourage you to send this kind of patch upstream, as it makes everyone's life much easier.

Debian on Chromebooks: What's New and What's Next?

A talk by Alper Nebi Yasak, who has done great work on running Debian and the Debian Installer on Chromebooks.

With Chromebooks being very popular machines in schools, it's nice to see people working on a path to liberate them.

Sequoia PGP, sq, gpg-from-sq, v6 OpenPGP, and Debian

I had the chance to see Justus' talk on Sequoia — an OpenPGP implementation in Rust — at DebConf22 in Kosovo. Back then, the conclusion was that sq wasn't ready for production yet.

Well it seems it now is! This in-depth talk goes through the history of the project and its goals. There is also a very good section on the current OpenPGP/LibrePGP schism.

Chameleon - the easy way to try out Sequoia - OpenPGP written in Rust

A very short talk by Holger on Chameleon, a tool to make migration to Sequoia easier.

TL;DW: apt install gpg-from-sq

Protecting OpenPGP keyservers from certificate flooding

Although I used to enjoy signing people's OpenPGP keys, I completely gave up on this practice around 2019 when dkg's key was flooded with bogus certifications and have been refusing to do so since.

In this talk, Gunnar talks about his PhD work on fixing this issue and making sure we can eventually restore this important function on keyservers.

Bits from the DPL

Bits from the DPL! A DebConf classic.

Linux live patching in Debian

Having to reboot servers after kernel upgrades is a hassle, especially with machines that have encrypted disk drives.

Although kernel live patching in Debian is still a work in progress, it is encouraging to see people trying to fix this issue.

"I use Debian BTW": fzf, tmux, zoxide and friends

A fun talk by Samuel Henrique on little changes and tricks one can make to their setup to make life easier.

Ideas to Move Debian Installer Forward

Another in-depth talk by Alper, this time on the Debian Installer and his ideas to try to make it better. I learned a lot about the d-i internals!

Lightning Talks

Lighting talks are always fun to watch! This year, the following talks happened:

1. Customizing your Linux icons
2. A Free Speech tracker by SFLC.IN
3. Desktop computing is irrelevant
4. An introduction to wcurl
5. Aliasing in dpkg
6. A DebConf art space
7. Tiny Tapeout, Fomu, PiCI
8. Data processing and visualisation in the shell

Is there a role for Debian in the post-open source era?

As an economist, I've been interested in Copyright and business models in the Free Software ecosystem for a while. In this talk, Hatta-san and Bruce Perens discuss the idea of alternative licences that are not DFSG-free, like Post-Open.

I attended my first DebConf this year! It was held in Busan, South Korea, and the whole experience was a blast. It was really fun meeting new people, and I’m looking forward to DebConf 25 in Brest!

Here is a list of some of the things I did (and that I can remember) during DebCamp/DebConf:

• Reviewed pending merge requests for GCES students.
  • capistrano
  • aionotify
  • gtts-token
  • subliminal
• Helped with the ruby 3.3 transition.
  • ruby-rantly
  • ruby-liquid-c
• Started packaging build-deps for brutespray.
  • golang-github-atomicgo-cursor
  • golang-github-atomicgo-keyboard
  • golang-github-atomicgo-schedule
  • golang-github-pterm-pterm
  • golang-github-wenerme-astgo
• Prepared the Debian Brasilia section for the Bits from Brazil talk.
• Attended the Debian Curl Maintainers BoF.
• Tried to get curl pytest tests running when building the Curl package.
• Submitted my first patch to the Linux kernel (thanks, Helen!).
• Volunteered in the Video Team (I got a T-Shirt!).
• Participated in the (joke-ish) bid for DebConf Brasília.

The KIO Framework has gained support for de-facto standard, cross-desktop thumbnail generators. This means that we have a support for thumbnails from 3rd party applications! On Linux systems, many applications that produce some kind of output, such as a 3D file or text document, ship a thumbnailer file that tells file managers how to create thumbnails of their files. One specific example I've used here in the images are STL files, for which we don't have our own KDE-specific thumbnailer plugin.

These thumbnailer files are currently used by Nautilus and Thunar, so we felt like we were missing out and wanted to join the party! :)

Thumbnailer files

Thumbnailer files are simple text files that tell the system what program we should run to generate a thumbnail. You can check what thumbnailers you have installed by running ls /usr/share/thumbnailers

For example, the STL thumbnailer file looks like this:

[Thumbnailer Entry] TryExec=stl-thumb Exec=xvfb-run --auto-servernum -w 0 stl-thumb -f png -s %s %i %o MimeType=model/stl;model/x.stl-ascii;model/x.stl-binary;application/sla;

It tells the software running the thumbnailer what commands to use to generate the thumbnail, and what mimetypes it supports.

KDE Thumbnailer Plugins

On KDE side, we have used plugins for KIO, that reside in the kio-extras repository. They work just fine for our usecase in KDE apps, but nobody should need to write a KIO specific plugin for their application.

The changes to KIO

You can check the merge request for more in-depth details, but here's a summary of how I made it work side-by-side with our plugin system:

We utilize the KIO plugins always first if possible, since we know for sure they work. This is to avoid any possible regressions and oddities, and to keep the change as unintrusive as possible. When we encounter a mimetype that is not supported by our plugins, like STL files, we utilize a thumbnailer file instead.

This also means that it's transparent to users. Users do not have to worry which one they have installed.

Why make support for thumbnailer files then?

As mentioned earlier, no application should need to create a plugin for KIO just to make their thumbnails show up in our applications.

Thumbnailer files offer other benefits too, such as easing future transitions, (like from KF6 to KF7); working nicely with sandboxing, and being distributable in Flatpak bundles.

I am also working on moving our own plugins into thumbnailers, so we get the benefits from that too.

How can I test it out?

Currently it's only in the master branch of KIO, so if you really want to try it out, you will have to set up KDE Plasma development environment: https://develop.kde.org/docs/getting-started/building/kdesrc-build-setup/

When inside in the development environment, open Dolphin and enable the thumbnailers from preview settings.

Any help testing it would be very welcome! :) Let me know of any possible improvements and bugs!

The Chair of the Board of the OSI has acknowledged the resignation offered by Secretary of the Board, Aeva Black. The Chair and the entire Board would like to thank Black for their invaluable contribution to the success of OSI, as well of the entire Open Source Community, and for their service as board member and officer of the Initiative.

In lieu of a formal announcement about the Third Edition, how about a progress update?

Core technology updates: Django + Python

Embarrassment-Driven Development

One of the main motivations for a third edition was that the 2e is based on Django 1.11, which dropped out of support back in 2017, and that's been a big turnoff for readers for a while, and quite embarrassing really.

So, the plan is to upgrade to Django 5.x, and progress is good -- I've already updated most of the core chapters to Django 4.2, and upgrade Python to 3.12 while I was at it. Django 5 is next, and I'm hoping/assuming it will be a smaller leap that 1->4 was, so that won't be far behind.

New Deployment Technologies: Docker + Ansible

I've always been proud that the book includes several chapters on how to actually deploy our app to production, and make the app live on the actual public Internet. But the deployment process from the first and second editions--broadly speaking, SSH in to your server, hack about to figure out how to get your app deployed manually, and then automate what you did with glorified shellscripts, aka Fabric--was starting to look less and less like what modern deployment looks like, or my experience of it at least.

I uhmmed and ahhed about it for a while, but in the end I decided to go with a deployment process that looks like this:

• Package up our app into a Docker container, and use our tests to confirm it really works
• Use Ansible to automate pushing that container onto a server and running it.

Check out the latest version of the deployment chapters here:

• Chapter 9: Containerization aka Docker
• Chapter 10: Making our App Production-Ready
• Chapter 11: Infrastructure As Code: Automated Deployments With Ansbile

I think I like how it's turned out, a lot of the fiddliness and debugging of deployment/production-readiness can now happen locally (in Docker containers on your own machine), so I think that tightens and speeds up the feedback loop a fair bit.

JavaScript

The Javascript chapter was another head-scratcher. I wanted to move away from QUnit, and include some more modern/ES6 syntax. In the end, I decided to go with Jasmine, which is old but still popular, but to keep the browser-based test runner, which is a bit of an unconventional choice, but it does mean we can avoid the whole Node.js and node_modules learning curve.

Aside from that, I've wound down the "JavaScript is such a nightmare" jokes, because they're really not fair any more, and were probably never that funny besides.

Check out the new version here:

• Chapter 16: A Gentle Excursion Into JavaScript

Some changes of emphasis

The other main changes to the book are going to be around how I talk about some of the tradeoffs involved in the use of mocking, and unit vs integration vs functional/e2e tests. I think the first and second editions were perhaps a little too opinionated on this front (I still cringe to think how defensive I was when I first wrote the Hot Lava chapter, sorry CaseY!!), and my thinking has evolved a lot since I wrote my second book with Bob.

That's still very much on the drawing board though, so you'll have to watch this space for updates on that front.

Anyways, all the latest versions of the 3e chapters are live here on the site, and also as an Early Release on O'Reilly Learning, so do dive in and let me know what you think!

When you write asynchronous code in Python, you’ll likely need to create asynchronous iterators and iterables at some point. Asynchronous iterators are what Python uses to control async for loops, while asynchronous iterables are objects that you can iterate over using async for loops.

Both tools allow you to iterate over awaitable objects without blocking your code. This way, you can perform different tasks asynchronously.

In this tutorial, you’ll:

• Learn what async iterators and iterables are in Python
• Create async generator expressions and generator iterators
• Code async iterators and iterables with the .__aiter__() and .__anext__() methods
• Use async iterators in async loops and comprehensions

To get the most out of this tutorial, you should know the basics of Python’s iterators and iterables. You should also know about Python’s asynchronous features and tools.

Get Your Code: Click here to download the free sample code that you’ll use to learn about asynchronous iterators and iterables in Python.

Take the Quiz: Test your knowledge with our interactive “Asynchronous Iterators and Iterables in Python” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

Asynchronous Iterators and Iterables in Python

Take this quiz to test your understanding of how to create and use Python async iterators and iterables in the context of asynchronous code.

Getting to Know Async Iterators and Iterables in Python

Iterators and iterables are fundamental components in Python. You’ll use them in almost all your programs where you iterate over data streams using a for loop. Iterators power and control the iteration process, while iterables typically hold data that you want to iterate over.

Python iterators implement the iterator design pattern, which allows you to traverse a container and access its elements. To implement this pattern, iterators need the .__iter__() and .__next__() special methods. Similarly, iterables are typically data containers that implement the .__iter__() method.

Note: To dive deeper into iterators and iterables, check out the Iterators and Iterables in Python: Run Efficient Iterations tutorial.

Python has extended the concept of iterators and iterables to asynchronous programming with the asyncio module and the async and await keywords. In this scenario, asynchronous iterators drive the asynchronous iteration process, mainly powered by async for loops and comprehensions.

Note: In this tutorial, you won’t dive into the intricacies of Python’s asynchronous programming. So, you should be familiar with the related concepts. If you’re not, then you can check out the following tutorials:

• Getting Started With Async Features in Python
• Async IO in Python: A Complete Walkthrough

In these tutorials, you’ll gain the required background to prepare for exploring asynchronous iterators and iterables in more depth.

In the following sections, you’ll briefly examine the concepts of asynchronous iterators and iterables in Python.

Async Iterators

Python’s documentation defines asynchronous iterators, or async iterators for short, as the following:

An object that implements the .__aiter__() and .__anext__() [special] methods. .__anext__() must return an awaitable object. [An] async for [loop] resolves the awaitables returned by an asynchronous iterator’s .__anext__() method until it raises a StopAsyncIteration exception. (Source)

Similar to regular iterators that must implement .__iter__() and .__next__(), async iterators must implement .__aiter__() and .__anext__(). In regular iterators, the .__iter__() method usually returns the iterator itself. This is also true for async iterators.

To continue with this parallelism, in regular iterators, the .__next__() method must return the next object for the iteration. In async iterators, the .__anext__() method must return the next object, which must be awaitable.

Python defines awaitable objects as described in the quote below:

An object that can be used in an await expression. [It] can be a coroutine or an object with an .__await__() method. (Source)

In practice, a quick way to make an awaitable object in Python is to call an asynchronous function. You define this type of function with the async def keyword construct. This call creates a coroutine object.

Note: You can also create awaitable objects by implementing the .__await__() special method in a custom class. This method must return an iterator that yields control back to the event loop until the awaited result is ready. This topic is beyond the scope of this tutorial.

When the data stream runs out of data, the method must raise a StopAsyncIteration exception to end the asynchronous iteration process.

Here’s an example of an async iterator that allows iterating over a range of numbers asynchronously:

Read the full article at https://realpython.com/python-async-iterators/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]