Feeds

Note

Open-source efforts around scikit-learn at Inria are spinning off to a new enterprise, Probabl, in charge of sustainable development of a data-science commons.

Contents

• Prelude: funding scikit-learn is hard
• The birth of a new ambition
• Probabl, a mission-driven enterprise
• Probabl is already having an impact
• My position within Probabl, my vested interests
• More to come

Prelude: funding scikit-learn is hard

Scikit-learn is a central software component in today’s machine learning landscape, and it is open source, governed by a community, easy to install, and well documented. It started many years ago as a project that we did on the side, and we were joined by many volunteers, which was key to the success of the project. We soon decided to ensure that scikit-learn was not only a volunteer-based effort. Over more than a decade, I’ve dedicated a lot of energy to this, using a variety of funding mechanisms: first grants (as an academic), then sponsoring and related contracts with various actors.

Digital commons eliminate scarcity and exclusivity

Funding digital commons is really hard. People build fortunes by leveraging competitive advantages, by creating lock-ins, or selling access to data. What makes a great open-source library, as scikit-learn, is exactly what prevents these tricks: we are committed to being independent, easy to use and install, lightweight…

The birth of a new ambition

Scikit-learn is very successful, but it could be more. For instance, it does not facilitate pushing to production as much as tensorflow, which can be served, deployed to android… And scikit-learn is not very visible to top decision makers: it’s not a line on their budget, a brand that they know. As a consequence, it is not reaping the benefit of its success [1].

[1]Many commercial tools are sitting on top of open source software like scikit-learn (splunk, sagemaker, to name only a few), making profits, and not helping in any way the open source world that they build upon. The French government is backing us to push the envelope

3 years ago, the French government challenged us to go further, to consolidate the ecosystem into a consistent data-science commons. The strategic interest of France is to preserve some technological autonomy on data, eg sensitive data. Thus, the government offered us, at Inria, a funding opportunity to go further.

They promised us a lot of money (dozens of millions of Euros), but with a specific mission to develop a sustainable “data-science commons” [2] ecosystem around scikit-learn. I’ll spare you the details of the amount of meetings we had, documents that we wrote, to sketch the outline of the project. I pushed forward a vision of technical components that fit in the broader open-source ecosystem, complementing it.

[2]The letter that we received from the French government specifically defines the objective in these words: “data-science common” (“Communs numériques pour la Science des Données”)

As I moved forward, I faced a difficulty: the French government wanted a sustainability plan, and private investment to back it. To be honest, this is not what I’m good at. François Goupil, the COO of the scikit-learn consortium, was helping me, but we needed more for our ambitions. And this is when we started talking to Yann Lechelle, a tech entrepreneur with an impressive track record interested in the impact of France on the global tech world.

Probabl, a mission-driven enterprise

With Yann, we built a new vision. Our challenge is to be long-term sustainable and virtuous for scikit-learn, its broader ecosystem, and its community. Yann brought in a business point of view, and I tried to bring that of open-source communities beyond probabl [3], for instance avoiding to getting in the way of others building businesses that contribute to scikit-learn. Indeed, we are convinced that having a broad and diverse community around scikit-learn is central to its future.

[3]One of the first things that Probabl did (Guillaume Lemaître, to be specific), was submit a grant application (to the Chang-Zuckenberg Institute), to fund, via NumFocus, a developer employed by Quantsight, with no money transiting via Probabl (one reason being that we have no operations outside of Europe so far).

Our sustainability model is still being finetuned. What I can tell is that it will involve a mix of professional service, support & sponsorship agreement, as well as a product-based offer, where we supplement scikit-learn with enterprise features. Our focus will be on features that are typically not the focus of open-source developers: integration in large structures, such as access control, LDAP connection, regulatory compliance. We will not shoehorn scikit-learn in open core or dual licensing approaches: we want our incentives to be aligned with scikit-learn, and its ecosystem, being as complete as possible.

Foster growth and adoption of our open-source stack

In a sense, our inspiration is that of RedHat, where the growth of the company fosters the growth and adoption of the software (Linux in the case of RedHat), beyond the company, in an ecosystem, and for a wide variety of applications.

Strong growth will mean external capital. To ensure that we do not lose the focus on our mission, building data-science commons, Yann penciled down a specific governance of the company (and then validated it with many people, as we are a spin-off from a governmental organization). The ultimate share structure, and the board, are divided in three electoral colleges: one for outside investors, one for founders and employees, and one for public institutions. This ensures a balance of power that hopefully will keep us aligned to our mission. I think that this structure sends a strong signal that we are not just another for-profit that will go from creating useful tech to dark money-generating patterns.

Probabl is already having an impact

A strong open-source team In February, the whole team developing scikit-learn at Inria moved to Probabl, joined by Adrin Jalali, a Berlin-based core developer of scikit-learn and fairlearn. We’ve been hiring excellent people, and we now have 9 people on open-source (see the Probabl team), spending their time contributing to open source (Jérémie, for instance, has been doing the last releases for scikit-learn).

Fostering an ecosystem Probabl is not only about scikit-learn. We are prioritizing 8 libraries, central to the machine-learning and data science ecosystem: joblib, fairlearn, imbalanced-learn… In general, as we have always done, we will not hesitate contributing to upstream or related projects. Our goal is to have a healthy open-source ecosystem around data-science.

Not only software Not everybody sees the important lines of code. I’ve become increasingly aware of the need to do outreach and communication, to coders, but also to decision makers. At Probabl we dedicate energy to be in business meetings, to participate in the tech narrative, to teach how to best do data science, eg with didactic videos. We’re starting a mentioning program, we’ll be organizing sprints… I am convinced that all this is a useful long-term investment.

My position within Probabl, my vested interests

I am a French civil servant (a researcher at Inria, one of our national research institute). Such a position comes with strong responsibilities to control conflicts of interest. The creation of Probabl underwent strict scrutiny (that took a long long time). I have been recently cleared to take an active role: 10% of my time is allocated to be a scientific and open-source advisor for Probabl.

I am not paid by Probabl. 100% of my salary comes from Inria (and I was not given a raise because of my involvement in Probabl). I do have financial interests as a founder, but given that I have a small active part, I have one of the smallest amount of shares among founders.

My main interest in Probabl is really the success of its mission: the long-term growth of an open-source data-science ecosystem. Spinning-off from Inria actually continues my efforts in this direction, but with more agility and breadth. And having on top of open source a variety of complementary commercial activities makes it stronger, by answering better the needs of some actors.

More to come

There are many things that we are still ironing. Clearing out specific details takes time (for instance, clearing my role took a while). We are still to announce the future of the sponsorship program that we had set up at the Inria foundation. Its mission has been transferred to Probabl. Currently, Probabl’s open source team is ensuring continuity of our work with the existing sponsors. But we will set up broader partnership opportunities, with a similar governance, that enable third-parties to invest in open source on a roadmap decided jointly with the open-source community.

I believe that we need a lot of transparency in how we decide upon priorities in our open source team. Our 2024 priorities for scikit-learn are visible here.

I look forward to when Probabl will start adding value to scikit-learn for enterprises with an offer enriching scikit-learn and the broader open-source ecosystem.

I am acutely aware that good open source is made of communities, and that communities need trust and understanding of big players such as Probabl (well, so far we are not that big). I hope that with time our actions will become easy to read and speak of themselves.

Creating performance tests for Python Morsels exercises is a frequent annoyance

I loathe writing automated tests for performance-related exercises because they’re always flaky. How flaky depends on the exercise, what I’m testing, and the time variability inherent in the particular Python features that a learner might use.

I came up with a solution for flaky tests recently, but it also makes my tests less readable. I then came up with a tool to improve the readability, but that has its own trade-offs.

The code I eventually came up with is a beautiful Python monstrosity.

1 2 3 4 5 6 @attempt_n_times(10) def _(): nonlocal micro_time, tiny_time micro_time = time(micro_numbers) tiny_time = time(tiny_numbers) self.assertLess(tiny_time, micro_time*n)

I’ll explain what that code does, but first let’s talk about why it’s needed.

The flaky performance tests

My flaky performance tests initially looked like this:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 def test_some_test(self): n, m = 2.45, 2.04 micro_time = time(micro_numbers) tiny_time = time(tiny_numbers) self.assertLess(tiny_time, micro_time*n) small_time = time(small_numbers) self.assertLess(small_time, tiny_time*n) self.assertLess(small_time, micro_time*n*m) medium_time = time(medium_numbers) self.assertLess(medium_time, micro_time*n*m*m) self.assertLess(medium_time, tiny_time*n*m) self.assertLess(medium_time, small_time*n)

The first block runs a performance test for the user’s function on a very small list and on a slightly larger list and then asserting that the slightly larger list didn’t take too much longer to run. The next two blocks run the same code on even larger lists and make further assertions about the relative times that the code took to run.

This roughly approximates the time complexity of this code.

Running performance checks in a loop

These performance checks need to:

1. Predictably fail for inefficient solutions
2. Predictably pass for efficient solutions
3. Run fast (within just a few seconds) even when the code is inefficient
4. Avoid the use of threading because they’ll be running on WebAssembly in the browser
5. Run consistently on pretty much any computer

These 5 requirements together have caused me countless headaches. I get the tests passing well, but they don’t always fail when they should. I get the tests failing and passing when they should, but then they’re too slow. And so on…

Notice the n and m factors in the above assertions:

1 self.assertLess(small_time, micro_time*n*m)

If n and m are too big, we’ll get false positives (tests passing when they should fail). If n and m are too small, we’ll get false negatives (tests failing when they should pass).

To avoid both Type I and Type II errors, I decided to keep n and m small but attempt the assertion block multiple times.

Here’s the (far less flaky) revised code:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 def test_some_test(self): n, m = 2.45, 2.04 for attempts_left in reversed(range(10)): try: micro_time = time(micro_numbers) tiny_time = time(tiny_numbers) self.assertLess(tiny_time, micro_time*n) break except AssertionError: if attempts_left == 0: raise for attempts_left in reversed(range(5)): try: small_time = time(small_numbers) self.assertLess(small_time, tiny_time*n) self.assertLess(small_time, micro_time*n*m) break except AssertionError: if attempts_left == 0: raise for attempts_left in reversed(range(3)): try: medium_time = time(medium_numbers) self.assertLess(medium_time, micro_time*n*m*m) self.assertLess(medium_time, tiny_time*n*m) self.assertLess(medium_time, small_time*n) break except AssertionError: if attempts_left == 0: raise

The for loop runs the code multiple times, the break statement stops the code as soon as the assertions all pass, and the except and if ensure that any assertion errors are suppressed until/unless we’re on the final iteration of the loop.

Let’s call this a for-try-break-except-if-raise pattern. It’s an absurdly verbose name fitting of absurdly verbose code.

This for-try-break-except-if-raise pattern works pretty well! But it’s not pretty.

Like many programmers, I believe that Don’t Repeat Yourself (DRY) need not apply to tests. Tests are allowed to be repetitive if the verbosity improves readability.

But there is so much noise in that code! I decided that removing some noise might improve readability. So I devised a helper utility to reduce the repetition.

In search of a solution

While pondering the repetitive noise in this code, I wondered what Python features I could use to abstract away this for-try-break-except-if-raise pattern.

Could I make a context manager and use a with block? That might help with the try-except, but context managers can’t run their code block multiple times, so that wouldn’t help with the for and the break. So a context manager is out.

Could I abstract this away into a looping helper by implementing a generator function? We are looping and generator functions can break early. But, a generator function can’t catch an exception that’s raised within the body of a loop. So a generator function wouldn’t work either.

What about a decorator? 🤔

Context managers and decorators both sandwich a block of code. But decorators sandwich functions and they have the power to run the same function repeatedly. A decorator might work!

Here’s a decorator that will run a given function up to 10 times (until no AssertionError is raised):

1 2 3 4 5 6 7 8 9 def try_10_times(function): def wrapper(): for attempts_left in reversed(range(10)): try: return function() except AssertionError: if attempts_left == 0: raise return wrapper

To use this decorator, we would need to define a function and then call that function:

1 2 3 4 5 6 7 @try_10_times def assertions(): micro_time = time(micro_numbers) tiny_time = time(tiny_numbers) self.assertLess(tiny_time, micro_time*n) assertions()

This isn’t quite good enough though…

1. We need a pattern to run code N times (not necessarily exactly 10)
2. We reference the variables defined in each block in later blocks, so micro_time and tiny_time will need to be available outside that function
3. We need this function to run just one time right after it’s defined… could we do that automatically?

All 3 of these problems are solvable:

1. We need a decorator that accepts arguments
2. We need to use rarely seen nonlocal statement
3. We could have the decorator automatically call the decorated function

The final weird decorator

Here’s the decorator I ended up with:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 def attempt_n_times(n): """ Run tests multiple times if assertions are raised. Allows for more forgiving tests when assertions may be a bit flaky. """ def decorator(function): """This looks like a decorator, but it actually runs the function!""" for attempts_left in reversed(range(n)): try: return function() except AssertionError: if attempts_left == 0: raise return decorator

This decorator accepts an n argument which determines the maximum number of times the decorated function should be called. The decorator then calls the function repeatedly in a for loop and a try-except block. As soon as an AssertionError is not raised during one of these function calls, the looping stops.

The weirdest part about this decorator is that it calls the decorated function. Note that the decorator function doesn’t define a wrapper function within itself… it just runs code right away!

The resulting beautiful Python monstrosity

Here’s the final refactored test code:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 def test_some_test(self): n, m = 2.45, 2.04 micro_time = tiny_time = small_time = medium_time = 0 @attempt_n_times(10) def _(): nonlocal micro_time, tiny_time micro_time = time(micro_numbers) tiny_time = time(tiny_numbers) self.assertLess(tiny_time, micro_time*n) @attempt_n_times(5) def _(): nonlocal small_time small_time = time(small_numbers) self.assertLess(small_time, tiny_time*n) self.assertLess(small_time, micro_time*n*m) @attempt_n_times(3) def _(): nonlocal medium_time medium_time = time(medium_numbers) self.assertLess(medium_time, micro_time*n*m*m) self.assertLess(medium_time, tiny_time*n*m) self.assertLess(medium_time, small_time*n)

The attempt_n_times decorator immediately calls the function it decorates. Each function is defined and immediately called one or more times, in a try-except block within a loop.

That’s why we’ve named these functions with the throwaway _ name: we don’t care about the name of a function we’re never going to refer to again.

Also note the use of the nonlocal statement. Each function in Python has its own scope and all assignments assign to the local scope by default. That nonlocal variable pulls those variables to the scope of the outer function instead.

Compare the above code to the code just before this refactor:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 def test_some_test(self): n, m = 2.45, 2.04 for attempts_left in reversed(range(10)): try: micro_time = time(micro_numbers) tiny_time = time(tiny_numbers) self.assertLess(tiny_time, micro_time*n) break except AssertionError: if attempts_left == 0: raise for attempts_left in reversed(range(5)): try: small_time = time(small_numbers) self.assertLess(small_time, tiny_time*n) self.assertLess(small_time, micro_time*n*m) break except AssertionError: if attempts_left == 0: raise for attempts_left in reversed(range(3)): try: medium_time = time(medium_numbers) self.assertLess(medium_time, micro_time*n*m*m) self.assertLess(medium_time, tiny_time*n*m) self.assertLess(medium_time, small_time*n) break except AssertionError: if attempts_left == 0: raise

I find the refactored version easier to skim.

But that attempt_n_times decorator does abuse the decorator syntax. Decorators aren’t meant to call the function they’re decorating.

Is this misuse of decorators worth it?

Is this worth it?

Decorators aren’t supposed to immediately call the function they decorate. But there’s nothing stopping them from doing so. I feel that I’ve traded “normal code” for a beautiful monstrosity that’s easier to skim at a glance.

The attempt_n_times decorator is pretending that it’s a block-level tool by using a function because there’s no other way to invent such a tool in Python.

I think abstracting away the for-try-break-except-if-raise pattern was worth it, even though I ended up abusing Python’s decorator syntax in the process.

What do you think? Was that attempt_n_times abstraction worth it?

In this video I introduce Selenium AT-SPI for testing KDE applications. I present the KDE goals of sustainable software, accessibility, and automation in system testing, and how Selenium helps achieve all of them.

Selenium AT-SPI is an amazing piece of software written by KDE developer Harald Sitter. It is a tool used in KDE to automate tests of GUI applications. This enables developers to design applications that are accessible for all and increase their energy efficiency. As part of Season of KDE 2024 I decided to make a video tutorial for KDE developers.

If you find this video helpful, you can reach out to me on Gitter. I would love to hear back from you 😃

This video was made by Pradyot Ranjan (@pradyotranjan:gitter.im).

• Part 2: Setting up Selenium
• Part 3: Identifying Accessibility Issues
• Part 4: Writing Selenium Tests

In this video I deep dive into writing tests with Selenium. I create a simple test for the KCalc calculator application and run it with Selenium AT-SPI. Similar steps can be followed to write GUI tests for any KDE application.

Selenium AT-SPI is an amazing piece of software written by KDE developer Harald Sitter. It is a tool used in KDE to automate tests of GUI applications. This enables developers to design applications that are accessible for all and increase their energy efficiency. As part of Season of KDE 2024 I decided to make a video tutorial for KDE developers.

If you find this video helpful, you can reach out to me on Gitter. I would love to hear back from you 😃

This video was made by Pradyot Ranjan (@pradyotranjan:gitter.im).

• Part 1: An Introduction to Selenium
• Part 2: Setting up Selenium
• Part 3: Identifying Accessibility Issues

In this video I explain how the accerciser utility works. Accerciser is a tool used to identify and test GUI accessibility elements. I also run accerciser on KCalc, the KDE calculator application.

Selenium AT-SPI is an amazing piece of software written by KDE developer Harald Sitter. It is a tool used in KDE to automate tests of GUI applications. This enables developers to design applications that are accessible for all and increase their energy efficiency. As part of Season of KDE 2024 I decided to make a video tutorial for KDE developers.

If you find this video helpful, you can reach out to me on Gitter. I would love to hear back from you 😃

This video was made by Pradyot Ranjan (@pradyotranjan:gitter.im).

• Part 1: An Introduction to Selenium
• Part 2: Setting up Selenium
• Part 4: Writing Selenium Tests

In this video I set up Selenium AT-SPI locally on KDE Neon. This video follows the Selenium setup guide found here: https://community.kde.org/Selenium.

Selenium AT-SPI is an amazing piece of software written by KDE developer Harald Sitter. It is a tool used in KDE to automate tests of GUI applications. This enables developers to design applications that are accessible for all and increase their energy efficiency. As part of Season of KDE 2024 I decided to make a video tutorial for KDE developers.

If you find this video helpful, you can reach out to me on Gitter. I would love to hear back from you 😃

This video was made by Pradyot Ranjan (@pradyotranjan:gitter.im).

• Part 1: An Introduction to Selenium
• Part 3: Identifying Accessibility Issues
• Part 4: Writing Selenium Tests

G2 posted some fun logos for Kate on reddit.

I think they are nice and flashy and well suited if you want to show your appreciation for Kate and like that art style and a good addition to our awesome icon and mascot.

Static Version Animated Version Licensing

G2 licensed these files under the CC BY-NC-SA 4.0. Feel free to share the stuff with this license and credit for G2.

Comments?

A matching thread for this can be found here on r/KDE.

FTP master

This month I accepted 347 and rejected 49 packages. The overall number of packages that got accepted was 348.

Debian LTS

This was my hundred-nineteenth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian.

During my allocated time I uploaded or worked on:

• [#1070154] bullseye-pu: qtbase-opensource-src/5.15.2+dfsg-9+deb11u1 package upload
• [#1064550] bullseye-pu: libjwt 1.10.2-1+deb11u1 has been marked for accept
• [#1067544] bullseye-pu: libmicrohttpd 0.9.72-2+deb11u1 has been marked for accept

I also continued to work on tiff and last but not least did a week of FD and attended the monthly LTS/ELTS meeting.

Unfortunately I used lots of time to debug an issue with nghttp2. Please see my odyssey below.

Debian ELTS

This month was the seventieth ELTS month. During my allocated time I uploaded:

• [ELA-1104-1-1]nghttp2 security update for one CVEs to fix an DoS resulting from bad handling of CONTINUATION frames in Stretch

For some tests I installed the new nghttp2 package on my Stretch VM and started the daemon. Unfortunately I got an unexpected error from getaddrinfo() about ai_socktype not supported. The daemon was configured to listen on lo, the device was available, but the error remained. I was pretty sure that my patch was not the reason for this and indeed the unpatched version showed this error as well. I didn’t want to release an untested package, so nghttp2 had to start at least! Therefore I built a minimal example to reproduce the issue. getaddrinfo() failed for hints.ai_socktype=SOCK_STREAM and a numerical IP address. Having no hints at all or “localhost” instead of “127.0.0.1” made the error disappear (as a remark: “localhost” resolves to 127.0.0.1, the ipv6 variant is “ip6-localhost”). I could see that in nghttp2 as well. Configuring it with “localhost” let the error vanish but the daemon still exited due to other reasons. After some time of debugging, I added another network interface to my VM and configured it with a dummy IPv4 address. Voila, everything worked as expected. According to Wikipedia, IPv6 was ratified as standard in 2017 and Stretch was also released in 2017. No wonder that a IPv6-only-VM had problems back then and these problems survived to the present.

I also continued to work on an update for tiff in Jessie and Stretch, did a week of FD and attended the LTS/ELTS meeting.

Debian Printing

This month I uploaded new upstream or bugfix versions of:

• … cups-bjnp

This work is generously funded by Freexian!

Debian Astro

This month I uploaded a new upstream or bugfix version of:

• … astap
• … astap-cli
• … c-munipack
• … libahp-xc
• … planetary-system-stacker
• … fxload
• … virtualgps

Debian IoT

This month I uploaded new upstream or bugfix versions of:

• … pywws
• … openzwave
• … openoverlayrouter

Debian Mobcom

Due to more and more problems with time_t, I removed osmo-iuh and all dependencies from armel, armhf and i386, sorry. If there is really anybody using this software on 32-bit architectures don’t hesitate to get in touch.

It is official now, the GSoC student working on the Mobcom packages is Nathan Doris. He already finished the hardest part of the job and I could upload the latest version of libosmocore. I really enjoy working with him and look forward to a pleasant SoC :-).

misc

This month I uploaded new upstream or bugfix versions of:

• … dateutils
• … chktex
• … liburjtag
• … pollinate
• … usb-modeswitch
• … greylistd
• … entropybroker

Did I already mention that I love lists with topics I can work on. I print out such lists and enjoy checking off one after the other. End of May Helmut told me that I am a bit lazy and gave me such a list with all my packages that have one or the other issue with /usr-move. Most of the uploads above are packages on that list and I could check off a lot :-).

Welcome to the May 2024 report from the Reproducible Builds project! In these reports, we try to outline what we have been up to over the past month and highlight news items in software supply-chain security more broadly. As ever, if you are interested in contributing to the project, please visit our Contribute page on our website.

Table of contents:

1. A peek into build provenance for Homebrew
2. Distribution news
3. Mailing list news
4. Miscellaneous news
5. Two new academic papers
6. diffoscope
7. Website updates
8. Upstream patches
9. Reproducibility testing framework

A peek into build provenance for Homebrew

Joe Sweeney and William Woodruff on the Trail of Bits blog wrote an extensive post about build provenance for Homebrew, the third-party package manager for MacOS. Their post details how each “bottle” (i.e. each release):

[…] built by Homebrew will come with a cryptographically verifiable statement binding the bottle’s content to the specific workflow and other build-time metadata that produced it. […] In effect, this injects greater transparency into the Homebrew build process, and diminishes the threat posed by a compromised or malicious insider by making it impossible to trick ordinary users into installing non-CI-built bottles.

The post also briefly touches on future work, including work on source provenance:

Homebrew’s formulae already hash-pin their source artifacts, but we can go a step further and additionally assert that source artifacts are produced by the repository (or other signing identity) that’s latent in their URL or otherwise embedded into the formula specification.

Distribution news

In Debian this month, Johannes Schauer Marin Rodrigues (aka josch) noticed that the Debian binary package bash version 5.2.15-2+b3 was “uploaded to the archive twice. Once to bookworm and once to sid but with differing content.” This is problem for reproducible builds in Debian due its assumption that the package name, version and architecture triplet is unique. However, josch highlighted that

This example with bash is especially problematic since bash is Essential:yes, so there will now be a large portion of .buildinfo files where it is not possible to figure out with which of the two differing bash packages the sources were compiled.

In response to this, Holger Levsen performed an analysis of all .buildinfo files and found that this needs almost 1,500 binNMUs to fix the fallout from this bug.

Elsewhere in Debian, Vagrant Cascadian posted about a Non-Maintainer Upload (NMU) sprint to take place during early June, and it was announced that there is now a #debian-snapshot IRC channel on OFTC to discuss the creation of a new source code archiving service to, perhaps, replace snapshot.debian.org. Lastly, 11 reviews of Debian packages were added, 15 were updated and 48 were removed this month adding to our extensive knowledge about identified issues. A number of issue types have been updated by Chris Lamb as well. […][…]

Elsewhere in the world of distributions, deep within a larger announcement from Colin Percival about the release of version 14.1-BETA2, it was mentioned that the FreeBSD kernels are now built reproducibly.

In Fedora, however, the change proposal mentioned in our report for April 2024 was approved, so, per the ReproduciblePackageBuilds wiki page, the add-determinism tool is now running in new builds for Fedora 41 (‘rawhide’). The add-determinism tool is a Rust program which, as its name suggests, adds determinism to files that are given as input by “attempting to standardize metadata contained in binary or source files to ensure consistency and clamping to $SOURCE_DATE_EPOCH in all instances”. This is essentially the Fedora version of Debian’s strip-nondeterminism. However, strip-nondeterminism is written in Perl, and Fedora did not want to pull Perl in the buildroot for every package. The add-determinism tool eliminates many causes of non-determinism and work is ongoing to continue the scope of packages it can operate on.

Mailing list news

On our mailing list this month, regular contributor kpcyrd wrote to the list with an update on their source code indexing project, whatsrc.org. The whatsrc.org project, which was launched last month in response to the XZ Utils backdoor, now contains and indexes almost 250,000 unique source code archives. In their post, kpcyrd gives an example of its intended purpose, noting that it shown that whilst “there seems to be consensus about [the] source code for zsh 5.9” in various Linux distributions, it “does not align with the contents of the zsh Git repository”.

Holger Levsen also posted to the list with a ‘pre-announcement’ of sorts for the 2024 Reproducible Builds summit. In particular:

[Whilst] the dates and location are not fixed yet, however if you don’ help us with finding a suitable location soon, it is very likely that we’ll meet again in Hamburg in the 2nd half of September 2024 […].

Lastly, Frederic-Emmanuel Picca wrote to the list asking for help understanding the “non-reproducible status of the Debian silx package” and received replies from both Vagrant Cascadian and Chris Lamb.

Miscellaneous news

strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build. This month strip-nondeterminism version 1.14.0-1 was uploaded to Debian unstable by Chris Lamb chiefly to incorporate a change from Alex Muntada to avoid a dependency on Sub::Override to perform monkey-patching and break circular dependencies related to debhelper […]. Elsewhere in our tooling, Jelle van der Waa modified reprotest because the pipes module will be removed in Python version 3.13 […].

It was also noticed that a new blog post by Daniel Stenberg detailing “How to verify a Curl release” mentions the SOURCE_DATE_EPOCH environment variable. This is because:

The [curl] release tools document also contains another key component: the exact time stamp at which the release was done – using integer second resolution. In order to generate a correct tarball clone, you need to also generate the new version using the old version’s timestamp. Because the modification date of all files in the produced tarball will be set to this timestamp.

Furthermore, Fay Stegerman filed a bug against the Signal messenger app for Android to report that their ‘reproducible’ builds cannot, in fact, be reproduced. However, Fay is quick to note that she has:

… found zero evidence of any kind of compromise. Some differences are yet unexplained but everything I found seems to be benign. I am disappointed that Reproducible Builds have been broken for months but I have zero reason to doubt Signal’s security in any way.

Lastly, it was observed that there was a concise and diagrammatic overview of “supply chain threats” on the SLSA website.

Two new academic papers

Two new scholarly papers were published this month.

Firstly, Mathieu Acher, Benoît Combemale, Georges Aaron Randrianaina and Jean-Marc Jézéquel of University of Rennes on Embracing Deep Variability For Reproducibility & Replicability. The authors describe their approach as follows:

In this short [vision] paper we delve into the application of software engineering techniques, specifically variability management, to systematically identify and explicit points of variability that may give rise to reproducibility issues (e.g., language, libraries, compiler, virtual machine, OS, environment variables, etc.). The primary objectives are: i) gaining insights into the variability layers and their possible interactions, ii) capturing and documenting configurations for the sake of reproducibility, and iii) exploring diverse configurations to replicate, and hence validate and ensure the robustness of results. By adopting these methodologies, we aim to address the complexities associated with reproducibility and replicability in modern software systems and environments, facilitating a more comprehensive and nuanced perspective on these critical aspects.

(A PDF of this article is available.)

Secondly, Ludovic Courtès, Timothy Sample, Simon Tournier and Stefano Zacchiroli have collaborated to publish a paper on Source Code Archiving to the Rescue of Reproducible Deployment. Their paper was motivated because:

The ability to verify research results and to experiment with methodologies are core tenets of science. As research results are increasingly the outcome of computational processes, software plays a central role. GNU Guix is a software deployment tool that supports reproducible software deployment, making it a foundation for computational research workflows. To achieve reproducibility, we must first ensure the source code of software packages Guix deploys remains available.

(A PDF of this article is also available.)

diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made a number of changes such as uploading versions 266, 267, 268 and 269 to Debian, making the following changes:

• New features:

  • Use xz --list to supplement output when comparing .xz archives; essential when metadata differs. (#1069329)
  • Include xz --verbose --verbose (ie. double) output. (#1069329)
  • Strip the first line from the xz --list output. […]
  • Only include xz --list --verbose output if the xz has no other differences. […]
  • Actually append the xz --list after the container differences, as it simplifies a lot. […]

• Testing improvements:

  • Allow Debian testing to fail right now. […]
  • Drop apktool from Build-Depends; we can still test APK functionality via autopkgtests. (#1071410)
  • Add a versioned dependency for at least version 5.4.5 for the xz tests as they fail under (at least) version 5.2.8. (#374)
  • Fix tests for 7zip 24.05. […][…]
  • Fix all tests after additon of xz --list. […][…]

• Misc:

  • Update copyright years. […]

In addition, James Addison fixed an issue where the HTML output showed only the first difference in a file, while the text output shows all differences […][…][…], Sergei Trofimovich amended the 7zip version test for older 7z versions that include the string “[64]“ […][…] and Vagrant Cascadian relaxed the versioned dependency to allow version 5.4.1 for the xz tests […] and proposed updates to guix for versions 267, 268 and pushed version 269 to Guix. Furthermore, Eli Schwartz updated the diffoscope.org website in order to explain how to install diffoscope on Gentoo […].

Website updates

There were a number of improvements made to our website this month, including Chris Lamb making the “print” CSS stylesheet nicer […]. Fay Stegerman made a number of updates to the page about the SOURCE_DATE_EPOCH environment variable […][…][…] and Holger Levsen added some of their presentations to the “Resources” page. Furthermore, IOhannes zmölnig stipulated support for SOURCE_DATE_EPOCH in clang version 16.0.0+ […], Jan Zerebecki expanded the “Formal definition” page and fixed a number of typos on the “Buy-in” page […] and Simon Josefsson fixed the link to Trisquel GNU/Linux on the “Projects” page […].

Upstream patches

This month, we wrote a number of patches to fix specific reproducibility issues, including:

• Bernhard M. Wiedemann:

  • nauty (CPU-detection issue)
  • emacs (ASLR)

• Chris Lamb:

  • #1070754 filed against gensio.
  • #1071064 filed against tkgate.
  • #1072094 filed against ruby-pgplot.

Reproducibility testing framework

The Reproducible Builds project operates a comprehensive testing framework running primarily at tests.reproducible-builds.org in order to check packages and other artifacts for reproducibility. In May, a number of changes were made by Holger Levsen:

• Debian-related changes:

  • Enable the rebuilder-snapshot API on osuosl4. […]
  • Schedule the i386 architecture a bit more often. […]
  • Adapt cleanup_nodes.sh to the new way of running our build services. […]
  • Add 8 more workers for the i386 architecture. […]
  • Update configuration now that the infom07 and infom08 nodes have been reinstalled as “real” i386 systems. […]
  • Make diffoscope timeouts more visible on the #debian-reproducible-changes IRC channel. […]
  • Mark the cbxi4a-armhf node as down. […][…]
  • Only install the hdmi2usb-mode-switch package only on Debian bookworm and earlier […] and only install the haskell-platform package on Debian bullseye […].

• Misc:

  • Install the ntpdate utility as we need it later. […]
  • Document the progress on the i386 architecture nodes at Infomaniak. […]
  • Drop an outdated and unnoticed notice. […]
  • Add live_setup_schroot to the list of so-called “zombie” jobs. […]

In addition, Mattia Rizzolo reinstalled the infom07 and infom08 nodes […] and Vagrant Cascadian marked the cbxi4a node as online […].

If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

• IRC: #reproducible-builds on irc.oftc.net.

• Twitter: @ReproBuilds

• Mastodon: @reproducible_builds@fosstodon.org

• Mailing list: rb-general@lists.reproducible-builds.org

There hasn't been a boom like the AI boom since the .com days. And it may look like a space destined to be controlled by a couple of tech giants. But Ines Montani thinks open source will play an important role in the future of AI. I hope you join us for this excellent conversation about the future of AI and open source.<br/> <br/> <strong>Episode sponsors</strong><br/> <br/> <a href='https://talkpython.fm/sentry'>Sentry Error Monitoring, Code TALKPYTHON</a><br> <a href='https://talkpython.fm/porkbun'>Porkbun</a><br> <a href='https://talkpython.fm/training'>Talk Python Courses</a><br/> <br/> <strong>Links from the show</strong><br/> <br/> <div><b>Ines Montani on Twitter</b>: <a href="https://twitter.com/_inesmontani" target="_blank" rel="noopener">@_inesmontani</a><br/> <b>spaCy</b>: <a href="https://spacy.io" target="_blank" rel="noopener">spacy.io</a><br/> <b>Prodigy App</b>: <a href="https://prodi.gy" target="_blank" rel="noopener">prodi.gy</a><br/> <b>Ines' presentation at PyCon Lithuania</b>: <a href="https://www.youtube.com/watch?v=SsnDN7LI7IY" target="_blank" rel="noopener">youtube.com</a><br/> <b>LM Studio</b>: <a href="https://lmstudio.ai" target="_blank" rel="noopener">lmstudio.ai</a><br/> <b>Little Bobby Tables</b>: <a href="https://xkcd.com/327/" target="_blank" rel="noopener">xkcd.com</a><br/> <br/> <b>spaCy and NLP course</b>: <a href="https://talkpython.fm/spacy" target="_blank" rel="noopener">talkpython.fm</a><br/> <b>Watch this episode on YouTube</b>: <a href="https://www.youtube.com/watch?v=zaZrWZwKJH4" target="_blank" rel="noopener">youtube.com</a><br/> <b>Episode transcripts</b>: <a href="https://talkpython.fm/episodes/transcript/465/the-ai-revolution-wont-be-monopolized" target="_blank" rel="noopener">talkpython.fm</a><br/> <br/> <b>--- Stay in touch with us ---</b><br/> <b>Subscribe to us on YouTube</b>: <a href="https://talkpython.fm/youtube" target="_blank" rel="noopener">youtube.com</a><br/> <b>Follow Talk Python on Mastodon</b>: <a href="https://fosstodon.org/web/@talkpython" target="_blank" rel="noopener"><i class="fa-brands fa-mastodon"></i>talkpython</a><br/> <b>Follow Michael on Mastodon</b>: <a href="https://fosstodon.org/web/@mkennedy" target="_blank" rel="noopener"><i class="fa-brands fa-mastodon"></i>mkennedy</a><br/></div>

A week ago I attended the 22. Gulaschprogrammiernacht (GPN22) in Karlsruhe, Germany. That’s a bit of a smaller version of the Chaos Communication Congress, although with 1000+ attendees “small” doesn’t really do it justice. Below are some of my notes from conversations I had there.

ⓒ Entropia e.V. KDE Itinerary Debugging other people’s trips

Occasionally Itinerary produces weird results in its timeline for complex trips, and this can be hard to debug remotely without asking for a full export of all the personal data in there. Meeting users and contributors at events is a good opportunity to look at such issues on their devices and analyze the problems together.

Some things could be fixed on the spot, and verified shortly afterwards on the affected installations:

• Improved trip grouping for incomplete trips and closely adjacent trips.
• Improved recovery from wrong timezone information in manually added entries.
• Fixed timeline ordering with location change elements without arrival time.

Data syncing via Matrix

Syncing multiple Itinerary instances is an often requested feature. One of the most promising approaches is doing this via Matrix. An in-depth discussion on that at least gave me a much more refined idea on how that could look like, and what we’d need in order to get there.

Specifically:

• Use a special room type for this, so it’s not cluttering the room list in “normal” chat clients.
• Use one room per trip rather than one room for the entire data set. This avoids scalability problems with state events, and would allow sharing individual trips with other people for joint planning and traveling together.
• Use one state event per timeline element, so that there is always a known latest state without having to load/replay the entire room history.

iOS port

I also met someone looking into an iOS port of Itinerary. Obviously that’s in a very early exploratory state, but should that turn out viable it would probably also benefit many other KDE applications.

Project management

We also talks about how to clean up and revive task tracking for Itinerary. For various historic reasons relevant tasks are currently spread over two Phabricator boards, a Gitlab board, tasks in Itinerary’s Gitlab project and Bugzilla, neither of which is perfectly maintained, current nor complete.

Not sure yet how to best structure this, but it clearly needs to change, and I need to more actively use this rather than having things in my local notes.

KDE Platform & Frameworks Password and credential management

How could credential/secrets storage/management look like for our platform in a world beyond KWallet? KWallet’s design, UX and security model goes back more than 20 years by now, and it shows. And since this is a sensitive and complex topic to touch, it’s probably worth working out first how we would want this to look like eventually.

Things to consider include:

• Standardized access APIs for applications like XDG portals or the secret service API.
• Data exchange formats with other password management systems.
• Integration with cloud-based password managers for synchronization between different devices.
• Supporting external hardware tokens and TPMs.
• Supporting Passkeys and WebAuthn.
• Unlocking UI nicely integrated into the shell, and also supporting e.g. biometric unlocking.
• Hiding the complexity of all that from the user during setup to the extend possible.
• Browser integration and integration with SSH and GPG key stores.
• Should 2FA tools like Keysmith use this or remain separate?
• Minimizing the need for configuration, while keeping much of the above opt-in.

Given the complexity we probably should find a place to collect all the ideas and requirements, and then get a few interested people together for a few days to turn this into a plan.

Application debug and support infrastructure

We do have functionality in Frameworks to help us diagnosing issues happening on a user’s device, most prominently KCrash. Some of our applications have additional custom-built diagnostic functions as well though, such as:

• NeoChat has some debug output capturing system.
• Itinerary has a “debug mode” that shows some additional diagnostics and test functions.

Would it make sense to generalize/standardize some of this and upstream it to Frameworks?

Emergency and weather alerts

One of my main agenda points was meeting with FOSS Warn again, for our joint work on an emergency and weather alert aggregation server.

Topcis included:

• Quality metrics and monitoring of the input feeds. While we have feeds from 100+ countries, the activity on those ranges between 10+k alerts in the last 30 days to none. The latter could be valid for a small island nation with stable weather recently, but for a larger country this raises questions on how well that feed is maintained.
• The data format and data model for feed metadata and API to query that, something my prototype didn’t do at all. This is useful to show in clients what kinds of alerts (or any at all) could be expected for an area of interest.
• How to deal with split-up multilingual feeds. The Common Alerting Protocol (CAP) has support for multilingual entries, but in at least two cases (Russia and Saudi Arabia) those seem to be provided in separate single-language feeds instead.

Transitous

Improvements for Transitous also came out of this.

Chatting with a bunch of people the acquisition of Greyhound by FlixBus was mentioned, leading to somebody raising the question whether that would mean Transitous includes the Greyhound network (given FlixBus in Europe is covered). That turned out to not be the case, so the next person started to do a quick web search finding data for North America as well. Yet another person who had recently travelled that way was able to confirm that to be valid and current. A pull request for Transitous followed soon after, and within less than an hour this was available on the production instance.

Barely four months after this all started at FOSDEM this was just awesome to watch.

And that’s probably the biggest value of such events, a random hallway conversation can result in a significant improvement to public transport routing connectivity on another continent :)

Sorry for the interruption last week; I was on vacation. While I was vacating, my colleagues were in full-on fix-everything mode in preparation for the upcoming Plasma 6.1 release in a little over a week. And what a release it promises to be! I think this is going to be a good one, folks. Lots of great features, improved performance and smoothness, and oodles of fixes for all kinds of strange bugs with your wild and wacky hardware devices!

New Features

Plasma’s Networks widget now supports WebAuth for SAML-based authentication. I don’t know what this means, but if you do, that probably means you’re happy about it! (Joel Holdsworth, Plasma 6.2.0 Link)

The network QR code that you can show in Plasma’s Networks widget is now draggable, so you can get an image file out of it wherever you want! (Fushan Wen, Plasma 6.2.0. Link)

System Monitor’s data backend now supports getting CPU/memory/IO/etc. pressure data from /proc/pressure/ (when supported by the Linux kernel) and displaying it in sensors. Hopefully this data will start to show up in user-facing features soon! (Adrian Edwards, Plasma 6.2.0 Link)

UI Improvements

Dolphin once again recommends installing Filelight if you try to get information about free space but it’s not installed (Felix Ernst, Dolphin 24.08.0. Link)

Passwords copied from KWalletManager no longer appear visible in the Clipboard widget (Weng Xuetian, KWalletManager 24.08.0. Link)

Plasma’s Panel Settings dialog no longer obscures the hover pop-ups for individual widgets while it’s open (David Edmundson, Plasma 6.1.0. Link)

In Plasma’s Sticky Notes widget, the colors of the inline buttons already change with the system color scheme. Now they also change appropriately based on the color scheme you select for the sticky note itself! (Evgeniy Harchenko, Plasma 6.1.0. Link)

Removed the “Hide utility windows for inactive applications” option, because it was broken and apparently nobody had noticed or reported this, and also it’s kind of a weird thing to exist in the first place. This also fixes a Plasma panel auto-hide bug caused by using it (Xaver Hugl, Plasma 6.1.0. Link 1 and link 2)

Searching in System Settings no longer sometimes returns nonsensical matches based on keywords that got implicitly joined together in ways that didn’t make sense (Harald Sitter, Plasma 6.1.0. Link)

A better cursor icon is now used when dragging windows (Vlad Zahorodnii, Plasma 6.1.0. Link)

On X11 with an NVIDIA GPU, a floating panel, and adaptive panel transparency, an unfortunate bug in the NVIDIA driver causes windows to lag when moved and resized. For now we’ve added a warning in the panel settings dialog about this (Ivan Tkachenko, Plasma 6.2.0. Link)

Improved the accessibility of the common Kirigami.PlaceholderMessage UI component (Aleix Pol Gonzalez, Frameworks 6.4. Link)

The custom accent color feature (including “accent color from wallpaper”) now does a better job of picking colors for links that will be readable, no matter what colors your base color scheme uses (Akseli Lahtinen, Plasma 6.1.0. Link)

Bug Fixes

Fixed an unusual issue in Elisa that could cause it to not launch when certain DBus setups on certain distros (Jack Hill, Elisa 24.05.1. Link)

Fixed Elisa not launching at all on Windows after a Windows integration library we were using was removed (Jack Hill, Elisa 24.05.1. Link)

Elisa no longer crashes when you try to enqueue the contents of filesystem folders that don’t actually have any music in them (Jack Hill, Elisa 24.05.1. Link)

Spectacle’s highlight annotation once again actually highlights (Noah Davis, Spectacle 24.05.1. Link)

Fixed a way that KWin could crash after the system wakes from sleep with weird screens that turn on strangely (Vlad Zahorodnii, Plasma 6.1.0. Link)

KWin should no longer crash when apps do weird things and for some reason two drag operations end up happening at once (Vlad Zahorodnii, Plasma 6.1.0. Link)

Fixed a way that KWin could crash on X11 when you change Global Themes (Xaver Hugl, Plasma 6.1.0. Link)

Fixed a weird crash in KWin that could happen when you’re pressing keys on the keyboard during the exact moment that XWayland disconnects (David Edmundson, Plasma 6.1.0. Link)

Fixed a cause of Plasma using too much memory and/or freezing due to system notifications containing certain sizes of images (Akseli Lahtinen, Plasma 6.1.0. Link)

Changing a System Monitor widget located on a thick panel to the “Application Table” or “Process Table” display style no longer causes Plasma to freeze (Akseli Lahtinen, Plasma 6.1.0. Link)

Fixed the most common crash in System Settings’ Firewall page which could be experienced by changing settings and then switching pages (Nicolas Fella, Plasma 6.1.0. Link)

Fixed an extremely esoteric Plasma crash that was caused by repeatedly wiggling the pointer over certain menu items of the context menu of the sub-folder popup of a folder located on the desktop (Akseli Lahtinen, Plasma 6.1.0. Link)

When you try to add a sensor to System Monitor or one of its so-named widgets that you’ve already added, you’re now warned and prevented from doing so, instead of being allowed to continue, which would silently break the display of all sensors (Arjen Hiemstra, Plasma 6.1.0. Link)

Connected devices that are nice enough to report battery information to Plasma no longer sometimes randomly get disconnected for no good reason (Ivan Tkachenko, Plasma 6.1.0. Link)

Some monitors that do unusual and exotic things under the hood no longer get their resolutions set to 640×480 after resuming from sleep (Xaver Hugl, Plasma 6.1.0. Link)

The “Allow restore on future sessions” checkbox on the screen sharing dialog once again works (Nicolas Fella, Plasma 6.1.0. Link)

Titlebar context menus of non-maximized Aurorae-decorated windows are no longer misplaced on Wayland (Vlad Zahorodnii, Plasma 6.1.0. Link)

Kicker’s panel button is no longer too large with a thin panel, which was a regression caused by me trying to keep it from getting too large with thick panels! (Niccolò Venerandi, Plasma 6.1.0. Link)

Connecting a new screen that’s larger than any of the current screens no longer causes it to get a small constrained version of the default wallpaper; now it gets the correct size (Marco Martin, Plasma 6.1.0. Link)

When you’ve got a floating Plasma panel on a screen edge that’s shared with another screen, it no longer pushes an odd blurry rectangle onto that adjacent screen when it de-floats (Niccolò Venerandi, Plasma 6.1.0. Link)

Using the “Application” or “Window” layout switching mode no longer causes Plasma’s System Tray popup to close immediately when you try to open it while the widget is located on a panel on the bottom or right screen edge (Marco Martin, Plasma 6.1.0. Link)

Shutting down, restarting, etc. from within KWin’s Overview effect no longer causes it to freeze for 45 seconds before completing the operation (David Edmundson, Plasma 6.2.0. Link)

Fixed a bug that caused items on the Plasma desktop to disappear until Plasma was restarted after their icons changed, either because you changed them manually, or they got automatically changed by something else automatically—for example by putting items into the trash (Akseli Lahtinen, Frameworks 6.4. Link)

Worked around a Qt regression that prevented toolbar buttons in QtQuick apps from showing that they have keyboard focus (Aleix Pol Gonzales, Frameworks 6.4. Link)

Other bug information of note:

• 5 Very high priority Plasma bugs (up from 3 two weeks ago). Current list of bugs
• 33 15-minute Plasma bugs (down from 36 two weeks ago). Current list of bugs
• 294 KDE bugs of all kinds fixed over the last two weeks. Full list of bugs

Performance & Technical

Improved the performance of basically everything for cases where the ~/.cache folder is located on a slow disk, such as a hard drive (David Edmundson, Plasma 6.1.0. Link)

Plasma and KWin no longer rocket up in CPU usage for no good reason when using the using “Keep the selection and clipboard the same” and “Text selection: Always save in history” Clipboard widget settings (David Edmundson, Plasma 6.1.0. Link)

Re-enabled hardware-accelerated cursor support for Intel GPUs, since the bugs that were causing us to disable it have since been resolved. This should make cursors more responsive and not increase CPU usage when moving it (Vlad Zahorodnii, Plasma 6.2.0. Link)

Implemented Breeze styling for movable splitters in QML software, so we can now start porting things to use it. This will eventually result in, for example, resizable sidebars in places where you’d expect for sidebars to be resizable—imagine that! (Ivan Tkachenko, Frameworks 6.4. Link)

Automation & Systematization

Turned a manual test for the Plasma System Tray into an automatic one, so now it actually gets run regularly (Fushan Wen, link)

Added a few autotests for the Plasma Clipboard widget to verify recent fixes (Fushan Wen, link)

…And Everything Else

This blog only covers the tip of the iceberg! If you’re hungry for more, check out https://planet.kde.org, where you can find more news from other KDE contributors.

How You Can Help

The KDE organization has become important in the world, and your time and labor have helped to bring it there! But as we grow, it’s going to be equally important that this stream of labor be made sustainable, which primarily means paying for it. Right now the vast majority of KDE runs on labor not paid for by KDE e.V. (the nonprofit foundation behind KDE, of which I am a board member), and that’s a problem. We’ve taken steps to change this with paid technical contractors — but those steps are small due to growing but still limited financial resources. If you’d like to help change that, consider donating today!

Otherwise, visit https://community.kde.org/Get_Involved to discover other ways to be part of a project that really matters. Each contributor makes a huge difference in KDE; you are not a number or a cog in a machine! You don’t have to already be a programmer, either. I wasn’t when I got started. Try it, you’ll like it! We don’t bite!

GNUnet 0.21.2

This is a bugfix release for gnunet 0.21.1. It primarily addresses some connectivity issues introduced with our new transport subsystem.

Links

• Source: https://ftpmirror.gnu.org/gnunet/gnunet-0.21.2.tar.gz ( https://ftpmirror.gnu.org/gnunet/gnunet-0.21.2.tar.gz.sig )
• Source (meson): https://buildbot.gnunet.org/gnunet-0.21.2-meson.tar.gz ( https://buildbot.gnunet.org/gnunet-0.21.2-meson.tar.gz.sig )
• Detailed list of changes: https://git.gnunet.org/gnunet.git/log/?h=v0.21.2
• NEWS: https://git.gnunet.org/gnunet.git/tree/NEWS?h=v0.21.2
• The list of closed issues in the bug tracker: https://bugs.gnunet.org/changelog_page.php?version_id=440

The GPG key used to sign is: 3D11063C10F98D14BD24D1470B0998EF86F59B6A

Note that due to mirror synchronization, not all links may be functional early after the release. For direct access try https://ftp.gnu.org/gnu/gnunet/

Ansible Playbooks is the known term, now there is a new term which is being floted in the project, which is Ansible Rulebooks. Today we are going to discuss about Ansible&aposs journey from Playbook to Rulebook rather Playbook with Rulebook.

What is Event Driven Ansible?

What is Event Driven Ansible? In simple terms, some action is triggered by some events. The idea of EDA comes from Event driven architecture. Event driven ansible runs code automatically based on received event notifications.

Some important terms:

What is event in Event Driven Ansible?

The event is the notification of a certain incident.

Where do we get the events from?

We get the events from event sources. Ansible EDA provides different pulgins to support various event sources. There are several event source plugins such as :
url_check (checking the http status code), webhook (providing and checking events from webhook), journald (monitoring the journald logs) and the list goes on.

When to take actions?

Rulebook defines conditions and actions in case of fulfilling those actions. Conditions use operators as strings, boolean and numerical data. And actions are occurrence of events once the conditions are met. Running a playbook, setting a fact, running a module etc.

Small example Project

Here is a small example of Event Driven Ansible and how it is run. The idea is on receiving of a message (here the number 42) a playbook will run in the host. There are the following 3 files :

demo_rule.yml --- - name: Listen for events on a webhook hosts: all sources: - ansible.eda.webhook: host: 0.0.0.0 port: 8000 rules: - name: Say thank you condition: event.payload.message == "42" action: run_playbook: name: demo.yml

This is the rulebook. We are using the webhook plugin here as the event source. As a rule in the event of receiving the message 42 as json payload in the webhook, we run the playbook called demo.yml

demo.yml - hosts: localhost connection: local tasks: - debug: msg: "Thank you for the answer."

demo.yml, the playbook which run on the occurrence of the event mentioned in the rulebook and prints a debug message.

--- local: hosts: localhost

inventory.yml mentions the hosts to run the action against.

Further there are 2 files to one to test 42.json and 43.json to test the code.

{ "message" : "42" } { "message" : "43" }

First we have to install all related dependencies before we can run the rulebook.

$ python -m venv .venv $ source .venv/bin/activate $ python -m pip install ansible ansible-rulebook ansible-runner psycopg $ ansible-galaxy collection install ansible.eda $ ansible-rulebook --rulebook demo_rule.yml -i inventory.yml --verbose

Go to another terminal and on the same directory path and run the following command to test the Rulebook. After receiving the message, the playbook runs.

curl -X POST -H "Content-Type: application/json" -d @42.json 127.0.0.1:8000/endpoint Output 2024-06-07 16:48:53,868 - ansible_rulebook.app - INFO - Starting sources 2024-06-07 16:48:53,868 - ansible_rulebook.app - INFO - Starting rules ... TASK [debug] ******************************************************************* ok: [localhost] => { "msg": "Thank you for the answer." } PLAY RECAP ********************************************************************* localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 2024-06-07 16:50:08,224 - ansible_rulebook.action.runner - INFO - Ansible runner Queue task cancelled 2024-06-07 16:50:08,225 - ansible_rulebook.action.run_playbook - INFO - Ansible runner rc: 0, status: successful

Now if we run the other json file 43.json we see that the playbook does not run even after the http status code being 200.

curl -X POST -H "Content-Type: application/json" -d @43.json 127.0.0.1:8000/endpoint

Output :

2024-06-07 18:20:37,633 - aiohttp.access - INFO - 127.0.0.1 [07/Jun/2024:17:20:37 +0100] "POST /endpoint HTTP/1.1" 200 159 "-" "curl/8.2.1"

You can try this yourself follwoing this git repository.

PHPCamp 2024 in Pune offers an immersive experience where PHP developers of all levels can enhance their skills, expand their knowledge, and connect with a vibrant community. Attendees will engage in hands-on workshops, explore the latest trends, and network with industry leaders in a collaborative and inclusive environment. This event is a unique opportunity for developers to be part of a groundbreaking gathering that fosters innovation and professional growth.

How do you effectively break a software problem into individual steps? What are signs you're writing overly clever code? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Discover DrupalX with Jay Callicott on Medium, a toolkit tailored for enterprise website development. Learn how DrupalX integrates Bootstrap 5 and Storybook to empower developers to effortlessly create scalable, visually stunning websites.

Dear Translators:

Recently, the Licensing and Compliance Lab provided guidelines
for writing copyright notices in www.gnu.org translations:

https://www.gnu.org/s/trans-coord/w/Copyright-Notices.html

Please take them into account.

After received 2 translators‘ feedback plus my thought, I would put the following as advice for new translations:

1. add your name in the copyright notices in the translation if you think your contribution is enough for an article, like

Copyright &copy; 2024 Free Software Foundation, Inc.<br></br>
Copyright &copy; 2024 XIE Wensheng (translation)<

2. or optionally add your name in the TRANSLATOR'S CREDITS part as we always do.

<b>翻译</b>：李凡希，2010。<br></br>
<b>审校</b>：<a href="mailto:1945649519@qq.com">&lt;Nios34&gt;</a>，2020。<br></br>
<b>翻译团队</b>：<a rel="team" href="https://savannah.gnu.org/projects/www-zh-cn/">&lt;CTT&gt;</a>，2017-2024。<

best regards,
wxie

Let’s go for my web review for the week 2024-23.

Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art Large Language Models

Tags: tech, ai, gpt, machine-learning, safety, research

Another cruel reminder that basic reasoning is not to be expected from LLMs. Here is a quote from the conclusion of the paper which makes it clear:

“We think that observations made in our study should serve as strong reminder that current SOTA LLMs are not capable of sound, consistent reasoning, as shown here by their breakdown on even such a simple task as the presented AIW problem, and enabling such reasoning is still subject of basic research. This should be also a strong warning against overblown claims for such models beyond being basic research artifacts to serve as problem solvers in various real world settings, which are often made by different commercial entities in attempt to position their models as a strong mature product for end-users. […] Observed breakdown of basic reasoning capabilities, coupled with such public claims (which are also based on standardized benchmarks), present an inherent safety problem. Models with insufficient basic reasoning are inherently unsafe, as they will produce wrong decisions in various important scenarios that do require intact reasoning.”

https://arxiv.org/abs/2406.02061

A critique of Cara.app: the ‘No AI’ Instagram and Artstation copycat child. - David Revoy

Tags: tech, social-media, art, criticism

Interesting critique of this new platform… it’s the beginning of the hype cycle but will probably exhibit the same decay phenomenon than other platforms.

https://www.davidrevoy.com/article1032/a-critique-of-caraapp-the-no-ai-instagram-and-artstation-copycat-child

Why are vulnerabilities out of control in 2024? – Open Source Security

Tags: tech, foss, security, data, data-science

The more releases out there the more vulnerabilities are (and could be) discovered. Some actions are necessary to get things under control properly.

https://opensourcesecurity.io/2024/06/03/why-are-vulnerabilities-out-of-control-in-2024/

Engineering for Slow Internet

Tags: tech, networking, reliability

A good reminder of everything which might go wrong when connectivity is bad. Most tools let you down in such a case.

https://brr.fyi/posts/engineering-for-slow-internet

Why do CPUs have multiple cache levels? | The ryg blog

Tags: tech, hardware, cpu

Very nice explanation and metaphors on how CPUs cache levels work.

https://fgiesen.wordpress.com/2016/08/07/why-do-cpus-have-multiple-cache-levels/

BenchExec: A Framework for Reliable Benchmarking and Resource Measurement

Tags: tech, benchmarking, tools

Looks like an interesting benchmarking tool. To keep an eye on.

https://github.com/sosy-lab/benchexec

TIL #099 – order values of dictionary by iterable of keys with operator.itemgetter | mathspp

Tags: tech, programming, python

Definitely a nice Python trick. Fairly elegant, I’ll try to remember it.

https://mathspp.com/blog/til/order-values-of-dictionary-by-iterable-of-keys-with-operator.itemgetter

Message authentication codes for safer distributed transactions

Tags: tech, filesystem, distributed, safety, cryptography

Interesting use of cryptography without a security concern. It’s more about safety and ensuring something wasn’t missed by mistake.

https://mazzo.li/posts/mac-distributed-tx.html

The state of Vulkan apps in 2024

Tags: tech, 3d, vulkan, portability

The difficult path for Vulkan. The data obviously is biased since it includes games and most of them are still targeting Windows and so DirectX. I’d be curious to see something similar excluding games (and so focusing on medical, industrial etc.).

https://www.carette.xyz/posts/state_of_vulkan_2024/

How I learned Vulkan and wrote a small game engine with it

Tags: tech, 3d, game, vulkan

Interesting dive into the experience of writing a small Vulkan engine (almost) from scratch.

https://edw.is/learning-vulkan/

How to Build Engineering Strategy

Tags: tech, management, strategy, vision

Packed with useful information. Clearly some things I’m eager to test in there.

https://blog.practicalengineering.management/how-to-build-engineering-strategy-42e464018177

xkcd: Earth Temperature Timeline

Tags: science, history, data-visualization

So yes, the climate changed before… now slowly scroll until the end to appreciate how brutal it is this time.

https://xkcd.com/1732/

Bye for now!

Wordpress (Gutenberg) page editor in Drupal Zé Fri, 06/07/2024 - 08:34 Wordpress (Gutenberg) page editor in Drupal

WP's default Gutenberg editor is an intuitive, yet highly flexible page editor that exists as an independent open-source project, and has meanwhile matured and become popular, with integration into Drupal as well.