FLOSS Project Planets

Over the past six months, we’ve met hundreds of developers that are using Python to build AI chat apps for their own knowledge domains, using the RAG (Retrieval Augmented Generation) approach to send chunks of knowledge to an LLM model along with the user question.

We’ve also heard from many developers that they’d like to learn how to build their own RAG chat apps, but they don’t know where to start. So we’re hosting a virtual hackathon to help you learn how to build your own RAG chat app with Python!

.

From January 29th to February 12th, we’ll host live streams showing you how to build on our most popular RAG chat sample repository, while also explaining the core concepts underlying all modern RAG chat apps. Live stream topics will include vector search, access control, GPT-4 with vision. We’re hoping to get developers from all over the world involved, so we’ll also have live streams in Spanish, Portuguese, and Chinese. There will be prizes for the best chat apps, and even a prize for our most helpful community member.

To learn more, visit the AI Chat App Hack page, and follow the steps there to register and meet the community. Hope to see you there!

More RAG resources for Python developers

If you’re interested in learning more about RAG chat apps but can’t join the hack, here are some resources to get you started:

• Tutorial: Get started with the Python enterprise chat sample using RAG
• GitHub Universe: Quickly build and deploy OpenAI apps on Azure, infused with your own data
• Azure AI resources for Python developers
• Using Llamaindex with Azure AI Search
• AI Discord community

The post Join us for AI Chat App Hack from Jan. 29 – Feb.12 appeared first on Python.

Do you use NumPy, Pandas, or scikit-learn and want to get faster results? Nvidia has created GPU-based replacements for each of these with the shared promise of extra speed.

For example, if you visit the front page of NVidia’s RAPIDS project, you’ll see benchmarks showing cuDF, a GPU-based Pandas replacement, is 15× to 80× faster than Pandas!

Unfortunately, while those speed-ups are impressive, they are also misleading. GPU-based libraries might be the answer to your performance problems… or they might be an an unnecessary and expensive distraction.

Read more...

Defending against the PyTorch supply chain attack PoC About • Blog • Newsletter • Links Defending against the PyTorch supply chain attack PoC

Published 2024-01-17 by Seth Larson
Reading time: minutes

This critical role would not be possible without funding from the OpenSSF Alpha-Omega project. Massive thank-you to Alpha-Omega for investing in the security of the Python ecosystem!

Last week there which a publication into a proof-of-concept supply chain attack against PyTorch using persistence in self-hosted GitHub runners, capturing tokens from triggerable jobs as a third-party contributor, and modifying workflows. This report was #1 on Hacker News for most of Sunday. In the comments of this publication there was a lot of discussion and folks questioning "how do you defend from this type of attack"?

Luckily for open source users, there are already techniques that can be used today to mitigate the downstream impact of a compromised dependency:

• Using a lock file with pinned hashes like pip with --require-hashes, poetry.lock, or Pipfile.lock.
• Reviewing diffs between currently pinned and new candidate releases. The diff must be of the installed artifacts, not using git tags or source repository information. Tools like diffoscope are useful for diffing wheel files which are actually zip files in disguise.
• For larger organizations the cost of manual review can be amortized by mirroring PyPI and only updating dependencies that have been manually reviewed.
• Binary or compiled dependencies can be compiled from source to ensure malicious code isn't hidden from human inspection.

These are tried-and-true methods to protect yourself and ensure dependencies aren't compromised regardless of what happens upstream. Obviously the suggestions above take time and effort to implement. Generally there's desire from me and others to make the above steps easier for consumers like exposing build provenance for easier reviewing of source code or by improving the overall safety of PyPI content using malware scanning and reporting.

Part of my plans for 2024 is to create guidance for Python open source consumers and maintainers for how to safely use packaging tools both from the perspective of supply chain integrity but also for vulnerabilities, builds, etc. So stay tuned for that!

CPython Software Bill-of-Materials update

Last week I published a draft for CPython's SBOM document specifically for the source tarballs in order to solicit feedback from consumers of SBOMs and developers of SBOM tooling. I received great feedback from Adolfo Garcia Veytia and Ritesh Noronha including the following points:

• Strip version information from the fileName attribute
• The top-level CPython component had no relationships to non-file components, should have DEPENDS_ON relationships to all its dependent packages.
• Fix the formatting of the "Tool: " name and version. Correct format is {name}-{version}.
• Use the fileName attribute on the CPython package instead of using a separate file component for the tarball containing CPython source code.
• Include an email address for all "Person" identities.
• Guidance on alternatives to the documentNamespace field.

After applying this feedback we now have an SBOM which meets NTIA's Minimum Elements of an SBOM and scores 9.6 out of 10 for the SBOM Quality Score.

Next I'm working on the infrastructure for actually generating and making the SBOM available for consumers:

• Created a PR for generating the draft SBOM. Next need to hook into the actual release process and opportunistically generate an SBOM.
• Applied changes according to feedback from SBOM reviewers.
• Created a PR for enabling hosting an SBOM artifact on https://python.org/downloads

Other items

• Reviewed PEP 740 proposal for arbitrary attestation mechanism for PyPI artifacts.
• Triaged multiple reports to the Python Security Response Team.

That's all for this week! 👋 If you're interested in more you can read last week's report.

Thanks for reading! ♡ Did you find this article helpful and want more content like it? Get notified of new posts by subscribing to the RSS feed or the email newsletter.

This work is licensed under CC BY-SA 4.0

We are excited to announce the creation of a European project December 1st 2023, which will run for the next 36 months. This Next Generation Internet pilot named "NGI TALER" is operated by a consortium of 11 partners from 8 European countries with the mandate to roll out an innovative electronic payment system for the greater benefit of European citizens, merchants, and banks. This payment system is different from current online payment methods, like credit cards or bank transfers, in that it offers privacy for the buyer: neither merchants nor banks can trace or link the payments. It is also a no-risk payment option for the merchant as there is no equivalent of fake or stolen credit cards, as payments are cleared and confirmed instantly. The payment system is socially, ecologically and fiscally responsible: it is not a new currency, there is no energy-consuming proof-of-work or proof-of-stake method and clearing is processed much faster than payments by credit cards. NGI TA [...]

Disclaimer: I am not one of KDE's masterminds or spokespersons. I am a mere bystander with few unimportant commits. I follow KDE's ecosystem and other developments in the free software world. In the following, I share some thoughts and my personal opinion.

Talks about new programming languagesAfter 30 years of C code, the Linux kernel opens itself to a second high-level language: Rust. Since fall of 2022 the kernel mainly gained infrastructure work. Some experiments show promising results like a Rust-based network driver or a scheduler.Recently, Git developers started to discuss how to allow Rust code in our beloved version control system. Far from having reached a consensus, its media coverage and heated discussions in forums show how interested the public is in this topic.Other projects try to replace established software by rewritten from scratch Rust ones: uutils coreutils, sudo-rs, librsvg, Rustls. Heck, Rewrite it it Rust (RiiR) has become a meme.
We already have a new programming language!KDE is close to its 6th Megarelease, with one major change being based on Qt 6. Qt 6 requires C++17 which -- as of today -- is perceived as modern C++ and is a leap compared to C++11. It is possible to write modern software with C++17. Still, additional tools like C++ Core Guidelines or Cppcheck are advised to keep the number of preventable bugs low.Most of the projects mentioned in the introduction are using C. This inflicts more pain to the developers and thus using Rust is more attractive. For sure, a fair portion of RiiR arguments do not apply to KDE's C++ code base.Problems with C++ remainC++ cannot adapt to modern ways like including a borrow checker or a less complicated syntax, as this would break compatibility. As much as C++ improved as a language, its compilers, and its ecosystem, it is not enough to be considered a good choice for new projects. NIST and NSA advice to move away from C++.Other problems like complicated tooling with variations on different platforms (build systems, compiler, linker, debugger, dependency management), mixed-in C-style code, difficult to parse C++ code, cannot be solved.I fear that in a not to distant future, C++ might be perceived as an outdated choice to learn and people might less likely consider to join KDE as contributors.
What can be done?In the past, GNOME adopted Vala as a new language to solve the short-comings of C. Vala seems to be dead. Going with Rust did not lead to a project-wide adoption.Some people are working on Qt bindings for Rust, e.g., CXX-Qt from KDAB. I am not sure if Qt itself is working on something similar. At least there is no go-to binding.
Beside the hot topic Rust, two big players invest in ways to have good interoperability with existing code bases and a modern language: Cpp2 / cppfront and Carbon.Cpp2 is a new language from Herb Sutter, who chairs the C++ working group. The idea is to have a transpiler cppfront producing modern C++ code. Cpp2 is not backward compatible to C++ and thus not limited in introducing new ways or removing existing parts. Cpp2 promises to integrate seamlessly in existing C++ code bases as it is compiled into C++ code.Carbon is a project by Google developers and follows a different approach. It aims to provide a new language that can use all C++ features in interfaces, even templates with all bells and whistles.Discuss our futureI do not want to whine about C++. I want to start a discussion on how KDE's future might look like. KDE was always driving innovations. We helped CMake to become one of the most important build systems for C++. KDE 4.0 introduced the semantic desktop. KHTML's code base was the nucleus for today's big browsers.Probably we should have this discussion as a BoF at Akademy 2024 or other places where KDE's masterminds and people with a feeling for future trends come together and form/formulate future directions. In the meantime, I start a discourse thread.Personally, I would like to see some push for Cpp2. More important, I want to see that we are actively shaping KDE's future.

Here are my top 10 articles from The DropTimes in 2023. From igniting my Drupal journey with essential resources to exploring the visionary world of Drupal, these articles have shaped my understanding of Drupal's evolution.

#612 – JANUARY 16, 2024
View in Browser »

Exploring Python in Excel

Are you interested in using your Python skills within Excel? Would you like to share a data science project or visualization as a single Office file? This week on the show, we speak with Principal Architect John Lam and Sr. Cloud Developer Advocate Sarah Kaiser from Microsoft about Python in Excel.
REAL PYTHON podcast

Python 3.13 Gets a JIT

This article does a deeper dive into the JIT recently added to the CPython 3.13 pre-release. This JIT is a bit different, it is called a copy-and-patch JIT, and the post explains what that means.
ANTHONY SHAW

NumPy 2 Is Coming: Preventing Breakage, Updating Your Code

NumPy 2 is coming, and it’s backwards incompatible. Learn how to keep your code from breaking, and how to upgrade.
ITAMAR TURNER-TRAURING

Build Invincible Apps With Temporal’s Python SDK

Get an introduction to Temporal’s Python SDK by walking through our easy, free tutorials. Learn how to build Temporal applications using Python, including building a data pipeline Workflow and a subscription Workflow. Get started here →
TEMPORAL sponsor

PSF Says: EU’s Cyber Resilience Act Has Wins for Open Source

PYTHON SOFTWARE FOUNDATION

Articles & Tutorials Learn From 2023’s Most Popular Python Tutorials and Courses

Revisit your favorite Real Python tutorials and video courses from 2023. Explore various topics, from Python basics to web development, machine learning, and effective coding environments. It’s been a busy year of learning, and there’s something for everyone to discover and build upon in 2024.
REAL PYTHON

Python’s Array: Working With Numeric Data Efficiently

In this tutorial, you’ll dive deep into working with numeric arrays in Python, an efficient tool for handling binary data. Along the way, you’ll explore low-level data types exposed by the array module, emulate custom types, and even pass a Python array to C for high-performance processing.
REAL PYTHON

Data Deduplication in Python with RecordLinkage

Duplicate detection is a critical process in data preprocessing, especially when dealing with large datasets. In this tutorial, you will explore data deduplication using Python’s RecordLinkage package, paired with Pandas for data manipulation.
PATRYK SZLAGOWSKI • Shared by Izabela Pawlik

The Curious Case of Pydantic and the 1970s Timestamps

When parsing Unix timestamps, Pydantic guesses whether to interpret them in seconds or milliseconds. While this is certainly convenient and works most of the time, it can drastically (and silently) distort timestamps from a few decades ago.
ARIE BOVENBERG • Shared by Arie Bovenberg

A Critical Supply Chain Attack on PyTorch

This post describes how coders found an exploit in the PyTorch supply chain, leaving the repo and its maintainers up for attack. Full details on what the vulnerability was and how to avoid the problem in your own repos is covered.
JOHN STAWINSKI

Comparing Coroutines, by Example, in Kotlin and Python

This series of 3 articles compares Python and Kotlin, with a focus on coroutines and generators. It compares, through examples, how coroutines are used in both languages, to read files and perform network requests.
MEDIUM.COM • Shared by Carmen Alvarez

Enhance Your Flask Web Project With a Database

Adding a database to your Flask project elevates your web app to the next level. In this tutorial, you’ll learn how to connect your Flask app to a database and how to receive and store posts from users.
REAL PYTHON

SQLALchemy vs Django ORM

If you are working with Django ORM most of the time and then switching to SQLAlchemy, you may face some unexpected behavior. This post describes the most important differences between them.
ALEXEY EVSEEV

Annotating *args and **kwargs in Python

“Typing *args and **kwargs has always been a pain since you couldn’t annotate them precisely before.” This article shows you what your options are when typing function signatures.
REDOWAN DELOWAR

Python Gotcha: Modifying a List While Iterating

Python makes it easy to modify a list while you are iterating through it’s elements. This will bite you. Read on to find out how and what can be done about it.
ANDREW WEGNER

A Deep Dive Into Python’s functools.wraps Decorator

Take a deep dive into Python’s functools.wraps decorator to learn how it maintains metadata in your code. A concise guide to effective decorator use.
JACOB PADILLA

max() is broken

The built-in function max in Python is broken and this article explains why, drawing parallels with other programming and mathematics concepts.
MATHSPP.COM • Shared by Rodrigo Girão Serrão

All PyCon 2023 (US and AU) Talks Sorted by the View Count

A full list of PyCon talks given in the US and Australia which are available on YouTube, and sorted by popularity.
SUBSTACK.COM

Projects & Code PikaPython: Python Interpreter in 4KB of RAM

GITHUB.COM/PIKASTECH

Fontimize: Optimize Fonts to the Glyphs on Your Site

GITHUB.COM/VINTAGEDAVE

instructor: Structured Outputs for LLMS

GITHUB.COM/JXNL

Pint: Units for Python

PYPI.ORG

Events Weekly Real Python Office Hours Q&A (Virtual)

January 17, 2024
REALPYTHON.COM

PyData Bristol Meetup

January 18, 2024
MEETUP.COM

PyLadies Dublin

January 18, 2024
PYLADIES.COM

Chattanooga Python User Group

January 19 to January 20, 2024
MEETUP.COM

IndyPy: Models & AI For Dummies (Hybrid)

January 23, 2024
MEETUP.COM • Shared by Laura Stephens

Happy Pythoning!
This was PyCoder’s Weekly Issue #612.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

At The DropTimes (TDT), we constantly evolve. The formative steps for us were challenging. We might be just out of infancy, but still in childhood. That does not give us any excuse not to be a better version of what we were yesterday. As we align with the build-in-open philosophy, let us record our recent decisions in response to community feedback.

Authored by: Nadiia Nykolaichuk.

Upon entering the new year, many of us have started working towards resolutions or aspiring achievements. In the digital space, Drupal websites, too, can aspire to be on the cutting edge of trends and provide the most compelling online journeys to users.

My to-read shelf

This year, with respect to my ever-growing reading backlog, I'm going to try something new: when I acquire a new book, I'm going to try to read at least a few pages of it immediately. My theory is this will help me to have a better idea of what to expect when I come to pick the next book to start, later on. A few pages may not be very representative of a full book (In "How to read a Novel: A User's Guide John Sullivan suggests reading 69 pages before giving up on a book), but it may be better than nothing.

I'll report back if it seems to work.

<p><a target="_blank" href="https://drupal.org/project/rjsf">RJSF</a> is a bit of a unique module in that there is more JavaScript than php in the module. This presents some challenges when trying to host and package the module from a Drupal.org repository. Like that, a release on d.o is a copy of a specific commit and not a build artifact controlled by the repo. Because of this RJSF ships the JS part of the module as an NPM package that is only updated when a release is tagged. This has some significant downsides like if you want to use the module you need to install both the module and the npm package. Or to use the dev version of the module you have to build the JS yourself. Or to tag a release I have to manually package a release for NPM. Recently I've been thinking about how to simplify the installation and release process for RJSF and decided to start by attempting to include the built js with the module instead of as an NPM package.</p><h2>The Goal</h2><p><ul><li>Include built javascript when the module is installed via composer</li><li>Update the built javascript whenever the source changes</li><li>Do it all automatically without any manual steps</li></ul></p>

A classic childhood game is tic-tac-toe, also known as naughts and crosses. It’s simple and enjoyable, and coding a version of it with Python is an exciting project for a budding programmer. Now, adding some artificial intelligence (AI) using Python can make an old favorite even more thrilling.

In this comprehensive tutorial, you’ll construct a flexible game engine. This engine will include an unbeatable computer player that employs the minimax algorithm to play tic-tac-toe flawlessly. Throughout the tutorial, you’ll explore concepts such as immutable class design, generic plug-in architecture, and modern Python coding practices and patterns.

In this video course, you’ll learn how to:

• Develop a reusable Python library containing the tic-tac-toe game engine
• Create a Pythonic code style that accurately models the tic-tac-toe domain
• Implement various artificial players, including one using the powerful minimax algorithm
• Construct a text-based console front end for the game, enabling human players to participate
• Discover effective strategies for optimizing performance

Are you ready to embark on this step-by-step adventure of building an extensible game engine with an unbeatable AI player using the minimax algorithm?

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Pamela Fox is a Python Cloud Developer Advocate at Microsoft. 

Topics include:

• Girl Develop It
• Django Girls
• Girls Who Code
• Teaching a language vs teaching a tool
• What a dev advocate does
• Accessibility (A11y) testing
• Playwright
• axe-core
• Snapshot testing
• pytest plugin authoring
• Flask SQLAlchemy
• Relearning Go

Links from the show:

• Python Bytes 323 with Pamela: AI search wars have begun
• Python Test 199 with Pamela: Is Azure Right for a Side Project?
• gdi: Girl Develop It
• Django Girls
• Girls Who Code
• "Automated accessibility audits" - Pamela Fox (North Bay Python 2023)
• Playwright
• axe-core
• pytest-axe-playwright-snapshot, plugin from Pamela
• pytest-crayons plugin is from a PyCascades talk about building plugins
• pytest-check, yet another plugin
• FlaskSQLAlchemy
• Concurrency is not Parallelism by Rob Pike

The Complete pytest Course

• Level up your testing skills and save time during coding and maintenance.
• Check out courses.pythontest.com

★ Support this podcast on Patreon ★ <p>Pamela Fox is a Python Cloud Developer Advocate at Microsoft. </p><p><br>Topics include:</p><ul><li>Girl Develop It</li><li>Django Girls</li><li>Girls Who Code</li><li>Teaching a language vs teaching a tool</li><li>What a dev advocate does</li><li>Accessibility (A11y) testing</li><li>Playwright</li><li>axe-core</li><li>Snapshot testing</li><li>pytest plugin authoring</li><li>Flask SQLAlchemy</li><li>Relearning Go</li></ul><p>Links from the show:</p><ul><li><a href="https://pythonbytes.fm/episodes/show/323/ai-search-wars-have-begun">Python Bytes 323 with Pamela: AI search wars have begun</a></li><li><a href="https://podcast.pythontest.com/episodes/199-is-azure-right-for-a-side-project">Python Test 199 with Pamela: Is Azure Right for a Side Project?</a></li><li><a href="https://girldevelopit.com">gdi: Girl Develop It</a></li><li><a href="https://djangogirls.org/en/">Django Girls</a></li><li><a href="https://girlswhocode.com">Girls Who Code</a></li><li><a href="https://www.youtube.com/watch?v=J-4Qa6PSomM">"Automated accessibility audits" - Pamela Fox (North Bay Python 2023)</a></li><li><a href="https://playwright.dev">Playwright</a></li><li><a href="https://github.com/dequelabs/axe-core">axe-core</a></li><li><a href="https://github.com/pamelafox/pytest-axe-playwright-snapshot">pytest-axe-playwright-snapshot</a>, plugin from Pamela</li><li><a href="https://www.youtube.com/watch?v=kevcz8NRcQU">pytest-crayons plugin is from a PyCascades talk about building plugins</a></li><li><a href="https://github.com/okken/pytest-check">pytest-check</a>, yet another plugin</li><li><a href="https://flask-sqlalchemy.palletsprojects.com/en/3.1.x/">FlaskSQLAlchemy</a></li><li><a href="https://www.youtube.com/watch?v=oV9rvDllKEg">Concurrency is not Parallelism by Rob Pike</a></li></ul><p><br></p> <br><p><strong>The Complete pytest Course</strong></p><ul><li>Level up your testing skills and save time during coding and maintenance.</li><li>Check out <a href="https://courses.pythontest.com/p/complete-pytest-course">courses.pythontest.com</a></li></ul> <strong> <a href="https://www.patreon.com/PythonPeople" rel="payment" title="★ Support this podcast on Patreon ★">★ Support this podcast on Patreon ★</a> </strong>

As part of my role in the Drupal Association, we are trying to find new ways to unleash innovation. Innovation as it happens is a key goal for the Drupal Association. What surprised me when I started with the Drupal Association was to meet companies that were contributors, (some of them known for being long-time contributors) or that are very interested in contributing, but then not knowing how they could maximize their contributions or even where they should be contributing to.

I don’t think that these are a few isolated cases, as it’s not the first time I've seen this trend. Back when I was working for a 100+ developer consultancy firm there was a big corporate push to increase our contribution to open source. And contribute we did. We started “Pizza Fridays”, which meant we were spending Fridays contributing, doing presentations between us, and having pizza for lunch. We had fun, but we lacked structure, purpose, and higher goals (and a healthy diet on Fridays). Our plan was not aligned with anything other than our own appetite to experiment or learn something.

If we had a structure that aligned us to the project we were contributing to, our contributions would have been more impactful, business would have benefited in a more meaningful way, and the whole team would have probably been allowed to contribute even further and longer in time. We did amazing things, don’t get me wrong, but the impact of those could have been much bigger.

That’s why, today, we are introducing the credit bounty program. The idea is to do an initial experiment, and if it has an impact on Drupal moving forward, we’ll tweak it if needed and continue with new iterations.

I expect that the issues and projects that we are promoting will change over time, so we’ll share soon how you can get updated information.

If you are a maintainer and you would like us to include your issues in this pilot program, that may be a possibility as well, so please send me an email: alex.moreno@association.drupal.org. Depending on how this first phase goes, we may start promoting contributed module issues as well based on the popularity of the modules, usage on sites, complexity, how innovative they are, etc, etc

For now, this is the list of issues where (core for now) maintainers need your help. The amount of credit we’ll be given is 50 credits, which is 5 times the normal amount of credits we would grant (normal core issues get 10 credits).

• Cannot save translated nodes after upgrading to 8.8 due to invalid path
  • https://www.drupal.org/project/drupal/issues/3101344
• Composer scaffolding fails when permissions on default.settings.yml or default.settings.php is not writable
  •  https://www.drupal.org/project/drupal/issues/3091285
• Configuration language being overwritten during module install
  • https://www.drupal.org/project/drupal/issues/2905295
• Prevent saving config entities when configuration overrides are applied
  • https://www.drupal.org/project/drupal/issues/2910353

Maintainers will grant credit as normal on these issues, and *all* of the contributing organizations and individuals that the maintainers credit will receive the full bounty. 

Make sure to read Drupal Core's Issue Etiquette for core contribution, and the Contributor Guide. 

Have questions or ideas? Please ping me: alex.moreno@association.drupal.org
 

Mastering Drupal API for Your Next Big Project Success kanapatrick Tue, 01/16/2024 - 14:23

Embarking on a large-scale web development project demands a robust framework that not only supports your vision but also enhances your efficiency and scalability. With over 20 years of development, Drupal has become one of the leading content management systems for building complex and robust websites.

It’s a pleasure to be on the OpenUK New Year’s Honours list for 2024. There’s some impressive names on there such as Richard Hughes of Packagekit and other projects at Red Hat, Colin Watson who was at Ubuntu with me and I see is now freelance, Mike McQuaid was previously of KDE but is now trying a startup with Mac packager Workbrew for Homebrew.

OpenUK run various activities for open tech in UK countries and KDE currently needs some more helpers for a stall at their State of Open Con in London on Feb 6 and 7 February, if you can help do get in touch.

KDE’s 6th releases will happen next month bringing with it the refresh of code and people that a new major version number can bring, I think KDE’s software in the coming year will continue to impress.

My life fell apart after some family loss last year so I’ve run away to the end of the world at Finesterre in Galicia in Spain for now, let me know if you’re in the area.

Happy 2024!

DAIS have continued their programme of posthumous Coil remasters and re-issues. Constant Shallowness Leads To Evil was remastered by Josh Bonati in 2021 and re-released in 2022 in a dizzying array of different packaging variants. The original releases in 2000 had barely any artwork, and given that void I think Nathaniel Young has done a great job of creating something compelling.

A limited number of the original re-issue have special lenticular covers, although these were not sold by any distributors outside the US. I tried to find a copy on my trip to Portland in 2022, to no avail.

Last year DAIS followed Constant… with Queens Of The Circulating Library, same deal: limited lenticular covers, US only.

Both are also available digital-only, e.g. on Bandcamp: Constant…, Queens…. The original, pre-remastered releases have been freely available on archive.org for a long time: Constant…, Queens…

Both of these releases feel to me that they were made available by the group somewhat as an afterthought, having been produced primarily as part of their live efforts. (I'm speculating freely here, it might not be true). Live takes of some of this material exist in the form of Coil Presents Time Machines, which has not (yet) been reissued. In my opinion this is a really compelling recording. I vividly remember listening to this whilst trying to get an hour's rest in a hotel somewhere on a work trip. It took me to some strange places!

I'll leave you from one of my favourite moments from "Colour Sound Oblivion", Coil's video collection of live backdrops. When this was performed live it was also called "Constant Shallowness Leads To Evil", although it's distinct from the material on the LP:

also available on archive.org. A version of this Constant… made it onto a Russian live bootleg, which is available on Spotify and Bandcamp complete with some John Balance banter: we only do this on religious holidays…

Constant Shallowness Leads to Evil by Coil

Posted on January 16, 2024 Tags: debian, free software, nix, life

The nix package manager is available in Debian since May 2020. Why would one use it in Debian?

• learn about nix
• install software that might not be available in Debian
• install software without root access
• declare software necessary for a user’s environment inside $HOME/.config

Especially the last point nagged me every time I set up a new Debian installation. My emacs configuration and my Desktop setup expects certain software to be installed.

Please be aware that I’m a beginner with nix and that my config might not follow best practice. Additionally many nix users are already using the new flakes feature of nix that I’m still learning about.

So I’ve got this file at .config/nixpkgs/config.nix1:

with (import <nixpkgs> {}); { packageOverrides = pkgs: with pkgs; { thk-emacsWithPackages = (pkgs.emacsPackagesFor emacs-gtk).emacsWithPackages ( epkgs: (with epkgs.elpaPackages; [ ace-window company org use-package ]) ++ (with epkgs.melpaPackages; [ editorconfig flycheck haskell-mode magit nix-mode paredit rainbow-delimiters treemacs visual-fill-column yasnippet-snippets ]) ++ [ # From main packages set ] ); userPackages = buildEnv { extraOutputsToInstall = [ "doc" "info" "man" ]; name = "user-packages"; paths = [ ghc git (pkgs.haskell-language-server.override { supportedGhcVersions = [ "94" ]; }) nix stack thk-emacsWithPackages tmux vcsh virtiofsd ]; }; }; }

Every time I change the file or want to receive updates, I do:

nix-env --install --attr nixpkgs.userPackages --remove-all

You can see that I install nix with nix. This gives me a newer version than the one available in Debian stable. However, the nix-daemon still runs as the older binary from Debian. My dirty hack is to put this override in /etc/systemd/system/nix-daemon.service.d/override.conf:

[Service] ExecStart= ExecStart=@/home/thk/.local/state/nix/profile/bin/nix-daemon nix-daemon --daemon

I’m not too interested in a cleaner way since I hope to fully migrate to Nix anyways.

1. Note the nixpkgs in the path. This is not a config file for nix the package manager but for the nix package collection. See the nixpkgs manual.↩︎

2025 may seem like a distant future, but for anyone managing a website on Drupal 7, January 5th 2025 is a date to mark on your calendar: Support and security updates for Drupal 7 are scheduled to end on that day. To stay relevant and secure, it's time to upgrade.

The new year has just begun and we have six weeks left before the final release! The most noticeable change since my last post is obviously that we have decided on the wallpaper to be used in Plasma 6.0! But of course there’s more going on under the hood than just that.

My desktop isn’t usually that tidy

I actually spent most of my time in Qt Wayland rather than KDE code lately but more on that in an upcoming blog post once all my changes have been integrated. Nevertheless, there are still plenty of Wayland-related and other improvements on the Plasma, Frameworks, and KDE Gear side to talk about here.

XDG Foreign Everywhere

After my previous experience of revamping the KWin Window Killer and having learned how to use the XDG Foreign Protocol (a Wayland protocol for exporting a surface to enable a different process to attach to it), I looked at all the places we have a helper application show a window in another application. This is actually done a lot more often than I thought and thus I added API in KWindowSystem for both exporting and importing windows on Wayland.

Since the export side is only really needed on Wayland, I added (un)exportWindow(QWindow*) functions in KWaylandExtras (a utility namespace with Wayland-specific windowing system functionality) along with a windowExported signal once that has been performed. Setting a foreign parent windows can already be done by using KWindowSystem::setMainWindow or even just QWindow::fromWinId. Both of them take a WId (a long int usually) which means that a string-based handle received from the compositor doesn’t play well with the existing APIs.

In case of KWindowSystem I just added a QString overload. The clever part is that it also understands a long int in a string, thus you just feed a token received as a string from QCommandLineParser or stdin verbatim into the API (even supports using 0x and 0b prefixes) and have it do the right thing on all platforms. That way you only have to special-case the export part for Wayland but the importing side will “just work”. Lifetime of the objects is tied to the actual window and there’s no additional resource tracking logic needed on the application side besides calling those functions, pretty neat.

Both kdialog and keditfiletype support XDG Foreign handles for their relevant attach/parent argument now. It is important to have all “public” command-line tools adjusted for 6.0, thereby knowing it’s a Qt 6 build is enough to tell whether it will understand the string or refuse to start on being unable to parse it as a number. If you know of any other tools that might need adjustment, please do tell. The KAuth Framework for executing privileged tasks as well as the KDE PolicyKit agent also learned how to create and understand those handles. This ensures the password prompt is attached to the window it came from (e.g. when changing settings). There’s still plenty of places where KAuth is used without setting a parent window on the executed action but at least the infrastructure is all there now.

More fractional scaling goodness

Of course I accomplished a few more fixes for fractional scaling. The pixmap created by Item.grabToImage now captures it with the proper scale factor. I also made a fix for then using that grab result in a Drag handler (not merged yet). Together with another fix for Plasma’s Folder View and a KWin change I did a while ago we should finally have crisp pixmaps when dragging icons on the desktop. That was a lot more entangled than I anticipated with the scale factor discarded at pretty much every opportunity along the way.

Left to right: Original item, original blurry capture, fixed rendering (salmon for illustration)

TextInput and TextEdit also re-render themselves immediately when the scale factor changes. This fixes sticky notes on the desktop being blurry until interacted with as well as spin boxes and other editable controls in apps like System Settings. The issue was addressed for labels some time ago but similar treatment was needed for input fields, too.

It’s the little things…

I talked about Dolphin’s icon rendering in September and now file thumbnails are properly rendered with fractional scaling, too, both in the main file view and the information sidebar. Méven Car worked on high-dpi support for thumbnailers previously and the switch to Frameworks 6 was a good opportunity to change the wire-format used for communication between app and thumbnailer to use a floating-point number for the scale factor instead of an integer. While at it, I also fixed the “tick” icon to accept input on the address bar. Furthermore, mouse cursor theme previews in System Settings are also scaled smoothly. Incidentally, KWin’s bouncing cursor (startup feedback) is scaled according to the cursor size on Wayland, too.

Speaking of Dolphin, the Places sidebar no longer lets you drag one place into another one. While I found that Windows also lets you do that (macOS doesn’t), I don’t really see the point, unless you want to drag your Documents folder to an external hard drive directly? More importantly, though, it makes re-arranging places very finicky since the drop area in-between places is very small. Obviously, it is still possible to drag files and folders from the main view and elsewhere onto an entry in the Places panel to copy/move/link it to the folder or hard drive in question.

In order to fix KWin’s screen edge triggering inadvertently while selecting text in an application and nudging the corner of the screen, I disabled them completely when a mouse cursor is pressed. However, I wasn’t entirely happy with this (and people in the comments weren’t either) and now it’s possible to drag a file into a screen edge and peek at the desktop to drop it there. Making the new overview effects work with drag and drop is going to be a lot more work, so it’s left for a later time. I’d also love to be able to switch virtual desktops while dragging a file, just like we do when touching the screen edge while dragging a window.

That irksome Konsole font bug finally squashed!

On the subject of broken rendering, I also found a way to improve font rendering in Konsole after the removal of QFont::ForceIntegerMetrics in Qt 6. From what I can tell Konsole entirely relies on the fact that all characters are the same width. Apparently, even with a Monospace font under certain conditions thanks to Hinting and Kerning this may not be entirely the case. Forcing full hinting on the font used seems to ensure that no such trickery is going on. If you still see broken fonts when text is highlighted or selecting it, check that your distribution doesn’t force any particular font settings here.

Anything else Okular’s annotation bar now showing proper page numbers and annotation contents

As a heavy user of Okular’s fantastic annotation feature where you can add labels, sticky notes, shapes, lines, and all sorts of other markers to a (PDF) document, I slightly improved the Annotation sidebar: Custom page numbers are now displayed (pages in a PDF aren’t necessarily strictly numbered but there can be Roman numerals for the index, for example, just like in a real book) and the textual content of the annotation, if any, is shown as well to more easily identify which item is which.

Libksysguard learned SI prefixes for Ampere, Volt, Watt, and Watt-hour units. Should I ever upgrade to a Petawatt solar installation, System Monitor has got me covered. KMessageDialog gained a beep function for playing the relevant notification sound (warning, error, etc), for applications that implement a custom message box and already depend on KWidgetsAddons but don’t want to pull in KNotification just for this. It is now used by Kate’s “Save?” dialog which looks like a message box when saving a single file but because it may also display a list of files is a custom implementation. I also had a look at how to make Qt’s own QMessageBox play the KDE sounds but this is routed though the Qt accessibility framework and I wasn’t sure how to hook into that without jeopardizing more important components of it like the AT-SPI interface.

Do you want to install updates or what?

With the prevalence of Offline Updates (i.e. restarting the system and installing updates in a minimal environment) there’s now a dedicated “Reboot & Install Updates” button on the logout screen to skip installing updates and just reboot. This could surely be extended in the future, e.g. shut down without installing updates or install them now instead of on next boot and so on but at least you can now easily reboot the system without installing updates if you want to.

One of the first changes exclusive to Plasma 6.1: WiFi channel display.

Finally, even though we’re all busy squashing any remaining bugs for the 6.0 release, it has been branched off into the “stable” release branch and the repositories are again open for gentle feature development (which includes anything that needs new translations). I just merged a tiny change exclusive to Plasma 6.1: displaying the WiFi channel number next to its frequency in connection details.

Discuss this post on KDE Discuss.