FLOSS Project Planets
Taavi Väänänen: Bulk downloading Wikimedia Commons categories
Wikimedia Commons, the Wikimedia project for freely licensed media files, also contains a bunch of photos by me and photos of me at various events. While I don't think Commons is going away anytime soon, I would still like to have a local copy of those images available on my own storage hardware.
Obviously this requires some way to query for photos you want to download. I'm using Commons categories for this, since that's easy to implement and works for both use cases. The Commons community tends to come up with very specific categories that you can use, and if not, you can usually categorize the files yourself.
thankfully Commons has no such thing as a Conflict of interest (COI) policy
There is almost an existing tool for this: Sam Wilson's mwcli project has support for exporting images one has uploaded to Commons. However I couldn't use that to upload photos of me others have uploaded, plus it's written in PHP and I don't exactly want to deal with the problem of figuring out how to package it in a way I could neatly install it on my NAS.
So I wrote my own tool for it, called comload. It's written in Python because Python is easy to deploy (I can just throw it in a .deb and upload it to my internal repository), and because I did not find a Go library to handle Action API pagination for me. The basic usage is like this:
$ comload --subcats "Taavi Väänänen"This will download any files in Category:Taavi Väänänen and its sub-categories to the current directory. Former image versions, as well as the image description and SDC data, if any, is also included. And it's smart enough to not download any files that are already there on future runs, so you can just throw it in a systemd timer to get any future files. I'd still like it to handle moved files without creating a duplicate copy, but otherwise I'm really happy with the current state.
comload is available from PyPI and from my Git server directly, and is licensed under the GPLv3.
Jonathan Dowland: Code formatting in documents
I've been exploring typesetting and formatting code within text documents such as papers, or my thesis. Up until now, I've been using the listings package without thinking much about it. By default, some sample Haskell code processed by listings looks like this (click any of the images to see larger, non-blurry versions):
It's formatted with a monospaced font, with some keywords highlighted, but not syntactic symbols.
There are several other options for typesetting and formatting code in LaTeX documents. For Haskell in particular, there is the preprocessor lhs2tex, The default output of which looks like this:
A proportional font, but it's taken pains to preserve vertical alignment, which is syntactically significant for Haskell. It looks a little cluttered to me, and I'm not a fan of nearly everything being italic. Again, symbols aren't differentiated, but it has substituted them for more typographically pleasing alternatives: -> has become →, and \ is now λ.
Another option is perhaps the newest, the LaTeX package minted, which leverages the Python Pygments program. Here's the same code again. It defaults to monospace (the choice of font seems a lot clearer to me than the default for listings), no symbolic substitution, and liberal use of colour:
An informal survey of the samples so far showed that the minted output was the most popular.
All of these packages can be configured to varying degrees. Here are some examples of what I've achieved with a bit of tweaking
listings adjusted with colour and some symbols substituted (but sadly not the two together)
lhs2tex adjusted to be less italic, sans-serif and use some colour
All of this has got me wondering whether there are straightforward empirical answers to some of these questions of style.
Firstly, I'm pretty convinced that symbolic substitution is valuable. When writing Haskell, we write ->, \, /= etc. not because it's most legible, but because it's most practical to type those symbols on the most widely available keyboards and popular keyboard layouts.1 Of the three options listed here, symbolic substitution is possible with listings and lhs2tex, but I haven't figured out if minted can do it (which is really the question: can pygments do it?)
I'm unsure about proportional versus monospaced fonts. We typically use monospaced fonts for editing computer code, but that's at least partly for historical reasons. Vertical alignment is often very important in source code, and it can be easily achieved with monospaced text; it's also sometimes important to have individual characters (., etc.) not be de-emphasised by being smaller than any other character.
lhs2tex, at least, addresses vertical alignment whilst using proportional fonts. I guess the importance of identifying individual significant characters is just as true in a code sample within a larger document as it is within plain source code.
From a (brief) scan of research on this topic, it seems that proportional fonts result in marginally quicker reading times for regular prose. It's not clear whether those results carry over into reading computer code in particular, and the margin is slim in any case. The drawbacks of monospaced text mostly apply when the volume of text is large, which is not the case for the short code snippets I am working with.
I still have a few open questions:
- Is colour useful for formatting code in a PDF document?
- does this open up a can of accessibility worms?
- What should be emphasised (or de-emphasised)
- Why is the minted output most popular: Could the choice of font be key? Aspects of the font other than proportionality (serifs? Size of serifs? etc)
- The Haskell package Data.List.Unicode lets the programmer use a range of unicode symbols in place of ASCII approximations, such as ∈ instead of elem, ≠ instead of /=. Sadly, it's not possible to replace the denotation for an anonymous function, \, with λ this way.↩
KDE Android News (October 2024)
Here’s an overview of recent work around Android platform support for KDE Frameworks and KDE applications, most of which is a direct result of discussions and work at Akademy and the Matrix conference.
Notification permission fixesPorting Itinerary and NeoChat to use the KNotification permission API identified two issues around permission checks and callbacks on permission changes that resulted in the application seeing the wrong permission state. That’s fixed now.
Retirement of the Qt 5 CIWith the 24.08 KDE Gear release all our Android apps are based on Qt 6, including their stable release branches. We have therefore started with retiring the Android Qt 5 CI/CD infrastructure, which should save us both maintenance and computing resources.
As Qt 5 is meanwhile lagging behind several Android SDK versions it’s no longer a viable platform for producing APKs that work on up-to-date devices anymore anyway.
Android CI has meanwhile been removed from the KDE Frameworks 5 maintenance branches as well as from a few other libraries that still used it. The removal of Gitlab CI templates, Craft caches and container images will follow.
In particular this means the invent-registry.kde.org/sysadmin/ci-images/android-qt515 container image is deprecated and will be removed eventually. Please get in touch if you are still using this externally. Poppler’s CI was such a case for example.
QML file installationThe QML module macros in ECM used to install QML files to disk, besides bundling those via the Qt resource system as well. While that is still needed on most platforms due to some specific code in Kirigami, it’s unnecessary on Android where we rely solely on the bundled data.
This resulted in unnecessary content in the APKs, which has been fixed and makes all our APKs a bit smaller now.
Translation lookup orderThe probably most visible change is a fix for a long standing bug in KI18n’s multi-language fallback lookup order, which would result in applications showing a wild mix of languages under certain conditions.
This happened when the primary device language was set to English but the country to anything but the US or the UK and one or more secondary languages were also selected. While not strictly limited to Android, there’s two factors that made it particularly prone to happen there:
- Newer Android versions allow language and country to be set fully independently, while on many other systems only certain predefined combinations are available.
- The language setting doesn’t just impact application translations but also text input, so many more users have multiple languages configured.
KI18n first looks for a translation for the specific language/country pair and then just the language, before falling back to the next configured language/country pair. If no translation is found it’ll eventually use the English/US source text.
Conceptually this is not wrong, but the implementation missed the fact that there is no “country-less” English translation but only the English/US source text. A configuration of English/Canada and French/Canada therefore previously resulted in a French translation rather than an English one, as it does now.
This has been backported and should be available in all our APKs with 24.08.2 latest.
Runtime language changeThanks to input from Fabian during Akademy there’s also significant progress on having applications react to system language changes at runtime.
This basically consists of three parts:
- Propagate the native Android system configuration change to Qt, done in Qt CR 596175.
- Reset cached values inside KI18n on system language changes, done in KI18n MR 124.
- Trigger QML binding re-evaluation for i18n() calls on language changes, implemented in KI18n MR 127.
With those three changes applied and a few lines adjusted in the application code to make use of this large parts of the UI already follow system language changes automatically.
It’s far not perfect yet, as there’s more things that need to update in this case than just translated strings. Date/time formatting for example, as discussed in QTBUG-129727. But overall this is already much better than what I had expected and assumed to be feasible with realistic effort.
Dark mode supportAs reported previously we have working support for dark mode since 24.08.1, thanks to Julius’ work on icon recoloring.
So far this required minimal changes to applications to enable it though. That has also been fixed, dark mode support is now automatically enabled for all applications using the Breeze style.
OutlookThere’s still more to do regarding Android platform integration. I’d say the two probably most pressing issues are the following:
- On some devices the font size is unusably small, caused by the display scale factor being wrong. Based on some investigation during Akademy the current working theory is that this is a race condition in Qt’s code reading that information. I have no device/setup that reproduces this problem unfortunately.
- Selecting files in the platform file dialog that are located on a cloud storage such as Nextcloud silently fails. That is, to the application selecting such a file looks as if the user had canceled the dialog. Here we know exactly why this happens (it’s explicit code in Qt doing this, for valid reasons), the challenge is rather to find a proper solution.
If you are interested in Android integration for KDE applications, feel free to join us in the #kde-android Matrix channel!