Feeds

DocKnot is my static web site generator, with additional features for managing software releases and package documentation.

This release switches to semantic versioning for the Perl modules, hence the v prefix to the version number. This appears to be the standard in the Perl world for indicating that the version follows the semantic versioning standard. That also required adding support to DocKnot for release tarballs whose version string starts with v. I plan to convert all of my Perl modules to semantic versioning in their next releases.

The main change in this release is that it incorporates my old faq2html script that was previously used to convert text documents to HTML for publication. This code was imported into DocKnot and integrated with the rest of the package, so text documents can now be referenced by *.spin pointers rather than the (now-deprecated) *.faq files.

The long delay in the release is because I was hoping to refactor the faq2html code to work as a proper module with no global state and to follow my current Perl coding guidelines before releasing a version of DocKnot containing it, but the refactoring is taking forever and support for v-prefixed versions was blocking other releases, so I'm releasing it in a less-than-ideal state and hoping to fix it later.

There are also a few other bug fixes and improvements, the most notable of which is probably that the footer on generated web pages now points properly to the DocKnot distribution page rather than my old spin page.

You can get the latest version from CPAN or from the DocKnot distribution page.

I’m interested to find out who is reading my blog. Following the lead of Jamie Tanna who was in turn copying Terence Eden (both of whose blogs I read), I’d like to ask people who read this to drop me an email or leave a comment on this post if you read this blog and see this post. I have basic analytics on this blog, and I seem to get a reasonable number of views – but I don’t know how many of those are real people, and how many are bots etc.

Feel free to just say hi, but if you have chance then I’d love to find out a bit more about you and how you read this. Specifically, feel free to answer any or all of the following questions:

• Do you read this on the website or via RSS?
• Do you check regularly/occasionally for new posts, or do you follow me on social media (if so, which one?) to see new posts?
• How did you find my blog in the first place?
• Are you interested in and/or working in one of my specialisms – like geospatial data, general data science/data processing, Python or similar?
• Which posts do you find most interesting? Programming posts on how to do things? Geographic analyses? Book reviews? Rare unusual posts (disability, recipes etc)?
• Have you met me in real life?
• Is there anything particular you’d like me to write about?

The comments box should be just below here, and my email is robin@rtwilson.com

Thanks!

Debian packaging consists of a directory (debian/) containing a number of "hard-coded" filenames such as debian/control, debian/changelog, debian/copyright. In addition to these, many packages will also use a number of optional files that are named via a pattern such as debian/{{PACKAGE}}.install.

At a high level the patterns looks deceptively simple. However, if you start working on trying to automatically classify files in debian/ (which could be helpful to tell the user they have a typo in the filename), you will quickly realize these patterns are not machine friendly at all for this purpose.

The patterns deconstructed

To appreciate the problem fully, here is a primer on the pattern and all its issues. If you are already well-versed in these, you might want to skip the section.

The most common patterns are debian/package.stem or debian/stem and usually the go to example is the install stem ( a concrete example being debian/debhelper.install). However, the full pattern consists of 4 parts where 3 of them are optional.

• The package name followed by a period. Optional, but must be the first if present.
• The name segment followed by a period. Optional, but must appear between the package name (if present) and the stem. If the package name is not present, then the name segment must be first.
• The stem. Mandatory.
• An architecture restriction prefixed by a period. Optional, must appear after the stem if present.

To visualize it with [foo] to mark optional parts, it looks like debian/[PACKAGE.][NAME.]STEM[.ARCH]

Detecting whether a given file is in fact a packaging file now boils down to reverse engineering its name against this pattern. Again, so far, it might still look manageable. One major complication is that every part (except ARCH) can contain periods. So a trivial "split by period" is not going to cut it. As an example:

debian/g++-3.0.user.service

This example is deliberately crafted to be ambiguous and show this problem in its full glory. This file name can be in multiple ways:

• Is the stem service or user.service? (both are known stems from dh_installsystemd and dh_installsystemduser respectively). In fact, it can be both at the same time with "clever" usage of --name=user passed to dh_installsystemd.
• The g++-3.0 can be a package prefix or part of the name segment. Even if there is a g++-3.0 package in debian/control, then debhelper (until compat 15) will still happily match this file for the main package if you pass --name=g++-3.0 to the helper. Side bar: Woe is you if there is a g++-3 and a g++-3.0 package in debian/control, then we have multiple options for the package prefix! Though, I do not think that happens in practice.

Therefore, there are a lot of possible ways to split this filename that all matches the pattern but with vastly different meaning and consequences.

Making detection practical

To make this detection practical, lets look at the first problems that we need to solve.

1. We need the possible stems up front to have a chance at all. When multiple stems are an option, go for the longest match (that is, the one with most periods) since --name is rare and "code golfing" is even rarer.
2. We can make the package prefix mandatory for files with the name segment. This way, the moment there is something before the stem, we know the package prefix will be part of it and can cut it. It does not solve the ambiguity if one package name is a prefix of another package name (from the same source), but it still a lot better. This made its way into debhelper compat 15 and now it is "just" a slow long way to a better future.

A simple solution to the first problem could be to have a static list of known stems. That will get you started but the debhelper eco-system strive on decentralization, so this feels like a mismatch.

There is also a second problem with the static list. Namely, a given stem is only "valid" if the command in question is actually in use. Which means you now need to dumpster dive into the mess that is Turning-complete debhelper configuration file known as debian/rules to fully solve that. Thanks to the Turning-completeness, we will never get a perfect solution for a static analysis.

Instead, it is time to back out and instead apply some simplifications. Here is a sample flow:

1. Check whether the dh sequencer is used. If so, use some heuristics to figure out which addons are used.
2. Delegate to dh_assistant to figure out which commands will be used and which debhelper config file stems it knows about. Here we need to know which sequences are in use from step one (if relevant). Combine this with any other sources for stems you have.
3. Deconstruct all files in debian/ against the stems and known package names from debian/control. In theory, dumpster diving after --name options would be helpful here, but personally I skipped that part as I want to keep my debian/rules parsing to an absolute minimum.

With this logic, you can now:

• Provide typo detection of the stem (debian/foo.intsall -> debian/foo.install) provided to have adequate handling of the corner cases (such as debian/*.conf not needing correction into debian/*.config)
• Detect possible invalid package prefix (debian/foo.install without foo being a package). Note this has to be a weak warning unless the package is using debhelper compat 15 or you dumpster dived to validate that dh_install was not passed dh_install --name foo. Agreed, no one should do that, but they can and false positives are the worst kind of positives for a linting tool.
• With some limitations, detect files used without the relevant command being active. As an example, the some integration modes of debputy removes dh_install, so a debian/foo.install would not be used.
• Associate a given file with a given command to assist users with the documentation look up. Like debian/foo.user.service is related to dh_installsystemduser, so man dh_installsystemduser is a natural start for documentation.

I have added the logic for all these features in debputy though the documentation association is currently not in a user facing command. All the others are now diagnostics emitted by debputy in its editor support mode (debputy lsp server) or via debputy lint. In the editor mode, the diagnostics are currently associated with the package name in debian/control due to technical limitations of how the editor integration works.

Some of these features will the latest version of debhelper (moving target at times). Check with debputy lsp features for the Extra dh support feature, which will be enabled if you got all you need.

Note: The detection is currently (mostly) ignoring files with architecture restrictions. That might be lifted in the future. However, architecture restricted config files tend to be rare, so they were not a priority at this point. Additionally, debputy for technical reasons ignores stem typos with multiple matches. That sadly means that typos of debian/docs will often be unreported due to its proximity to debian/dirs and vice versa.

Diving a bit deeper on getting the stems

To get the stems, debputy has 3 primary sources:

1. Its own plugins can provide packager provided files. These are only relevant if the package is using debputy.
2. It is als possible to provide a debputy plugin that identifies packaging files (either static or named ones). Though in practice, we probably do not want people to roll their own debputy plugin for this purpose, since the detection only works if the plugin is installed. I have used this mechanism to have debhelper provide a debhelper-documentation plugin to enrich the auto-detected data and we can assume most people interested in this feature would have debhelper installed.
3. It asks dh_assistant list-guessed-dh-config-files for config files, which is covered below.

The dh_assistant command uses the same logic as dh to identify the active add-ons and loads them. From there, it scans all commands mentioned in the sequence for the PROMISE: DH NOOP WITHOUT ...-hint and a new INTROSPECTABLE: CONFIG-FILES ...-hint. When these hints reference a packaging file (as an example, via pkgfile(foo)) then dh_assistant records that as a known packaging file for that helper.

Additionally, debhelper now also tracks commands that were removed from the sequence. Several of the dh_assistant subcommand now use this to enrich their (JSON) output with notes about these commands being known but not active.

The end result

With all of this work, you now get:

$ apt satisfy 'dh-debputy (>= 0.1.43~), debhelper (>= 13.16~), python3-lsprotocol, python3-levenshtein' # For demo purposes, pull two known repos (feel free to use your own packages here) $ git clone https://salsa.debian.org/debian/debhelper.git -b debian/13.16 $ git clone https://salsa.debian.org/debian/debputy.git -b debian/0.1.43 $ cd debhelper $ mv debian/debhelper.install debian/debhelper.intsall $ debputy lint warning: File: debian/debhelper.intsall:1:0:1:0: The file "debian/debhelper.intsall" is likely a typo of "debian/debhelper.install" File-level diagnostic $ mv debian/debhelper.intsall debian/debhleper.install $ debputy lint warning: File: debian/debhleper.install:1:0:1:0: Possible typo in "debian/debhleper.install". Consider renaming the file to "debian/debhelper.debhleper.install" or "debian/debhelper.install" if it is intended for debhelper File-level diagnostic $ cd ../debputy $ touch debian/install $ debputy lint --no-warn-about-check-manifest warning: File: debian/install:1:0:1:0: The file debian/install is related to a command that is not active in the dh sequence with the current addons File-level diagnostic

As mentioned, you also get these diagnostics in the editor via the debputy lsp server feature. Here the diagnostics appear in debian/control over the package name for technical reasons.

The editor side still needs a bit more work. Notably, changes to the filename is not triggered automatically and will first be caught on the next change to debian/control. Likewise, changes to debian/rules to add --with to dh might also have some limitations depending on the editor. Saving both files and then triggering an edit of debian/control seems to work reliable but ideally it should not be that involved.

The debhelper side could also do with some work to remove the unnecessary support for the name segment with many file stems that do not need them and announce that to debputy.

Anyhow, it is still a vast improvement over the status quo that was "Why is my file silently ignored!?".

Review: Welcome to Boy.Net, by Lyda Morehouse

Series: Earth's Shadow #1 Publisher: Wizard's Tower Press Copyright: April 2024 ISBN: 1-913892-71-9 Format: Kindle Pages: 355

Welcome to Boy.Net is a science fiction novel with cyberpunk vibes, the first of a possible series.

Earth is a largely abandoned wasteland. Humanity has survived in the rest of the solar system and spread from Earth's moon to the outer planets. Mars is the power in the inner system, obsessed with all things Earth and effectively run by the Earth Nations' Peacekeeping Force, the ENForcers. An ENForcer soldier is raised in a creche from an early age, implanted with cybernetic wetware and nanite enhancements, and extensively trained to be an elite fighting unit. As befits a proper military, every ENForcer is, of course, male.

The ENForcers thought Lucia Del Toro was a good, obedient soldier. They also thought she was a man. They were wrong about those and many other things. After her role in an atrocity that named her the Scourge of New Shanghai, she went AWOL and stole her command ship. Now she and her partner/girlfriend Hawk, a computer hacker from Luna, make a living with bounty hunting jobs in the outer system. The ENForcers rarely cross the asteroid belt; the United Miners see to that.

The appearance of an F-class ENForcer battle cruiser in Jupiter orbit is a very unpleasant surprise. Lucia and Hawk hope it has nothing to do with them. That hope is dashed when ENForcers turn up in the middle of their next job: a bounty to retrieve an AI eye.

I first found Lyda Morehouse via her AngeLINK cyberpunk series, the last of which was published in 2011. Since then, she's been writing paranormal romance and urban fantasy as Tate Hallaway. This return to science fiction is an adventure with trickster hackers, throwback anime-based cowboy bars, tense confrontations with fascist thugs, and unexpected mutual aid, but its core is a cyberpunk look at the people who are unwilling or unable to follow the rules of social conformity. Gender conformity, specifically.

Once you understand what this book is about, Welcome to Boy.Net is a great title, but I'm not sure it serves its purpose as a marketing tool. This is not the book that I would have expected from that title in isolation, and I'm a bit worried that people who would like it might pass it by.

Inside the story, Boy.Net is the slang term for the cybernetic network that links all ENForcers. If this were the derogatory term used by people outside the ENForcers, I could see it, but it's what the ENForcers themselves call it. That left me with a few suspension of disbelief problems, since the sort of macho assholes who are this obsessed with male gender conformance usually consider "boys" to be derogatory and wouldn't call their military cybernetic network something that sounds that belittling, even as a joke. It would be named after some sort of Orwellian reference to freedom, or something related to violence, dominance, brutality, or some other "traditional male" virtue.

But although this term didn't work for me as world-building, it's a beautiful touch thematically.

What Morehouse is doing here is the sort of concretized metaphor that science fiction is so good at: an element of world-building that is both an analogy for something the reader is familiar with and is also a concrete piece of world background that follows believable rules and can be manipulated by the characters. Boy.Net is trying to reconnect to Lucia against her will. If it succeeds, it will treat the body modifications she's made as damage and try to reverse all of them, attempting to convert her back to the model of an ENForcer. But it is also a sharp metaphor for how gender roles are enforced in our world: a child assigned male is connected to a pervasive network of gender expectations and is programmed, shaped, and monitored to match the social role of a boy. Even if they reject those expectations, the gender role keeps trying to reconnect and convert them back.

I really enjoyed Morehouse's handling of the gender dynamics. It's an important part of the plot, but it's not the only thing going on or the only thing the characters think about. Lucia is occasionally caught by surprise by well-described gender euphoria, but mostly gender is something other people keep trying to impose on her because they're obsessed with forcing social conformity.

The rest of the book is a fun romp with a few memorable characters and a couple of great moments with unexpected allies. Hawk and Lucia have an imperfect but low drama relationship that features a great combination of insight and the occasional misunderstanding. It's the kind of believable human relationship that I don't see very much in science fiction, written with the comfortable assurance of an author with over a dozen books under her belt. Some of the supporting characters are also excellent, including a non-binary deaf hacker that I wish had been a bit more central to the story.

This is not the greatest science fiction novel I've read, but it was entertaining throughout and kept me turning the pages. Recommended if you want some solar-system cyberpunk in your life.

Welcome to Boy.Net reaches a conclusion of sorts, but there's an obvious hook for a sequel and a lot of room left for more stories. I hope enough people buy this book so that I can read it.

Rating: 7 out of 10

I have a very.. unusual notetaking and task setup with Kate, using Orgmode files. I wanted to showcase it and explain how it works, maybe someone else has similar needs.

Here's a small screenshot of my journal notes, I blurred out some things I didn't want to share.

Why OrgMode?

I actually prefer markdown much more. I enjoy writing markdown files and it just works in most editors. However, there is zero markdown note taking apps on android, that allow me to sync from my nextcloud and have notifications.

I write very journal style notes. I have one big journal file (used to be a file per day before), where I write down all the interesting things that happened, plus most importantly, my work tasks.

I tried to use calendar with tasks before.. But theres way too much context switching between notetaking apps, calendars, tasks (and on android you have to have two apps for tasks and calendar!!!). I was drowning in various apps and windows, constantly hopping around..

All I wanted to do was write down a task in my journal, then have that task notify me on my phone and on my desktop whenever it was scheduled.

So eventually I found an app called Orgzly Revived. It lets me take the outline style notes where everything is a bulletpoint, with tasks and such, timers, repeats.. And it has the dang phone notifications no other note taking app has. So it's perfect. Works awesome for my needs.

Only thing is that it requires Orgmode files. Okay, I can do that. I first used Logseq with both markdown and orgmode files, but it's quite slow app, even on my nice PC. And no notifications on mobile of course.

I then realised that I am trying to overcomplicate things. I live a lot of my work life in Kate editor, so I decided to start using that. I had to update the Orgmode Kate syntax file a bit to make it work better for me (it's been upstreamed now yay), but other than that it works really nicely.

And why not Emacs? I need an editor, not an OS. Well, jokes aside, I am just not really into learning yet another tool for something like note taking. Last time I tried to set up Neovim config that works for me, I just ended up doing everything in Kate.. I wager same would happen with emacs. I am very happy with Kate, that's all. And I contribute to it now and then, so it just makes sense to use it. :)

Now I just sync all my note files through my Nextcloud to my phone and my computers. I have couple scripts that make figuring out my agenda and have notifications on my PC as well.

My Kate setup

My setup took some time to get together and with couple scripts and snippets it works pretty well.

• First, you need to git clone https://codeberg.org/akselmo/kate-orgmode.git
  • In there I have couple python and shell scripts, and snippets for Kate
  • You need to create a .venv for it with python -m venv .venv and then pip install -r requirements.txt
  • Check the Readme file for how to use the snippets
• Create session called Notes.
  • In Plasma, you can just search "Notes" in KRunner and open the Notes session, that just opens the files.
• Symlink the org mode snippets to ~/.local/share/ktexteditor_snippets/data/ and make sure they work in Kate
  • Snippets are incredibly powerful in Kate, i recommend using them for many things!
  • These snippets automate stuff like "worktask" entries.
  • Unfortunately any timestamps need to be modified by hand, i have not yet had time to make an external tool that spawns a calendar view and spits out a timestamp.
  • For example, "worktask" adds a following item.

* TODO [#ABC] task_name :work: SCHEDULED: <2024-07-07 Sun 21:12> DEADLINE: <2024-07-07 Sun 21:12> :PROPERTIES: :CREATED: [2024-07-07 Sun 21:12] :product: product :kdebug: [[https://bugs.kde.org/show_bug.cgi?id=bugid][bugid]] :merge-request: UPDATE_ME :END:

• Add an "external tool" entry for the run-orgagenda.sh item in the files.
  • Set the "Output" setting to "Display in a panel"
  • For the scripts, make sure they're reading right folder (where your notes are)
  • Now you can press ctrl+alt+i and type Org Agenda and it shows you a fun lil overview in a panel of all your items in agenda
• In your DE's autostart, add the run-orgnotifier.sh script.
  • This will send notification for anything you have set as scheduled or deadline before 30 min, and before/after 5 min of the timestamp.

That should be it for the setup, the code for these scripts is absolutely horrid but you should check that all filepaths match yours etc. I need to clean up the code when I feel like it but honestly it Just Works:tm: for now.. lol.

Journaling?

I have one single file that is called Journal-2024 and I create new journal every year. In this journal file, I prepend new items on top of the file, like this.

* Daily notes 2024-07-07 :PROPERTIES: :CREATED: [2024-07-07 Sun 21:20] :END: Some notes for this day ** Some specific subnote for this daily note blabla * TODO [#A] do this task ples :personal: SCHEDULED: <2024-07-06 Sat 21:22> DEADLINE: <2024-07-07 Sun 21:23> :PROPERTIES: :CREATED: [2024-07-05 Fri 21:22] :END: blalbla * DONE [#B] ancient task :work: :PROPERTIES: :CREATED: [2000-01-01 someDayIdk 21:23] :END: its been 24 years #+BEGIN_SRC code ancient code #+END_SRC

This means that you can just quickly see all the new items. Also in Kate, there's a handy feature for folding all toplevel nodes when you press ctrl+alt+i so you get just list of all headers. I use it all the time. Folding the nodes only works with the newest ksyntaxhighlighting since i recently updated the orgmode syntax file to support that lol.

Anything else

Any other items go to their respective note files.

For example anything related to programming goes to "Programming notes" file. Then I use tags for those items to categorize them based on language, framework, etc..

Instead of having many small files, I have multiple big files, where I categorize items with tags. I can then easily search for items by typing :tag: in the search field in Kate, and it shows me all items related to that tag. Just, well, remember to tag your items! :D

Wrap up

To wrap it up, this system has made things much easier to remember for me, and I can handle more complicated tasks like repeating workout tasks with my phone. (I can do it in Kate too but it requires manual number changing).

I wouldn't mind changing back to markdown if there was a markdown app for phone that has actual notifications.

Sorry this post is a bit all over the place, but there's really not much more interesting things to share about this. It's just couple scripts reminding me or showing all the items in my agenda, and the rest is snippets in Kate and writing orgmode files.

And I am quite happy with my system. Maybe it'll help you to get some ideas too!

Also please tell me about any markdown note taking apps that have notifications on mobile and desktop, if you know any. You can ping me on fedi, which you can find on my about section.

Thanks for reading!

Kids ‘n Billies is a yearly music festival in Nijmegen. The genre is rockabilly, folk, ’50s, punk. It is a family friendly festival, outdoors in the fairly small (seats 900) open-air theatre the Goffert. I got “married” – by two Elvis impersonators – to my nepgenoot at the festival in 2012, so it’s our yearly musical outing. Here are some notes from this year’s edition, which was two weeks ago already.

The festival is arranged around two stages. Main stage is the central theatre, and there is a bospodium (forest stage) in the shrubbery where maybe 50 people can crowd around a stage that is about 4-by-3 metres. My preference goes out to the bospodium, because it is such a more intimate experience.

On the whole, this is also a kid-friendly festival (like there’s buckets where you can re-load your water pistols, if it’s a sunny day). The birds are singing, you sit on the grass or stand under the trees. That does make some genres feel a bit out-of-place.

Notes from the Main Stage

• The Gories felt out-of-place. If I was at a dark and boozy get-together and hanging around at the edge of the pit, it’d be fine. Not by the light of day.
• King Salami and the Cumberland 3 seemed like a racist caricature of themselves. I felt a bit uncomfortable watching them – because is it me projecting racist expectations, is it them playing with my expectations, or are we just all having fun – but they absolutely had a blast on stage and were really good with the kids. King Salami is a top-notch front-man.
• Slim Cessna’s Auto Club did not resonate with me at all. Kinda slick country, and I thought George Cessna’s solo set (Bospodium) was way better.
• Tornado Beat! have all the right moves and a very traditional rockabilly line-up. Danceable, and a tight set.
• Angry Zeta are a pack of raccoons (mapaches, as they translated for me later) from Buenos Aires in punk rock clothes with rockabilly sensibilities and a bunch of really sweet people, too. I learned some Spanish, and was reminded of various independence movements by their tour poster – as in, their tour locations were in Euskal Herria, Catalunya, Schweiz, Nederland, .. . They look scary, make a lot of noise, and have fun doing it.

Notes from the Park

Thanks to Korte Metta for doing my hair, so I had lovely flowers and ’50s girly bows and things. At the festival she mostly does kids and a few adults, and I commend her for making one big hairy dude totes adorable.

Notes from the Bospodium

• Posessed by Paul James had two sets, and they played in Eindhoven a couple of days later and was good enough that I had to go see him again. One guy with a guitar, a banjo, and a violin. “Posessed” is a really accurate description. It’s personal, it’s wild.
• Hymn for Her was a trio – no dog on this tour – for the set and projected really well. With a synth piano, which I thought was a bit daring at a rockabilly festival, but Diver (the pianist) nailed it. The way the family plays together is really nice to see.
• George Cessna looked like a hipster, had a backing track he recorded earlier, and was both soulful and country and I really liked it. Raised my hopes for the Auto Club, later, but that was a disappointment.
• Van Tastik reminded me a bit of the Reverend Deadeye. It’s a one-man band, good stage presence and connects to the audience. My nepgenoot didn’t like some of the first set, but the good fallen Reverend Van Tastik promised a more friendly second set, and delivered. That’s one really nice part of a small festival like this – you literally can grab a beer with the artists and sit a spell. Anyway, for grunchy-punchy one-man noise, this is a man to follow.

A recurring theme from the artists is that the socials are really important to them from a financial perspective. I’m personally not on any of those socials (YouTube, Twitter, Instagram, …) so I can’t help them there, but I figure I can at least make a little noise for them all. And buy the T-shirt.

The seventh release of the binb package, and first in four years, is now on CRAN. binb regroups four rather nice themes for writing LaTeX Beamer presentations much more easily in (R)Markdown. As a teaser, a quick demo combining all four themes is available; documentation and examples are in the package.

This release contains a CRAN-requested fix for a just-released new pandoc version which deals differently with overflowing bounding boxes from graphics; an added new LaTeX command is needed. We also polished continuous integration and related internals a few times but this does not contain directly user-facing changes in the package.

Changes in binb version 0.0.7 (2024-07-06)

• Several rounds of small updates to ge continuous integration setup.

• An additional LaTeX command needed by pandoc (>= 3.2.1) has been added.

CRANberries provides a summary of changes to the previous version. For questions or comments, please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Recently, we worked on performance improvements for a complex enterprise-scale Drupal system. Performance bottlenecks can appear in many different parts of such systems. Many possible performance improvements merit their respective discussion - each with a high level of detail. In this blog post, we want to share what we have learned from a more distant perspective. For some improvements, the Drupal ecosystem already offers well-working solutions in core and the contribution space. Some other changes sound easy on paper, but the implementation requires much effort. That is why we need to zoom out more to make good decisions on the software architecture level. Here are some of the noteworthy changes and decisions we made.

The open fonts for Malayalam developed by Rachana Institute of Technology use our independently developed advanced shaping rules since 2020. A conscious decision was made to support only the revised OpenType specification for Indic scripts (the script tag mlm2, which fixed issues with the v1 specification such as halant shifting). Our shaping rules provide precise, exact and definite shaping for Malayalam Unicode fonts on all major software platforms.

And yet, there are many users who still use either old or buggy softwares/platforms. Hussain and Bhattathiri have expressed angst and displeasure in seeing their beautifully and meticulously designed fonts not shaped correctly on some typeset works and prints (for instance, Ezhuthu is used by Mathrubhumi newspaper showing detached u-signs). I have received many requests over the years to add support for those obsolete (or sometimes proprietary) platforms, but have always refused.

Fig. 1: Detached ു-sign & other shaping issues with InDesign

Few weeks ago, CVR and I were trying to generate Malayalam epub content to read on a Kobo ebook reader (which supports loading user’s own fonts, unlike Kindle). We found that Kobo’s shaping software (quite possibly an old version of Pango) does not support the v2 OpenType specification. That did irk me and I knew it is going to be a rabbit hole. A little bit of reverse engineering and a day later, we were happy to read Malayalam properly shaped in Kobo, by adding rudimentary support for v1 spec.

Fig. 2: RIT Rachana shaped perfectly with Kobo ebook reader (ignore the book title).

Out of curiosity, I checked whether those small additions work with Windows XP, but it did not (hardly surprising). But now that the itch has been scratched; a bunch of shaping rules were added to support XP era applications as well (oh, well).

Fig. 3: RIT Rachana shaped perfectly in Windows XP.

Few days later, a user also reported (known) shaping issue with Adobe InDesign. Though I was inclined to close it as NOTABUG pointing to use HarfBuzz instead, the user was willing to help test a few attempts I promised to make. Adobe 2020/2021 (and earlier) products use Lipika shaper, but recent versions are using HarfBuzz natively. Lipika seems to support v2 OpenType specification, yet doesn’t work well with our existing shaping rules. Quite some reverse engineering and half a dozen attempts later, I have succeeded in writing shaping rules that support Lipika along with other shapers.

Fig.4: RIT Rachana shaped perfectly with InDesign 2021 (note: the characters outside margins is a known issue only with InDesign, and it is fixed with a workaround).

All published (and in progress) RIT Malayalam fonts are updated with these new set of shaping rules; which means all of them will be shaped exactly, precisely and correctly (barring the well-known limitation of v1 specification and bugs in legacy shapers ) all the way from Windows XP (2002) to HarfBuzz 8.0 (present day) and all applications in between.

Supported shaping engines

With this extra engineering work, RIT fonts now tested to work well with following shaping engines/softwares. Note: old Pango and Qt4 have shaping issues (with below base ല forms and ു/ൂ forms of conjuncts, in respective shapers), but those won’t be fixed. Any shaper other than HarfBuzz (and to a certain extent Uniscribe) is best effort only.

• Shaping engines:
  • HarfBuzz
  • Allsorts
  • DirectWrite/Uniscribe (Windows old & new)
  • Lipika (Adobe)
• GUI toolkits & typesetting applications:
  • XeTeX, LuaHBTeX
  • Scribus
  • Qt5 onward (KDE applications)
  • Pango/GTK (GNOME applications)
  • LibreOffice
  • Prince XML
  • Microsoft Office
  • Emacs
  • Adobe InDesign, Illustrator, Photoshop (with HarfBuzz or Lipika)
  • Firefox, Chrome/Chromium, Edge browsers

New releases

New releases are made available for all the fonts developed by Rachana Institute of Typography, viz.

• RIT Rachana (version 1.5.0)
• RIT MeeraNew (version 1.6)
• RIT Panmana (version 1.4)
• RIT TNJoy (version 1.6)
• RIT Keraleeyam (version 3.2)
• RIT Sundar (version 2.3)
• RIT Uroob (version 2.1)
• RIT Karuna (version 1.2)
• RIT Chingam (version 1.2) — with many other improvements
• RIT Ezhuthu (version 1.5.0)
• RIT Kutty (version 1.3)

Acknowledgements

A lot of invaluable work was done by Narayana Bhattathiri, Ashok Kumar and CV Radhakrishnan in testing and verifying the fonts with different platforms and typesetting systems.

End users who reported issues and helped with troubleshooting have also contributed heavily in shaping (pun intended) community software like RIT Malayalam open fonts.

Who am I?

I am Ajay Chauhan (IRC: hisir:matrix.org), currently in my second year of undergraduate studies in Computer Science & Engineering. I'll be working on porting Arianna to Foliate-js for my Google Summer of Code project. I have previously worked on Kdenlive as part of the Season of KDE '24.

I hope to help bring the said feature into reality, but also to sharpen my own development skills through real-world projects, contribute back to the open source community and gain more inner confidence.

What am I working on for this year’s GSoC?

Arianna uses epub.js, which is no longer actively maintained. This creates a challenge as the epub.js may not be able to keep up with the evolving standards and new changes. To address this issue, the proposed solution is to port Arianna to the Foliate-js, which is an actively maintained epub renderer and has more features compared to epub.js and is used by the Foliate ebook reader also.

The problem that this project aims to solve is the need for a reliable and up-to-date epub rendering solution for the Arianna ebook reader. By porting Arianna to Foliate-js, the project will ensure that Arianna can continue to provide support for the latest epub standards and features.

My mentors for the project is Carl Schwan, and I appreciate the opportunity to collaborate with and learn from him during this process.

My work done so far Setting up the development environment

First step was to setup the dev environment, I used Qt Creator for that, it’s fairly easy to set up the development environment.

Reading through existing codebase

The journey began with a thorough review of the existing implementation of epub.js (epub-viewer.js) in Arianna. I identified the features and functionalities that needed to be ported to Foliate-js, understanding how epub.js was used in Arianna for rendering, navigation, and user interactions, and the C++ backend code.

Next, I focused on identifying all the places where epub.js was integrated with Qt in Arianna. This involved comparing the existing implementation with Foliate-js's approach to implementing the same features. To document the changes, I prepared a draft merge request.

Familiarised myself with the current Foliate reader implementation of the foliate-js. However, challenging for me to understand the backend architecture and how different components are interconnected within Foliate-js, requiring additional time to fully grasp the codebase.

Implementing changes

• Fixing the tricky part of the loading process. I had a hard time wrapping my head around how all the pieces fit together. My mentor helped me in this.

Implemented Table of Contents (TOC) functionality: const { book } = action.payload; if (book && book.toc) { applicationWindow().contextDrawer.model.importFromJson( JSON.stringify(book.toc) ); } else { console.warn("Book or TOC not available"); }

It takes the TOC data from the book and imports it into the app's context drawer. This way, users can easily access and navigate through the book's structure.

Added metadata handling: const metadata = action.payload.book.metadata; if (metadata) { backend.metadata = metadata; root.bookReady(backend.metadata.title); Database.addBook(backend.file, JSON.stringify(metadata)); }

Let me break down what this does:

• First, it grabs the metadata from the book that's been loaded.
• If there's metadata, it does a few important things:
  • Updates the backend with this new info.
  • Tells the app that the book is ready to go, passing along the title.
  • Stores all this metadata in the database.

What’s next?

In the following weeks of GSoC, I plan to first to focus on tackling the rendition setup. This is a crucial step - it's what will allow our ebook reader to display the content of ebooks correctly, and also setting the style for the reader. This will involve integrating the Foliate-js rendition module with Arianna's Qt implementation.

Thanks for your reading, stay tuned for updates on my progress via my blog and feel free to connect with me.

With Akademy 2024 hosted in a venue with OSM indoor mapping, what happens if we put KDE’s conference companion app Kongress and Itinerary’s train station indoor map view together?

Venue maps

Out of the box we get a multi-floor map, which is already an improvement compared to just showing building outlines on a conventional (outdoor) map. Visually it’s still far off from handmade conference venue maps though.

Indoor venue map in Kongress.

This is using vanilla OSM data (via KDE’s raw data tile server), which is following upstream within 24 hours. Any work done on the map data is thus not only benefiting a specific event but all OSM users.

Finding rooms

Using OSM raw data and client-side rendering rather than pre-rendered raster graphics allows for additional ways to interact with the map or browse the data.

Itinerary has a nearby amenity search feature for example. What you are also often looking for at a conference venue are rooms though. We can apply the same concept here, with a few adjustments on which data to display and how to group it.

Room search dialog in Kongress.

This is also the foundation for matching room information from the schedule to the map. That’s not implemented yet in the app though, as we’ll need the actual schedule for testing that.

Custom styles

For a conference in e.g. a university building only a small subset of rooms might be relevant for an event. Custom made venue maps often highlight those somehow, e.g. with different colors.

As our indoor map renderer uses MapCSS stylesheets to control the visual appearance, that’s easy enough to do here as well. And since MapCSS stylesheets can include other files, we also only need a small event-specific addition that then includes the default style.

Even-specific stylesheet highlighting and labeling relevant rooms. Custom content

While stylesheet can do a lot, they can’t generate new content on the map. That’s also needed though, as there might be structures that are only there for the event and thus too short-lived to be added to OSM.

This can be solved by applying an OSM changeset file on top of the loaded map data. That’s basically a “diff” for the map data with event-specific additions, changes or removals.

Event-specific additional map content.

This way event-specific content becomes part of the map data itself rather than being in an app-specific overlay. We can therefore make use of all the existing infrastructure for the custom content as well, from support for multi-lingual labels to being searchable.

Outlook

And there’s still more we could do.

Time-dependent elements

Some elements might only be available or applicable for a certain time of the event, or certain rooms might have different purposes on different days.

OSM has a complex mechanism to model time-dependent information, using opening hours expressions. We can parse and interpret those as well, but their use is currently limited to showing actual opening hours for amenities or shops. This is accessible to styles, so we can e.g. gray out currently closed things.

Generalizing this and also allowing to interpret opening hours expression on arbitrary tags in combination with custom styles should not be too far out and give us a really powerful mechanism to model time-dependent features.

Routing

Kongress also has our experimental indoor router integrated. For the Akademy venue this works ok-ish (apart from access to the elevator, the map data quality around that isn’t good enough yet).

Indoor map routing in Kongress.

An important still missing part here however is the ability to select and customize routing profiles, as routing becomes particularly useful when considering mobility constraints.

Also note that we are only talking about routing here, full navigation is much further away still as we need to have working indoor localization for that as well, which is a much harder problem to solve.

Do we actually need all this?

For a comparatively small event like Akademy in a relatively simple two-floor venue, probably not. But think about your first visit at a massive event like FOSDEM, or consider stairs being an insurmountable obstacle. Having support with finding your way around becomes much more important then.

Akademy is a great opportunity to experiment with this though, both from the software and the mapping perspective, and to show what’s possible with the data and technology we already have.

New Features

You can now turn on the “autoscrolling” feature of the Libinput driver, which lets you scroll on any scrollable view by holding down the middle button of your mouse and moving the whole mouse (Evgeniy Chesnokov, Plasma 6.2.0. Link)

UI Improvements

When zooming into or out of a document in Okular using Ctrl+Scroll, it now zooms into or out of the actual cursor position, not the center of the page (Alexis Murzeau, Okular 24.08.0. Link)

Okular now scales radio buttons and checkboxes to the size of the form fields they inhabit, which looks better for forms that have huge or tiny versions of these (Pratham Gandhi, Okular 24.08.0. Link)

Dolphin now supports the systemwide “disable smooth scrolling” setting (Nathan Misner, Dolphin 24.08.0 Link)

Opening and closing Elisa’s playlist panel is no longer somewhat choppy (Jack Hill, Elisa 24.08. Link)

When quick-tiling two adjacent windows and resizing one, the other will resize too. The location of the split between them is now reset to its default position after all adjacent quick-tiled windows are closed or un-tiled (Erwin Saumweber, Plasma 6.1.2. Link)

.Desktop files in sub-folders below your desktop are now shown as they are on the desktop itself (Alexander Wilms, Plasma 6.2.0. Link)

On System Settings’ Accessibility page, the Open dialog for choosing custom bell sounds now accepts .oga files, and also tells you what types of files it supports (me: Nate Graham, Plasma 6.2.0. Link)

On System Settings Desktop Effects page, “internal” effects are no longer listed at all (even in a hidden-by-default state), which makes it more difficult for people to break their systems by accident, and also fixes an odd interaction whereby clicking the “Defaults” button would reset the default settings of internal effects changed elsewhere. You can still see the internal effects in KWin’s debug console window if needed (Vlad Zahorodnii, Plasma 6.2.0. Link)

Made a bunch of small changes to System Settings pages to align them better with the new human interface guidelines (me: Nate Graham, Plasma 6.2.0. Link 1, link 2, link 3, and link 4)

Improved the legibility of the text in Kirigami.NavigationTabBar buttons, especially on low or medium DPI screens (me: Nate Graham, Frameworks 6.4. Link)

Bug Fixes

Fixed a recent regression that caused the Powerdevil power management daemon to sometimes crash randomly when the system has any monitors connected that support DDC-based brightness control (Jakob Petsovits, Plasma 6.1.2. Link)

On the System Settings’ recently re-done Keyboard page, table columns in the layout table are once again resizable, and also have more sensible default widths now (Wind He, Plasma 6.1.2. Link)

Fixed one source of the recent issue with certain System Settings pages being sometimes broken when opened — this one being the issue where opening the Touchpad or Networks pages would break other ones opened afterwards. We’re still investigating the other issues, which frankly make no sense and shouldn’t be happening. Some of them may be Qt regressions. Investigation is ongoing (Marco Martin, Plasma 6.1.3. Link)

Icons in the new Edit Mode’s toolbar buttons are no longer slightly blurry (Akseli Lahtinen, Plasma 6.1.3. Link)

KWin’s “open new windows under pointer” feature now actually does, and ignores the active screen when that screen differs from the screen with the pointer on it (Xaver Hugl, Plasma 6.1.3. Link)

Fixed multiple recent regressions and longstanding issues with System Monitor widgets displayed on panels (Arjen Hiemstra, Plasma 6.2.0):

• Text in small pie charts overflowing onto the next line awkwardly (link)
• Adjacent pie charts overlapping at certain panel thicknesses (link)
• Graphs not taking enough space on a thick panel (link)

With wide color gamut turned on or an ICC color profile in use, transparent windows are no longer too transparent (Xaver Hugl, Plasma 6.2.0. Link)

Showing and hiding titlebars and frames on a scaled display no longer causes XWayland windows to move diagonally by about 1px every time (Vlad Zahorodnii, Plasma 6.2.0. Link)

Fixed multiple issues and glitches affecting floating panels via a significant code refactor (Marco Martin, Plasma 6.2.0. Link 1, link 2, and link 3)

Fixed a recent Qt regression that caused Plasma to sometimes crash when screens were disconnected (David Edmundson, Qt 6.7.3. Link 1 and link 2)

Fixed a Qt regression that caused web pages rendered by QtWebEngine (most notably in KMail’s HTML message viewer window) to display have blocky, blurry, or pixelated text and graphics (David Edmundson, Qt 6.8.0. Link)

Other bug information of note:

• 3 Very high priority Plasma bugs (down from 4 last week). Current list of bugs
• 31 15-minute Plasma bugs (up from 29 last week). Current list of bugs
• 98 KDE bugs of all kinds fixed over the last week. Full list of bugs

Performance & Technical

Made the pam_kwallet library able to build with libgcrypt 1.11, restoring its ability to let the system wallet unlock automatically on login again (Daniel Exner, Plasma 6.1.2. Link)

Automation & Systematization

Added some UI tests to KCalc, ensuring that the recent prominent regression in functionality can’t happen again (Gabriel Barrantes, link)

…And Everything Else

This blog only covers the tip of the iceberg! If you’re hungry for more, check out https://planet.kde.org, where you can find more news from other KDE contributors.

How You Can Help

As I mentioned last week, if you use have multiple systems or an adventurous personality, you can really help us out by installing beta versions of Plasma using your distro’s available repos and reporting bugs. Arch, Fedora, and openSUSE Tumbleweed are examples of great distros for this purpose. So please please do try out Plasma beta versions. It truly does help us! Heck, if you’re very adventurous, live on the nightly repos. I’ve been doing this full-time for 5 years with my sole computer and it’s surprisingly stable.

Does that sound too scary? Consider donating today instead! That helps too.

Otherwise, visit https://community.kde.org/Get_Involved to discover other ways to be part of a project that really matters. Each contributor makes a huge difference in KDE; you are not a number or a cog in a machine! You don’t have to already be a programmer, either. I wasn’t when I got started. Try it, you’ll like it! We don’t bite!

Last post featured the DAG Hamilton generated graphviz graph shown below. I'll be dressing this up a little and highlighting some functionality. For the toy example here, the script employed is a bit of overkill. For a bigger workflow, it may come in handy.

I'll start with the finished products:

1) A Hamilton logo and a would be company logo get added (manual; the Data Inputs Highlighted subtitle is there for later processing when we highlight functionality.)
2) through 4) are done programmatically (code is shown further down). I saw an example on the Hamilton web pages that used aquamarine as the highlight color; I liked that, so I stuck with it.

2) Data source and data source function highlighted.



3) Web scraping functions highlighted.



4) Output nodes highlighted.


A few observations and notes before we look at configuration and code: I've found the charts to be really helpful in presenting my workflow to users and leadership (full disclosure: my boss liked some initial charts I made; my dream of the PowerPoint to solve all scripter<->customer communication challenges is not yet reality, but for the first time in a long time, I have hope.)

In the web scraping highlighted diagram, you can pretty clearly see that data_with_company node has an input into the commodity_word_counts node. The domain specific rationale from the last blog post is that I don't want to count every "Barrick Gold" company name occurrence as another mention of "Gold" or "gold."

Toy example notwithstanding, in real life, being able to show where something branches critically is a real help. Assumptions about what a script is actually doing versus what it is doing can actually be costly in terms of time and productivity for all parties. Being able to say and show ideas like, "What it's doing over here doesn't carry over to that other mission critical part you're really concerned with; it's only for purposes of the visualization which lies over here on the diagram" or "This node up here representing <the real life thing> is your sole source of input for this script; it is not looking at <other real world thing> at all."

graphviz and diagrams like this have been around for decades - UML, database schema visualizations, etc. What makes this whole DAG Hamilton thing better for me is how easy and accessible it is. I've seen C++ UML diagrams over the years (all respect to the C++ people - it takes a lot of ability, discipline, and effort); my first thought is often, "Oh wow . . . I'm not sure I have what it takes to do that . . . and I'm not sure I'd want to . . ."

Enough rationalization and qualifying - on to the config and the code!

I added the title and logos manually. The assumption that the graphviz dot file output of DAG Hamilton will always be in the format shown would be premature and probably wrong. It's an implementation detail subject to change and not a feature. That said, I needed some features in my graph outputs and I achieved them this one time.

Towards the top of the dot file is where the title goes:

// Dependency Graphdigraph {        labelloc="t"        label=<<b>Toy Web Scraping Script Run Diagram<BR/>Data Inputs Highlighted</b>> fontsize="36" fontname=Helvetica

labelalloc="t" puts the text at the top of the graph (t for top, I think).
// Dependency Graphdigraph {        labelloc="t"        label=<<b>Toy Web Scraping Script Run Diagram<BR/>Data Inputs Highlighted</b>> fontsize="36" fontname=Helvetica        hamiltonlogo [label="" image="hamiltonlogolarge.png" shape="box", width=0.6, height=0.6, fixedsize=true]        companylogo [label="" image="fauxcompanylogo.png" shape="box", width=5.10 height=0.6 fixedsize=true]

The DAG Hamilton logo listed first appears to end up in the upper left part of the diagram most of the time (this is an empirical observation on my part; I don't have a super great handle on the internals of graphviz yet).

Getting the company logo next to it requires a bit more effort. A StackOverflow exchange had a suggestion of connecting it invisibly to an initial node. In this case, that would be the data source. Inputs in DAG Hamilton don't get listed in the graphviz dot file by their names, but rather by the node or nodes they are connected to: _parsed_data_inputs instead of "datafile" like you might expect. I have a preference for listing my input nodes only once (deduplicate_inputs=True is the keyword argument to DAG Hamilton's driver object's display_all_functions method that makes the graph).

The change is about one third of the way down the dot file where the node connection edges start getting listed:

parsed_data -> data_with_wikipedia _parsed_data_inputs [label=<<table border="0"><tr><td>datafile</td><td>str</td></tr></table>> fontname=Helvetica margin=0.15 shape=rectangle style="filled,dashed" fillcolor="#ffffff"]        companylogo -> _parsed_data_inputs [style=invis]

DAG Hamilton has a dashed box for script inputs. That's why there is all that extra description inside the square brackets for that node. I manually added the fillcolor="#ffffff" at the end. It's not necessary for the chart (I believe the default fill of white /#ffffff was specified near the top of the file), but it is necessary for the code I wrote to replace the existing color with something else. Otherwise, it does not affect the output.

I think that's it for manual prep.

Onto the code. Both DAG Hamilton and graphviz have API's for customizing the graphviz dot file output. I've opted to approach this with brute force text processing. For my needs, this is the best option. YMMV. In general, text processing any code or configuration tends to be brittle. It worked this time.

# python 3.12
"""Try to edit properties of graphviz output."""
import sys
import re
import itertools
import graphviz
INPUT = 'ts_with_logos_and_colors'
FILLCOLORSTRLEN = 12AQUAMARINE = '7fffd4'COLORLEN = len(AQUAMARINE)
BOLDED = ' penwidth=5'BOLDEDEDGE = ' [penwidth=5]'
NODESTOCOLOR = {'data_source':['_parsed_data_inputs',                               'parsed_data'],                'webscraping':['data_with_wikipedia',                               'colloquial_company_word_counts',                               'data_with_company',                               'commodity_word_counts'],                'output':['info_output',                          'info_dict_merged',                          'wikipedia_report']}
EDGEPAT = r'\b{0:s}\b[ ][-][>][ ]\b{1:s}\b'
TITLEPAT = r'Toy Web Scraping Script Run Diagram[<]BR[/][>]'ENDTITLEPAT = r'</b>>'
# Two tuples as values for edges.EDGENODESTOBOLD = {'data_source':[('_parsed_data_inputs', 'parsed_data')],                   'webscraping':[('data_with_wikipedia', 'colloquial_company_word_counts'),                                  ('data_with_wikipedia', 'data_with_company'),                                  ('data_with_wikipedia', 'commodity_word_counts'),                                  ('data_with_company', 'commodity_word_counts')],                   'output':[('data_with_company', 'info_output'),                             ('colloquial_company_word_counts', 'info_dict_merged'),                             ('commodity_word_counts', 'info_dict_merged'),                             ('info_dict_merged', 'wikipedia_report'),                             ('data_with_company', 'info_dict_merged')]}
OUTPUTFILES = {'data_source':'data_source_highlighted',               'webscraping':'web_scraping_functions_highlighted',               'output':'output_functions_highlighted'}
TITLES = {'data_source':'Data Sources and Data Source Functions Highlighted',          'webscraping':'Web Scraping Functions Highlighted',          'output':'Output Functions Highlighted'}
def get_new_source_nodecolor(src, nodex):    """    Return new source string for graphviz    with selected node colored aquamarine.
    src is the original graphviz text source    from file.
    nodex is the node to have it's color edited.    """    # Full word, exact match.    wordmatchpat = r'\b' + nodex + r'\b'    pat = re.compile(wordmatchpat)    # Empty string to hold full output of edited source.    src2 = ''    match = re.search(pat, src)    # nodeidx = src.find(nodex)    nodeidx = match.span()[0]    print('nodeidx = ', nodeidx)    src2 += src[:nodeidx]    idxcolor = src[nodeidx:].find('fillcolor')    print('idxcolor = ', idxcolor)    # fillcolor="#b4d8e4"    # 012345678901234567    src2 += src[nodeidx:nodeidx + idxcolor + FILLCOLORSTRLEN]    src2 += AQUAMARINE    currentposit = nodeidx + idxcolor + FILLCOLORSTRLEN + COLORLEN    src2 += src[currentposit:]    return src2
def get_new_title(src, title):    """    Return new source string for graphviz    with new title part of header.
    src is the original graphviz text source    from file.
    title is a string.    """    # Empty string to hold full output of edited source.    src2 = ''    match = re.search(TITLEPAT, src)    titleidx = match.span()[1]    print('titleidx = ', titleidx)    src2 += src[:titleidx]    idxendtitle = src[titleidx:].find(ENDTITLEPAT)    print('idxendtitle = ', idxendtitle)    src2 += title    currentposit = titleidx + idxendtitle    print('currentposit = ', currentposit)    src2 += src[currentposit:]    return src2
def get_new_source_penwidth_nodes(src, nodex):    """    Return new source string for graphviz    with selected node having bolded border.
    src is the original graphviz text source    from file.
    nodex is the node to have its box bolded.    """    # Full word, exact match.    wordmatchpat = r'\b' + nodex + r'\b'    pat = re.compile(wordmatchpat)    # Empty string to hold full output of edited source.    src2 = ''    match = re.search(pat, src)    nodeidx = match.span()[0]    print('nodeidx = ', nodeidx)    src2 += src[:nodeidx]    idxbracket = src[nodeidx:].find(']')    src2 += src[nodeidx:nodeidx + idxbracket]    print('idxbracket = ', idxbracket)    src2 += BOLDED    src2 += src[nodeidx + idxbracket:]    return src2
def get_new_source_penwidth_edges(src, nodepair):    """    Return new source string for graphviz    with selected node pair having bolded edge.
    src is the original graphviz text source    from file.
    nodepair is the two node tuple to have    its edge bolded.    """    # Full word, exact match.    edgepat = EDGEPAT.format(*nodepair)    print(edgepat)    pat = re.compile(edgepat)    # Empty string to hold full output of edited source.    src2 = ''    match = re.search(pat, src)    edgeidx = match.span()[1]    print('edgeidx = ', edgeidx)    src2 += src[:edgeidx]    src2 += BOLDEDEDGE     src2 += src[edgeidx:]    return src2
def makehighlightedfuncgraphs():    """    Cycle through functionalities to make specific    highlighted functional parts of the workflow    output graphs.
    Returns dictionary of new filenames.    """    with open(INPUT, 'r') as f:        src = f.read()
    retval = {}        for functionality in TITLES:        print(functionality)        src2 = src        retval[functionality] = {'dot':None,                                 'svg':None,                                 'png':None}        src2 = get_new_title(src, TITLES[functionality])        # list of nodes.        to_process = (nodex for nodex in NODESTOCOLOR[functionality])        countergenerator = itertools.count()        count = next(countergenerator)        print('\nDoing node colors\n')        for nodex in to_process:            print(nodex)            src2 = get_new_source_nodecolor(src2, nodex)            count = next(countergenerator)        to_process = (nodex for nodex in NODESTOCOLOR[functionality])        countergenerator = itertools.count()        count = next(countergenerator)        print('\nDoing node bolding\n')        for nodex in to_process:            print(nodex)            src2 = get_new_source_penwidth_nodes(src2, nodex)            count = next(countergenerator)        print('Bolding edges . . .')        to_process = (nodex for nodex in EDGENODESTOBOLD[functionality])        countergenerator = itertools.count()        count = next(countergenerator)        for nodepair in to_process:            print(nodepair)            src2 = get_new_source_penwidth_edges(src2, nodepair)            count = next(countergenerator)        print('Writing output files . . .')        outputfile = OUTPUTFILES[functionality]        with open(outputfile, 'w') as f:            f.write(src2)        graphviz.render('dot', 'png', outputfile)        graphviz.render('dot', 'svg', outputfile)
makehighlightedfuncgraphs()

Thanks for stopping by.

This tutorial details how to integrate GraphQL with Django using Strawberry.

Authored by Nadiia Nykolaichuk.

Drupal is ceaselessly evolving, with the best Drupal minds nurturing brilliant ideas and implementing them in the new releases. Six months ago, Drupal 10.2 rolled out with a set of exciting enhancements. Now it’s time to celebrate that Drupal 10.3 has been officially released on June 20.

Have you ever wondered how to get a friend or colleague or even a complete stranger hooked up with free software? Here's the ultimate guide.

Have you ever wondered how to get a friend or colleague or even a complete stranger hooked up with free software? Here's the ultimate guide.

Continuous forward compatibility checking of extensions for Drupal 12, 13, etc using @gitlab CI 😍 https://www.hojtsy.hu/blog/2024-jul-05/continuous-forward-compatibility-checking-extensions-drupal-12-13-etc Get inline and ongoing feedback about readiness for upcoming major versions of Drupal as you develop extensions!

1.5 week prior, Lee “larowlan” + Jesse fixed a bug in the undo/redo functionality and added the first unit test (#3452895), and added it to CI (where it now runs immediately, no need to wait for composer to run JS unit tests!) Except … the unit tests didn’t actually run — oops! Rectifying that revealed a whole range of new Cypress challenges, which Ben “bnjmnm” worked tirelessly to solve this during the entire 5th week, and it was merged on Wednesday of this week :)

Anybody who has contributed to the drupal.org GitLab CI templates knows how painful this can be!

Missed a prior week? See all posts tagged Experience Builder.

Goal: make it possible to follow high-level progress by reading ~5 minutes/week. I hope this empowers more people to contribute when their unique skills can best be put to use!

For more detail, join the #experience-builder Slack channel. Check out the pinned items at the top!

As alluded to in last week’s update, I’m shifting my focus to coordinating.

That, together with quite a few people being out or preparing for webinars or Drupal Dev Days Burgas means this is an unusually short update — also because I left on vacation on Friday the 21st of June.

Before going on vacation, I wanted to ensure work could continue in my absence because we need to get to the point where Lauri’s vision is accessible in both UX wireframe form (Lauri’s working on that with Acquia UX) and technical diagram form (up to me to get that going). So, since last week:

1. Lauri created #3454094: Milestone 0.1.0: Experience Builder Demo. I created #3455753: Milestone 0.2.0: Experience Builder-rendered nodes. Anything that is not necessary for either of those two should not be worked on at this time. They together form “the early phase” — the milestone 0.1.0 has a hard DrupalCon Barcelona 2024 deadline in September, the 0.2.0 does not have a similar firm date.
2. As much of 0.2.0 as possible should already be in 0.1.0, which means ideally the back end is ahead of the front end. That’s what #3450586: [META] Early phase back-end work coordination and #3450592: [META] Early phase front-end work coordination. are for. I did a big update to have the next ~10 or so back-end things to build spelled out in detail in concrete issues, with vaguer descriptions for the things that are further out and subject to change anyway.
3. Initial diagram of the data model as currently partially implemented and the direction we’ve been going in … followed by a significant expansion of detail. You can see the diagrams on GitLab, in the docs/diagrams directory.
4. The JSON-based data storage model is influenced significantly by some of the product requirements, and what those are and their exact purpose has not been very clear. To fix that, I created [later phase] [META] 7. Content type templates — aka “default layouts” — affects the tree+props data model (which updates the data model diagram!) — and Lauri recorded a video with his thinking around this, in which he walks through two diagrams: one for data sources + design system, one that breaks down a concrete content type.
5. A lot of discussion happened between Lauri, catch and I on [META] Configuration management: define needed config entity types, which needs a lot more clarity before all config entity types can be implemented.

One pretty cool issue landed this week that drives home that second item : #3455898: Connect client & server, with zero changes to client (UI): rough working endpoints that mimic the UI’s mocks — thanks to that, the PoC UI is now optionally populated by the first article node, unless you enable development mode (see ui/README.md), then it uses dummy data not served by Drupal. A small but hacky change but an important pragmatic step in the right direction :) And it unblocks Jesse on next steps on the UI!

Try it yourself locally if you like, but there’s not much you can do yet.
Install the 0.x branch — the “Experience Builder PoC” toolbar item takes you there!

Weeks 7 and 8 will be special editions: I will have been completely absent during week 7 (plus, it’ll be Drupal Dev Days!), and present only for the last day of week 8. I’ll catch up what happened and do a write-up both for myself as well as all of you!

Thanks to Lauri for reviewing this!

• Drupal
• Acquia
• Experience Builder

• The UI starts being powered by the server side.

Continuous forward compatibility checking of extensions for Drupal 12, 13, etc

We still keep improving the ecosystem readiness and tooling with each new major Drupal core version. Drupal 11 is to be released in a few weeks on the week of July 29 (so probably the first days of August) and already almost half of the top 200 modules are ready. But we need to keep thinking ahead.

The Project Update Bot (originally built by Ted Bowman at Acquia and since then very actively owned and improved by Björn Brala at SWIS) posted into more than 7600 project issue queues on Drupal.org with merge request suggestions to improve and in many cases solve compatibility with the upcoming major version. 

The bot is a clever combination of Upgrade Status and drupal-rector with some custom decision logic. So humans can also run those tools! But what if we automate it even more? What if we help pre-empt forwards incompatible code getting into modules in the first place? 

Gábor Hojtsy Fri, 07/05/2024 - 20:34