Feeds

GNU anubis version 4.3 is available for download. This is a maintenance release, including some new features:

anubisusr requires GnuTLS

New configuration statement: use-pam

 
Used in CONTROL section, this boolean statement enables or disables the use of the Pluggable Authentication Module interface for accounting and session management.

New configuration statement: identd-keyfile

Sets the name of the file with shared keys used for decrypting replies from the auth service.  It is used in traditional mode if anubis receives an encrypted response from the client's identd server (e.g. if they are running pidentd with encryption).

A week has passed since I attended the 37th Chaos Communication Congress (37C3) in Hamburg, Germany, together with a bunch of other KDE people.

KDE Assembly

For the first time KDE had a larger presence, with a number of people and our own assembly. We also ended up hosting the Linux on Mobile assembly as their originally intended space didn’t materialize. The extra space demand was compensated by assimilating neighbouring tables of a group that didn’t show up for the most part.

KDE assembly featuring the event-typical colorful LEDs and sticker piles (photo by Victoria).

Just for the logistical convenience of having a meeting point and a place to store your stuff alone this was a big help, and it also made a number of people find us that we’d otherwise probably wouldn’t have met.

Talks

There was one KDE talk in the main program, Joseph’s Software Licensing For A Circular Economy covering some of the KDE Eco work. I’d estimate about 500 attendees, despite the “early” morning slot.

Beyond that I actually managed to attend very few talks, Breaking “DRM” in Polish trains being clearly one of my favorites.

Emergency and Weather Alerts

I got a chance to meet the author of FOSS Warn. FOSS Warn is a free alternative to the proprietary emergency and weather alert apps in Germany. That topic had originally motivated my work on UnifiedPush support for KDE, and an emergency and weather alert service that allows subscribing to areas of interest and receiving push notifications for new alerts, covering almost 100 countries.

The latter is a proof of concept demo at best though. The idea is to collaborate with the FOSS Warn team on a joint backend implementation, evolving this into something production-ready and usable for all of us.

While there’s still a couple of technicalities to resolve, after having met in person I’m very confident that this will all work out.

Open Transport Community

Other groups I’m involved with were present at 37C3 as well, like the Open Transport community, with the Bahnbubble Meetup being the event that brought everyone together.

Discussion topics included:

• Identifying and getting access to missing data tables and documentations for DB and VDV ticket barcodes.
• Helping others with implementing the ERA FCB ticket format.
• Evolving the Transport API Repository.
• International collaboration and networking between the Open Transport communities in Europe,
• Integration between Itinerary and Träwelling.
• Train station identifiers in Wikidata, including a recent property proposal for DB station numbers, as well as a yet to be written one for IFOPT identifiers.

Many more topics ended up being discussed in parallel, overall I’d say there’s more than enough content and interested people for its own dedicated event/conference on this. Until somebody organizes that there’s the open-transport Matrix channel and the Railways and Open Transport track at FOSDEM.

Indoor Localization and Routing

The OSM community was at 37C3 as well of course, and in that context I got one of the probably most unexpected contacts, by meeting one of the authors of simpleLoc. That’s an indoor navigation solution, which is one of the big open challenges in our indoor map view used in Itinerary.

Like other such solutions it’s unfortunately not fully available as Free Software, but the available information, components and published papers nevertheless turned out very valuable, in particular since this is using a localization approach that requires only commonly available smartphone sensors.

Much more immediately applicable were the hints on how they implemented indoor routing. Unlike for “outdoor” navigation graph-based algorithms are usually not an option, we need something that works on polygons in order to use OSM data as input directly. Existing solutions I encountered for that in a mapping/geography context were all proprietary unfortunately, but there’s a free library that solves that problem for 3D game engines: Recast Navigation.

While I’m not entirely sure yet how to map all relevant details to that (e.g. directional paths for escalators, tactile paving guides, etc), the initial experiments look very promising.

Routing through a complex indoor corridor.

There’s obviously a ton of work left, as this essentially requires mapping all relevant bits of OSM indoor data to a 3D model, and that’s at the very limit of what can be extracted from the OSM data and data model in many places.

KDE Outreach

While people working on technology make up a significant part of the audience of 37C3, there’s many more people from other areas attending as well, so this also was an opportunity for a bit of user research, following the pattern of the kde.org/for pages.

KDE for activists

On the KDE for Activists page we focus a lot on tools for secure and self-hosted infrastructure. The need for that seems like a given or even a hard requirement for people in that field, not something that needs to be argued for (the event itself might provide a certain selection bias for that though).

What we however need to improve on is making this much more robust and easy to setup and manage for people that might not be familiar with all the technical details, jargon and abbreviations we expose them to. Even worse, that can pose the risk of making dangerous mistakes, up to causing people physical harm.

KDE for FOSS projects

Meeting other FOSS developers is also interesting, in particular those not well connected to our usual bubble. That’s often single person projects, some of them even quite successful ones. Around those KDE is then the odd outlier, due to our size and choice of tools/infrastructure (ie. not Github).

Topics that typically come up then are handling of finances, legal risks/liability, shortcomings of Github, moderation of communication channels, etc., all things far less painful from the perspective of someone under the umbrella of a big organization like KDE.

I think there’s some interesting discussions to be had on how widely we want to extend the KDE umbrella and how we want to promote that, what other umbrella organizations there are and whether there are still uncovered gaps between those, and how to manage the scope and governance of such umbrella organizations.

It’s probably also worth talking more about what we already have and do in that regard, it’s not even clear to everyone apparently that joining a larger organization is even an option.

KDE for public administration

We got questions for pointers to material/support for doing medium-sized Linux/Plasma deployments in public administration. This is unfortunately something we don’t have anywhere in a well structured form currently I think.

It would seem very useful to have, beyond a KDE focus even. There’s people on the inside fighting for this, and while the upcoming Windows 11 induced large-scale hardware obsolescence is working in our favor there, the increasingly pervasive use of MS Teams is making a migration to FOSS infrastructure and workspaces much harder.

Conclusion

I had very high expectations for this after the experience at 36C3, but by day 3 this had exceeded all of them. The extreme breadth of people there is just unmatched, coming from FOSS/Open Data/Open Hardware projects tiny to large, public administration and infrastructure, education/universities, funding organizations, politics and lobbying, civil/social initiatives, you name it.

And all of that in a fun atmosphere that never stops to amaze. While walking down a corridor you might find yourself overtaken by a person driving a motorized couch, and if you have an urgent need for an electron microscope for whatever reason someone over in the other hall brought one just in case. And all of that is just “normal”, I could fill this entire post with anecdotes like that.

Ideas and plans for 38C3 were already discussed :)

The following article demonstrates a useful trick in Tkinter to create multiple “screens” (e.g. a login screen, register screen, main page screen) within a single window! Instead of creating a new window for each screen, we will use a single tkinter window, which swaps dynamically between multiple “screens” (represented by frames).

This is more efficient and faster than creating a new window every time.

Complete Code:

The core idea is simple, we have multiple classes, each of which represents a window. These classes inherit from the Frame class, essentially making them frames as well. To “swap” between screens, we destroy the existing frame (including all of its children objects) and then create the new frame to take its place.

A YouTube video explaining this code step-by-step is included at the bottom of this article.

import tkinter as tk def center_window(width, height): x = (root.winfo_screenwidth() // 2) - (width // 2) y = (root.winfo_screenheight() // 2) - (height // 2) root.geometry(f'{width}x{height}+{x}+{y}') class WelcomeWindow(tk.Frame): def __init__(self, master): super().__init__() self.master = master self.master.title("Welcome") center_window(200, 150) login_button = tk.Button(self, text="Login", width=10, command = self.on_login) login_button.pack(padx=20, pady=(20, 10)) register_button = tk.Button(self, text="Register", width=10, command = self.on_register) register_button.pack(pady=10) self.pack() def on_login(self): for widget in self.winfo_children(): widget.destroy() self.destroy() LoginWindow(self.master) def on_register(self): for widget in self.winfo_children(): widget.destroy() self.destroy() RegisterWindow(self.master) class LoginWindow(tk.Frame): def __init__(self, master): super().__init__() self.master = master self.master.title("Login") self.master.resizable(False, False) center_window(250, 150) tk.Label(self, text="Username:").grid(row=0, column=0) self.username_entry = tk.Entry(self) self.username_entry.grid(row=0, column=1, padx=10, pady=10) tk.Label(self, text="Password:").grid(row=1, column=0) self.password_entry = tk.Entry(self, show="*") self.password_entry.grid(row=1, column=1, padx=10, pady=10) submit_button = tk.Button(self, text="Submit", width=8,command = self.on_successful_login) submit_button.grid(row=2, column=1, sticky="e", padx=10, pady=(10, 0)) submit_button = tk.Button(self, text="Back", width=8, command = self.on_back) submit_button.grid(row=2, column=0, sticky="w", padx=10, pady=(10, 0)) self.pack() def on_back(self): for widget in self.winfo_children(): widget.destroy() self.destroy() WelcomeWindow(self.master) def on_successful_login(self): for widget in self.winfo_children(): widget.destroy() self.destroy() MainWindow(self.master) class RegisterWindow(tk.Frame): def __init__(self, master): super().__init__() self.master = master self.master.title("Register") self.master.resizable(False, False) center_window(300, 250) tk.Label(self, text="First Name:").grid(row=0, column=0, sticky="w") self.first_name_entry = tk.Entry(self, width=26) self.first_name_entry.grid(row=0, column=1, padx=10, pady=10, sticky="e") tk.Label(self, text="Last Name:").grid(row=1, column=0, sticky="w") self.last_name_entry = tk.Entry(self, width=26) self.last_name_entry.grid(row=1, column=1, padx=10, pady=10, sticky="e") tk.Label(self, text="Password:").grid(row=2, column=0, sticky="w") self.password_entry = tk.Entry(self, show="*", width=26) self.password_entry.grid(row=2, column=1, padx=10, pady=10, sticky="e") tk.Label(self, text="Email:").grid(row=3, column=0, sticky="w") self.email_entry = tk.Entry(self, width=26) self.email_entry.grid(row=3, column=1, padx=10, pady=10, sticky="e") submit_button = tk.Button(self, text="Submit", width=8) submit_button.grid(row=7, column=1, padx=10, pady=10, sticky="e") submit_button = tk.Button(self, text="Back", width=8, command = self.on_back) submit_button.grid(row=7, column=0, sticky="w", padx=10, pady=(10, 10)) self.pack() def on_back(self): for widget in self.winfo_children(): widget.destroy() self.destroy() WelcomeWindow(self.master) class MainWindow(tk.Frame): def __init__(self, master): super().__init__() self.master = master center_window(500, 500) self.pack() root = tk.Tk() root.eval('tk::PlaceWindow . center') WelcomeWindow(root) root.mainloop()

This marks the end of the Switching between multiple Screens in Tkinter Tutorial. Any suggestions or contributions for CodersLegacy are more than welcome. Questions regarding the tutorial content can be asked in the comments section below.

The post Switching between Multiple Screens in Tkinter (dynamically) appeared first on CodersLegacy.

I was just recently starting to get this on projects after (at least I think that's why) I updated to Chrome 120.

Could not open connection: unknown error: cannot find Chrome binary (Driver info: chromedriver=120.0.6099.71 (9729082fe6174c0a371fc66501f5efc5d69d3d2b-refs/branch-heads/6099_56@{#13}),platform=Linux 6.2.0-37-generic x86_64) (Behat\Mink\Exception\DriverException)

That's the error message in behat at least, which I think originates from the actual webdriver (chromedriver) response. If I look at the debug information from the chromedriver logs it says this:

[1703837483,910][INFO]: [0ca18bb59db30d5acd358de02a01da0a] RESPONSE InitSession ERROR unknown error: cannot find Chrome binary

Well the error is clear enough. It can not find the binary. That's fine by me, but where would I go about informing about the binary? Well, for me that would be in behat.yml:

@@ -33,8 +33,9 @@ default: w3c: false marionette: null chrome: + binary: /usr/bin/google-chrome switches:

This probably translates to something like this, while initiating the session with chromedriver (slightly edited for relevance and brevity):

[1703887172,675][INFO]: [95b2908582293fa560a7301661f5e741] COMMAND InitSession { "desiredCapabilities": { "chrome.binary": "/usr/bin/google-chrome", "chrome.extensions": [ ], "chrome.switches": [ "--ignore-certificate-errors", "--disable-gpu", "--no-sandbox", "--disable-dev-shm-usage" ], "goog:chromeOptions": { "args": [ "--ignore-certificate-errors", "--disable-gpu", "--no-sandbox", "--disable-dev-shm-usage" ], "binary": "/usr/bin/google-chrome", "extensions": [ ] }, "ignoreZoomSetting": false, "marionette": true, } }

If you are using some other tool that interacts with chromedriver, I am sure you are already setting some parameters there, which you could append this new parameter to.

Here’s my scenario: I have a static site generator that is building HTML pages for a community project that I’m working on. How can I make sure, automatically, that all the links to other internal pages within the site continue to work? In this article, I’ll show you how I managed to do that using Scrapy, a web scraping tool, and GitHub Actions, the project’s Continuous Integration system. To solve this problem, I decided to use a web scraper.

Posted on January 6, 2024
Tags: madeof:bits, meta

After just a tiny1 delay I’ve finally added support for tags to this blog.

In the next few days I may go back and add / change tags to the older posts, or I may not, I’ll decide.

Also, I still need to render a tag cloud somewhere; maybe it will happen soon, maybe it will take another year. :D

I hope I’ve also succesfully worked around the bug in Friendica where the src for images got deleted instead of properly converted.

1. less than one year!↩︎

Check out the important work our volunteers accomplished at today's Free Software Directory (FSD) IRC meeting.

Check out the important work our volunteers accomplished at today's Free Software Directory (FSD) IRC meeting.

Amanda Luker

Senior Frontend Engineer

As a Senior Frontend Engineer at Four Kitchens, Amanda is responsible for translating visual designs and applying them to Drupal sites.

January 1, 1970

Drupal provides a system for site administrators to add their own images and have them appear uniformly on the website. This system is called Image Styles. This tool can resize, crop, and scale images to fit any aspect ratio required by a design.

When creating responsive websites, a single image style for each image variation is insufficient. Each image, such as a hero image, a card image, a WYSIWYG image, or a banner image, requires multiple versions of one image. This ensures that the website delivers only what visitors need based on their screen size. For instance, a mobile user may only require a 320-pixel-wide image, while a large desktop user may want an 1,800-pixel-wide image (doubled for double-pixel density). For this reason, Drupal has Responsive Image Styles, which will group your images into a set of styles that will each show under different conditions.

Practical approach to convert images from design to Drupal

• Determine your image’s aspect ratio. If you find that the images in the design are not in a common aspect ratio (like 1:1, 2:1, 4:3, or 16:9) or if they vary by a little bit, consider running the dimensions through a tool that will find the closest reasonable aspect ratio.
• Determine the smallest and largest image sizes. For example, for a 16:9 aspect ratio, the smallest size might be 320 pixels x 180 pixels, while the largest could be 3,200 pixels x 1,800 pixels (doubled for high-density screens).
• To generate all variations, you can use an AI tool to print images with 160-pixel increments between each size. 160 increments tend to hit a lot of common breakpoints. Here’s an example using GitHub CoPilot:

There are likely more ways to streamline this process with Copilot. I’ve also used ChatGPT to rewrite them using a prefix, making it easy to add them in Drupal like this:

If adding all of these steps seems like a lot of work, consider using the Easy Responsive Images module! This module can create image styles for you, allowing you to set your aspect ratios and the increment between each style.

Once you have all your styles in place, create your responsive image styles by following these steps:

• Choose a name for your responsive image style based on its usage
• Select the “responsive image” breakpoint group
• Usually, I choose to select multiple image styles and use the sizes attribute. Use the sizes attribute to craft your “sizes.” For example:

(min-width:960px) 50vw, (min-width:1200px) 30vw, 100vw

In this example, choosing an image that is smaller than 960 pixels will best fit the full width of the viewport. At 960 pixels, the image will be selected to best fill half of the viewport width, and at 1,200 pixels, 30%. This approach is nimble and allows the browser to choose the most appropriate image for each case.

After setting the size rules, choose all of the image styles that you want the browser to be able to use. You don’t have to use them all. In some cases, you might have two responsive image styles that are pulling from the same aspect ratio image styles, but one uses all of them and the other uses a subset of them.

After adding your responsive image style, you need to map your Media View Mode:

1. Go to https://[your-site.local]/admin/structure/display-modes/view/add/media
2. Add the media view mode as a new Display for Images: https://[your-site.local]/admin/structure/media/manage/image/display
3. Choose “Responsive image” as the Format and select your new responsive image style

Once you have set this up, you are ready to use the View Mode to display the image field for your entity.

In this example, all the images have the same breakpoint. There may be times when you need to have different aspect ratios at different breakpoints. In those cases, you may want to use your custom theme’s Breakpoint Group. This will allow you to manually select each image style on for each breakpoint (instead of letting Drupal choose it for you).

The post Responsive image best practices for Drupal appeared first on Four Kitchens.

In a recent dispatch, I explained the whole-program compilation strategy used in Whiffle and Hoot. Today’s note explores what a correct solution might look like.

being explicit

Consider a module that exports an increment-this-integer procedure. We’ll use syntax from the R6RS standard:

(library (inc) (export inc) (import (rnrs)) (define (inc n) (+ n 1)))

If we then have a program:

(import (rnrs) (inc)) (inc 42)

Then the meaning of this program is clear: it reduces to (+ 42 1), then to 43. Fine enough. But how do we get there? How does the compiler compose the program with the modules that it uses (transitively), to produce a single output?

In Whiffle (and Hoot), the answer is, sloppily. There is a standard prelude that initially has a number of bindings from the host compiler, Guile. One of these is +, exposed under the name %+, where the % in this case is just a warning to the reader that this is a weird primitive binding. Using this primitive, the prelude defines a wrapper:

... (define (+ x y) (%+ x y)) ...

At compilation-time, Guile’s compiler recognizes %+ as special, and therefore compiles the body of + as consisting of a primitive call (primcall), in this case to the addition primitive. The Whiffle (and Hoot, and native Guile) back-ends then avoid referencing an imported binding when compiling %+, and instead produce backend-specific code: %+ disappears. Most uses of the + wrapper get inlined so %+ ends up generating code all over the program.

The prelude is lexically splatted into the compilation unit via a pre-expansion phase, so you end up with something like:

(let () ; establish lexical binding contour ... (define (+ x y) (%+ x y)) ... (let () ; new nested contour (define (inc n) (+ n 1)) (inc 42)))

This program will probably optimize (via partial evaluation) to just 43. (What about let and define? Well. Perhaps we’ll get to that.)

But, again here I have taken a short-cut, which is about modules. Hoot and Whiffle don’t really do modules, yet anyway. I keep telling Spritely colleagues that it’s complicated, and rightfully they keep asking why, so this article gets into it.

is it really a big letrec?

Firstly you have to ask, what is the compilation unit anyway? I mean, given a set of modules A, B, C and so on, you could choose to compile them separately, relying on the dynamic linker to compose them at run-time, or all together, letting the compiler gnaw on them all at once. Or, just A and B, and so on. One good-enough answer to this problem is library-group form, which explicitly defines a set of topologically-sorted modules that should be compiled together. In our case, to treat the (inc) module together with our example program as one compilation unit, we would have:

(library-group ;; start with sequence of libraries ;; to include in compilation unit... (library (inc) ...) ;; then the tail is the program that ;; might use the libraries (import (rnrs) (inc)) (inc 42))

In this example, the (rnrs) base library is not part of the compilation unit. Presumably it will be linked in, either as a build step or dynamically at run-time. For Hoot we would want the whole prelude to be included, because we don’t want any run-time dependencies. Anyway hopefully this would expand out to something like the set of nested define forms inside nested let lexical contours.

And that was my instinct: somehow we are going to smash all these modules together into a big nested letrec, and the compiler will go to town. And this would work, for a “normal” programming language.

But with Scheme, there is a problem: macros. Scheme is a “programmable programming language” that allows users to extend its syntax as well as its semantics. R6RS defines a procedural syntax transformer (“macro”) facility, in which the user can define functions that run on code at compile-time (specifically, during syntax expansion). Scheme macros manage to compose lexical scope from the macro definition with the scope at the macro instantiation site, by annotating these expressions with source location and scope information, and making syntax transformers mostly preserve those annotations.

“Macros are great!”, you say: well yes, of course. But they are a problem too. Consider this incomplete library:

(library (ctinc) (import (rnrs) (inc)) (export ctinc) (define-syntax ctinc (lambda (stx) ...)) // ***

The idea is to define a version of inc, but at compile-time: a (ctinc 42) form should expand directly to 43, not a call to inc (or even +, or %+). We define syntax transformers with define-syntax instead of define. The right-hand-side of the definition ((lambda (stx) ...)) should be a procedure of one argument, which returns one value: so far so good. Or is it? How do we actually evaluate what (lambda (stx) ...) means? What should we fill in for ...? When evaluating the transformer value, what definitions are in scope? What does lambda even mean in this context?

Well... here we butt up against the phasing wars of the mid-2000s. R6RS defines a whole system to explicitly declare what bindings are available when, then carves out a huge exception to allow for so-called implicit phasing, in which the compiler figures it out on its own. In this example we imported (rnrs) for the default phase, and this is the module that defines lambda (and indeed define and define-syntax). The standard defines that (rnrs) makes its bindings available both at run-time and expansion-time (compilation-time), so lambda means what we expect that it does. Whew! Let’s just assume implicit phasing, going forward.

The operand to the syntax transformer is a syntax object: an expression annotated with source and scope information. To pick it apart, R6RS defines a pattern-matching helper, syntax-case. In our case ctinc is unary, so we can begin to flesh out the syntax transformer:

(library (ctinc) (import (rnrs) (inc)) (export ctinc) (define-syntax ctinc (lambda (stx) (syntax-case stx () ((ctinc n) (inc n)))))) // ***

But here there’s a detail, which is that when syntax-case destructures stx to its parts, those parts themselves are syntax objects which carry the scope and source location annotations. To strip those annotations, we call the syntax->datum procedure, exported by (rnrs).

(library (ctinc) (import (rnrs) (inc)) (export ctinc) (define-syntax ctinc (lambda (stx) (syntax-case stx () ((ctinc n) (inc (syntax->datum #'n)))))))

And with this, voilà our program:

(library-group (library (inc) ...) (library (ctinc) ...) (import (rnrs) (ctinc)) (ctinc 42))

This program should pre-expand to something like:

(let () (define (inc n) (+ n 1)) (let () (define-syntax ctinc (lambda (stx) (syntax-case stx () ((ctinc n) (inc (syntax->datum #'n)))))) (ctinc 42)))

And then expansion should transform (ctinc 42) to 43. However, our naïve pre-expansion is not good enough for this to be possible. If you ran this in Guile you would get an error:

Syntax error: unknown file:8:12: reference to identifier outside its scope in form inc

Which is to say, inc is not available as a value within the definition of ctinc. ctinc could residualize an expression that refers to inc, but it can’t use it to produce the output.

modules are not expressible with local lexical binding

This brings us to the heart of the issue: with procedural macros, modules impose a phasing discipline on the expansion process. Definitions from any given module must be available both at expand-time and at run-time. In our example, ctinc needs inc at expand-time, which is an early part of the compiler that is unrelated to any later partial evaluation by the optimizer. We can’t make inc available at expand-time just using let / letrec bindings.

This is an annoying result! What do other languages do? Well, mostly they aren’t programmable, in the sense that they don’t have macros. There are some ways to get programmability using e.g. eval in JavaScript, but these systems are not very amenable to “offline” analysis of the kind needed by an ahead-of-time compiler.

For those declarative languages with macros, Scheme included, I understand the state of the art is to expand module-by-module and then stitch together the results of expansion later, using a kind of link-time optimization. You visit a module’s definitions twice: once to evaluate them while expanding, resulting in live definitions that can be used by further syntax expanders, and once to residualize an abstract syntax tree, which will eventually be spliced into the compilation unit.

Note that in general the expansion-time and the residual definitions don’t need to be the same, and indeed during cross-compilation they are often different. If you are compiling with Guile as host and Hoot as target, you might implement cons one way in Guile and another way in Hoot, choosing between them with cond-expand.

lexical scope regained?

What is to be done? Glad you asked, Vladimir. But, I don’t really know. The compiler wants a big blob of letrec, but the expander wants a pearl-string of modules. Perhaps we try to satisfy them both? The library-group paper suggests that modules should be expanded one by one, then stitched into a letrec by AST transformations. It’s not that lexical scope is incompatible with modules and whole-program compilation; the problems arise when you add in macros. So by expanding first, in units of modules, we reduce high-level Scheme to a lower-level language without syntax transformers, but still on the level of letrec.

I was unreasonably pleased by the effectiveness of the “just splat in a prelude” approach, and I will miss it. I even pled for a kind of stop-gap fat-fingered solution to sloppily parse module forms and keep on splatting things together, but colleagues helpfully talked me away from the edge. So good-bye, sloppy: I repent my ways and will make amends, with 40 hail-maries and an alpha renaming thrice daily and more often if in moral distress. Further bulletins as events warrant. Until then, happy scheming!

Explore the evolution of Lyubomir Filipov's career from his initial encounter with Drupal to his current role as a Group Architect at FFW Agency, delving into his strategies for team management, overcoming multilingual challenges, community involvement, and the intersection of artificial intelligence with the digital field. Filipov's diverse experiences reflect a multifaceted professional journey, offering insights into Drupal, community engagement, academic pursuits, and the future landscape of digital expertise.

Get your session in now...

 

Don't Miss the Last Chance to Submit Your Idea

With over 40 sessions submitted to date, we have a couple of days left for you to get your ideas in for MidCamp 2024!

We’re open to talks for all levels of Drupal users, from beginner through advanced, as well as end users and business owners.

See our session tracks page for a full description of the kinds of talks we are looking for.  If you’re new to speaking, check out the recording of our Speaker Workshop for ideas.

Submit a session now!

Important Dates:

• Proposal Deadline: January 7 (Sunday), 2024 at midnight CST
• Tickets on sale: very soon!
• Early-bird deadline and speakers announced:  February (all speakers are eligible for a free ticket, and anyone who submits a session that is not accepted will be eligible for early-bird pricing even after it closes)
• MidCamp 2024: March 20-22

Sponsors Get Early, Privileged Access

Get early and privileged access to new talent and customers by sponsoring MidCamp. We have a variety of sponsorship packages available.

Starting at $600, sponsoring organizations can target their jobs to a select group of experienced Drupal talent, maximize exposure by sharing space with dozens of jobs instead of millions, and have three days of being face-to-face with applicants.

Our sponsorship packages are designed to showcase your organization as a supporter of the Drupal community and provide opportunities to:

• grow your brand,
• generate leads,
• and recruit Drupal talent.

Check out the sponsorship packages here, we look forward to working with you to get your organization involved for 2024!

Stay In The Loop

Join the MidCamp Slack and come hang out with the community online. We will be making announcements there from time to time. We’re also on Twitter and Mastodon.

We can’t wait to see you soon! Don’t forget, cancel all those other plans and make MidCamp the only thing happening on your calendar from March 20-22, 2024.

Drupal is an open-source content management system (CMS) that has gained significant popularity for its ability to power some of the most prominent and frequently visited websites globally. With a vast developer community of 1.4 million strong, contributing over 50,000 free modules, Drupal offers unparalleled flexibility and power, making it our preferred CMS choice for […]

How to Leverage Drupal's Layout Builder to Create Complex Pages Editor Fri, 01/05/2024 - 15:42

Sometimes, creating a modern design with high functionality is problematic. Especially when it comes to large-scale and complex projects. There are various solutions to make the process of creating layouts easier and more convenient. One of the most influential and popular is Drupal Layout Builder.

Comparing two lists is a common task in programming, and it becomes crucial when you need to find differences, and intersections, or validate data consistency. In this tutorial, we will explore various methods to compare two lists in Python. We’ll cover simple equality checks, membership testing, element-wise comparisons, and advanced techniques using built-in functions and […]

The post Comparing Two Lists in Python appeared first on TechBeamers.

Let’s go for my web review for the week 2024-01. First one of 2024, Happy New Year to all!

In 2024, please switch to Firefox – Roy Tanck

Tags: tech, mozilla, foss, web

Alright, it’s 2024… Looking for a New Year Resolution? This is definitely the one you should contemplate! Maybe also prepare your switch to Plasma 6, but different story. 😉

https://roytanck.com/2023/12/23/in-2024-please-switch-to-firefox/

 The Internet Is About to Get Weird Again

Tags: tech, web, culture, social-media

Very inspiring article. I really wish we’ll indeed see more and more weird and creative websites. People apparently feels uneasy in their current social media jails, the time is ripe to leap in the unknown again.

https://www.rollingstone.com/culture/culture-commentary/internet-future-about-to-get-weird-1234938403/

EU CRA: What does it mean for open source? - Bert Hubert’s writings

Tags: tech, foss, law

The current compromise might not be as bad as anticipated. Some things could be improved still. The community needs to be involved in the process.

https://berthub.eu/articles/posts/eu-cra-what-does-it-mean-for-open-source/

Birb + Fossil: An RSS Revival? - Tim Kellogg

Tags: tech, fediverse, rss, ai, nlp

Indeed there are a few trends at play right now which lead to RSS being in use again. I can only hope it’ll keep growing.

https://timkellogg.me/blog/2024/01/03/birb

How bad are search results? Let’s compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

Tags: tech, search, web

This looks like now is the time for smaller search engines to shine. Clearly the dominant ones are not doing a really good job anymore.

https://danluu.com/seo-spam/

Tesla blamed drivers for failures of parts it long knew were defective

Tags: tech, tesla, automotive, safety

The results of this investigation are actually super scary… Even if you don’t own one, those are in our streets. In any case it is a clear statement that you can’t and shall not run an automotive company like a startup…

https://www.reuters.com/investigates/special-report/tesla-musk-steering-suspension/

The I in LLM stands for intelligence | daniel.haxx.se

Tags: tech, ai, machine-learning, gpt, security

When bug bounty programs meet LLM hallucinations… developer time is wasted.

https://daniel.haxx.se/blog/2024/01/02/the-i-in-llm-stands-for-intelligence/

How I forked SteamOS for my living room PC — iliana.fyi

Tags: tech, valve, foss, linux, system

Funny reverse engineering of the latest SteamOS to use it in a different context. Give some information on their system design.

https://iliana.fyi/blog/build-your-own-steamos-updates/

Why I’m skeptical of low-code - Nick Scialli | Senior Software Engineer

Tags: tech, low-code

Proper word of caution. Those tools ain’t necessarily bad… but know what the limits will be.

https://nick.scialli.me/blog/why-im-skeptical-of-low-code/

How Standard Ebooks serves millions of requests per month with a 2GB VPS; or, a paean to the classic web - Alex Cabal

Tags: tech, complexity, craftsmanship, dependencies, php, web

Nice ode to simplifying web projects, to great effects. You can go a long way serving millions of requests if you choose your tools properly.

https://alexcabal.com/posts/standard-ebooks-and-classic-web-tech

A Closer Look to a Key-Value Storage Engine | Yihang’s Blog

Tags: tech, system, design, architecture, databases

Very interesting exploration of the design choices behind the creation of a key value storage engine.

https://silhding.github.io/2021/08/20/A-Closer-Look-to-a-Key-Value-Storage-Engine/

SQLALchemy vs Django ORM | Alexey Evseev

Tags: tech, databases, python, orm, django

Overview of the differences in behavior between the two most popular ORMs in the Python ecosystem. This shows interesting different design options for such things.

https://st4lk.github.io/en/blog/2023/12/09/sqlalchemy-vs-django-orm/

The ugrep file pattern searcher

Tags: tech, tools, command-line

Looks like another nice grep alternative. This one keeps its command line interface compatible with grep which is nice, not the case for other alternatives and that turns out to be my main gripe with them.

https://ugrep.com/

Container to WASM converter

Tags: tech, docker, webassembly

This makes for quite a few emulation layers stacked on each other. Still this is an interesting experiment.

https://github.com/ktock/container2wasm

Profilerpedia

Tags: tech, profiling

Nice reference for the tools available to profile software.

https://profilerpedia.markhansen.co.nz/

Occupancy explained

Tags: tech, amd, gpu, shader, performance

A very precise and thorough article about GPU occupancy. What it is, how it relates to perceived performances, it’s potentisl relationship with cache thrashing and the tools you can use to measure it on AMD GPUs.

https://gpuopen.com/learn/occupancy-explained/

Speed up your code: don’t pass structs bigger than 16 bytes on AMD64 · GitHub

Tags: tech, c++, assembly, performance

Not necessarily a practical advice in most of our daily code. Still this exhibits interesting low level details about argument passing. Might come in handy in a few cases, to be kept in mind.

https://gist.github.com/FeepingCreature/5dff669aad380a123b15659e195fb96c

Ambient Co-presence

Tags: tech, web, design

Good food for thought regarding shared presence on the web. A few good ideas in the design space. Obviously very web centric, I wonder what we could do on the desktop with shared documents for instance.

https://maggieappleton.com/ambient-copresence

The value of canonicity - Building Nubank

Tags: tech, engineering, management, developer-experience

Nice writeup about the benefits of homogeneity in an engineering organization. It also shows how it is a balancing act though, since you need experiments to happen in a controlled way for evolution to still happen.

https://building.nubank.com.br/the-value-of-canonicity/

Bye for now!

Are you interested in using your Python skills within Excel? Would you like to share a data science project or visualization as a single Office file? This week on the show, we speak with Principal Architect John Lam and Sr. Cloud Developer Advocate Sarah Kaiser from Microsoft about Python in Excel.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Ramon Miranda has published a new video on the Krita channel: Krita 5.2 Features. Take a look and learn what’s new in our latest release:

 

 

The post New Video: Krita 5.2 Features! appeared first on Krita.

People love PyCharm’s debugger, and we work hard to make it both fast and easy to use. But what if it could be even faster and easier to use? Python 3.12 adds the new low-impact monitoring API described in PEP 669, enabling debuggers, profilers, and similar tools to run code at almost full speed. As […]

A remembered set is used by a garbage collector to identify graph edges between partitioned sub-spaces of a heap. The canonical example is in generational collection, where you allocate new objects in newspace, and eventually promote survivor objects to oldspace. If most objects die young, we can focus GC effort on newspace, to avoid traversing all of oldspace all the time.

Collecting a subspace instead of the whole heap is sound if and only if we can identify all live objects in the subspace. We start with some set of roots that point into the subspace from outside, and then traverse all links in those objects, but only to other objects within the subspace.

The roots are, like, global variables, and the stack, and registers; and in the case of a partial collection in which we identify live objects only within newspace, also any link into newspace from other spaces (oldspace, in our case). This set of inbound links is a remembered set.

There are a few strategies for maintaining a remembered set. Generally speaking, you start by implementing a write barrier that intercepts all stores in a program. Instead of:

obj[slot] := val;

You might abstract this away:

write_slot(obj, sizeof obj, &obj[slot], val);

As you can see, it’s quite an annoying transformation to do by hand; typically you will want some sort of language-level abstraction that lets you keep the more natural syntax. C++ can do this pretty well, or if you are implementing a compiler, you just add this logic to the code generator.

Then the actual write barrier... well its implementation is twingled up with implementation of the remembered set. The simplest variant is a card-marking scheme, whereby the heap is divided into equal-sized power-of-two-sized cards, and each card has a bit. If the heap is also divided into blocks (say, 2 MB in size), then you might divide those blocks into 256-byte cards, yielding 8192 cards per block. A barrier might look like this:

void write_slot(ObjRef obj, size_t size, SlotAddr slot, ObjRef val) { obj[slot] := val; // Start with the store. uintptr_t block_size = 1<<21; uintptr_t card_size = 1<<8; uintptr_t cards_per_block = block_size / card_size; uintptr_t obj_addr = obj; uintptr_t card_idx = (obj_addr / card_size) % cards_per_block; // Assume remset allocated at block start. void *block_start = obj_addr & ~(block_size-1); uint32_t *cards = block_start; // Set the bit. cards[card_idx / 32] |= 1 << (card_idx % 32); }

Then when marking the new generation, you visit all cards, and for all marked cards, trace all outbound links in all live objects that begin on the card.

Card-marking is simple to implement and simple to statically allocate as part of the heap. Finding marked cards takes time proportional to the size of the heap, but you hope that the constant factors and SIMD minimize this cost. However iterating over objects within a card can be costly. You hope that there are few old-to-new links but what do you know?

In Whippet I have been struggling a bit with sticky-mark-bit generational marking, in which new and old objects are not spatially partitioned. Sometimes generational collection is a win, but in benchmarking I find that often it isn’t, and I think Whippet’s card-marking barrier is at fault: it is simply too imprecise. Consider firstly that our write barrier applies to stores to slots in all objects, not just those in oldspace; a store to a new object will mark a card, but that card may contain old objects which would then be re-scanned. Or consider a store to an old object in a more dense part of oldspace; scanning the card may incur more work than needed. It could also be that Whippet is being too aggressive at re-using blocks for new allocations, where it should be limiting itself to blocks that are very sparsely populated with old objects.

what v8 does

There is a tradeoff in write barriers between the overhead imposed on stores, the size of the remembered set, and the precision of the remembered set. Card-marking is relatively low-overhead and usually small as a fraction of the heap, but not very precise. It would be better if a remembered set recorded objects, not cards. And it would be even better if it recorded slots in objects, not just objects.

V8 takes this latter strategy: it has per-block remembered sets which record slots containing “interesting” links. All of the above words were to get here, to take a brief look at its remembered set.

The main operation is RememberedSet::Insert. It takes the MemoryChunk (a block, in our language from above) and the address of a slot in the block. Each block has a remembered set; in fact, six remembered sets for some reason. The remembered set itself is a SlotSet, whose interesting operations come from BasicSlotSet.

The structure of a slot set is a bitvector partitioned into equal-sized, possibly-empty buckets. There is one bit per slot in the block, so in the limit the size overhead for the remembered set may be 3% (1/32, assuming compressed pointers). Currently each bucket is 1024 bits (128 bytes), plus the 4 bytes for the bucket pointer itself.

Inserting into the slot set will first allocate a bucket (using C++ new) if needed, then load the “cell” (32-bit integer) containing the slot. There is a template parameter declaring whether this is an atomic or normal load. Finally, if the slot bit in the cell is not yet set, V8 will set the bit, possibly using atomic compare-and-swap.

In the language of Blackburn’s Design and analysis of field-logging write barriers, I believe this is a field-logging barrier, rather than the bit-stealing slot barrier described by Yang et al in the 2012 Barriers Reconsidered, Friendlier Still!. Unlike Blackburn’s field-logging barrier, however, this remembered set is implemented completely on the side: there is no in-object remembered bit, nor remembered bits for the fields.

On the one hand, V8’s remembered sets are precise. There are some tradeoffs, though: they require off-managed-heap dynamic allocation for the buckets, and traversing the remembered sets takes time proportional to the whole heap size. And, should V8 ever switch its minor mark-sweep generational collector to use sticky mark bits, the lack of a spatial partition could lead to similar problems as I am seeing in Whippet. I will be interested to see what they come up with in this regard.

Well, that’s all for today. Happy hacking in the new year!