Feeds

FreeIPMI 1.6.12 - 11/19/23
--------------------------
o Use poll() over select() to avoid fd limit in openipmi driver.
o Fix potential portability problems on systems without cbrt().
o Minor documentation updates.

FreeIPMI 1.6.13 - 01/26/24
--------------------------
o Fix build issues on systems where inb/outb are declared with
  inline assembly.
o Add additional sensor/event interpretations.

Check out the important work our volunteers accomplished at today's Free Software Directory (FSD) IRC meeting.

Check out the important work our volunteers accomplished at today's Free Software Directory (FSD) IRC meeting.

Check out the important work our volunteers accomplished at today's Free Software Directory (FSD) IRC meeting.

Aegir is a hosting system built in Drupal, for Drupal.

It lets you easily create new Drupal sites and create databases, filesystems, virtual hosts etc. for the sites. You can manage hundreds or thousands of sites using a simple Drupal based UI. As simple as you would manage a list of 100 blog posts, you can manage 100 Drupal websites.

Currently the latest released version of Aegir is: Aegir 3.

Aegir 3 relies on Drush 8, which means it can work with: Drupal 7; Drupal 8 and Drupal 9. But not Drupal 10. Oh.

I have need for it to work with Drupal 10. Specifically we have a large collection of Drupal sites, that currently Drupal 9, but ready and waiting to jump to Drupal 10. Yes, we've left it too late, we should have done this a long time ago, but we are where we are.

I've spent the day looking into various options, they are roughly:

• Move to BOA
• Get Aegir 3 to work with Drupal 10
• Move to Aegir 4
• Move to Aegir 5
• Move to something else

My terms of reference

I'm looking for a solution that changes as little as possible in the current stack. We've got a big, beefy server, and lots and lots of Drupal sites on it. The Drupal sites themselves run fine and we're after the management of them really. We don't need to scale out particularly, and if we do, we've got options on the hardware side vertically. We also don't need some fancy multi-datacenter approach, and ideally keep with our Nginx, Varnish and Apache sandwich.

I do want to manage things in the old school way, but I need a few, select, non-technical users to be able to manage the sites in a friendly UI like Aegir's hosting system.

Lots of the new hotness out there are essentially orchestrating Kubernetes, and that's great, but I don't actually want 100 containers all running PHP which is how I understand all of these systems would essentially function.

I used to be an Aegir maintainer, and I have a deep knowledge of how Aegir works and even helped write some of the inter-process communications stuff in Drush 8. 

Move to BOA

The first option is one that has shown to be working, and requires a custom fork of Drush 8, some core patches and using BOA. BOA is quite an aggressive fork of Aegir, and I'm not sure I want all the changes and whatnot that comes with it. I think I have to discount it, because the last time I looked there would simply be too much change on the nginx side of things at least, and it was doing all kinds of things I don't simply understand and thus won't be able to reason about.

The main inspiration I take from BOA is that they've got it working! Well, they've got something working. It looks like they've essentially forked Drush 8, applied some Drupal core and Aegir patches. This makes it so that Aegir can still use it's aliases, code and talk to Drupal 10 sites. How they've been able to do this is very impressive indeed. I feel like it's just a matter of time before it breaks though, surely Drupal 10 is going to change something in it's lifecycle and cause issues. I don't quite see how this can work with Drush commands in contrib modules. For example, during deployments on our sites we revert features, but that's a features module provided Drush command, which won't be callable from Drush 8 in Drupal 10.

Get Aegir 3 to work with Drupal 10

This I feel is the least effort option. Find some way to get standard Aegir 3 to communicate with Drupal 10. Now, Drush 8 can't directly bootstrap a Drupal 10 site. Aegir 3 uses Drush 8 to be both the backend storage for all the data about the sites to host, and to bootstrap the sites it manages.

This worked really well in the Drupal 5/6/7 days, and just about coped with Drupal 8/9, but hasn't for Drupal 10. Think of it this way, what if instead of hosting a Drupal site, it was a Wordpress site? Aegir simply wouldn't be able to neatly bootstrap into a Drupal site and do it's stuff. It's the same with Drupal 10.

How much stuff does it really do though? I'm not 100% sure. I know that it gets the currently installed packages for example, but what else does it do. It looks to me like it often hands off a Drush subprocess do the heavy lifting. Maybe we can hand off to another subprocess, one that runs a site-local Drush 12+ instance, passing in all the options it needs to bootstrap and run the actual commands on the site.

Please, if this is a completely crazy way to go, someone please comment and let me know!

Move to Aegir 4

So there is a 4.x branch of Aegir, and apparently it can work with Drupal 10 sites. It does this by replicating the aliases to YAML files and then running a global Drush 11, which can then bootstrap the Drupal sites. Apparently it also has replaced some of the Drush invocations with standard process invocations. I guess it had to do this because the Drush to Drush process communication stuff was in Drush 8 and got removed.

It seems like Aegir 4 also brought in a lot of other changes too, and that's not particularly something I want or need. Also, it's never been officially released/tested by the community etc.

Move to Aegir 5

This got announced in a few blog posts a while back, and I've not heard anything since. It's possible that it's there and ready to go, but as far as I understand it, it's a whole change to how the sites would be managed and hosted, and while an import from Aegir 3 is on the roadmap, it seems like it would import the sites into something completely different, not some simple vhosts and a DB server.

Also, even if it does exist, it'll have had minimal testing and eyes on it.

Move to something else

I think this is my preferred long-term option. These sites don't really need Drupal. They could actually be static sites plugged into a central content repository. That would drastically simplify lots and lots of things. Or we keep the Drupal sites, but move them to Kubernetes and using something like Amazee's Lagoon to host them etc. This would be very cool, but probably quite expensive in terms of having enough resources to host all the containers.

One for the long term future I think.

 

Get Aegir 3 to work with Drupal 10

I might give this a go in that I think what I can do is this:

1. Get Aegir running normally.
2. Install a Drupal 9 site.
3. Get a single Aegir task running that doesn't bootstrap the Drupal 9 site with Drush 8. I'm thinking something like getting the one-time login link. This has data flowing in both directions, and I think, Drush 8 trying to bootstrap Drupal to the run the provision command. Getting this working would involve calling a site local Drush based on the data from the alias, but not actually using the alias.
4. If that works, put something into settings.php that throws an exception if Drush 8 is detected, and then try to get everything else working for my use-cases.
5. Then try and set up a Drupal 10 site.

I'd love to know your thoughts on this. Do you have an old Aegir running sites, and you don't know what to do with it? Or do you think I'm crazy and want to offer me a better way to go, use the comments below, please!

 

Updates/notes

So looking at the provision commands, the actual provision command to say, get the one-time login link bootstraps the Drupal site. So that would need to change that command so that it no-longer bootstrapped Drupal at all, but instead changed to be a pure Drush command. Then it could load the data from the alias etc.
I'm tempted to go ahead and get a dev instance of Aegir running and the add a new Drush command, that doesn't bootstrap Drupal, but does attempt to grab data from an alias and see what I can do. That possibly an even simpler proof of concept, as it's an entirely new Drush command, but one I then know could be called by provision's task runner instead of the current one.

The Web Chefs

January 1, 1970

At Four Kitchens we keep several lists of “Hot Topics” to share our learnings across the dozens of sites that we care for. Are you upgrading a Drupal site to CKEditor5? We’ve tidied up one of these internal wiki documents into this set of general upgrade guidelines that might pertain to your website.

Rough steps to upgrade

The level of effort needed for this upgrade will be different for each site. It may take some time to figure out. CKEditor 5 is available in Drupal 9.5 and beyond. You can try switching/upgrading on a local site or multidev and assess the situation.

First, create a list of CKEditor enhancement modules on the site and check if they are Drupal 10 ready (the reports from Upgrade Status and this Drupal.org page may help). Common modules to look for include Linkit, Anchor Link, Advanced Link, IMCE, Entity Embed, Video Embed Field, Footnotes, and anything with the word “editor” in the title.

As a best practice, you should test both the creation of new content, and editing existing content in several places. This will help make sure that some lesser used HTML isn’t treated differently in the new CKEditor. Run visual regression tests (if available).

You may need to point out key interface changes to your clients or stakeholders (e.g., contextual windows for links/media/tables instead of modals, etc.). While it is a bit of a change, it’s overall an improved user experience, especially for new people who are coming in cold.

Anchor links

Anchor link gives editors the ability to create links to different sections within a page.

For “better integration with Drupal 10, CKEditor 5, and LinkIt” there is a 3.0.0@alpha version. If your project isn’t using wikimedia/composer-merge-plugin, you must require northernco/ckeditor5-anchor-drupal package and add the following to the repositories section of composer.json:

{ "type": "package", "package": { "name": "northernco/ckeditor5-anchor-drupal", "version": "0.3.0", "type": "drupal-library", "dist": { "url": "https://registry.npmjs.org/@northernco/ckeditor5-anchor-drupal/-/ckeditor5-anchor-drupal-0.3.0.tgz", "type": "tar" } } }

Issue

Branch

Embedded media

Depending on the age of your site, it might be using one of several techniques to embed media into the WYSIWYG:

• Entity embed
• Video embed field
• The embedding functionality built into more recent versions of Drupal Core

If your site is using the video_embed_field module (most sites are probably using Drupal core’s media module), there is a patch that adds support for CKE5. Insert Image works slightly different (though this is probably not the case if your site uses core’s media module). It’s worth considering if there is a way to enhance this for user experience, if necessary.

If your site uses custom Entity Embed for media, consider switching to the core media library. It may provide a better administrative user experience in some cases.

The insert image button in CKEditor functions a little differently than it used to. Rather than bringing up a modal with fields to upload an image like the image below:

It now immediately pulls up your computer’s file system for you to search for images like so:

After adding your image, the alt tag box prompts you underneath the image:

After submitting your alt tag, you can adjust alignment and sizing:

Moving general styles to link styles

It was common in CKEditor4 to use its “Styles” feature to provide a way to add variations of links (to make them look like buttons, or to add icons).

There are a few UX problems with that approach. Either the styles are set to apply on <span>, which means that they can be applied to non-links, or the styles are set to apply on <a>, which means that they are mysteriously grayed out most of the time (until you select a link). Either way, it’s not intuitive how to apply a link style. In CKEditor5, we can switch to using the Link Styles module.

Change in Styles dropdown behavior

In CKEditor4, when integrated with Drupal, the Styles dropdown only allowed applying one style to an element (e.g., “external link”). If you tried to apply a different style, such as “locked link,” the previous style would be removed.

The Drupal implementation of CKEditor5 allows for multiple styles to be applied to elements via the Styles dropdown. This change may be unexpected for some, and could result in elements that look broken, such as when a link has both the “external link” and “locked link” styles.

CKEditor5 introduced a new API for adding theme-specific styles. The new architecture might cause the CKEditor5 theme to bleed into the admin theme. To know how to deal with these issues, review new API for adding theme-specific styles in CKEditor5.

You’ll likely run into an issue with styles bleeding outside of the editor, so see the other section within this page.

Cut and paste

Paste-from-Word, paste-from-Google-Docs, etc. is now built-in to CKEditor5. (At least for 90% of use cases.) There’s a paid plugin for more esoteric needs.

There is no paste-as-plain-text plugin for CKEditor5. You can use Ctrl-Shift-V (or Cmd-Shift-V) to paste as plain text. If you want to get rid of all formatting (including bold, links, etc.) in existing text, you can highlight the text, use Ctrl-C to copy, then Ctrl-Shift-V to paste it back as plain text.

Behat

Many of our Behat automated test broke after the update because there were multiple structural changes, so this is how we solved it: First, here is the doc about how to get the editor instance in case you want to know more about it. This is how we rewrite our custom step to fill out the CKEditor during testing. (We found the code in an article post-post).

/** * A step to help fill in a ckeditor wysiwyg. * * @param string $locator * The css locator of the field that ckeditor has taken over. * @param string $value * The html value you wish to fill in to ckeditor. * * @Then I fill in wysiwyg on field :locator with :value */ public function iFillInWysiwygOnFieldWith($locator, $value) { // https://ckeditor.com/docs/ckeditor5/latest/support/faq.html#how-to-get-the-editor-instance-object-from-the-dom-element $ckeditor5_drupal_editable_element = "div.form-item-$locator .ck-editor__editable"; $this->getSession() ->executeScript( " var domEditableElement = document.querySelector(\"$ckeditor5_drupal_editable_element\"); if (domEditableElement.ckeditorInstance) { const editorInstance = domEditableElement.ckeditorInstance; if (editorInstance) { editorInstance.setData(\"$value\"); } else { throw new Exception('Could not get the editor instance!'); } } else { throw new Exception('Could not find the element!'); } " ); }

and the mink step for regular field:

And I fill in wysiwyg on field "field-summary-0-value" with "Some Teaser Text"

And for a field inside a paragraph:

And I fill in wysiwyg on field "field-sidebar-content-0-subform-field-simple-text-0-value" with "Behat Side Nav Body Text"

Preventing custom styles from bleeding into admin theme with CKEditor5

See the new API documentation about implementing theme styles in the new way. This may require some adjustments on your end.

One of the major changes with CKEditor5 is that it pulls WYSIWYG styles onto the whole page when there is a WYSIWYG on the page. In CKEditor4, styles were only pulled into the CKEditor iframe. This can be extremely frustrating when the admin theme looks odd or different on pages that contain a WYSIWYG.

Limit the number of stylesheets being pulled into the WYSWIYG. (First, note that this method has only been confirmed to work on newer versions of Sous using specific webpack settings. If you are having problems with it, make sure your webpack settings allow for multiple manifests to be generated. You may need to refer to a newer site to see how it is configured.)

The first step is to create a new stylesheet (a manifest) called wysiwyg.scss in the same directory as your styles.scss file, which assembles all the stylesheets used in your theme. For this stylesheet, we’ll only want to include the stylesheets that our WYSIWYG needs. For example, I have one that looks like this:

@import url('https://fonts.googleapis.com/css2?family=Poppins:ital,wght@0,400;0,700;1,400;1,700&display=swap'); @import '~normalize.css/normalize'; @import '~breakpoint-sass/stylesheets/breakpoint'; // Components @import '00-base/**/*.scss'; // Include all atoms except form items. @import '01-atoms/00-links/**/*.scss'; @import '01-atoms/01-text/**/*.scss'; @import '01-atoms/02-lists/*.scss'; @import '01-atoms/tables/*.scss'; @import '01-atoms/images/**/*.scss'; @import '05-pages/colors.scss'; @import '05-pages/base.scss';

In this example, we are pulling in a couple needed files from node_modules (normalize and breakpoint), and then any .scss files from base, and then select files from atoms (links, text, lists, tables, and images).

Compile and make sure that it has created the new files at /dist/css/wysiwyg.css. If you get any errors, you may need to include another file that has a variable you need, or something along those lines.

1.) Update your .info file In your theme’s .info file, set CKEditor5 to use your new stylesheet:

ckeditor5-stylesheets: - dist/css/wysiwyg.css

2.) Review the WYSIWYG. Visit a page with a WYSIWYG on the page, and verify that the limited styles are loading properly within the WYSIWYG. Try all the dropdowns and buttons that are included in the WYSIWYG settings. If anything appears unthemed, review your styles to see if there’s a stylesheet missing from your manifest.

3.) Review the rest of the page. Now review the page around the WYSIWYG and note how if differs from other pages that do not have a WYSIWYG. Common differences to look for are: heading styles, text styles, buttons — basically anything that you included in your manifest.

4.) Limit styles

• Find the page’s body class for node edit pages (in our test case, .gin--edit-form). It may depend on your admin theme.
• Find the wrapper class for the WYSIWYG. Most likely the best choice is .ck-content. Our approach will be to hide styles from .gin--edit-form, but then add them to .ck-content.

For example:

body { background-color: clr(background); color: clr(text); @include body-copy; }

becomes:

body:not(.gin--edit-form), .ck-content { background-color: clr(background); color: clr(text); @include body-copy; }

And for buttons:

.main .button { @include button-base; @include button-color-primary; @include button-medium; }

it becomes:

body:not(.gin--edit-form) .button, .main .button, .ck-content a.button { @include button-base; @include button-color-primary; @include button-medium; }

With any luck, the styles used apply mixins, which makes it easy to filter out where to apply the styles. In some cases, the overriding of styles may become hard because of the order in which the stylesheets are loaded. Try to avoid !importants and instead use things like an additional element or class to firm up your override.

One issue that may come up is your overrides here end up overriding things in your custom theme, depending on how they are defined. In this case, don’t wrap the styles in the body classes, but rather undo the custom theme’s style on the admin page items manually. Luckily, since we’re narrowly applying custom styles, only things used in the WYSIWYG will need to be addressed.

For instance:

// Apply general link styles to all links. a { @include link; } // Overrides for Admin pages containing CKEditor (you will need a body class only on admin pages). .user-logged-in { a { background-image: none; transition: none; } .horizontal-tabs-list a, .toolbar a { font-weight: normal; } } // Reapply link styles to links within the WYSIWYG .ck-editor a { @include link; }

Continue to review your page and adjust it until it no longer differs from other admin pages.

Editor explodes out of its container in deeper paragraphs

This issue seems to occur only with rich text fields within a paragraph. It might be limited to the Gin theme.

This issue might be because of the container’s width. If input fields inside the container have a specified size exceeding the screen width, it can lead the editor to inherit the container’s width, extending beyond the screen. You can see this as a Drupal Core/CKEditor5 bug in Drupal.org: CKEditor5 toolbar items of multivalue field (typically Paragraphs) overflowing on narrow viewports and overlapping with node form’s sidebar on wide viewports.

To resolve this quickly, set the input fields to 100% width, making sure everything works seamlessly. Be sure to include this in a stylesheet of your admin theme.

.node-form input[size] { width: 100%; }

We can also modify the ‘flex-wrap’ property of the CKEditor buttons to make sure they stay within the container’s width:

.ck-editor .ck.ck-toolbar.ck-toolbar_grouping > .ck-toolbar__items { flex-wrap: wrap; } Additional resources

• Five Can’t Miss Articles on CKEditor 5 in Drupal
• Upgrading to CKEditor5
• Upgrading my site from CKEditor4 to CKEditor5
• How to upgrade from CKEditor4 to CKEditor5

The post Tips for upgrading to CKEditor5 appeared first on Four Kitchens.

Inspired by a Mastodon post by Françoise Conil, who investigated the current popularity of build backends used in pyproject.toml files, I wanted to investigate how the popularity of build backends used in pyproject.toml files evolved over the years since the introduction of PEP-0517 in 2015.

Getting the data

Tom Forbes provides a huge dataset that contains information about every file within every release uploaded to PyPI. To get the current dataset, we can use:

curl -L --remote-name-all $(curl -L "https://github.com/pypi-data/data/raw/main/links/dataset.txt")

This will download approximately 30GB of parquet files, providing detailed information about each file included in a PyPI upload, including:

1. project name, version and release date
2. file path, size and line count
3. hash of the file

The dataset does not contain the actual files themselves though, more on that in a moment.

Querying the dataset using duckdb

We can now use duckdb to query the parquet files directly. Let’s look into the schema first:

describe select * from '*.parquet'; ┌─────────────────┬─────────────┬─────────┐ │ column_name │ column_type │ null │ │ varchar │ varchar │ varchar │ ├─────────────────┼─────────────┼─────────┤ │ project_name │ VARCHAR │ YES │ │ project_version │ VARCHAR │ YES │ │ project_release │ VARCHAR │ YES │ │ uploaded_on │ TIMESTAMP │ YES │ │ path │ VARCHAR │ YES │ │ archive_path │ VARCHAR │ YES │ │ size │ UBIGINT │ YES │ │ hash │ BLOB │ YES │ │ skip_reason │ VARCHAR │ YES │ │ lines │ UBIGINT │ YES │ │ repository │ UINTEGER │ YES │ ├─────────────────┴─────────────┴─────────┤ │ 11 rows 6 columns │ └─────────────────────────────────────────┘

From all files mentioned in the dataset, we only care about pyproject.toml files that are in the project’s root directory. Since we’ll still have to download the actual files, we need to get the path and the repository to construct the corresponding URL to the mirror that contains all files in a bunch of huge git repositories. Some files are not available on the mirrors; to skip these, we only take files where the skip_reason is empty. We also care about the timestamp of the upload (uploaded_on) and the hash to avoid processing identical files twice:

select path, hash, uploaded_on, repository from '*.parquet' where skip_reason == '' and lower(string_split(path, '/')[-1]) == 'pyproject.toml' and len(string_split(path, '/')) == 5 order by uploaded_on desc

This query runs for a few minutes on my laptop and returns ~1.2M rows.

Getting the actual files

Using the repository and path, we can now construct an URL from which we can fetch the actual file for further processing:

url = f"https://raw.githubusercontent.com/pypi-data/pypi-mirror-{repository}/code/{path}"

We can download the individual pyproject.toml files and parse them to read the build-backend into a dictionary mapping the file-hash to the build backend. Downloads on GitHub are rate-limited, so downloading 1.2M files will take a couple of days. By skipping files with a hash we’ve already processed, we can avoid downloading the same file more than once, cutting the required downloads by circa 50%.

Results

Assuming the data is complete and my analysis is sound, these are the findings:

There is a surprising amount of build backends in use, but the overall amount of uploads per build backend decreases quickly, with a long tail of single uploads:

>>> results.backend.value_counts() backend setuptools 701550 poetry 380830 hatchling 56917 flit 36223 pdm 11437 maturin 9796 jupyter 1707 mesonpy 625 scikit 556 ... postry 1 tree 1 setuptoos 1 neuron 1 avalon 1 maturimaturinn 1 jsonpath 1 ha 1 pyo3 1 Name: count, Length: 73, dtype: int64

We pick only the top 4 build backends, and group the remaining ones (including PDM and Maturin) into “other” so they are accounted for as well.

The following plot shows the relative distribution of build backends over time. Each bin represents a time span of 28 days. I chose 28 days to reduce visual clutter. Within each bin, the height of the bars corresponds to the relative proportion of uploads during that time interval:

Looking at the right side of the plot, we see the current distribution. It confirms Françoise’s findings about the current popularity of build backends:

• Setuptools ~50%
• Poetry: ~33%
• Hatch: ~10%
• Flit: ~3%
• Other: ~4%

Between 2018 and 2020 the graph exhibits significant fluctuations, due to the relatively low amount uploads utizing pyproject.toml files. During that early period, Flit started as the most popular build backend, but was eventually displaced by Setuptools and Poetry.

Between 2020 and 2020, the overall usage of pyproject.toml files increased significantly. By the end of 2022, the share of Setuptools peaked at 70%.

After 2020, other build backends experienced a gradual rise in popularity. Amongh these, Hatch emerged as a notable contender, steadily gaining traction and ultimately stabilizing at 10%.

We can also look into the absolute distribution of build backends over time:

The plot shows that Setuptools has the strongest growth trajectory, surpassing all other build backends. Poetry and Hatch are growing at a comparable rate, but since Hatch started roughly 4 years after Poetry, it’s lagging behind in popularity. Despite not being among the most widely used backends anymore, Flit maintains a steady and consistent growth pattern, indicating its enduring relevance in the Python packaging landscape.

The script for downloading and analyzing the data can be found in my GitHub repository. It contains the results of the duckb query (so you don’t have to download the full dataset) and the pickled dictionary, mapping the file hashes to the build backends, saving you days for downloading and analyzing the pyproject.toml files yourself.

This year, we are trying out a new approach with our releases, moving away from a quarterly schedule to more regular monthly feature-rich releases. This change is intended to deliver new features more rapidly and streamline the feedback process.

Download PyCharm 2023.3.3

Visual diff for Jupyter notebooks

There is no need to dig into JSONs anymore – PyCharm now provides a diff view for Jupyter notebooks that renders cells with inputs as if you’re looking through the actual notebook.

AI Assistant: Unit test generation

Use JetBrains AI Assistant to speed up your Python development with automated unit test generation for methods in classes.

Jupyter notebooks: Widget rendering

While PyCharm 2023.3.3 already displays interactive graphics for libraries such as Matplotlib, Bokeh, Plotly, pyecharts, and TensorBoard, we plan on improving this functionality even further in the upcoming releases.

Async viewer for the debugger

Get information about the program’s state and track the evaluation of coroutines while debugging in PyCharm. You can now also use the await keyword outside of functions in the debug console.

Fixes for the DataFrame viewer

In PyCharm 2023.3, we introduced the ability to view DataFrames and series in a separate editor tab. Following user feedback, we’ve improved this view, which now offers new color coding for data and better performance with large datasets.

These are the most notable updates featured in the PyCharm 2023.3.3 release. For a detailed overview of all the changes, we recommend reviewing the release notes.

Your feedback is invaluable to us as we work to improve PyCharm. We encourage you to share your thoughts and suggestions on the latest features and updates. Connect with us on X (formerly Twitter) or drop a comment below to let us know what you think. If you come across any bugs while working with the IDE, please report them to our issue tracker.

An increasingly common setup is to use Drupal as the CMS / backend of an application with a frontend Node.js application: Drupal holds the content and manages certain things like search indexing and integration with certain third-party tools (DAM, for example), and the Node application creates the presentation layer. An assumption in this post is that the Node.js application is making JSON:API requests to the Drupal application via the Drupal core JSON:API module’s functionality.

With this setup, it’s critically important to plan a cache strategy, to avoid making excessive requests to the backend and potentially sacrificing performance. Here are some considerations.

Objective: For the best results in terms of performance and utilization, both the Drupal backend and the Node.js frontend should have a cache strategy in place. 

Being able to know what your users are doing on your website is a critical component of maintaining and running your Drupal website.  For years, site maintainers have relied on Google Analytics to gather that information and with the recent launch of GA4 it's important to make sure your site is setup for success.  The Acquia Drupal Integration Team has worked closely to upgrade the Google Tag module as part of the GA4 transition and it is better than ever.  Leveraging the latest GTag script and supporting GTM all from within the same module eliminates confusion for developers and maintainers.  During this tutorial we will step you through installing the new Google Tag module and some basic configuration items you need to consider.

1. Add/Enable the Google Tag module Add the Module

   Depending on how your Drupal site is setup, there are different ways to add the module to your

Speed to market is often critical for applications especially when rolling out new sites, features for an offering, or need to create campaign pages, landing pages, and other one off content that doesn't necessarily need to follow a strict template. 

The tool you use to achieve this could make or break this goal depending on your needs as well as how it is used. 

In this tutorial we aim to give you the questions you need to ask yourself when trying to decide whether using Drupal Core's Layout Builder or Acquia's Site Studio offering for your next project.

1. Cost (both upfront and maintenance)

   Acquia Site Studio is a proprietary tool provided by Acquia to our Acquia hosted customers. One license is included

The requirement is to cache content for as long as possible, but update promptly when content changes are made in Drupal. 

Note that in this setup I’m using Acquia Edge powered by Cloudflare, but the strategy would be similar for Acquia Edge powered by Akamai.

The objective is to bootstrap Drupal as infrequently as possible, by relying on the Varnish and CDN layers, but always to serve fresh content: Cache pages in Varnish for a long time, but provide a method (Purge) to invalidate Varnish when a content change is made in Drupal. At the CDN layer, the goal is to make sure the current version of a page is delivered to the browser, by checking Varnish frequently to see if a new version is available. This check is a lightweight call to Varnish. If Varnish does not have a new version, the CDN-cached version is delivered.

Drupal Modules:

After bcachefs suddenly got merged into mainline in 6.7 (after years and years of development and arguing on LKML), I've been curious; could this be an interesting thing to test out? I gave up btrfs many years ago as just way too slow, and to be honest I've been quite fine with ext4/xfs + LVM + md, but there was something in it that spoke to me nevertheless.

So the last month or so, I've been having /home on my primary machine (server) as bcachefs, with a small 10GB SSD backing. Surprisingly… it's been quite OK? It doesn't really speak all that well with LVM (since it's very much intended as a replacement, you can't do stuff like let one SSD volume back two different HDD volumes), and its erasure coding parts are not ready yet (so I'll still need MD underneath for the RAID-6), but it's been stable. It hasn't eaten any of my files, it hasn't crashed, it hasn't had weird performance hiccups. For something that's still so new in mainline, that's actually pretty good.

There have been bugs. The built-in compression doesn't always manage to compress everything that it should (although it is definitely an improvement over no compression at all!), rebalance hasn't always woken up when it should. There were scary fsck warnings on a mount once. bcachefs migrate (which is one of those things that sound totally bonkers, and I guess is), whose concept I assume was copied from btrfs, didn't work at first for me until I bugfixed it.

And I see sometimes people on #bcache report errors, although it seems they do get timely help (and not just of the sort “blow away your entire filesystem and restore from backups”). You can run this as your primary filesystem right now, but you are definitely an early adopter then, and you have to be ready for some cuts. Parts of online fsck is coming in 6.8, but it's still missing stuff like scrub, the aforementioned erasure coding (RAID that isn't RAID-0 or RAID-1 or a combination thereof), send/receive, and a few other things.

Integration into Debian is also pretty young still. On another computer (my main desktop machine, although that's not my most used one), I now have / on a multi-device bcachefs, and that is… rough. Upstream is increasingly writing things in Rust, which needs some working out. GRUB can't read it (not that I really care about /boot on bcachefs). There's obviously no installer support. But again, it hasn't eaten anything yet.

So if you like living life on the edge, I guess you could give it a spin. But remember those backups :-)

How does a debugger work? What can you learn about Python by building one from scratch? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Introduction

One cool object-oriented programming (OOP) technique / pattern is enforcing consistent interfaces.

In Python you can use Abstract Base Classes (ABCs) for that.

Using ABCs ensures that all subclasses implement the required methods.

This can make it easier to maintain and extend the existing code base.

A real world Pybites example

Check out this code example:

from abc import ABCMeta, abstractmethod class PybitesSearch(metaclass=ABCMeta): @abstractmethod def match_content(self, search: str) -> list: """Implement in subclass to search Pybites content""" class IncompleteSearch(PybitesSearch): pass search = IncompleteSearch() # TypeError: Can't instantiate abstract class IncompleteSearch # with abstract methods match_content

Note: In the example above, we’ve used the ABCMeta metaclass directly for defining the abstract base class.

However, Python provides a more modern and succinct way to achieve the same result using the ABC class from the abc module.

Note that the type of ABC is still ABCMeta, therefore inheriting from ABC requires the usual precautions regarding metaclass usage, as multiple inheritance may lead to metaclass conflicts.

https://docs.python.org/3/library/abc.html How does this work?

This error might be a bit confusing at first, no?

When a class is declared as an abstract class in Python, it means that the class serves as a template for other classes.

This template can define abstract methods, which are methods that are declared but not implemented. Any subclass derived from this abstract class must implement these abstract methods.

When you try to instantiate a subclass that hasn’t implemented all of the abstract methods of its parent class, Python prevents it.

This is because the subclass is still considered abstract, as it doesn’t provide concrete implementations for all the abstract methods it inherited.

In this example:

• PybitesSearch is an abstract class because it has at least one abstract method, match_content.
• When you create a subclass like IncompleteSearch and don’t implement the match_content method, IncompleteSearch inherits the abstract status from PybitesSearch. It’s like an incomplete blueprint.
• As mentioned before Python enforces that you cannot create instances of abstract classes. Therefore, when you attempt to instantiate IncompleteSearch, Python raises shown TypeError, indicating that IncompleteSearch can’t be instantiated because it’s still “abstract” due to the unimplemented match_content method.

Key to this pattern is the @abstractmethod decorator. This decorator, applied to a method within an abstract base class, marks the method as one that must be overridden in any concrete subclass.

Without implementing these abstract methods, the subclass remains abstract, and Python will prevent its instantiation. It’s a clear signal to developers about the essential methods that need to be defined in any subclass.

ABCs in the wild

Abstract Base Classes are widely used in Python, both in the Standard Library as well as in third-party libraries, to provide a formal way to define interfaces. Let’s look at a few examples …

Collections Module

Iterable, Iterator, Sequence (and others): These ABCs in the collections.abc module are used to define the expected methods for iterable objects (like lists and tuples), iterators, and sequence types.

For instance, if you create a custom collection type and want it to be treated as a sequence, you can inherit from Sequence and implement __getitem__.

Here is another example if Iterable

# collections._collections_abc class Iterable(metaclass=ABCMeta): __slots__ = () @abstractmethod def __iter__(self): while False: yield None ... # another module from collections.abc import Iterable class CustomList(Iterable): def __init__(self, items): self.items = items def __iter__(self): return iter(self.items) # CustomList must implement __iter__ adhering to the Iterable interface io Module

The io module provides a hierarchy of ABCs for handling streaming data (like files and streams).

These ABCs can be used to enforce the implementation of standard stream methods.

from io import IOBase class CustomTextIO(IOBase): def __init__(self, text): self.text = text self.index = 0 def read(self, size=-1): if size == -1: return self.text else: data = self.text[self.index:self.index + size] self.index += size return data file_like_object = CustomTextIO("Hello, World!") print(file_like_object.read(5)) # Hello # take read() method out and you'll get an error # AttributeError: 'CustomTextIO' object has no attribute 'read' contextlib Module

AbstractContextManager can be used to create robust and reusable context managers that adhere to the context manager protocol, ensuring the proper implementation of the required methods:

class AbstractContextManager(abc.ABC): """An abstract base class for context managers.""" __class_getitem__ = classmethod(GenericAlias) def __enter__(self): """Return `self` upon entering the runtime context.""" return self @abc.abstractmethod def __exit__(self, exc_type, exc_value, traceback): """Raise any exception triggered within the runtime context.""" return None ...

As demonstrated before leaving off the abstract method in the subclass raises a TypeError:

from contextlib import AbstractContextManager class chdir(AbstractContextManager): def __init__(self, path): self.path = path ch = chdir("/tmp") # TypeError: Can't instantiate abstract class chdir with abstract method __exit__

If you’re curious why only __exit__ is enforced and not __enter__, note that the latter is already provided with a default implementation that simply returns self, as this is often all you need. Of course subclasses can still override __enter__ if they need different behavior, but they are not forced to do so.

Flexibility

This is similar to the reason why in the PybitesSearch ABC shown earlier, I chose to make only the match_content method abstract and not get_data or show_matches.

These latter methods offer good default implementations that are sufficient for general use cases, so I didn’t require them to be implemented in every subclass.

This approach aligns with the philosophy of providing sensible defaults while allowing for flexibility.

Subclasses are free to override get_data and show_matches if specialized behavior is needed, thanks to regular inheritance.

This mix of some methods being abstract (enforcing their implementation) and others being regular (providing defaults but allowing overrides) exemplifies a flexible and powerful design pattern.

It allows for the creation of a robust and versatile framework where the essential parts are guaranteed to be implemented, while still offering customization where it’s beneficial.

When to not use ABCs

While ABCs are powerful, they come with nuances. One common pitfall is overusing them, especially when simple functions could suffice (related article: When to write classes).

ABCs are ideal for larger, more complex systems where multiple objects share a common interface. That said, Pybites Search is a relatively small project and I think they are a nice fit there.

Conclusion

ABCs are a way of enforcing a contract: “If you want to be a type of PybitesSearch, you must know how to match_content.

Pybites Search has gotten quite a few subtypes since (ArticleSearch, BiteSearch, etc.) and lived happily ever after.

You can look at the code here.

Please use it next time you want to look for Pybites content from your terminal.

And if you see any improvements please open an issue on the repo …

Contributing to open-source projects like the Pybites search tool not only enhances your coding skills but also allows you to be part of a collaborative community.

You gain experience in real-world projects, receive feedback from fellow developers, and contribute to tools that benefit the wider community. It’s a rewarding way to learn, grow, and connect with others in the field.

What is your experience?

I encourage you to explore ABCs more in your projects if you see a good use case to enforce an interface.

When you do, or if you have already experienced their power, join our community and tell us.

Let’s go for my web review for the week 2024-04.

Where have all the flowers gone? | daverupert.com

Tags: tech, web, blog, culture

Good continuation of “where have all the websites gone?”. They’re still here but we changed, all the more reason for curating.

https://daverupert.com/2024/01/where-have-all-the-websites-gone/

The Open Source Sustainability Crisis

Tags: tech, foss, sustainability

Yes, there’s something to do in this space. More funding is necessary, some form of platform might help… but it definitely won’t be enough.

https://openpath.chadwhitacre.com/2024/the-open-source-sustainability-crisis/

Platform Tilt: Documenting the Uneven Playing Field for an Independent Browser Like Firefox - Open Policy & Advocacy

Tags: tech, firefox, web, browser, google, apple, microsoft

Nice call from Mozilla to make this public. This way it is very obvious where the blockers are on some platforms.

https://blog.mozilla.org/netpolicy/2024/01/19/platform-tilt/

Victory! Ring Announces It Will No Longer Facilitate Police Requests for Footage from Users | Electronic Frontier Foundation

Tags: tech, surveillance

Still a long way to go but definitely a move in the right direction.

https://www.eff.org/deeplinks/2024/01/ring-announces-it-will-no-longer-facilitate-police-requests-footage-users

Fairly Trained launches certification for generative AI models that respect creators’ rights

Tags: tech, ai, machine-learning, gpt, copyright, licensing

This is an interesting move, we’ll see if this certification gets any traction.

https://www.fairlytrained.org/blog/fairly-trained-launches-certification-for-generative-ai-models-that-respect-creators-rights

Nightshade: Protecting Copyright

Tags: tech, ai, machine-learning, copyright

The tooling to protect against the copyright theft of image generator models training is making progress. This will clearly turn into an arm race.

https://nightshade.cs.uchicago.edu/whatis.html

Reading QR codes without a computer!

Tags: tech, barcode, qrcode

Another great way to understand how QR codes work.

https://qr.blinry.org/

Journey to the Centre of the JVM — Daniel Spiewak

Tags: tech, java, multithreading, memory

Very interesting talk. It gives a good idea of some properties of the JVM memory model. It also shows how the CPU architecture properties can leak all the way through changing some of the behavior of JVM code. Too bad this particular thing seems badly documented on the JDK side.

https://www.youtube.com/watch?v=EFkpmFt61Jo

That’s it! I’m making my own C++ package manager…

Tags: tech, c++, dependencies, buildsystems

Clearly I can understand the feeling. That’s yet another attempt at a solution for this… it’s young, will it get traction? Has the benefit of being kind of simple, too simplistic maybe?

https://david-delassus.medium.com/thats-it-i-m-making-my-own-c-package-manager-555eecbf7d2e

Re: The Case for Rust (in the base system)

Tags: tech, rust, c++, safety, system

Very interesting contribution to the FreeBSD hackers mailing list. Gives quite a good background about Rust, C++ and safety. Debunks a few claims you can easily hear in many places as if they were common knowledge.

https://lists.freebsd.org/archives/freebsd-hackers/2024-January/002876.html

On‐demand JSON: A better way to parse documents? - Keiser - Software: Practice and Experience - Wiley Online Library

Tags: tech, json, parsing, performance, c++

Very interesting approach to JSON parsing. Comes with a very thorough performance analysis.

https://onlinelibrary.wiley.com/doi/10.1002/spe.3313

Handling external API errors: A resumable approach

Tags: tech, api, services, consistency

On the difficulties of dealing with third party APIs. How to handle failures and reach eventual consistency? A few good solutions and patterns are proposed here.

https://thoughtbot.com/blog/handling-errors-when-working-with-external-apis

The most important goal in designing software is understandability | nicole@web

Tags: tech engineering, craftsmanship, quality

Understandability is indeed a very important goal. There are easy ways to improve it in a system.

https://ntietz.com/blog/the-most-important-goal-in-designing-software-is-understandability/

TDD Revisited - Ian Cooper - NDC Porto 2023 - YouTube

Tags: tech, tdd, tests

Nice talks, debunks very well quite a bit of the fallacies around people wrongly practicing TDD. I never realized how the root cause of those fallacies was the misusing of the “unit tests” term instead of “developers test”. This was indeed the wrong term, knew it, but first time I realize how profound the effects were.

https://www.youtube.com/watch?v=IN9lftH0cJc

Trunk Based Development

Tags: tech, version-control

A nice knowledge base about what is probably my favorite branching model. Goes in the variations you can have, the trade-offs and the other techniques you need to bring in for it to work well.

https://trunkbaseddevelopment.com/

On “owning” software - avdi.codes

Tags: tech, economics, cost, licensing, services, foss

Good exploration on how the total cost of ownership is spread depending on how is licensed the software you use and where you get your support from. I think there’s one point a bit too glanced over in the analysis of the cost for the proprietary SaaS case: what’s the cost of fixing a bug that affect your team? You might be a tiny fish in a large pond, good luck getting attention from support in this case.

https://avdi.codes/on-owning-software/

Lessons learned: 1,000 days of distributed at Atlassian

Tags: tech, atlassian, remote-working

Interesting report about distributed and remote work at Atlassian. They really did their homework. I recommend reading the whole report, they came up with a few original ideas.

https://www.atlassian.com/blog/distributed-work/distributed-work-report

Cancel your meetings if you can live with the outcome - Andy Grunwald

Tags: tech, meetings

Interesting approach to handling meetings. Start thinking about what happens if you’re not there.

https://andygrunwald.com/blog/cancel-your-meetings-if-you-can-live-with-the-outcome/

Bye for now!

OSI announced a public forum to widen the conversations that will lead to version 1.0 of the Open Source AI Definition. The forums are part of our commitment to inclusiveness and transparency, matching the public town hall meetings that started two weeks ago.

The public forum‘s goal is to welcome the broader community to engage in the conversations surrounding AI. There is only one category at the moment, but we plan to expand the forum’s scope over time.

Access to the forum is restricted to OSI members: if you’re not already a member, you can register now for free or you can use this as an opportunity to support OSI’s work and become a full member: Donating $50 or more will give you the option to vote in the board’s upcoming election and support our programs. OSI’s membership also allows you to submit a story to OpenSource.net, the community-based magazine.

The video recordings and slides from the first two town halls are on the forum:

• Open Source AI Definition Town Hall Jan 12
• Open Source AI Definition Town Hall Jan 26

By making the discussions accessible to the public, we believe that we can encourage greater participation, diversity of perspectives, and ultimately, a more comprehensive understanding of the challenges and opportunities in Open Source AI.

We have an aggressive timeline, we know we must get to a version 1.0 quickly but we also want to get there with the right amount of support from a wide set of stakeholders. Join the forum today and help us speed up the process.

The post <span class='p-name'>A public forum to discuss the Open Source AI Definition</span> appeared first on Voices of Open Source.

Innovative Approaches to Creating Slideshows Using Drupal Views Editor Fri, 01/26/2024 - 10:28

We all know such an effective and common way of demonstrating content as a slideshow. Similarly, visual content was shown even before the invention of computers on ancient mechanical devices that used slides. This is where the name comes from.
But, sometimes, a simple carousel of pictures is boring. Today, the Drupal web development agency tells you how to add slideshows using Drupal views effectively and creatively.

Why is Python so popular? There is plenty of room for debate on this but one solid reason is it's easy to adopt, easy to use, and caters to people who are not quite developers/data scientists but need to do some computing. Do you know where there largest untapped set of that group hang out? Excel. That's why it's super exciting that Python is now going to be built directly into Excel. Just go into a cell and type =PY and you're off writing full Python 3 code that is backed by a lite Anaconda distribution of Python. And we have Dr. Sarah Kaiser here to give us the rundown on Python in Excel.<br/> <br/> <strong>Episode sponsors</strong><br/> <br/> <a href='https://talkpython.fm/posit'>Posit</a><br> <a href='https://talkpython.fm/pdm2024-v2'>Pybites PDM</a><br> <a href='https://talkpython.fm/training'>Talk Python Courses</a><br/> <br/> <strong>Links from the show</strong><br/> <br/> <div><b>Sarah's website</b>: <a href="https://www.sckaiser.com" target="_blank" rel="noopener">sckaiser.com</a><br/> <b>Sarah on Mastodon</b>: <a href="https://mathstodon.xyz/@crazy4pi314" target="_blank" rel="noopener">@crazy4pi314@mathstodon.xyz</a><br/> <br/> <b>Get started with Python in Excel</b>: <a href="https://support.microsoft.com/en-us/office/get-started-with-python-in-excel-a33fbcbe-065b-41d3-82cf-23d05397f53d" target="_blank" rel="noopener">microsoft.com</a><br/> <b>Python in SQL Server</b>: <a href="https://cloudblogs.microsoft.com/sqlserver/2017/04/19/python-in-sql-server-2017-enhanced-in-database-machine-learning/" target="_blank" rel="noopener">microsoft.com</a><br/> <b>8 of the Biggest Excel Mistakes of All Time</b>: <a href="https://blog.hurree.co/8-of-the-biggest-excel-mistakes-of-all-time" target="_blank" rel="noopener">blog.hurree.co</a><br/> <b>Security and Python in Excel</b>: <a href="https://support.microsoft.com/en-us/office/data-security-and-python-in-excel-33cc88a4-4a87-485e-9ff9-f35958278327" target="_blank" rel="noopener">microsoft.com</a><br/> <b>Episode transcripts</b>: <a href="https://talkpython.fm/episodes/transcript/446/python-in-excel" target="_blank" rel="noopener">talkpython.fm</a><br/> <br/> <b>--- Stay in touch with us ---</b><br/> <b>Subscribe to us on YouTube</b>: <a href="https://talkpython.fm/youtube" target="_blank" rel="noopener">youtube.com</a><br/> <b>Follow Talk Python on Mastodon</b>: <a href="https://fosstodon.org/web/@talkpython" target="_blank" rel="noopener"><i class="fa-brands fa-mastodon"></i>talkpython</a><br/> <b>Follow Michael on Mastodon</b>: <a href="https://fosstodon.org/web/@mkennedy" target="_blank" rel="noopener"><i class="fa-brands fa-mastodon"></i>mkennedy</a><br/></div>

mrcal 2.4 is out: the release notes. Once again, this is mostly a bug-fix release en route to the big new features coming in 3.0. The most noteworthy fixes:

• mrcal can be built with clang. Try it out like this: CC=clang CXX=clang++ make. This opens up some portability improvements, such as making it easier to run on Windows.
• Full dense stereo pipeline in C.
• Tools to support more file formats:
  • mrcal-from-kalibr
  • mrcal-to-kalibr
  • mrcal-from-ros

  These are experimental. Please let me know if these are or aren't useful

The portability work was motivated by Matt Morley, who was interested in integrating mrcal into PhotonVision, the toolkit used by students in the FIRST Robotics Competition. Matt completed that work, and mrcal is now a part of PhotonVision 2024.1.2! Thanks, Matt!

I don't know if there will be a mrcal 2.5, but the next interesting release will be mrcal 3.0. The biggest internal rework is complete: the new cross-reprojection uncertainty quantification method is implemented, tested and documented. The results are very promising, but lots needs to happen before we can reliably compute intrinsics without chessboards and produce full SFM solves in mrcal and all the related things.