Feeds

Using JavaScript Frameworks - React, Vue, Angular in Drupal Editor Mon, 10/28/2024 - 15:03

The integration of JavaScript frameworks, like React, Vue, and Angular, with Drupal has sparked a wave of creativity and innovation. It goes beyond building websites. This blog explores the benefits and methods of integrating these frameworks with Drupal, demonstrating how this fusion enhances front-end development and user engagement.
Traditional static websites and strict CMS constraints are becoming a thing of the past. Nowadays, developers are embracing the adaptability and engagement provided by JavaScript frameworks to design UI within the Drupal environment.

Drupal's Frontend Landscape: General Overview

To really understand how JavaScript frameworks such as React, Vue, and Angular work with Drupal, you need to know about the frontend environment of Drupal and the difficulties developers meet while creating complicated user interfaces in this strong content management system.

The number of FAIme jobs has reached 30.000. Yeah!
At the end of this November the FAIme web service for building customized ISOs turns 7 years old. It had reached 10.000 jobs in March 2021 and 20.000 jobs were reached in June 2023. A nice increase of the usage.

Here are some statistics for the jobs processed in 2024:

Type of jobs 3%     cloud image 11%     live ISO 86%     install ISO Distribution 2%     bullseye 8%     trixie 12%     ubuntu 24.04 78%     bookworm Misc

• 18%   used a custom postinst script
• 11%   provided their ssh pub key for passwordless root login
• 50%   of the jobs didn't included a desktop environment at all, the others used GNOME, XFCE or KDE or the Ubuntu desktop the most.
• The biggest ISO was a FAIme job which created a live ISO with a desktop and some additional packages This job took 30min to finish and the resulting ISO was 18G in size.

Execution Times

The cloud and live ISOs need more time for their creation because the FAIme server needs to unpack and install all packages. For the install ISO the packages are only downloaded. The amount of software packages also affects the build time. Every ISO is build in a VM on an old 6-core E5-1650 v2. Times given are calculated from the jobs of the past two weeks.

Job type     Avg     Max install no desktop     1 min     2 min install GNOME     2 min     5 min

The times for Ubuntu without and with desktop are one minute higher than those mentioned above.

Job type     Avg     Max live no desktop     4 min     6 min live GNOME     8 min     11 min

The times for cloud images are similar to live images.

A New Feature

For a few weeks now, the system has been showing the number of jobs ahead of you in the queue when you submit a job that cannot be processed immediately.

The Next Milestone

At the end of this years the FAI project will be 25 years old. If you have a success story of your FAI usage to share please post it to the linux-fai mailing list or send it to me. Do you know the FAI questionnaire ? A lot of reports are already available.

Here's an overview what happened in the past 20 years in the FAI project.

About FAIme

FAIme is the service for building your own customized ISO via a web interface. You can create an installation or live ISO or a cloud image. Several Debian releases can be selected and also Ubuntu server or Ubuntu desktop installation ISOs can be customized. Multiple options are available like selecting a desktop and the language, adding your own package list, choosing a partition layout, adding a user, choosing a backports kernel, adding a postinst script and some more.

<strong>Topics covered in this episode:</strong><br> <ul> <li><strong><a href="https://pythoninsider.blogspot.com/2024/10/python-3140-alpha-1-is-now-available.html?featured_on=pythonbytes">Python 3.14.0 alpha 1 is now available</a></strong></li> <li><a href="https://github.com/astral-sh/uv/pull/8272?featured_on=pythonbytes"><strong>uv supports dependency groups</strong></a></li> <li><strong><a href="https://github.com/wagoodman/dive?featured_on=pythonbytes">dive: A tool for exploring each layer in a docker image</a></strong></li> <li><a href="https://pypi.org/project/pytest-metadata/?featured_on=pythonbytes"><strong>pytest-metadata</strong></a></li> <li><strong>Extras</strong></li> <li><strong>Joke</strong></li> </ul><a href='https://www.youtube.com/watch?v=70OO7BMV1KE' style='font-weight: bold;'data-umami-event="Livestream-Past" data-umami-event-episode="407">Watch on YouTube</a><br> <p><strong>About the show</strong></p> <p>Sponsored by us! Support our work through:</p> <ul> <li>Our <a href="https://training.talkpython.fm/?featured_on=pythonbytes"><strong>courses at Talk Python Training</strong></a></li> <li><a href="https://courses.pythontest.com/p/the-complete-pytest-course?featured_on=pythonbytes"><strong>The Complete pytest Course</strong></a> &amp; <a href="https://courses.pythontest.com/hello-pytest?featured_on=pythonbytes"><strong>Hello, pytest!</strong></a></li> <li><a href="https://www.patreon.com/pythonbytes"><strong>Patreon Supporters</strong></a></li> </ul> <p><strong>Connect with the hosts</strong></p> <ul> <li>Michael: <a href="https://fosstodon.org/@mkennedy"><strong>@mkennedy@fosstodon.org</strong></a></li> <li>Brian: <a href="https://fosstodon.org/@brianokken"><strong>@brianokken@fosstodon.org</strong></a></li> <li>Show: <a href="https://fosstodon.org/@pythonbytes"><strong>@pythonbytes@fosstodon.org</strong></a></li> </ul> <p>Join us on YouTube at <a href="https://pythonbytes.fm/stream/live"><strong>pythonbytes.fm/live</strong></a> to be part of the audience. Usually <strong>Monday</strong> at 10am PT. Older video versions available there too.</p> <p>Finally, if you want an artisanal, hand-crafted digest of every week of the show notes in email form? Add your name and email to <a href="https://pythonbytes.fm/friends-of-the-show">our friends of the show list</a>, we'll never share it.</p> <p><strong>Michael #1:</strong> <a href="https://pythoninsider.blogspot.com/2024/10/python-3140-alpha-1-is-now-available.html?featured_on=pythonbytes">Python 3.14.0 alpha 1 is now available</a></p> <ul> <li>First of seven planned alpha releases.</li> <li>Many new features for Python 3.14 are still being planned and written. Among the new major new features and changes so far: <ul> <li><a href="https://peps.python.org/pep-0649/?featured_on=pythonbytes">PEP 649</a>: <a href="https://docs.python.org/3.14/whatsnew/3.14.html#pep-649-deferred-evaluation-of-annotations">deferred evaluation of annotations</a></li> <li><a href="https://docs.python.org/3.14/whatsnew/3.14.html#improved-error-messages">Improved error messages</a></li> </ul></li> </ul> <p><strong>Brian #2:</strong> <a href="https://github.com/astral-sh/uv/pull/8272?featured_on=pythonbytes"><strong>uv supports dependency groups</strong></a></p> <ul> <li><a href="https://pythonbytes.fm/episodes/show/406/whats-on-django-tv-tonight">we covered dependency groups in episode 406</a></li> <li>as of <a href="https://github.com/astral-sh/uv/blob/main/CHANGELOG.md?featured_on=pythonbytes">0.4.27</a>, uv supports dependency groups</li> <li>docs show <a href="https://docs.astral.sh/uv/concepts/dependencies/?featured_on=pythonbytes">how to add dependencies</a> with uv add --group <ul> <li>also “The --dev, --only-dev, and --no-dev flags are equivalent to --group dev, --only-group dev, and --no-group dev respectively.”</li> </ul></li> <li>To install a group, uv pip install --group doesn’t work yet. <ul> <li>It’s waiting for PyPA to decide on an interface for pip, and uv pip will use that interface.</li> </ul></li> <li>But sync works. <pre><code>$ uv init # create a pyproject.toml $ uv add --group foo pytest $ uv venv # create venv $ uv sync --group foo # will install all dependencies, including group "foo" </code></pre></li> </ul> <p><strong>Michael #3:</strong> <a href="https://github.com/wagoodman/dive?featured_on=pythonbytes">dive: A tool for exploring each layer in a docker image</a></p> <ul> <li>via Mike Fiedler</li> <li>Features: <ul> <li>Show Docker image contents broken down by layer</li> <li>Indicate what's changed in each layer</li> <li>Estimate "image efficiency"</li> <li>Quick build/analysis cycles</li> <li>CI Integration</li> </ul></li> </ul> <p><strong>Brian #4:</strong> <a href="https://pypi.org/project/pytest-metadata/?featured_on=pythonbytes"><strong>pytest-metadata</strong></a></p> <ul> <li>An incredibly useful plugin for adding, you guessed it, metadata, to your pytest results.</li> <li>Required for <a href="https://pypi.org/project/pytest-html/?featured_on=pythonbytes">pytest-html</a> but also useful on it’s own</li> <li>Adds metadata to <ul> <li>text output with --verbose</li> <li>xml output when using --junit-xml, handy for CI systems that support junit.xml</li> </ul></li> <li>Other plugins depend on this and report in other ways, such as pytest-html</li> <li>By default, already grabs <ul> <li>Python version</li> <li>Platform info</li> <li>List of installed packages</li> <li>List of installed pytest plugins</li> </ul></li> <li>You can add your own metadata</li> <li>You can access all metadata (and add to it) from tests, fixtures, and hook functions via a metadata fixture.</li> <li>This is in the <a href="https://pythontest.com/top-pytest-plugins/?featured_on=pythonbytes">Top pytest Plugins list</a>, currently #5.</li> </ul> <p><strong>Extras</strong> </p> <p>Brian:</p> <ul> <li>I’ve started filtering deprecated plugins from <a href="https://pythontest.com/top-pytest-plugins/?featured_on=pythonbytes">the pytest plugin list</a>.</li> <li>I’m also going to start reviewing the list and pulling out interesting plugins as the topic of the <a href="https://testandcode.com?featured_on=pythonbytes">next season of Test &amp; Code</a>.</li> </ul> <p>Michael:</p> <ul> <li><a href="https://mastodon.social/@hugovk/113312137194438039?kjy=spring&featured_on=pythonbytes">Pillow 11 is out</a></li> <li><a href="https://hachyderm.io/@graham_knapp/113351051856672146?featured_on=pythonbytes">pip install deutschland</a></li> <li><a href="https://talkpython.fm/blog/?featured_on=pythonbytes">Talk Python has a dedicated blog</a>, please subscribe!</li> </ul> <p><strong>Joke:</strong> Dog names</p>

Salesforce API integrations and connected apps 2024-10-28, by Dariusz Suchojad Overview

This instalment in a series of articles about API integrations with Salesforce covers connected apps - how to create them and how to obtain their credentials needed to exchange REST messages with Salesforce.

In Salesforce's terminology, a connected app is, essentially, an API client. It has credentials, a set of permissions, and it works on behalf of a user in an automated manner.

In particular, the kind of a connected app that I am going to create below is one that can be used in backend, server-side integrations that operate without any direct input from end users or administrators, i.e. the app is created once, its permissions and credentials are set once, and then it is able to work uninterrupted in the background, on server side.

Server-side systems are quite unlike other kinds of apps, such as mobile ones, that assume there is a human operator involved - they have their own work characteristics, related yet different, and I am not going to cover them here.

Note that permission types and their scopes are a separate, broad subject and they will described in a separate how-to article.

Finally, I assume that you are either an administrator in a Salesforce organization or that you are preparing information for another person with similar grants in Salesforce.

Conceptually, there is nothing particularly unusual about Salesforce connected apps, it is just its own mini-world of jargon and, at the end of the day, it simply enables you to invoke APIs that Salesforce is built on. It is just that knowing where to click, what to choose and how to navigate the user interface can be a daunting challenge that this article hopes to make easier to overcome.

The steps

For an automated, server-side connected app to make use of Salesforce APIs, the requirements are:

• Having access to username/password credentials
• Creating a connected app
• Granting permissions to the app (not covered in this article)
• Obtaining a customer key and customer secret for the app

You will note that there are four credentials in total:

• Username
• Password
• Customer key
• Customer secret

Also, depending on what chapter of the Salesforce documentation you are reading, you will note that the customer key can be also known as "client_id" whereas another name for the customer secret is "client_secret". These two pairs mean the same.

Access to username/password credentials

For starters, you need to have an account in Salesforce, a combination of username + password that you can log in with and on whose behalf the connected app will be created:

Creating a connected app

Once you are logged in, go to Setup in the top right-hand corner:

In the search box, look up "app manager":

Next, click the "New Connected App" button to the right:

Fill out the basic details such as "Connect App Name" and make sure that you select "Enable OAuth Settings". Then, given that in this document we are not dealing with the subject of permissions at all, grant full access to the connected app and finally click "Save" at the bottom of the page.

Obtaining a customer key and customer secret

We have a connected app but we still do not know what its customer key and secret are. To reveal it, go to the "App Manager" once more, either via the search box or using the menu on the left hand side.

Find your app in the list and click "View" in the list of actions. Observe that it is "View", not "Edit" or "Manage", where you can check what the credentials are:

The customer key and secret van be now revealed in the "API (Enable OAuth Settings)" section:

This concludes the process - you have a connected app and all the credentials needed now.

Testing

Seeing as this document is part of a series of how-tos in the context of Zato, if you would like to integrate with Salesforce in Python, at this point you will be able to follow the steps in another where everything is detailed separately.

Just as a quick teaser, it would look akin to the below.

... # Salesforce REST API endpoint to invoke path = '/sobjects/Campaign/' # Build the request to Salesforce based on what we received request = { 'Name': input.name, 'Segment__c': input.segment, } # Create a reference to our connection definition .. salesforce = self.cloud.salesforce['My Salesforce Connection'] # .. obtain a client to Salesforce .. with salesforce.conn.client() as client: # type: SalesforceClient # .. create the campaign now. response = client.post(path, request) ...

On a much lower level, however, if you would just like to quickly test out whether you configured the connected app correctly, you can invoke from command line a Salesforce REST endpoint that will return an OAuth token, as below.

Note that, as I mentioned it previously, client_id is the same as customer key and client_secret is the same as customer secret.

curl https://example.my.salesforce.com/services/oauth2/token \ -H "X-PrettyPrint: 1" \ --header 'Content-Type: application/x-www-form-urlencoded' \ --data-urlencode 'grant_type=password' \ --data-urlencode 'username=hello@example.com' \ --data-urlencode 'password=my.password' \ --data-urlencode 'client_id=my.customer.key' \ --data-urlencode 'client_secret=my.client.secret'

The result will be, for instance:

{ "access_token" : "008e0000000PTzLPb!4Vzm91PeIWJo.IbPzoEZf2ygEM.6cavCt0YwAGSM", "instance_url" : "https://example.my.salesforce.com", "id" : "https://login.salesforce.com/id/008e0000000PTzLPb/0081fSUkuxPDrir000j1", "token_type" : "Bearer", "issued_at" : "1649064143961", "signature" : "dwb6rwNIzl76kZq8lQswsTyjW2uwvTnh=" }

Above, we have an OAuth bearer token on output - this can be used in subsequent, business REST calls to Salesforce but how to do it exactly in practice is left for another article.

Next steps:

➤ Read about how to use Python to build and integrate enterprise APIs that your tests will cover
➤ Python API integration tutorial
➤ Python Integration platform as a Service (iPaaS)
➤ What is an Enterprise Service Bus (ESB)? What is SOA?

More blog posts➤

The End of Drupal 7: Why Are So Many Sites Choosing WordPress? kanapatrick Mon, 10/28/2024 - 10:48

Since January 5, 2025, Drupal 7 has officially reached its end of life. This iconic Content Management System (CMS), used by thousands of sites worldwide, no longer receives security updates or official support. This deadline has prompted many site administrators to rethink their strategy and select a new platform to ensure continuity.

How can you perform a product data migration from an external database to Drupal using the tools available within the Migrate API? In this blog post, I’ll show you how to connect to the database, prepare the data structure, and use the migration tools available in Drush. This entry is aimed at people who have already had experience with migrations as well as those who are just getting started with them. I encourage you to read the article or watch the video of the “Nowoczesny Drupal” series.

The Drupal Association has published client guides to RFPs that prioritise open source software solutions. 

by fiona.crowson / 28 October 2024

In a recent blog post, 'How to write an RFP for Open Source Solutions: Featuring Drupal Certified Partners', the Drupal Association outlines:

• the advantages of open source software
• tips for finding the ideal service provider (and why Drupal Certified Partners like PreviousNext make for good partners)
• guidance for crafting a successful RFP
• strategies for evaluating proposals

Clients also have access to a downloadable open source Request For Proposal (aka Request For Quote) template.

The core of the guide provides a detailed overview about why choosing a Drupal Certified Partner is the key to the technical expertise, smooth collaboration and commitment to quality and innovation that helps ensure the success of your projects. PreviousNext has been able to attain the current top ranked Drupal Certified Partner status globally by demonstrating our proven track record, commitment to the Drupal open source community and our verifiable capabilities.  

Our team is highly experienced and happy to answer your questions about the advantages of Drupal, so please feel free to get in touch.

Quick links

• How to write an RFP article
• RFP template download

Kwave Update - October 2024

Kwave is an audio editor based on the KDE Frameworks. It was started in 1998 by Martin Wilz, and Thomas Eschenbacher has been the main developer since 1999. In recent years development has slowed. I wanted to do some software development and contribute to KDE, and I’m interested in audio, so towards the end of 2023 I started working on Kwave.

Kwave had not been ported to Qt 6 and KDE Frameworks 6 yet, so that’s what I started working towards. My first merge requests were to update deprecated code. (MR Convert plugin desktop files to json, MR Port away from deprecated Qt API, MR port away from deprecated I18N_NOOP macros, MR bump KF5_MIN_VERSION and update where KMessageBox API has been deprecated, MR port QRegExp to QRegularExpression)

With that preparatory work done, I worked at porting Kwave to Qt 6 and KDE Frameworks 6. Most of that work was straight-forward. The biggest changes were in Qt Multimedia, which Kwave can use for playback and recording. I finally got that done and merged in August 2024, just after version 24.08 was branched, so that change will get released in version 24.12 in December 2024. (MR port to Qt6 and KF6)

Next I did some code cleanup. (MR use ECMGenerateExportHeader, MR add braces to avoid ambiguous else, MR call KCrash::initialize() after KAboutData::setApplicationData())

Laurent Montel added the FreeBSD job to the Continuous Integration configuration, but the build failed initially. I’ve never ran FreeBSD, but with a few tries and pushing changes to trigger CI, I managed to get the CI to pass. I’m glad Laurent took the initiative here, because the FreeBSD job uses clang, so with the existing Linux job using gcc, CI makes sure Kwave builds with both compilers now. (MR Add freebsd)

I applied for a KDE Developer account and got approved on August 24, 2024. Now I could commit changes myself instead of having to remind others to do it.

Carl Schwan cleaned up some code and updated the zoom toolbar to use standard icons, which enabled removing the built-in zoom icons. (MR Modernize ZoomToolbar) That was the incentive I needed to remove the rest of the built-in icons and use standard icons instead, which helps Kwave fit the users theme better. (MR use icons from current theme) I also reordered the playback toolbar in a way that seemed more logical to me. (MR update player toolbar)

I investigated a bug: Kwave Playback settings dialog loads incorrectly until you switch playback methods and fixed it in MR: make sure a valid method gets selected when PlayBackDialog opens

I have some more work in progress, and I plan to continue working on Kwave. I will try to blog about what I’m doing, but I’m not going to commit to any regular schedule.

Get Involved

Kwave depends on the rest of KDE. It is built on the Frameworks, and the KDE sysadmin team keeps the infrastructure running. You can help KDE by getting involved, or at least donate.

If you would like to help improve Kwave, use it and try out its features! If you have questions or ideas, discuss them. If you find bugs, report them. If you want to get involved with development, download the source code and start hacking!

If you think my own work on Kwave is worth something and you can afford it, you can donate to me through Liberapay, Stripe, or PayPal.

At Acquia Engage NYC this week, our partner and customer conference, we shared how Acquia's Digital Experience Platform (DXP) helps organizations deliver digital experiences through three key capabilities:

• Content: Create, manage and deliver digital content and experiences - from images and videos to blog posts, articles, and landing pages - consistently across all your digital channels.
• Optimize: Continuously improve your digital content and experiences by improving accessibility, readability, brand compliance, and search engine optimization (SEO).
• Insights: Understand how people interact with your digital experiences, segment audiences based on their behavior and interests, and deliver personalized content that drives better engagement and conversion rates.

Since our last Acquia Engage conference in May, roughly six months ago, we've made some great progress, and we announced some major innovations and updates across our platform.

The Acquia Open DXP platform consists of three pillars - Content, Optimize, and Insight - with specialized products in each category to help organizations create, improve, and personalize digital experiences. Simplify video creation in Acquia DAM

Video is one of the most engaging forms of media, but it's also one of the most time-consuming and expensive to create. Producing professional, branded videos has traditionally required significant time, budget, and specialized skills. Our new Video Creator for DAM changes this equation. By combining templating, AI, and DAM's workflow functionality, organizations can now create professional, on-brand videos in minutes rather than days.

Make assets easier to find in Acquia DAM

Managing large digital asset libraries can become increasingly overwhelming. Traditional search methods rely on extensive metadata tagging and manual filtering options. Depending on what you are looking for, it might be difficult to quickly find the right assets.

To address this, we introduced Acquia DAM Copilot, which transforms the experience through conversational AI. Instead of navigating complicated filter menus, users can now simply type natural requests like "show me photos of bikes outside" and refine their search conversationally with commands like "only show bikes from the side view". This AI-powered approach eliminates the need for extensive tagging and makes finding the right content intuitive and fast.

Easier site building with Drupal

I updated the Acquia Engage audience on Drupal CMS (also known as Drupal Starshot), a major initiative I'm leading in the Drupal community with significant support from Acquia. I demonstrated several exciting innovations coming to Drupal: "recipes" to simplify site building, AI-powered site creation capabilities, and a new Experience Builder that will transform how we build Drupal websites.

Many in the audience had already watched my DrupalCon Barcelona keynote and expressed continued enthusiasm for the direction of Drupal CMS and our accelerated pace of innovation. Even after demoing it multiple times the past month, I'm still very excited about it myself. If you want to learn more, be sure to check out my DrupalCon presentation!

Improving content ranking with Acquia SEO

Creating content that ranks well in search engines traditionally requires both specialized SEO expertise and skilled content writers - making it an expensive and time-consuming process. Our new SEO Copilot, powered by Conductor, integrated directly into Drupal's editing experience, provides real-time guidance on keyword optimization, content suggestions, length recommendations, and writing complexity for your target audience. This helps content teams create search-engine-friendly content more efficiently, without needing deep SEO expertise.

Improving content quality with Acquia Optimize

We announced the rebranding of Monsido to Acquia Optimize and talked about two major improvements to this offering.

First, we improved how organizations create advanced content policies. Creating advanced content policies usually requires some technical expertise, as it can involve writing regular expressions. Now, users can simply describe in plain language what they want to monitor. For example, they could enter something like "find language that might be insensitive to people with disabilities", and AI will help create the appropriate policy rules. Acquia Optimize will then scan content across all your websites to detect any violations of those rules.

Second, we dramatically shortened the feedback loop for content checking. Previously, content creators had to publish their content and then wait for scheduled scans to discover problems with accessibility, policy compliance or technical SEO - a process that could take a couple of days. Now, they can get instant feedback. Authors can request a check while they work, and the system immediately flags accessibility issues, content policy violations, and other problems, allowing them to fix problems while the content is being written. This shift from "publish and wait" to "check and fix" helps teams maintain higher content quality standards, allows them to work faster, and can prevent non-compliant content from ever going live.

FedRAMP for Acquia Cloud Next

We were excited to announce that our next-generation Drupal Cloud, Acquia Cloud Next (ACN), has achieved FedRAMP accreditation, just like our previous platform, which remains FedRAMP accredited.

This means our government customers can now migrate their Drupal sites onto our latest cloud platform, taking advantage of improved autoscaling, self-healing, and cutting-edge features. We already have 56 FedRAMP customers hosting their Drupal sites on ACN, including Fannie Mae, The US Agency for International Development, and the Department of Education, to name a few.

Improved fleet management for Drupal

Acquia Cloud Site Factory is a platform that helps organizations manage fleets of Drupal sites from a single dashboard, making it easier to launch, update, and scale sites. Over the past two years, we've been rebuilding Site Factory on top of Acquia Cloud Next, integrating them more closely. Recently, we reached a major milestone in this journey. At Engage, we showcased Multi-Experience Operations (MEO) to manage multiple Drupal codebases across your portfolio of sites.

Previously, all sites in a Site Factory instance had to run the same Drupal code, requiring simultaneous updates across all sites. Now, organizations can run sites on different codebases and update them independently. This added flexibility is invaluable for large organizations managing hundreds or thousands of Drupal sites, allowing them to update at their own pace and maintain different Drupal versions where needed.

Improved conversion rates with Acquia Convert

Understanding user behavior is key to optimizing digital experiences, but interpreting the data and deciding on next steps can be challenging. We introduced some new Acquia Convert features (powered by VWO) to solve this.

First, advanced heat-mapping shows exactly how users interact with your pages, where they click first, how far they scroll, and where they show signs of frustration (like rage clicks).

Next, and even more powerful, is our new Acquia Convert Copilot that automatically analyzes this behavioral data to suggest specific improvements. For example, if the AI notices high interaction with a pricing slider but also signs of user confusion, it might suggest an A/B test to clarify the slider's purpose. This helps marketers and site builders make data-driven decisions and improve conversion rates.

Privacy-first analytics with Piwik Pro

As data privacy regulations become stricter globally, organizations face growing challenges with web analytics. Google Analytics has been banned in several European countries for not meeting data sovereignty requirements, leaving organizations scrambling for compliant alternatives.

We announced a partnership with Piwik Pro to address this need. Piwik Pro offers a privacy-first analytics solution that maintains compliance with global data regulations by allowing organizations to choose where their data is stored and maintaining full control over their data.

This makes it an ideal solution for organizations that operate in regions with strict data privacy laws, or any organization that wants to ensure their analytics solution remains compliant with evolving privacy regulations.

After the Piwik Pro announcement at Acquia Engage, I spoke with several customers who are already using Piwik Pro. Most worked in healthcare and other sectors handling sensitive data. They were excited about our partnership and a future that brings deeper integration between Piwik Pro, Acquia Optimize, Drupal, and other parts of our portfolio.

Conclusion

The enthusiasm from our customers and partners at Acquia Engage always reinvigorates me. None of these innovations would be possible without the dedication of our teams at Acquia. I'm grateful for their hard work in bringing these innovations to life, and I'm excited for what is next!

Drupal 11: Batch Operations Built Into Drupal

This is the sixth article in a series of articles about the Batch API in Drupal. The Batch API is a system in Drupal that allows data to be processed in small chunks in order to prevent timeout errors or memory problems.

So far in this series we have looked at creating a batch process using a form, followed by creating a batch class so that batches can be run through Drush, using the finished state to control batch processing, processing CSV files through a batch process and finally adding to running batch processes. These articles give a good grounding of how to use the Drupal Batch API.

In this article we will look at how the Batch API is used within Drupal. The Batch API in Drupal is either used to perform a task, which I will call "direct", or to pass on the batch operations to a hook, which I will call "indirect". These aren't official terms you understand, I'm just using them here to separate how Drupal uses the Batch API. I find these terms useful to describe where the batch is running.

Let's look at direct usage first.

Direct

Direct usage just means that a method in Drupal creates a BatchBuilder object and then uses that object to setup and trigger the batch run (via the batch_set() function). This is used in a variety of situations all over Drupal, including:

philipnorton42 Sun, 10/27/2024 - 18:23

As we have been doing yearly, a few weeks ago we had the 2024 edition of Linux App Summit (LAS). For those of you who don’t know, the Linux App Summit is a conference co-organised between KDE and GNOME among others where to bring together the different stakeholders of the linux ecosystem to make sure we have all the collaboration tools in place to have a great state of the art platform for the uses the world needs from us.

This year it was special in that since it became the Linux App Summit, we held it outside of Europe! We had LAS in Monterrey, the northernmost part of México in the ITESM (Instituto Tecnológico y de Estudios Superiores de Monterrey), a really nice campus and a beautiful venue.

Besides meeting people from other projects as I’d normally do in LAS, this year I had the added opportunity to meet the community the other side of the ocean. There certainly are good differences between how we organise FOSS communities Europe and there, it’s always useful to experience and iterate to ensure we are truly creating a community for everyone.

LAS ’25, Call for hosts

Are you considering bringing such communities to your home town? Please reach out, you can find some information how to do that here: https://linuxappsummit.org/bids/.

In my opinion, organising a conference is one of the best ways to bring FOSS talent to where you are. It’s often hard to find new ways to be part of what we do, this is a great one. I personally had the opportunity to do it in LAS’19 in Barcelona and it was a great experience.

LAS has the added value of being a melting pot of people from different communities so it’s a great opportunity to meet people that you might not crossed paths with before!

This looks straightforward and is far from it. I expect tool support will improve in the future. Meanwhile, this blog post serves as a step by step explanation for what is going on in code that I'm about to push to my team.

Let's take this relatively straightforward python code. It has a function printing an int, and a decorator that makes it argument optional, taking it from a global default if missing:

from unittest import mock default = 42 def with_default(f): def wrapped(self, value=None): if value is None: value = default return f(self, value) return wrapped class Fiddle: @with_default def print(self, value): print("Answer:", value) fiddle = Fiddle() fiddle.print(12) fiddle.print() def mocked(self, value=None): print("Mocked answer:", value) with mock.patch.object(Fiddle, "print", autospec=True, side_effect=mocked): fiddle.print(12) fiddle.print()

It works nicely as expected:

$ python3 test0.py Answer: 12 Answer: 42 Mocked answer: 12 Mocked answer: None

It lacks functools.wraps and typing, though. Let's add them.

Adding functools.wraps

Adding a simple @functools.wraps, mock unexpectedly stops working:

# python3 test1.py Answer: 12 Answer: 42 Mocked answer: 12 Traceback (most recent call last): File "/home/enrico/lavori/freexian/tt/test1.py", line 42, in <module> fiddle.print() File "<string>", line 2, in print File "/usr/lib/python3.11/unittest/mock.py", line 186, in checksig sig.bind(*args, **kwargs) File "/usr/lib/python3.11/inspect.py", line 3211, in bind return self._bind(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/inspect.py", line 3126, in _bind raise TypeError(msg) from None TypeError: missing a required argument: 'value'

This is the new code, with explanations and a fix:

# Introduce functools import functools from unittest import mock default = 42 def with_default(f): @functools.wraps(f) def wrapped(self, value=None): if value is None: value = default return f(self, value) # Fix: # del wrapped.__wrapped__ return wrapped class Fiddle: @with_default def print(self, value): assert value is not None print("Answer:", value) fiddle = Fiddle() fiddle.print(12) fiddle.print() def mocked(self, value=None): print("Mocked answer:", value) with mock.patch.object(Fiddle, "print", autospec=True, side_effect=mocked): fiddle.print(12) # mock's autospec uses inspect.getsignature, which follows __wrapped__ set # by functools.wraps, which points to a wrong signature: the idea that # value is optional is now lost fiddle.print() Adding typing

For simplicity, from now on let's change Fiddle.print to match its wrapped signature:

# Give up with making value not optional, to simplify things :( def print(self, value: int | None = None) -> None: assert value is not None print("Answer:", value) Typing with ParamSpec # Introduce typing, try with ParamSpec import functools from typing import TYPE_CHECKING, ParamSpec, Callable from unittest import mock default = 42 P = ParamSpec("P") def with_default(f: Callable[P, None]) -> Callable[P, None]: # Using ParamSpec we forward arguments, but we cannot use them! @functools.wraps(f) def wrapped(self, value: int | None = None) -> None: if value is None: value = default return f(self, value) return wrapped class Fiddle: @with_default def print(self, value: int | None = None) -> None: assert value is not None print("Answer:", value)

mypy complains inside the wrapper, because while we forward arguments we don't constrain them, so we can't be sure there is a value in there:

test2.py:17: error: Argument 2 has incompatible type "int"; expected "P.args" [arg-type] test2.py:19: error: Incompatible return value type (got "_Wrapped[P, None, [Any, int | None], None]", expected "Callable[P, None]") [return-value] test2.py:19: note: "_Wrapped[P, None, [Any, int | None], None].__call__" has type "Callable[[Arg(Any, 'self'), DefaultArg(int | None, 'value')], None]" Typing with Callable

We can use explicit Callable argument lists:

# Introduce typing, try with Callable import functools from typing import TYPE_CHECKING, Callable, TypeVar from unittest import mock default = 42 A = TypeVar("A") # Callable cannot represent the fact that the argument is optional, so now mypy # complains if we try to omit it def with_default(f: Callable[[A, int | None], None]) -> Callable[[A, int | None], None]: @functools.wraps(f) def wrapped(self: A, value: int | None = None) -> None: if value is None: value = default return f(self, value) return wrapped class Fiddle: @with_default def print(self, value: int | None = None) -> None: assert value is not None print("Answer:", value) if TYPE_CHECKING: reveal_type(Fiddle.print) fiddle = Fiddle() fiddle.print(12) # !! Too few arguments for "print" of "Fiddle" [call-arg] fiddle.print() def mocked(self, value=None): print("Mocked answer:", value) with mock.patch.object(Fiddle, "print", autospec=True, side_effect=mocked): fiddle.print(12) fiddle.print()

Now mypy complains when we try to omit the optional argument, because Callable cannot represent optional arguments:

test3.py:32: note: Revealed type is "def (test3.Fiddle, Union[builtins.int, None])" test3.py:37: error: Too few arguments for "print" of "Fiddle" [call-arg] test3.py:46: error: Too few arguments for "print" of "Fiddle" [call-arg]

typing's documentation says:

Callable cannot express complex signatures such as functions that take a variadic number of arguments, overloaded functions, or functions that have keyword-only parameters. However, these signatures can be expressed by defining a Protocol class with a call() method:

Let's do that!

Typing with Protocol, take 1 # Introduce typing, try with Protocol import functools from typing import TYPE_CHECKING, Protocol, TypeVar, Generic, cast from unittest import mock default = 42 A = TypeVar("A", contravariant=True) class Printer(Protocol, Generic[A]): def __call__(_, self: A, value: int | None = None) -> None: ... def with_default(f: Printer[A]) -> Printer[A]: @functools.wraps(f) def wrapped(self: A, value: int | None = None) -> None: if value is None: value = default return f(self, value) return cast(Printer, wrapped) class Fiddle: # function has a __get__ method to generated bound versions of itself # the Printer protocol does not define it, so mypy is now unable to type # the bound method correctly @with_default def print(self, value: int | None = None) -> None: assert value is not None print("Answer:", value) if TYPE_CHECKING: reveal_type(Fiddle.print) fiddle = Fiddle() # !! Argument 1 to "__call__" of "Printer" has incompatible type "int"; expected "Fiddle" fiddle.print(12) fiddle.print() def mocked(self, value=None): print("Mocked answer:", value) with mock.patch.object(Fiddle, "print", autospec=True, side_effect=mocked): fiddle.print(12) fiddle.print()

New mypy complaints:

test4.py:41: error: Argument 1 to "__call__" of "Printer" has incompatible type "int"; expected "Fiddle" [arg-type] test4.py:42: error: Missing positional argument "self" in call to "__call__" of "Printer" [call-arg] test4.py:50: error: Argument 1 to "__call__" of "Printer" has incompatible type "int"; expected "Fiddle" [arg-type] test4.py:51: error: Missing positional argument "self" in call to "__call__" of "Printer" [call-arg]

What happens with class methods, is that the function object has a __get__ method that generates a bound versions of itself. Our Printer protocol does not define it, so mypy is now unable to type the bound method correctly.

Typing with Protocol, take 2

So... we add the function descriptor methos to our Protocol!

A lot of this is taken from this discussion.

# Introduce typing, try with Protocol, harder! import functools from typing import TYPE_CHECKING, Protocol, TypeVar, Generic, cast, overload, Union from unittest import mock default = 42 A = TypeVar("A", contravariant=True) # We now produce typing for the whole function descriptor protocol # # See https://github.com/python/typing/discussions/1040 class BoundPrinter(Protocol): """Protocol typing for bound printer methods.""" def __call__(_, value: int | None = None) -> None: """Bound signature.""" class Printer(Protocol, Generic[A]): """Protocol typing for printer methods.""" # noqa annotations are overrides for flake8 being confused, giving either D418: # Function/ Method decorated with @overload shouldn't contain a docstring # or D105: # Missing docstring in magic method # # F841 is for vulture being confused: # unused variable 'objtype' (100% confidence) @overload def __get__( # noqa: D105 self, obj: A, objtype: type[A] | None = None # noqa: F841 ) -> BoundPrinter: ... @overload def __get__( # noqa: D105 self, obj: None, objtype: type[A] | None = None # noqa: F841 ) -> "Printer[A]": ... def __get__( self, obj: A | None, objtype: type[A] | None = None # noqa: F841 ) -> Union[BoundPrinter, "Printer[A]"]: """Implement function descriptor protocol for class methods.""" def __call__(_, self: A, value: int | None = None) -> None: """Unbound signature.""" def with_default(f: Printer[A]) -> Printer[A]: @functools.wraps(f) def wrapped(self: A, value: int | None = None) -> None: if value is None: value = default return f(self, value) return cast(Printer, wrapped) class Fiddle: # function has a __get__ method to generated bound versions of itself # the Printer protocol does not define it, so mypy is now unable to type # the bound method correctly @with_default def print(self, value: int | None = None) -> None: assert value is not None print("Answer:", value) fiddle = Fiddle() fiddle.print(12) fiddle.print() def mocked(self, value=None): print("Mocked answer:", value) with mock.patch.object(Fiddle, "print", autospec=True, side_effect=mocked): fiddle.print(12) fiddle.print()

It works! It's typed! And mypy is happy!

Chat Changes, Editor Enhancements and Audio Advancements

Welcome to a new issue of "This Week in KDE Apps"! Every week we cover as much as possible of what's happening in the world of KDE apps.

This week's changes and improvements cover a wide range of applications, from audio apps (including the classic Amarok, which is making a comeback) to Kate getting improvements to its integrated Git features.

In between, you have everything from new functionalities for note-taking utilities and media players, to upgrades in financial software and mobile apps.

Let's dig in!

Amarok A powerful music player that lets you rediscover your music

Tuomas Nurmi worked on making the codebase Qt6-compatible. (Tuomas Nurmi, Link)

Ark Archiving Tool

Jin Liu disabled the "Compress to tar.gz/zip" service menu items in read-only directories. (Jin Liu, 24.12.0. Link)

Dolphin Manage your files

You can now sort your videos by duration. (Somsubhra Bairi, 24.12.0. Link)

Eren Karakas added more standard actions (Sort By, View Mode, Cut and Copy) to the context menu in the trash view. (Eren Karakas, 24.12.0. Link)

Elisa Play music and listen to online radio stations

Elisa now supports loading lyrics from .lrc files sitting alongside the song files. (Gary Wang, 24.12.0. Link)

Manuel Roth fixed the bug in which the metadata for webradio http streams was not getting displayed. (Manuel Roth, 24.12.0. Link)

Haruna Media player

You now have the option to open videos in full screen mode. (Rikesh Patel, Link)

KDE Itinerary Digital travel assistant

Volker Krause was at the OSM Hack Weekend last week and worked on the support of MOTIS v2 API support in the public transport client library used by KDE Itinerary. He also added a map view of an entire trip to Itinerary and the KPublicTransport demo application.

Read more on his blog!

Kate Advanced Text Editor

In large repos, a git status update can be slow. The least we can do for the user is show that something is happening. Hence, now, if the git status is being refreshed you will see the refresh button become unclickable and start spinning. (Waqar Ahmed, 24.12.0. Link)

In the project tree view, files will now show their status in git. The status is shown minimally, i.e. via a small circle displayed in front of the file name. If the file has been modified, the circle is red; if the file is staged, it's green. (Waqar Ahmed, 24.12.0. Link)

We simplified the git panel by hiding the project combobox. The git panel will now show the status of the currently opened project. (Waqar Ahmed, 24.12.0. Link)

We fixed the SQL plugin's SQL export being randomly ordered. (Waqar Ahmed, 24.08.3. Link)

Clock Keep time and set alarms

KClock's timer now shows the remaining time instead of the elapsed time. (Zhangzhi Hu, 24.12.0. Link)

KMix Sound Mixer

We fixed the Audio Setup button, which didn't open the System Settings Audio page correctly. (Sergio Basto, 24.12.0. Link)

KMyMoney Personal finance manager based on double-entry bookkeeping

It's once again possible to download stock quotes from yahoo.com after they changed their output format. (Ralf Habacker, Link)

Reports can now be exported as PDF and XML. (Ralf Habacker, KMyMoney 5.2.0. Link 1, link 2)

Photos Image Gallery

We improved the design of the properties panel. (Carl Schwan, 24.12.0. Link)

Kleopatra Certificate manager and cryptography app

The name of the "KWatchGnuPG" utility provided by Kleopatra has been updated to "GnuPG Log Viewer" (Carl Schwan, 24.12.0. Link) and we gave it a new logo.

KleverNotes Take and manage your notes

KleverNotes' painting mode has been completely rewritten. It is now possible to add circles, rectangles, labels, and to choose the stroke size. The UI also uses a new floating toolbar. (Louis Schul, 1.2.0. Link)

We improved the animation when switching pages. (Luis Schul, 1.2.0. Link)

The note preview in the appearance settings was simplified to only show the important parts. (Luis Schul, 1.2.0. Link)

KMail A feature-rich email application

Fix a crash in the Exchange Web Services (EWS) backend. (Louis Moureaux, 24.08.3. Link)

KRDC Connect with RDP or VNC to another computer

We fixed sharing folders. (Fabio Bas, 24.08.3. Link)

Merkuro Calendar Manage your tasks and events with speed and ease

Claudio Cambra fixed adding and creating sub-todos (Claudio Cambra, 24.08.3. Link and Link)) and a bug that made clicking on the month view unreliable. (Claudio Cambra, 24.08.3, Link).

We also added back the maps showing the location of individual events. This was disabled during the Qt6 migration and never enabled back afterwards. (Claudio Cambra, 24.08.3, Link)

NeoChat Chat on Matrix

Support for libQuotient 0.9 has been backported to NeoChat 24.08. This brings, among other things, cross-signing support and support for the Matrix 1.12 API, including most importantly content repo functionality switching to authenticated media. (James Graham, 24.08.0, Link)

Okular View and annotate documents

Albert Astals fixed switching between pages in the single-page mode when using a mouse with a "high resolution" scroll wheel. (Albert Astals Cid, 24.12.0. Link)

You can now use any image type as a signature background. (Sune Vuorela, 24.12.0. Link)

We removed the last CHM support mention in Okular and on the website. CHM support was dropped when transitioning to the Qt6 version. (Albert Astals Cid, 24.12.0. Link 1, link 2)

Zanshin To Do Management Application

Fixed an issue where projects would be displayed twice when toggling on and off their data source. (David Faure, 24.08.3. Link)

And all this too...

Justin Zobel fixed various appstream files to use the new way of declaring the developer's name. (Justin Zobel, KRuler, Gwenview, KEuroCalc, ...)

We ported various projects to use declarative QML declaration for better maintainance and performance (Carl Schwan, Koko, Francis, Kalk).

... And Everything Else

This blog only covers the tip of the iceberg! If you’re hungry for more, check out Nate's blog about Plasma and be sure not to miss his This Week in Plasma series, where every Saturday he covers all the work being put into KDE's Plasma desktop environment.

For a complete overview of what's going on, visit KDE's Planet, where you can find all KDE news unfiltered directly from our contributors.

Get Involved

The KDE organization has become important in the world, and your time and contributions have helped us get there. As we grow, we're going to need your support for KDE to become sustainable.

You can help KDE by becoming an active community member and getting involved. Each contributor makes a huge difference in KDE — you are not a number or a cog in a machine! You don’t have to be a programmer either. There are many things you can do: you can help hunt and confirm bugs, even maybe solve them; contribute designs for wallpapers, web pages, icons and app interfaces; translate messages and menu items into your own language; promote KDE in your local community; and a ton more things.

You can also help us by donating. Any monetary contribution, however small, will help us cover operational costs, salaries, travel expenses for contributors and in general just keep KDE bringing Free Software to the world.

To get your application mentioned here, please ping us in invent or in Matrix.

Are you near London and want to learn about automated testing in Drupal?

I'll be presenting a session and Q&A on automated testing and test-driven development in Drupal.

This is one of my favourite topics to present and teach, so I'm looking forward to this event.

If you want to attend, RSVP on the meetup event page.

If you can't, check out my free Drupal testing email course or book a 1-on-1 consulting call and I'll get you started.

Drupal 11.1.0 and 10.4.0 release dates

Drupal core typically has a minor release window the second week of December. This is to provide enough time after PHP and Symfony's release dates for core compatibility to be updated, but still far enough before the major end-of-year holidays to avoid interfering with vacations and travel.

This year, DrupalCon Singapore is scheduled for the same week as the minor release. Normally, we would avoid having a minor release the same week as a DrupalCon, but in this case we are unable to move the release date. We will aim to release 11.1.0 and 10.4.0 later in the week to avoid having the release during the actual days of the conference. The release window is now December 12-13 UTC.

Drupal 11.0 and 10.3 will continue to have security coverage until June 2025. So, it is safe for site owners to wait until January 2025 or later, if necessary, to update their sites.

Drupal 11.1 alpha phase begins October 28

In preparation for the minor release, Drupal 11.1.x will enter the alpha phase the week of October 28, 2024. Core developers should plan to complete changes that are only allowed in minor releases prior to the alpha release. The 11.1.0-alpha1 deadline for most core patches is October 28, 2024.

The 10.5.x release branch of core will be created for the next maintenance minor release.

• Developers and site owners can begin testing the alpha after its release.

• The 11.1.x release branch of core will be created before the alpha is tagged. Future feature and API additions will continue to be targeted against 11.x.

• After 11.1.x is branched but before 11.1.0-alpha1 is tagged, alpha experimental modules will be removed from the 11.1.x codebase. Their development will continue in 11.x only.

• Following the release of Drupal 11.1 and 10.4, only security issues will be fixed in Drupal 11.0 and 10.3. Additionally, Drupal 10.2 will become end-of-life (EOL).

• During the alpha phase, core issues will be committed according to the following policy:

  1. Most issues that are allowed for patch releases will be committed to 11.1.x and 10.4.x. Such issues may also be committed to 11.0.x and 10.3.x until the final normal bugfix releases of 11.0 and 10.3 on December 4, 2024.
  2. Most issues that are only allowed in minor releases will be committed to 11.x only. (Such issues may be released in 11.2 or another future minor.). A few strategic issues may be backported to 11.1.x, but only at committer discretion after the issue is fixed in 11.x (so leave them set to 11.x unless you are a committer), and only up until the beta deadline.
  3. Most issues that are allowed in maintenance minor releases will be committed to 11.x and 10.5.x only. A few strategic issues may be backported to 11.1.x and 10.4.x, but only at committer discretion after the issue is fixed in 11.x (so leave them set to 11.x unless you are a committer), and only up until the beta deadline.

Roughly two weeks after the alpha release, the first beta release will be created. All the restrictions of the alpha release apply to beta releases as well. The release of the first beta is a firm deadline for all feature and API additions. Even if an issue is pending in the Reviewed & Tested by the Community (RTBC) queue when the commit freeze for the beta begins, it will be committed to the next minor release only.

The release candidate phase will begin the week of November 25.

Security support of Drupal 10 and 11 Drupal 10.2.x Security releases will be provided until December 12, 2024. Drupal 10.3.x and 11.0.x Security releases will be provided until June 18, 2025.

See the Drupal core release process overview, the Drupal core release schedule, allowed changes during the Drupal 10 and 11 release cycles, and Drupal 10 and 11 backwards compatibility and internal API policy for more information.

Again this year, Arm offered to host us for a mini-debconf in Cambridge. Roughly 60 people turned up on 10-13 October to the Arm campus, where they made us really welcome. They even had some Debian-themed treats made to spoil us!

Hacking together

For the first two days, we had a "mini-debcamp" with disparate group of people working on all sorts of things: Arm support, live images, browser stuff, package uploads, etc. And (as is traditional) lots of people doing last-minute work to prepare slides for their talks.

Sessions and talks

Saturday and Sunday were two days devoted to more traditional conference sessions. Our talks covered a typical range of Debian subjects: a DPL "Bits" talk, an update from the Release Team, live images. We also had some wider topics: handling your own data, what to look for in the upcoming Post-Quantum Crypto world, and even me talking about the ups and downs of Secure Boot. Plus a random set of lightning talks too! :-)

Video team awesomeness

Lots of volunteers from the DebConf video team were on hand too (both on-site and remotely!), so our talks were both streamed live and recorded for posterity - see the links from the individual talk pages in the wiki, or http://meetings-archive.debian.net/pub/debian-meetings/2024/MiniDebConf-Cambridge/ for the full set if you'd like to see more.

A great time for all

Again, the mini-conf went well and feedback from attendees was very positive. Thanks to all our helpers, and of course to our sponsor: Arm for providing the venue and infrastructure for the event, and all the food and drink too!

Photo credits: Andy Simpkins, Mark Brown, Jonathan Wiltshire. Thanks!

Read more

At Drupalcamp Spain I had this moment of inspiration where I saw a further comparison between Drupal and the USS Enterprise from Star Trek.

Enjoy this creative exercise :)

 

Drupal and the USS enterprisedrupalSaturday, October 26, 2024 - 20:42

In one way or another, I have developed, configured, and worked with Drupal for over 15 years. On almost every website I’ve had the privilege of working on, there have been various forms of forms—comment fields, contact forms, membership requests, and so on. And something that’s always been present is spam.

Regardless of the size of the site, bots eventually find the forms. I’ve moved from module to module trying to prevent forms from being overtaken by bots and their often offensive content, which 99.9% of the time includes a link to some obscure website, often on the darker parts of the web. But where there are spam bots, there are also services and modules to stop them. Over the years, I’ve moved from module to module as bots have become smarter and some modules have become outdated technologically.

About six months ago, I revamped my own site, AdamEvertsson.se, and just recently realized that I’d forgotten to add a spam prevention module. How did I notice? I happened to see that I had over 3,500 comments spread across a very small number of posts—all 100% spam.

I quickly activated one of the classic modules I’ve used, but the spam posts continued to pour in by the dozens every day. Even though I have some go-to modules, I thought it might be interesting to see what’s new among spam prevention modules since it had been a while since I updated myself on the state of Drupal spam-blocking modules.

I quickly found the Antibot module, a new discovery for me, and within just a couple of days of testing, it proved to be 100% effective against spam. Since it worked so incredibly well, I stopped searching. I haven’t received a single spam post since activating it earlier this week, and I now have a new favorite to add to my collection of modules when building Drupal sites.

Here are the modules I currently consider relevant for blocking spam posts:

Antibot

As mentioned, this is now my go-to for spam-fighting and will be my standard module for spam management for a good while—until it loses effectiveness and another module steps up.

Visit the module’s project page on drupal.org.

Honeypot

A classic module that monitors how quickly a form is filled in, with some other functions as well. It’s been a favorite for many years and keeps pace with Drupal’s development. I highly recommend it and still use it on my sites that run on Drupal 7, for instance.

Visit the module’s project page on drupal.org.

Google reCAPTCHA

The classic box with prompts like "select all boxes with a moped" or "choose the images showing a bridge" is something we’ve all seen. It’s one of the internet’s most effective and widely used systems for ensuring “I am not a robot.” In Drupal alone, there are over 168,000 registered sites using this module and the reCAPTCHA system.

Visit the module’s project page on drupal.org.

Anti-Spam by CleanTalk

This is a new module I came across during my search but didn’t get around to testing since I found Antibot, which worked well. It has a bit more modest stats in terms of usage, with just over 3,000 sites using it, but it’s maintained and appears reliable.

Visit the module’s project page on drupal.org.

SpamSpan

While it doesn’t block spam directly, it prevents email addresses displayed on the site from being picked up by bots. It can and should be combined with one of the modules above.

Visit the module’s project page on drupal.org

Last weekend I attended the bi-annual OSM Hack Weekend in Karlsruhe again, organized by Geofabrik and this time hosted at a nearby university building due to the large number of participants.

Transitous

My main focus has been getting the public transport client library used by KDE Itinerary ready for MOTIS v2, as Transitous, our community-run public transport routing service, will switch to that in the not too distant future.

One big new feature in MOTIS v2 is support for GTFS shapes. That is, getting detailed paths for public transport sections, beyond just positions of intermediate stops, which allows for a much more useful map display for example.

Even more importantly, MOTIS now also provides detailed multi-floor paths for transfers or other parts of a trip where you have to move yourself (walking, biking, etc). This is all based on OSM data and thus matches perfectly to the map data, but since practically no other backend provides this level of detail it also required a few changes in our data model and API.

Besides the new MOTIS API being much more intuitive than the previous one having had Felix from the MOTIS team around (even if just online) who instantly implemented all suggested improvements in the server made this super productive.

If your region isn’t covered by Transitous yet, check out the contributor documentation on how to change that.

Itinerary

For debugging parsing of paths provided by MOTIS I added a map view to the KPublicTransport demo app. That ended up getting close to what we’d need for a map view of an entire trip in Itinerary, so we also have that now. It’s not where I’d like it yet e.g. regarding interactivity and the look of bi-directional paths it’s a good start.

Trip map view prototype in Itinerary.

A full trip map view was also one of the feature requests I got from other participants. Another suggestions that came up and that meanwhile has been implemented is pre-filling the stop location history with all locations involved in the current trip, which is quite helpful during trip planning.

Indoor Routing

Following a discussion on detailed mapping of hedges in outdoor mazes I learned there’s an OSM wiki page on that subject, which also lists a bunch of examples.

While I don’t really have any particular interest in outdoor mazes and/or fancy hedge art, these things just ask for being used as a test case for our indoor router.

Indoor router finding a way through a maze made out of hedges. You can help!

Hack weekends how this is called in the OSM community or sprints as this is known in the KDE community are immensely valuable and productive. There’s a great deal of knowledge transfer happening, and they are a big motivational boost.

However, physical meetings incur costs, and that’s where your donations help! KDE e.V. and local OSM chapters like the FOSSGIS e.V. support these activities.

The Announcement

Late last month there was an announcement of a “severity 9.9 vulnerability” allowing remote code execution that affects “all GNU/Linux systems (plus others)” [1]. For something to affect all Linux systems that would have to be either a kernel issue or a sshd issue. The announcement included complaints about the lack of response of vendors and “And YES: I LOVE hyping the sh1t out of this stuff because apparently sensationalism is the only language that forces these people to fix”.

He seems to have a different experience to me of reporting bugs, I have had plenty of success getting bugs fixed without hyping them. I just report the bug, wait a while, and it gets fixed. I have reported potential security bugs without even bothering to try and prove that they were exploitable (any situation where you can make a program crash is potentially exploitable), I just report it and it gets fixed. I was very dubious about his ability to determine how serious a bug is and to accurately report it so this wasn’t a situation where I was waiting for it to be disclosed to discover if it affected me. I was quite confident that my systems wouldn’t be at any risk.

Analysis Not All Linux Systems Run CUPS

When it was published my opinion was proven to be correct, it turned out to be a series of CUPS bugs [2]. To describe that as “all GNU/Linux systems (plus others)” seems like a vast overstatement, maybe a good thing to say if you want to be a TikTok influencer but not if you want to be known for computer security work.

For the Debian distribution the cups-browsed package (which seems to be the main exploitable one) is recommended by cups-daemon, as I have my Debian systems configured to not install recommended packages by default that means that it wasn’t installed on any of my systems. Also the vast majority of my systems don’t do printing and therefore don’t have any part of CUPS installed.

CUPS vs NAT

The next issue is that in Australia most home ISPs don’t have IPv6 enabled and CUPS doesn’t do the things needed to allow receiving connections from the outside world via NAT with IPv4. If inbound port 631 is blocked on both TCP and USP as is the default on Australian home Internet or if there is a correctly configured firewall in place then the network is safe from attack. There is a feature called uPnP port forwarding [3] to allow server programs to ask a router to send inbound connections to them, this is apparently usually turned off by default in router configuration. If it is enabled then there are Debian packages of software to manage this, the miniupnpc package has the client (which can request NAT changes on the router) [4]. That package is not installed on any of my systems and for my home network I don’t use a router that runs uPnP.

The only program I knowingly run that uses uPnP is Warzone2100 and as I don’t play network games that doesn’t happen. Also as an aside in version 4.4.2-1 of warzone2100 in Debian and Ubuntu I made it use Bubblewrap to run the game in a container. So a Remote Code Execution bug in Warzone 2100 won’t be an immediate win for an attacker (exploits via X11 or Wayland are another issue).

MAC Systems

Debian has had AppArmor enabled by default since Buster was released in 2019 [5]. There are claims that AppArmor will stop this exploit from doing anything bad.

To check SE Linux access I first use the “semanage fcontext” command to check the context of the binary, cupsd_exec_t means that the daemon runs as cupsd_t. Then I checked what file access is granted with the sesearch program, mostly just access to temporary files, cupsd config files, the faillog, the Kerberos cache files (not used on the Kerberos client systems I run), Samba run files (might be a possibility of exploiting something there), and the security_t used for interfacing with kernel security infrastructure. I then checked the access to the security class and found that it is permitted to check contexts and access-vectors – not access that can be harmful.

The next test was to use sesearch to discover what capabilities are granted, which unfortunately includes the sys_admin capability, that is a capability that allows many sysadmin tasks that could be harmful (I just checked the Fedora source and Fedora 42 has the same access). Whether the sys_admin capability can be used to do bad things with the limited access cupsd_t has to device nodes etc is not clear. But this access is undesirable.

So the SE Linux policy in Debian and Fedora will stop cupsd_t from writing SETUID programs that can be used by random users for root access and stop it from writing to /etc/shadow etc. But the sys_admin capability might allow it to do hostile things and I have already uploaded a changed policy to Debian/Unstable to remove that. The sys_rawio capability also looked concerning but it’s apparently needed to probe for USB printers and as the domain has no access to block devices it is otherwise harmless. Below are the commands I used to discover what the policy allows and the output from them.

# semanage fcontext -l|grep bin/cups-browsed /usr/bin/cups-browsed regular file system_u:object_r:cupsd_exec_t:s0 # sesearch -A -s cupsd_t -c file -p write allow cupsd_t cupsd_interface_t:file { append create execute execute_no_trans getattr ioctl link lock map open read rename setattr unlink write }; allow cupsd_t cupsd_lock_t:file { append create getattr ioctl link lock open read rename setattr unlink write }; allow cupsd_t cupsd_log_t:file { append create getattr ioctl link lock open read rename setattr unlink write }; allow cupsd_t cupsd_runtime_t:file { append create getattr ioctl link lock open read rename setattr unlink write }; allow cupsd_t cupsd_rw_etc_t:file { append create getattr ioctl link lock open read rename setattr unlink write }; allow cupsd_t cupsd_t:file { append create getattr ioctl link lock open read rename setattr unlink write }; allow cupsd_t cupsd_tmp_t:file { append create getattr ioctl link lock open read rename setattr unlink write }; allow cupsd_t faillog_t:file { append getattr ioctl lock open read write }; allow cupsd_t init_tmpfs_t:file { append getattr ioctl lock read write }; allow cupsd_t krb5_host_rcache_t:file { append create getattr ioctl link lock open read rename setattr unlink write }; [ allow_kerberos ]:True allow cupsd_t print_spool_t:file { append create getattr ioctl link lock open read relabelfrom relabelto rename setattr unlink write }; allow cupsd_t samba_var_t:file { append getattr ioctl lock open read write }; allow cupsd_t security_t:file { append getattr ioctl lock open read write }; allow cupsd_t security_t:file { append getattr ioctl lock open read write }; [ allow_kerberos ]:True allow cupsd_t usbfs_t:file { append getattr ioctl lock open read write }; # sesearch -A -s cupsd_t -c security allow cupsd_t security_t:security check_context; [ allow_kerberos ]:True allow cupsd_t security_t:security { check_context compute_av }; # sesearch -A -s cupsd_t -c capability allow cupsd_t cupsd_t:capability net_bind_service; [ allow_ypbind ]:True allow cupsd_t cupsd_t:capability { audit_write chown dac_override dac_read_search fowner fsetid ipc_lock kill net_bind_service setgid setuid sys_admin sys_rawio sys_resource sys_tty_config }; # sesearch -A -s cupsd_t -c capability2 allow cupsd_t cupsd_t:capability2 { block_suspend wake_alarm }; # sesearch -A -s cupsd_t -c blk_file Conclusion

This is an example of how not to handle security issues. Some degree of promotion is acceptable but this is very excessive and will result in people not taking security announcements seriously in future. I wonder if this is even a good career move by the researcher in question, will enough people believe that they actually did something good in this that it outweighs the number of people who think it’s misleading at best?

• [1] https://threadreaderapp.com/thread/1838169889330135132.html
• [2] https://tinyurl.com/26rjd5ex
• [3] https://tinyurl.com/2ckyvpyq
• [4] https://packages.debian.org/sid/miniupnpc
• [5] https://wiki.debian.org/AppArmor/HowToUse

Related posts:

1. SE Linux audit2allow -R and Milter policy Since the earliest days there has been a command named...
2. SE Linux File Context Precedence In my previous post I expressed a desire to use...
3. SE Linux Things To Do At the end of my talk on Monday about the...