Feeds

EuroPython 2024 talks about security About • Blog • Newsletter • Links EuroPython 2024 talks about security

Published 2024-10-04 by Seth Larson
Reading time: minutes

EuroPython 2024 which occurred back in July 2024 has published the talk recordings to YouTube earlier this week. I've been under the weather for most of this week, but have had a chance to listen to a few of the security-related talks in-between resting.

Counting down for Cyber Resilience Act: Updates and expectations

This talk was delivered by Python Software Foundation Executive Director Deb Nicholson and and Board Member Cheuk Ting Ho. The Cyber Resilience Act (CRA) is coming, and it'll affect more software than just the software written in the EU. Deb and Cheuk describe the recent developments in the CRA like the creation of a new entity called the "Open Source Steward" and how open source foundations and maintainers are preparing for the CRA.

For the rest of this year and next year I am focusing on getting the Python ecosystem ready for software security regulations like the CRA and SSDF from the United States.

Starting with improving the Software Bill-of-Materials (SBOM) story for Python, because this is required by both (and likely, future) regulations. Knowing what software you are running is an important first step towards being able to secure that same software.

To collaborate with other open source foundations and projects on this work, I've joined the Open Regulatory Compliance Working Group hosted by the Eclipse Foundation.

Towards licensing standardization in Python packaging

This talk was given by Karolina Surma and it detailed all the work that goes into researching, writing, and having a Python packaging standard accepted (spoiler: it's a lot!). Karolina is working on PEP 639 which is for adopting the SPDX licensing expression and identifier standards in Python as they are the current state of the art for modeling complex licensing situations accurately for machine (and human) consumption.

This work is very important for Software Bill-of-Materials, as they require accurate license information in this exact format. Thanks to Karolina, C.A.M. Gerlach, and many others for working for years on this PEP, it will be useful to so many uers once adopted!

The Update Framework (TUF) joins PyPI

This talk was given by Kairo de Araujo and Lukas Pühringer and it detailed the history and current status of The Update Framework (TUF) integration into the Python Package Index.

TUF provides better integrity guarantees for software repositories like PyPI like making it more difficult to "compel" the index to serve the incorrect artifacts and to make a compromise of PyPI easier to roll-back and be certain that files hadn't been modified. For a full history and latest status, you can view PEP 458 and the top-level GitHub issue for Warehouse.

I was around for the original key-signing ceremony for the PyPI TUF root keys which was live-streamed back in October 2020. Time flies, huh.

Writing Python like it's Rust: more robust code with type hints

This talk was given by Jakub Beránek about using type hints for more robust Python code. Having written a case-study on urllib3's adoption of type hints to find defects that testing and other tooling missed I highly recommend type hints for Python code as well:

Accelerating Python with Rust: The PyO3 Revolution

This talk was given by Roshan R Chandar about using PyO3 and Rust in Python modules.

Automatic Trusted Publishing with PyPI

This talk was given by Facundo Tuesca on using Trusted Publishing for authenticating with PyPI to publish packages.

Zero Trust APIs with Python

This talk was given by Jose Haro Peralta on how to design and implement secure web APIs using Python, data validation with Pydantic, and testing your APIs using tooling for detecting common security defects.

Best practices for securely consuming open source in Python

This talk was given by Cira Carey which highlights many of today's threats targetting open source consumers. Users should be aware of these when selecting projects to download and install.

Thanks for reading! ♡ Did you find this article helpful and want more content like it? Get notified of new posts by subscribing to the RSS feed or the email newsletter.

This work is licensed under CC BY-SA 4.0

Earlier this week I considered whether I should finally switch away from virtualenvwrapper to using local .venv managed by direnv.

I’ve never seriously used direnv, but I’ve been hearing Jeff and Hynek talk about their use of direnv for a while.

After a few days, I’ve finally stumbled into a setup that works great for me. I’d like to note the basics of this setup as well as some fancy additions that are specific to my own use case.

My old virtualenvwrapper workflow

First, I’d like to note my old workflow that I’m trying to roughly recreate:

1. I type mkvenv3 <project_name> to create a new virtual environment for the current project directory and activate it
2. I type workon <project_name> when I want to workon that project: this activates the correct virtual environment and changes to the project directory

The initial setup I thought of allows me to:

1. Run echo layout python > .envrc && direnv allow to create a virtual environment for the current project and activate it
2. Change directories into the project directory to automatically activate the virtual environment

The more complex setup I eventually settled on allows me to:

1. Run venv <project_name> to create a virtual environment for the current project and activate it
2. Run workon <project_name> to change directories into the project (which automatically activates the virtual environment)

The initial setup

First, I installed direnv and added this to my ~/.zshrc file:

1 eval "$(direnv hook zsh)"

Then whenever I wanted to create a virtual environment for a new project I created a .envrc file in that directory, which looked like this:

1 layout python

Then I ran direnv allow to allow, as direnv instructed me to, to allow the new virtual environment to be automatically created and activated.

That’s pretty much it.

Unfortunately, I did not like this initial setup.

No shell prompt?

The first problem was that the virtual environment’s prompt didn’t show up in my shell prompt. This is due to a direnv not allowing modification of the PS1 shell prompt. That means I’d need to modify my shell configuration to show the correct virtual environment name myself.

So I added this to my ~/.zshrc file to show the virtual environment name at the beginning of my prompt:

1 2 3 4 5 6 7 # Add direnv-activated venv to prompt show_virtual_env() { if [[ -n "$VIRTUAL_ENV_PROMPT" && -n "$DIRENV_DIR" ]]; then echo "($(basename $VIRTUAL_ENV_PROMPT)) " fi } PS1='$(show_virtual_env)'$PS1 Wrong virtual environment directory

The next problem was that the virtual environment was placed in .direnv/python3.12. I wanted each virtual environment to be in a .venv directory instead.

To do that, I made a .config/direnv/direnvrc file that customized the python layout:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 layout_python() { if [[ -d ".venv" ]]; then VIRTUAL_ENV="$(pwd)/.venv" fi if [[ -z $VIRTUAL_ENV || ! -d $VIRTUAL_ENV ]]; then log_status "No virtual environment exists. Executing \`python -m venv .venv\`." python -m venv .venv VIRTUAL_ENV="$(pwd)/.venv" fi # Activate the virtual environment . $VIRTUAL_ENV/bin/activate } Loading, unloading, loading, unloading…

I also didn’t like the loading and unloading messages that showed up each time I changed directories. I removed those by clearing the DIRENV_LOG_FORMAT variable in my ~/.zshrc configuration:

1 export DIRENV_LOG_FORMAT= The more advanced setup

I don’t like it when all my virtual environment prompts show up as .venv. I want ever prompt to be the name of the actual project… which is usually the directory name.

I also really wanted to be able to type venv to create a new virtual environment, activate it, and create the .envrc file for my automatically.

Additionally, I thought it would be really handy if I could type workon <project_name> to change directories to a specific project.

I made two aliases in my ~/.zshrc configuration for all of this:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 venv() { local venv_name=${1:-$(basename "$PWD")} local projects_file="$HOME/.projects" # Check if .envrc already exists if [ -f .envrc ]; then echo "Error: .envrc already exists" >&2 return 1 fi # Create venv if ! python3 -m venv --prompt "$venv_name"; then echo "Error: Failed to create venv" >&2 return 1 fi # Create .envrc echo "layout python" > .envrc # Append project name and directory to projects file echo "${venv_name} = ${PWD}" >> $projects_file # Allow direnv to immediately activate the virtual environment direnv allow } workon() { local project_name="$1" local projects_file="$HOME/.projects" local project_dir # Check for projects config file if [[ ! -f "$projects_file" ]]; then echo "Error: $projects_file not found" >&2 return 1 fi # Get the project directory for the given project name project_dir=$(grep -E "^$project_name\s*=" "$projects_file" | sed 's/^[^=]*=\s*//') # Ensure a project directory was found if [[ -z "$project_dir" ]]; then echo "Error: Project '$project_name' not found in $projects_file" >&2 return 1 fi # Ensure the project directory exists if [[ ! -d "$project_dir" ]]; then echo "Error: Directory $project_dir does not exist" >&2 return 1 fi # Change directories cd "$project_dir" }

Now I can type this to create a .venv virtual environment in my current directory, which has a prompt named after the current directory, activate it, and create a .envrc file which will automatically activate that virtual environment (thanks to that ~/.config/direnv/direnvrc file) whenever I change into that directory:

1 $ venv

If I wanted to customized the prompt name for the virtual environment, I could do this:

1 $ venv my_project

When I wanted to start working on that project later, I can either change into that directory or if I’m feeling lazy I can simply type:

1 $ workon my_project

That reads from my ~/.projects file to look up the project directory to switch to.

Switching to uv

I also decided to try using uv for all of this, since it’s faster at creating virtual environments. One benefit of uv is that it tries to select the correct Python version for the project, if it sees a version noted in a pyproject.toml file.

Another benefit of using uv, is that I should also be able to update the venv to use a specific version of Python with something like --python 3.12.

Here are the updated shell aliases for the ~/.zshrc for uv:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 venv() { local venv_name local dir_name=$(basename "$PWD") # If there are no arguments or the last argument starts with a dash, use dir_name if [ $# -eq 0 ] || [[ "${!#}" == -* ]]; then venv_name="$dir_name" else venv_name="${!#}" set -- "${@:1:$#-1}" fi # Check if .envrc already exists if [ -f .envrc ]; then echo "Error: .envrc already exists" >&2 return 1 fi # Create venv using uv with all passed arguments if ! uv venv --seed --prompt "$@" "$venv_name"; then echo "Error: Failed to create venv" >&2 return 1 fi # Create .envrc echo "layout python" > .envrc # Append to ~/.projects echo "${venv_name} = ${PWD}" >> ~/.projects # Allow direnv to immediately activate the virtual environment direnv allow } Switching to starship

I also decided to try out using Starship to customize my shell this week.

I added this to my ~/.zshrc:

1 eval "$(starship init zsh)"

And removed this, which is no longer needed since Starship will be managing the shell for me:

1 2 3 4 5 6 7 # Add direnv-activated venv to prompt show_virtual_env() { if [[ -n "$VIRTUAL_ENV_PROMPT" && -n "$DIRENV_DIR" ]]; then echo "($(basename $VIRTUAL_ENV_PROMPT)) " fi } PS1='$(show_virtual_env)'$PS1

I also switched my python layout for direnv to just set the $VIRTUAL_ENV variable and add the $VIRTUAL_ENV/bin directory to my PATH, since the $VIRTUAL_ENV_PROMPT variable isn’t needed for Starship to pick up the prompt:

1 2 3 4 5 layout_python() { VIRTUAL_ENV="$(pwd)/.venv" PATH_add "$VIRTUAL_ENV/bin" export VIRTUAL_ENV }

I also made a very boring Starship configuration in ~/.config/starship.toml:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 format = """ $python\ $directory\ $git_branch\ $git_state\ $character""" add_newline = false [python] format = '([(\($virtualenv\) )]($style))' style = "bright-black" [directory] style = "bright-blue" [character] success_symbol = "[\\$](black)" error_symbol = "[\\$](bright-red)" vimcmd_symbol = "[❮](green)" [git_branch] format = "[$symbol$branch]($style) " style = "bright-purple" [git_state] format = '\([$state( $progress_current/$progress_total)]($style)\) ' style = "purple" [cmd_duration.disabled]

I setup such a boring configuration because when I’m teaching, I don’t want my students to be confused or distracted by a prompt that has considerably more information in it than their default prompt may have.

The biggest downside of switching to Starship has been my own earworm-oriented brain. As I update my Starship configuration files, I’ve repeatedly heard David Bowie singing “I’m a Starmaaan”. 🎶

Ground control to major TOML

After all of that, I realized that I could additionally use different Starship configurations for different directories by putting a STARSHIP_CONFIG variable in specific layouts. After that realization, I made my configuration even more vanilla and made some alternative configurations in my ~/.config/direnv/direnvrc file:

1 2 3 4 5 6 7 8 9 10 11 12 layout_python() { VIRTUAL_ENV="$(pwd)/.venv" PATH_add "$VIRTUAL_ENV/bin" export VIRTUAL_ENV export STARSHIP_CONFIG=/home/trey/.config/starship/python.toml } layout_git() { export STARSHIP_CONFIG=/home/trey/.config/starship/git.toml }

Those other two configuration files are fancier, as I have no concern about them distracting my students since I’ll never be within those directories while teaching.

You can find those files in my dotfiles repository.

The necessary tools

So I replaced virtualenvwrapper with direnv, uv, and Starship. Though direnv was is doing most of the important work here. The use of uv and Starship were just bonuses.

I am also hoping to eventually replace my pipx use with uv and once uv supports adding python3.x commands to my PATH, I may replace my use of pyenv with uv as well.

Thanks to all who participated in my Mastodon thread as I fumbled through discovering this setup.

KStars v3.7.3 is released on 2024.10.03 for Windows, MacOS & Linux. It's a bi-monthly bug-fix release with a couple of exciting features.

Extension Interface

Ed Lee contributed the Extension Interface for KStars.

Extensions are small programs that can be added to interact with KStars/Ekos/INDI in order to provide extra functions and features. Extensions are separate from KStars. They are not provided as part of this software. Only a means to call them is provided for convenience. Make sure that you understand the requirements and risks of using an extension.

Three extensions are ready for release:

• FireCapture launcher (FC_launcher): disconnects the current primary camera INDI driver and launches FireCapture. Upon close restarts the INDI driver.
• KStars Backup (KS_backup): provides a GUI for the archiving and restoration of KStars/INDI (and optionally others) configuration directories to/from .tar.gz archives/
• Sirial_EEA: provides live stacking of the preview job from the Capture Module and displays it auto-stretched in the Ekos preview window.

Each extension must have a companion configuration file also located in the extensions directory, named the same as the executable with the addition of a .conf eg: an extension named example must also have a configuration file named example.conf A configuration file is a plain text file that provides configuration settings to the extension program and usage information to the user. A configuration file is only valid if it contains a line starting with: minimum_kstars_version=x.y.z The x.y.z is the minimum release of KStars that the extension is designed/tested against. This value is checked against the current KStars KSTARS_VERSION macro defined in version.h and must be equal or lower for the extension to be considered valid. The extension should also check that this minimum_kstars_version string matches what it expects.

Optionally each extension can also provide an icon file for display in the Extension drop down list. Again the naming should match the extension executable with a valid file extension (.jpg, .bmp, .gif, .png or .svg) and be placed in the same extensions directory. A default icon is used for any extension that does not provide it's own icon.

Several new DBus functions/signals are added to enable general extension use and for a specific upcoming extension.
Multi-Target SchedulerWolfgang Reissenberger continued the development of multi-camera acquisition and now scheduling with many exciting updates!
Multi-Camera SchedulingWith this new release it is possible to create and run schedules for two or more optical trains in parallel within the same KStars instance. All mount related events like slewing, dithering, alignment and meridian flip are synchronized, i.e. capturing on all optical trains takes these events into account.
Lead and follower jobsIf you want to run capturing on multiple optical trains in parallel, you need to decide, which of the optical trains is the lead job, which defines the target and the scheduling criteria. All other optical trains will be follower jobs, for which only the capture sequence to be used is relevant.
Job SynchronizationEkos ensures that no mount motion happens while one of the optical trains is capturing to avoid star trails.

In case of dithering, the capture module decides what to do as soon as dithering is requested, depending on the setup of the lead job. If the remaining capture time of a follower train is more than 50% of the lead job exposure time, capturing of the follower train is stopped and restarted after dithering. If the remaining capture time is less than 50%, the Capture module waits for the follower to finish its capture.

For the meridian flip and re-alignment: both are controlled by the lead job. All running follower capturing sequences are stopped and restarted after a successful completion of the respective action.
Recommendations

• The lead job should have the longest exposure times, especially if dithering is used.
• If you use re-focusing, consider to not suspend guiding during focusing, since the other optical train might capture in parallel.
• Configure the same optical train in the Align module that you use in the lead job. This is important if you use the re-alignment check option of the scheduler.

Do not use the Force re-alignment before re-starting jobs, since each new iteration of the lead job stops capturing of the follower trains. Consider using Verify captured image position every n frames instead, since this option triggers re-alignment only if it is necessary.Adhoc focus with Multi-Camera setupsWolfgang Reissenberger added support for simultaneous parallel focusing when you are running multi-camera setup. It would be a good idea to disable Suspend Guiding so that guiding is unaffected when focusing operation commences.Invent Multi-Star Guide Star

Hy Murveit introduced an experimental Multi-Star parameter. In the original (conservative) multi-star guiding implementation, although many stars contributed to the computed guiding drift computation, there still was one more important star that anchored the scheme.

With this change, all the guide stars contribute equally to the guiding calculations. This removes the risk that an unfortunate selection of a double star for the more-important main guide star would degrade performance.

Improvements & Bug fixes

• Fix crash on ekos reconnect
• Scheduler end-time display fix
• Focus Filter Offsets fix
• Add altitude graphs to scheduler
• Allow Flat re-calibration on the fly

This week, I spent my time mostly looking at issues tagged with "Editor Experience".

We’re pleased to announce that Release Candidate 1 of the Open Source AI Definition has been confirmed and published! If you’d like to add your name to the list of endorsers published online, please let us know. 

We traveled four continents presenting to diverse audiences and soliciting feedback on the draft definition: Deep Learning Indaba in Dakar, Senegal; IndiaFOSS in Bangalore, India; Open Source Summit EU in Vienna, Austria; and Nerdearla in Buenos Aires, Argentina.

The work continues this month as we continue to seek input at the Data in OSAI in Paris, France; at OCX in Mainz, Germany; and during our weekly town hall meetings. And, finally, we’ll be in Raleigh, North Carolina, at the end of the month for All Things Open, where we plan to present the Open Source AI Definition version 1.0!

My thanks to everyone who is contributing to this community-led process. Please continue to let your voice be heard.

Stefano Maffulli

Executive Director, OSI 

I hold weekly office hours on Fridays with OSI members: book time if you want to chat about OSI’s activities, if you want to volunteer or have suggestions.

News from the OSI The Open Source AI Definition RC1 is available for comments

The Open Source AI Definition first Release Candidate has been published and collaboration continues.

Other highlights:

• Co-designing the OSAID: a highlight from Nerdearla
• A Journey toward defining Open Source AI: presentation at Open Source Summit Europe
• Copyright law makes a case for requiring data information rather than open datasets for Open Source AI
• Data Transparency in Open Source AI: Protecting Sensitive Datasets
• Is “Open Source” ever hyphenated?
• Jordan Maris joins OSI

OSI in the news Defining Open Source AI Will Solve a Million Headaches

Article from Mark Surman at The New Stack

Other highlights:

• Elastic founder on returning to open source four years after going proprietary (TechCrunch)
• Europe’s Tech Future Hinges on Open Source AI (The New Stack)
• How big new AI regulatory pushes could affect open source (Tech Brew)
• Does Open Source Software Still Matter? (Datanami)
• AI2’s new model aims to be open and powerful yet cost effective (VentureBeat)
• Is that LLM Actually “Open Source”? We need to talk Open-Washing in AI Governance (HackerNoon)
• AI Models From Google, Meta, Others May Not Be Truly ‘Open Source’ (PCMag)
• What’s Behind Elastic’s Unexpected Return to Open Source? (The New Stack)

Join Open Policy Alliance

The Open Policy Alliance reaches 100 members on LinkedIn.

Other news

News from OSI affiliates:

• The Eclipse Foundation Launches the Open Regulatory Compliance Working Group to Help Open Source Participants Navigate Global Regulations
• LPI and OSI Unite to Professionalize the Global Linux and Open Source Ecosystem
• Apache Software Foundation Initiatives to Fuel the Next 25 Years of Open Source Innovation 
• Open source orgs strengthen alliance against patent trolls
• Open Source Foundations Considered Helpful
• Solving the Maker-Taker problem

News from OpenSource.net:

• OpenSource.Net turns one with a redesign
• Beyond the binary: The nuances of Open Source innovation
• Steady in a shifting Open Source world: FreeBSD’s enduring stability

Surveys

Tidelift’s 2024 State of the Open Source Maintainer Report

More than 400 maintainers responded and shared details about their work.

The State of Open Source Survey

In collaboration with the Eclipse Foundation and Open Source Initiative (OSI).

Jobs

OSI US Policy Manager

Lead OSI’s public policy agenda and education.

Bloomberg OSPO is Hiring

Bloomberg is seeking a Technical Architect to join their OSPO team.

Events

Upcoming events:

• Hacktoberfest (October – Online)
• SOSS Fusion (October 22-23, 2024 – Atlanta)
• Open Community Experience (October 22-24, 2024 – Mainz)
• All Things Open (October 27-29 – Raleigh)
• Nerdearla Mexico (November 7-9, 2024 – Mexico City)
• SeaGL (November 8-9, 2024 – Seattle)
• OpenForum Academy Symposium (November, 13-14, 2024 – Boston)

CFPs:

• FOSDEM 2025 call for devrooms (February 1-2, 2024 – Brussels)
• Consul Conference (February, 4-6, 2025 – Las Palmas de Gran Canaria)
• SCALE 22x (March 6-9, 2025 – Pasadena)

Thanks to our sponsors New sponsors and renewals

• Mercado Libre
• FerretDB
• Word Unscrambler

Interested in sponsoring, or partnering with, the OSI? Please see our Sponsorship Prospectus and our Annual Report. We also have a dedicated prospectus for the Deep Dive: Defining Open Source AI. Please contact the OSI to find out more about how your company can promote open source development, communities and software.

Support OSI by becoming a member!

Let’s build a world where knowledge is freely shared, ideas are nurtured, and innovation knows no bounds! 

Join the Open Source Initiative!

At the 10th anniversary of Nerdearla, one of the largest Open Source conferences in Latin America, Mer Joyce, Co-Design Facilitator of the Open Source AI Definition (OSAID), delivered a key presentation titled “Defining Open Source AI”. Held in Buenos Aires from September 24-28, 2024, this major event brought together 12,000 in-person participants and over 30,000 virtual attendees, with more than 200 speakers from 20 countries. Organized as a free-to-attend event, Nerdearla 2024 exemplified the spirit of Open Source collaboration by providing a platform for developers, enthusiasts, and thought leaders to share knowledge and foster community engagement.

Why is a definition so important?

Mer Joyce took the stage at Nerdearla to present “Defining Open Source AI”. Mer’s presentation focused on the organization’s ongoing work to establish a global Open Source AI Definition (OSAID). She emphasized the importance of co-designing this definition through a collaborative, inclusive process that ensures input from stakeholders across industries and continents.

Her talk underscored the significance of defining Open Source AI in the context of increasing AI regulations from governments in the EU, the U.S., and beyond. In her view, defining OSAI is essential for combating “open-washing”—where companies falsely market their AI systems as Open Source while imposing restrictive licenses—and for promoting true openness, transparency, and innovation in the AI space. 

A global and inclusive process

Mer Joyce highlighted the co-design process for the Open Source AI Definition, which has been truly global in scope. Workshops, talks, and activities were held on five continents, including Africa, Europe, Asia, North, and South America, with participants from over 35 countries. These in-person and virtual sessions ensured that voices from a wide range of backgrounds—especially those from underrepresented regions—contributed to shaping the OSAID.

The four freedoms

The core of the OSAID rests on the “Four Freedoms” of Open Source AI:

1. Use the system for any purpose and without having to ask for permission.
2. Study how the system works and inspect its components.
3. Modify the system for any purpose, including to change its output.
4. Share the system for others to use with or without modifications, for any purpose.

The working groups and their recommendations

Four working groups were formed with the intention of identifying what components must be open in order for an AI system to be used, studied, modified, and shared. The working groups focused on Bloom, OpenCV, Llama 2, and Pythia, four systems with different approaches to OSAI.

Each working group voted on the required components and evaluated legal frameworks and legal documents for each component. Subsequently, each working group proceeded to publish a recommendation report.

The end result is the OSAID with a comprehensive definition checklist encompassing a total of 17 components. As part of the validation process, more working groups are being formed to evaluate how well other AI systems align with the definition.

Nerdearla: a platform for open innovation

Mer Joyce’s presentation at Nerdearla exemplified the broader theme of the conference—creating a more open and collaborative future for technology. As one of the largest Open Source conferences in Latin America, Nerdearla serves as a vital hub for fostering innovation across the Open Source community. By bringing together experts like Mer Joyce to discuss pivotal issues such as AI transparency and openness, the event highlights the importance of defining shared standards for emerging technologies.

Moving forward: the future of the OSAID

The OSAID is currently in its final stages of development, with version 1.0 expected to be launched at the All Things Open conference in October 2024. The OSI invites individuals and organizations to endorse the OSAID ahead of its official release. This endorsement signifies support for a global definition that aims to ensure AI systems are open, transparent, and aligned with the values of the Open Source movement.
To get involved, participants are encouraged to attend weekly town halls, contribute feedback, and participate in the public review process. Consider endorsing the OSAID to become a part of the movement to define and promote truly Open Source AI systems.

Mer Joyce at Nerdearla and MercadoLibre’s office in Buenos Aires.

In this quiz, you’ll test your understanding of Python’s import statement and related topics.

By working through this quiz, you’ll revisit how to use modules in your scripts and import modules dynamically at runtime.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

A lot has happened again around KDE Itinerary since the previous summary post two month ago: A new two-level trip/timeline view, extended public transport location search, a new website and more public transport data coverage to name just a few things.

New Features Per-trip timeline

The probably biggest change is the replacement of the single combined timeline view that Itinerary had since its beginning with a two-level view consisting of a list of trips and a per-trip timeline view.

Trip list view.

Work on prerequisites for this has been featured in the past two summary blog posts already, such as the more explicit trip grouping controls and the staging area for about to be imported data.

Per-trip actions.

As trip grouping has become more relevant with this, there’s now multiple ways to explicitly control this:

• Merge two adjacent trips.
• Split an existing trip.
• Select which trip newly added content belongs to.

New transport icons

Another visually very noticeable change are the new transport icons by Andy Betts, replacing the previously used incoherent mix of different icon styles.

Breeze icons for modes of transport. Extended location search

The location picker for public transport searches can now also search for addresses instead of just stop names. Whether a result is a stop or an address can be distinguished by an icon, and more information about location results are shown when available (such as city, state/region and/or country), to help with identifying different places with the same name.

Searching for public transport stops and addresses.

Address search is only supported with the Transitous backend so far.

Another new way of getting to a location for a public transport search is via geo: URIs passed from other applications, which Itinerary can now handle on Linux and on Android.

Infrastructure Work New Website

Thanks to work by Carl on allowing to customize the automatically generated apps.kde.org pages, Itinerary now has a much nicer website, reusing some of the great content created for the KDE for Travelers page.

Transitous

The work on Transitous and MOTIS would deserve its own post, so this is just scratching the surface here, focusing on changes most impactful for Itinerary users.

• New base schedule coverage in France, Latvia, Lithuania, Montenegro, Poland, Serbia, Turkey and USA.
• New real-time data coverage in Croatia and Germany.
• Support for via routing, transfer time settings and GTFS shapes in MOTIS (which yet has to be made available to our clients though).
• Upgraded hardware thanks to an SSD donation, which should improve routing performance.

Matrix-based trip synchronization

The foundational work around explicit trip management in the past months has also cleared the path for synchronizing trips over Matrix. As mentioned in a previous post the work on this has now started.

Synchronization increases the requirements on precise change tracking and change notification, and it adds another path how data can change. The current implementation has been a bit sloppy in that regard, and improvements for this have already been integrated. This should fix timeline entries not updating correctly after an edit or receiving public transport data updates.

Fixes & Improvements Travel document extractor

• New or improved extractors for BlablaBus, booking.com, DB, Entur, Eurostar, Eventim, Flixbus, Italo, Koleo, MAV, Reisnordland ferries, Reservix, SNCB, SNCF, Sunnycars, United Airlines and VDV e-tickets.
• Support for importing Deutsche Bahn journey sharing links.

All of this has been made possible thanks to your travel document donations!

Public transport data

• Added access to BLS (Basel, Switzerland) and KVB (Cologne, Germany) public transport data.
• Fixed access to BVG (Berlin, Germany) and ZVV (Zürich, Switzerland) public transport data.
• Fixed missing intermediate stops in French long-distance trains, caused by incomplete data reported by ÖBB.
• Migrated to a new train coach layout API in Germany, increasing the coverage for regional and local trains as well as increasing the level of detail for vehicle feature information such as the quantity rather than just the presence of bike or wheelchair spaces.
• Improve data merging when location names are provided in localized and ASCII-transliterated forms.

Indoor map

• Improved display of semi-transparent logos in dark mode in element info dialog.
• Avoid element info dialog resizes during logo or image loading.
• Show bus station quay numbers on the map.

Bus station quay numbers. Itinerary app

• Fix some combo boxes not opening correctly in mobile mode.
• Use the more compact seat display from the timeline view also on details pages.
• Fix particularly long URLs not being detected as such during importing.
• Allow to create events in any OSM building.
• Fix showing arrival/departure times for disembark-only intermediate stops.
• Android’s dark mode is now respected automatically.
• Allow editing flight boarding groups.
• Show per-day sections in public transport journey search results.

How you can help

Feedback and travel document samples are very much welcome, as are all other forms of contributions. Feel free to join us in the KDE Itinerary Matrix channel.

After (quite) a summer break, here comes the 4th article of the 5-episode blog post series on Polis, written by Guido Berhörster, member of staff at my company Fre(i)e Software GmbH.

Have fun with the read on Guido's work on Polis,
Mike

Table of Contents of the Blog Post Series

1. Introduction
2. Initial evaluation and adaptation
3. Issues extending Polis and adjusting our goals
4. Creating (a) new frontend(s) for Polis (this article)
5. Current status and roadmap

4. Creating (a) new frontend(s) for Polis Why a new frontend was needed...

Our initial experiences of working with Polis, the effort required to implement more invasive changes and the desire of iterating changes more rapidly ultimately lead to the decision to create a new foundation for frontend development that would be independent of but compatible with the upstream project.

Our primary objective was thus not to develop another frontend but rather to make frontend development more flexible and to facilitate experimentation and rapid prototyping of different frontends by providing abstraction layers and building blocks.

This also implied developing a corresponding backend since the Polis backend is tightly coupled to the frontend and is neither intended to be used by third-party projects nor supporting cross-domain requests due to the expectation of being embedded as an iframe on third-party websites.

The long-term plan for achieving our objectives is to provide three abstraction layers for building frontends:

• a stable cross-domain HTTP API
• a low-level JavaScript library for interacting with the HTTP API
• a high-level library of WebComponents as a framework-neutral way of rapidly building frontends

The Particiapp Project

Under the umbrella of the Particiapp project we have so far developed two new components:

• the Particiapi server which provides the HTTP API
• the example frontend project which currently contains both the client library and an experimental example frontend built with it

Both the participation frontend and backend are fully compatible and require an existing Polis installation and can be run alongside the upstream frontend. More specifically, the administration frontend and common backend are required to administrate conversations and send out notifications and the statistics processing server is required for processing the voting results.

Particiapi server

For the backend the Python language and the Flask framework were chosen as a technological basis mainly due to developer mindshare, a large community and ecosystem and the smaller dependency chain and maintenance overhead compared to Node.js/npm. Instead of integrating specific identity providers we adopted the OpenID Connect standard as an abstraction layer for authentication which allows delegating authentication either to a self-hosted identity provider or a large number of existing external identity providers.

Particiapp Example Frontend

The experimental example frontend serves both as a test bed for the client library and as a tool for better understanding the needs of frontend designers. It also features a completely redesigned user interface and results visualization in line with our goals. Branded variants are currently used for evaluation and testing by the stakeholders.

In order to simplify evaluation, development, testing and deployment a Docker Compose configuration is made available which contains all necessary components for running Polis with our experimental example frontend. In addition, a development environment is provided which includes a preconfigured OpenID Connect identity provider (KeyCloak), SMTP-Server with web interface (MailDev), and a database frontend (PgAdmin). The new frontend can also be tested using our public demo server.

If you have been following our series, you have already migrated view modes—a prerequisite for field formatters. In this article, we are completing field-related migrations by importing formatter settings. This step builds on our previous work with view modes and field groups, bringing us closer to a functional Drupal 10 site.

Read more mauricio Thu, 10/03/2024 - 04:01

Learn how to make entity theming a breeze using the Pinto module. If you haven’t already, check out the first part of this series for an introduction to all things Pinto.

by adam.bramley / 3 October 2024

In our last post, we discussed Pinto concepts and how to use Theme objects to encapsulate theming logic in a central place for a component. Next, we’ll apply that knowledge to theming an entity. This will demonstrate the power of Pinto and how it will dramatically improve the velocity of delivering new components. 

One of the hardest things about theming Drupal is outputting markup that matches your design system. 

For example:

• Removing the “div soup” of Drupal fields
• Adding custom classes or attributes to field output
• Wrapping fields in custom tags (e.g. an h2)

While there are plenty of modules to alleviate this, it can often mean you have a mix of YAML configuration for markup, preprocess hooks, overridden templates, etc., to pull everything together. Pinto allows you to easily render an entity while reusing your frontender’s perfect template!

We need to cover a few more concepts and set things up to pull this all together. Once set up, new bundles or entity types can be added with ease.

We'll continue our Card component example from the previous post and cover:

1. Setting up a bundle class. In this example, we will implement it as a Block Content bundle
2. Using a custom entity view builder
3. Theming a Card block using Pinto

Bundle classes

In case you’re not aware, Drupal introduced the concept of Bundle classes almost three years ago. They essentially allow business logic for each bundle to be encapsulated in its own PHP class and benefit from regular PHP concepts such as code sharing via Traits, Interfaces, etc.

At PreviousNext, our go-to for implementing bundle classes is the BCA module, which allows you to define a class as a custom Bundle class via an attribute, removing the need for hook_entity_bundle_info_alter.

Our standard setup on projects is:

• An Interface per entity type (e.g MyProjectBlockContentInterface)
• An abstract base class per entity type (e.g. MyProjectBlockContentBase)
• A Bundle class per bundle
• Traits and interfaces for any shared fields/logic (e.g. BodyTrait for all bundles that have a Body field)

My preferred approach is to have a directory structure that matches the entity type located inside the project’s profile (e.g. src/Entity/BlockContent/Card.php. Feel free to set this up however you like. For example, some people may prefer to separate entity types into different modules.

Let’s set up our Card bundle class:

namespace Drupal\my_project_profile\Entity\BlockContent; use Drupal\bca\Attribute\Bundle; use Drupal\my_project_profile\Traits\DescriptionTrait; use Drupal\my_project_profile\Traits\ImageTrait; use Drupal\my_project_profile\Traits\TitleTrait; #[Bundle(entityType: self::ENTITY_TYPE_ID, bundle: self::BUNDLE)] final class Card extends MyProjectBlockContentBase { use TitleTrait; use DescriptionTrait; use ImageTrait; public const string BUNDLE = 'card'; }

Here we use the Bundle attribute provided by the BCA module to automatically register this class as the bundle class for the card bundle. We’re using constants here to make it easy to reference this machine name anywhere in our codebase. The ENTITY_TYPE_ID constant comes from the parent interface.

NOTE: I won’t go into too much detail about how the interfaces, base classes, and traits are set up. There are plenty of examples of how you might write these. Check out the change record for some basic examples! 

In our case, each trait is a getter/setter pair for each of our fields required to build our Card component: 

• Title - a plain text field
• Description - another plain text field
• Image - a Media reference field.

Custom entity view builder

EntityViewBuilders are PHP classes that contain logic on how to build (or render) an entity. Entity types can have custom EntityViewBuilders; for example BlockContent has its own defined in core. These are defined in the view_builder handler in an entity type's annotation and can also be overridden by using hook_entity_type_alter.

By default, the view builder class takes all of your configuration in an entity view display (i.e. field formatter settings, view modes, etc.) and renders it. We are using a custom view builder class to bypass all of that and simply return a render array via a Pinto object.

The function that drives this is getBuildDefaults so that’s all we need to override.

For this example, a custom view builder for the block content entity type can be as simple as:

namespace Drupal\my_project_profile\Handler; use Drupal\Core\Cache\CacheableMetadata; use Drupal\Core\Entity\EntityInterface; use Drupal\block_content\BlockContentViewBuilder; use Drupal\my_project_profile\Entity\Interface\BuildableEntityInterface; class MyProjectBlockContentViewBuilder extends BlockContentViewBuilder { /**  * {@inheritdoc}  */ public function getBuildDefaults(EntityInterface $entity, $view_mode) {   $build = parent::getBuildDefaults($entity, $view_mode);   if (!$entity instanceof BuildableEntityInterface || !$entity->shouldBuild($view_mode)) {     return $build;   }   $cache = CacheableMetadata::createFromRenderArray($build);   $build = $entity->build($view_mode);   $cache->merge(CacheableMetadata::createFromRenderArray($build))     ->applyTo($build);   return $build; } }

Here, we check for a custom BuildableEntityInterface and call a shouldBuild method. If either of those are FALSE then we fall back to Drupal’s default behaviour. Otherwise, we gather cacheable metadata from both the default build and the result of calling the build method, and then return the output. We will cover these in more detail shortly.

Now we just need an alter hook to wire things up:

use Drupal\my_project_profile\Handler\MyProjectBlockContentViewBuilder; /** * Implements hook_entity_type_alter(). */ function my_project_profile_entity_type_alter(array &$entity_types): void {  /** @var \Drupal\Core\Entity\ContentEntityType $blockContentDefinition */  $blockContentDefinition = $entity_types['block_content'];  // Override view builder class.  $blockContentDefinition->setViewBuilderClass(MyProjectBlockContentViewBuilder::class); }

Pro tip: Use the Hux module to do this in a Hooks class.

Now, any BlockContent bundle class that implements BuildableEntityInterface and returns TRUE from its shouldBuild method will completely bypass Drupal’s standard entity rendering and instead just return whatever we want from its build method.

BuildableEntityInterfacenamespace Drupal\my_project_profile\Entity\Interface; /** * Interface for entities which override the view builder. */ interface BuildableEntityInterface { /**  * Default method to build an entity.  */ public function build(string $viewMode): array; /**  * Determine if the entity should be built for the given view mode.  */ public function shouldBuild(string $viewMode): bool; }

This interface can be added to the Bundle class itself or the custom entity type interface we discussed earlier to keep all bundles consistent. This doesn’t just apply to the Block content entity type; you can use this for Paragraphs, Media, or your custom entity types. You’ll just need to override the view builder for each. 

It is generally not recommended to use this for Node since you’re more likely to get value out of something like Layout Builder for rendering nodes. Those layouts would then have block content added to them, which in turn will be rendered via this method.

Back to our Card example. It was extending a custom base class MyProjectBlockContentBase. That class may look something like this:

namespace Drupal\my_project_profile\Entity\BlockContent; use Drupal\block_content\BlockContentTypeInterface; use Drupal\block_content\Entity\BlockContent; abstract class MyProjectBlockContentBase extends BlockContent implements MyProjectBlockContentInterface { /**  * {@inheritdoc}  */ public function shouldBuild(string $viewMode): bool {   return TRUE; } }

Our base class extends core’s BlockContent class and implements our custom interface.

That custom interface can then extend BuildableEntityInterface.

The shouldBuild method is an optional implementation detail, but it is nice if you have multiple view modes for a bundle, which need to have differing logic. For example, you might have a media_library view mode that you want to continue to use Drupal’s standard rendering.

Now, all we need to do is implement the build method on our BlockContent bundle classes.

Let’s look at the Card example:

use Drupal\my_project_ds\ThemeObject\Card as PintoCard; final class Card extends MyProjectBlockContentBase {    // Trimmed for easy reading.  /**   * {@inheritdoc}   */  public function build(string $viewMode): array {    return PintoCard::createFromCardBlock($this)();  } }

Here, we’re simply returning the render array that results from invoking our Card Pinto object (aliased as PintoCard via the use statement).

We have also introduced a factory method createFromCardBlock on the Pinto theme object, which takes the entity and injects its data into the object.

This is what the fully implemented Pinto object would look like

namespace Drupal\my_project_ds\ThemeObject; use Drupal\Core\Cache\CacheableDependencyInterface; use Drupal\my_project_profile\Entity\BlockContent\Card as CardBlock; use Drupal\my_project_ds\MyProjectDs\MyProjectObjectTrait; use Pinto\Attribute\ThemeDefinition; #[ThemeDefinition([ 'variables' => [   'title' => '',   'description' => '',   'image' => '', ], ])] final class Card implements CacheableDependencyInterface { use MyProjectObjectTrait; private function __construct(   private readonly string $title,   private readonly array $image,   private readonly ?string $description, ) {} public static function createFromCardBlock(CardBlock $card): static {   return new static(     $card->getTitle(),     $card->getImage(),     $card->getDescription(),   ); } protected function build(mixed $build): mixed {   return $build + [     '#title' => $this->title,     '#description' => $this->description,     '#image' => $this->image,   ]; } }

The build and constructor methods were covered in our previous Pinto post. All that’s new here is the createFromCardBlock method, where we use the getters from the bundle class traits to inject the entity’s data into the constructor.

We also briefly mentioned cacheable metadata in our last post. Since our Pinto object implements CacheableDependencyInterface, we can add that metadata directly to the theme object. For example, you should enhance the bundle class’ build method to add the Image media entity as a cacheable dependency. That way if the media entity is updated, the Card output is invalidated.

/** * {@inheritdoc} */ public function build(string $viewMode): array { $build = PintoCard::createFromCardBlock($this); $image = $this->image->entity; if ($image) {    $build->addCacheableDependency($image);  } return $build(); }

Now, we have end-to-end rendering of a Drupal entity using Pinto Theme objects to render templates defined in a Storybook design system.

New bundles are simple to implement. All that’s needed is to click together the fields in the UI to build the content model, add the new Theme object, and wire that together with a bundle class.

I can’t overstate how much this has sped up our backend development. My latest project utilised Pinto from the very beginning, and it has made theming the entire site extremely fast and even… fun! 😀

Recently, a public dispute has emerged between WordPress co-founder Matt Mullenweg and hosting company WP Engine. Matt has accused WP Engine of misleading users through its branding and profiting from WordPress without adequately contributing back to the project.

As the Founder and Project Lead of Drupal, another major open source Content Management System (CMS), I hesitated to weigh in on this debate, as this could be perceived as opportunistic. In the end, I decided to share my perspective because this conflict affects the broader open source community.

I've known Matt Mullenweg since the early days, and we've grown both our open source projects and companies alongside each other. With our shared interests and backgrounds, I consider Matt a good friend and can relate uniquely to him. Equally valuable to me are my relationships with WP Engine's leadership, including CEO Heather Brunner and Founder Jason Cohen, both of whom I've met several times. I have deep admiration for what they’ve achieved with WP Engine.

Although this post was prompted by the controversy between Automattic and WP Engine, it is not about them. I don't have insight into their respective contributions to WordPress, and I'm not here to judge. I've made an effort to keep this post as neutral as possible.

Instead, this post is about two key challenges that many open source projects face:

1. The imbalance between major contributors and those who contribute minimally, and how this harms open source communities.
2. The lack of an environment that supports the fair coexistence of open source businesses.

These issues could discourage entrepreneurs from starting open source businesses, which could harm the future of open source. My goal is to spark a constructive dialogue on creating a more equitable and sustainable open source ecosystem. By solving these challenges, we can build a stronger future for open source.

This post explores the "Maker-Taker problem" in open source, using Drupal's contribution credit system as a model for fairly incentivizing and recognizing contributors. It suggests how WordPress and other open source projects could benefit from adopting a similar system. While this is unsolicited advice, I believe this approach could help the WordPress community heal, rebuild trust, and advance open source productively for everyone.

The Maker-Taker problem

At the heart of this issue is the Maker-Taker problem, where creators of open source software ("Makers") see their work being used by others, often service providers, who profit from it without contributing back in a meaningful or fair way ("Takers").

Five years ago, I wrote a blog post called Balancing Makers and Takers to scale and sustain Open Source, where I defined these concepts:

The difference between Makers and Takers is not always 100% clear, but as a rule of thumb, Makers directly invest in growing both their business and the open source project. Takers are solely focused on growing their business and let others take care of the open source project they rely on.

In that post, I also explain how Takers can harm open source projects. By not contributing back meaningfully, Takers gain an unfair advantage over Makers who support the open source project. This can discourage Makers from keeping their level of contribution up, as they need to divert resources to stay competitive, which can ultimately hurt the health and growth of the project:

Takers harm open source projects. An aggressive Taker can induce Makers to behave in a more selfish manner and reduce or stop their contributions to open source altogether. Takers can turn Makers into Takers.

Solving the Maker-Taker challenge is one of the biggest remaining hurdles in open source. Successfully addressing this could lead to the creation of tens of thousands of new open source businesses while also improving the sustainability, growth, and competitiveness of open source – making a positive impact on the world.

Drupal's approach: the Contribution Credit System

In Drupal, we've adopted a positive approach to encourage organizations to become Makers rather than relying on punitive measures. Our approach stems from a key insight, also explained in my Makers and Takers blog post: customers are a "common good" for an open source project, not a "public good".

Since a customer can choose only one service provider, that choice directly impacts the health of the open source project. When a customer selects a Maker, part of their revenue is reinvested into the project. However, if they choose a Taker, the project sees little to no benefit. This means that open source projects grow faster when commercial work flows to Makers and away from Takers.

For this reason, it's crucial for an open source community to:

1. Clearly identify the Makers and Takers within their ecosystem
2. Actively support and promote their Makers
3. Educate end users about the importance of choosing Makers

To address these needs and solve the Maker-Taker problem in Drupal, I proposed a contribution credit system 10 years ago. The concept was straightforward: incentivize organizations to contribute to Drupal by giving them tangible recognition for their efforts.

We've since implemented this system in partnership with the Drupal Association, our non-profit organization. The Drupal Association transparently tracks contributions from both individuals and organizations. Each contribution earns credits, and the more you contribute, the more visibility you gain on Drupal.org (visited by millions monthly) and at events like DrupalCon (attended by thousands). You can earn credits by contributing code, submitting case studies, organizing events, writing documentation, financially supporting the Drupal Association, and more.

A screenshot of an issue comment on Drupal.org. You can see that jamadar worked on this patch as a volunteer, but also as part of his day job working for TATA Consultancy Services on behalf of their customer, Pfizer.

Drupal's credit system is unique and groundbreaking within the Open Source community. The Drupal contribution credit system serves two key purposes: it helps us identify who our Makers and Takers are, and it allows us to guide end users towards doing business with our Makers.

Here is how we accomplish this:

• Certain benefits, like event sponsorships or advertising on Drupal.org, are reserved for organizations with a minimum number of credits.
• The Drupal marketplace only lists Makers, ranking them by their contributions.
• Top contributors appear first, and organizations that stop contributing gradually drop in rankings or are removed.
• We encourage end users to require open source contributions from their vendors. Drupal users like Pfizer and the State of Georgia only allow Makers to apply in their vendor selection process.

A slide from my recent DrupalCon Barcelona State of Drupal keynote showcasing key contributors to Drupal Starshot. This slide showcases how we recognize and celebrate Makers in our community, encouraging active participation in the project. Governance and fairness

Fairness in the open source credit system requires oversight by an independent, neutral party. This entity must objectively assess contributions to maintain equity.

In the Drupal ecosystem, the Drupal Association fulfills this crucial role. The Drupal Association operates independently, free from control by any single company within the Drupal ecosystem. Some of the Drupal Association's responsibilities include:

1. Organizing DrupalCons
2. Managing Drupal.org
3. Overseeing the contribution tracking and credit system

It's important to note that while I serve on the Drupal Association's Board, I am just one of 12 members and have not held the Chair position for several years. My company, Acquia, receives no preferential treatment in the credit system; the visibility of any organization, including Acquia, is solely determined by its contributions over the preceding twelve months. This structure ensures fairness and encourages active participation from all members of the Drupal community.

Drupal's credit system certainly isn't perfect. It is hard to accurately track and fairly value diverse contributions like code, documentation, mentorship, marketing, event organization, etc. Some organizations have tried to game the system, while others question whether the cost-benefit is worthwhile.

As a result, Drupal's credit system has evolved significantly since I first proposed it ten years ago. The Drupal Association continually works to improve the system, aiming for a credit structure that genuinely drives positive behavior.

Recommendations for WordPress

WordPress has already taken steps to address the Maker-Taker challenge through initiatives like the Five for the Future program, which encourages organizations to contribute 5% of their resources to WordPress development.

Building on this foundation, I believe WordPress could benefit from adopting a contribution credit system similar to Drupal's. This system would likely require the following steps to be taken:

1. Expanding the current governance model to be more distributed.
2. Providing clear definitions of Makers and Takers within the ecosystem.
3. Implementing a fair and objective system for tracking and valuing various types of contributions.
4. Implementing a structured system of rewards for Makers who meet specific contribution thresholds, such as priority placement in the WordPress marketplace, increased visibility on WordPress.org, opportunities to exhibit at WordPress events, or access to key services.

This approach addresses both key challenges highlighted in the introduction: it balances contributions by incentivizing major involvement, and it creates an environment where open source businesses of all sizes can compete fairly based on their contributions to the community.

Conclusion

Addressing the Maker-Taker challenge is essential for the long-term sustainability of open source projects. Drupal's approach may provide a constructive solution not just for WordPress, but for other communities facing similar issues.

By transparently rewarding contributions and fostering collaboration, we can build healthier open source ecosystems. A credit system can help make open source more sustainable and fair, driving growth, competitiveness, and potentially creating thousands of new open source businesses.

As Drupal continues to improve its credit system, we understand that no solution is perfect. We're eager to learn from the successes and challenges of other open source projects and are open to ideas and collaboration.

A little over a month after v.0.0.9, we have a Release Candidate version of the Open Source AI Definition. This was reached with lots of community feedback: 5 town hall meetings, several comments on the forum and on the draft, and in person conversations at events in Austria, China, India, Ghana, and Argentina.

There are three relevant changes to the part of the definition pertaining to the “preferred form to make modifications to a machine learning system.”

The feature that will draw most attention is the new language of Data Information. It clarifies that all the training data needs to be shared and disclosed. The updated text comes from many conversations with several individuals who engaged passionately with the design process, on the forum, in person and on hackmd. These conversations helped describe four types of data: open, public, obtainable and unshareable data, well described in the FAQ. The legal requirements are different for each. All are required to be shared in the form that the law allows them to be shared. 

Two new features are equally important. RC1 clarifies that Code must be complete, enough for downstream recipients to understand how the training was done. This was done to reinforce the importance of the training, both for transparency, security and other practical reasons. Training is where innovation is happening at the moment and that’s why you don’t see corporations releasing their training and data processing code. We believe, given the current status of knowledge and practice, that this is required to meaningfully fork (study and modify) AI systems.

Last, there is new text that is meant to explicitly acknowledge that it is admissible to require copyleft-like terms for any of the Code, Data Information and Parameters, individually or as bundled combinations. A demonstrative scenario is a consortium owning rights to training code and a dataset deciding to distribute the bundle code+data with legal terms that tie the two together, with copyleft-like provisions. This sort of legal document doesn’t exist yet but the scenario is plausible enough that it deserves consideration. This is another area that OSI will monitor carefully as we start reviewing these legal terms with the community.

A note about science and reproducibility

The aim of Open Source is not and has never been to enable reproducible software. The same is true for Open Source AI: reproducibility of AI science is not the objective. Open Source’s role is merely not to be an impediment to reproducibility. In other words, one can always add more requirements on top of Open Source, just like the Reproducible Builds effort does.

Open Source means giving anyone the ability to meaningfully “fork” (study and modify) a system, without requiring additional permissions, to make it more useful for themselves and also for everyone. This is why OSD #2 requires that the “source code” must be provided in the preferred form for making modifications. This way everyone has the same rights and ability to improve the system as the original developers, starting a virtuous cycle of innovation. Forking in the machine learning context has the same meaning as with software: having the ability and the rights to build a system that behaves differently than its original status. Things that a fork may achieve are: fixing security issues, improving behavior, removing bias. All these are possible thanks to the requirements of the Open Source AI Definition.

What’s coming next

With the release candidate cycle starting today, the drafting process will shift focus: no new features, only bug fixes. We’ll watch for new issues raised, watching for major flaws that may require significant rewrites to the text. The main focus will be on the accompanying documentation, the Checklist and the FAQ. We also realized that in our zeal to solve the problem of data that needs to be provided but cannot be supplied by the model owner for good reasons, we had failed to make clear the basic requirement that “if you can share the data you must.” We have already made adjustments in RC1 and will be seeking views on how to better express this in an RC2. 

In the next weeks until the 1.0 release of October 28, we’ll focus on:

• Getting more endorsers to the Definition
• Continuing to collect feedback on hackmd and forum, focusing on new, unseen-before concerns
• Preparing the artifacts necessary for the launch at All Things Open
• Iterating on the Checklist and FAQ, preparing them for deployment.

Link to the Open Source AI Definition Release Candidate 1

This blog has been re-posted and edited with permission from Dries Buytaert's blog.

DrupalCon Barcelona 2024 Driesnote presentation

Approximately 1,100 Drupal enthusiasts gathered in Barcelona, Spain, last week for DrupalCon Europe. As per tradition, I delivered my State of Drupal keynote, often referred to as the "DriesNote".

If you missed it, you can watch the video or download my slides (177 MB).

In my keynote, I gave an update on Drupal Starshot, an ambitious initiative we launched at DrupalCon Portland 2024. Originally called Drupal Starshot, inspired by President Kennedy's Moonshot challenge, the product is now officially named Drupal CMS.

The goal of Drupal CMS is to set the standard for no-code website building. It will allow non-technical users, like marketers, content creators, and site builders, to create digital experiences with ease, without compromising on the power and flexibility that Drupal is known for.

A four-month progress report

A preview of Drupal.org's front page with the updated Drupal brand and content.

While Kennedy gave NASA eight years, I set a goal to deliver the first version of Drupal CMS in just eight months. It's been four months since DrupalCon Portland, which means we're halfway through.

So in my keynote, I shared our progress and gave a 35-minute demo of what we've built so far. The demo highlights how a fictional marketer, Sarah, can build a powerful website in just hours with minimal help from a developer. Along her journey, I showcased the following key innovations:

1. A new brand for a new market: A brand refresh of Drupal.org, designed to appeal to both marketers and developers. The first pages are ready and available for preview at new.drupal.org, with more pages launching in the coming months.
2. A trial experience: A trial experience that lets you try Drupal CMS with a single click, eliminating long-standing adoption barriers for new users. Built with WebAssembly, it runs entirely in the browser – no servers to install or manage.
3. An improved installer: An installer that lets users install recipes – pre-built features that combine modules, configuration, and default content for common website needs. Recipes bundle years of expertise into repeatable, shareable solutions.
4. Events recipe: A simple events website that used to take an experienced developer a day to build can now be created in just a few clicks by non-developers.
5. Project Browser support for recipes: Users can now browse the Drupal CMS recipes in the Project Browser, and install them in seconds.
6. First page of documentation: New documentation created specifically for end users. Clear, effective documentation is key to Drupal CMS's success, so we began by writing a single page as a model for the quality and style we aim to achieve.
7. AI for site building: AI agents capable of creating content types, configuring fields, building Views, forms, and more. These agents will transform how people build and manage websites with Drupal.
8. Responsible AI policy: To ensure responsible AI development, we've created a Responsible AI policy. I'll share more details in an upcoming blog, but the policy focuses on four key principles: human-in-the-loop, transparency, swappable large language models (LLMs), and clear guidance.
9. SEO Recipe: Combines and configures all the essential Drupal modules to optimize a Drupal site for search engines.
10. 14 recipes in development: In addition to the Events and SEO recipes, 12 more are in development with the help of our Drupal Certified Partners. Each Drupal CMS recipe addresses a common marketing use case outlined in our product strategy. We showcased both the process and progress during the Initiative Lead Keynote for some of the tracks. After DrupalCon, we'll begin developing even more recipes and invite additional contributors to join the effort.
11. AI-assisted content migration: AI will crawl your source website and handle complex tasks like mapping unstructured HTML to structured Drupal content types in your destination site, making migrations faster and easier. This could be a game-changer for website migrations.
12. Experience Builder: An early preview of a brand new, out-of-the-box tool for content creators and designers, offering layout design, page building, basic theming and content editing tools. This is the first time I've showcased our progress on stage at a DrupalCon.
13. Future-proof admin UI with React: Our strategy for modernizing Drupal's backend UI with React.
14. The "Adopt-a-Document" initiative: A strategy and funding model for creating comprehensive documentation for Drupal CMS. If successful, I'm hopeful we can expand this model to other areas of Drupal. For more details, please read the announcement on drupal.org.
15. Global Documentation Lead: The Drupal Association's commitment to hire a dedicated Documentation Lead, responsible for managing all aspects of Drupal's documentation, beyond just Drupal CMS.

The feedback on my presentation has been incredible, both online and in-person. The room was buzzing with energy and positivity! I highly recommend watching the recording.

Attendees were especially excited about the AI capabilities, Experience Builder, and recipes. I share their enthusiasm as these capabilities are transformative for Drupal.

Many of these features are designed with non-developers in mind. Our goal is to broaden Drupal's reach beyond its traditional user base and reach more people than ever before.

Release schedule

Our launch plan targets Drupal CMS's release on Drupal's upcoming birthday: January 15, 2025. It's also just a couple of weeks after the Drupal 7 End of Life, marking the end of one era and the beginning of another.

The next milestone is DrupalCon Singapore, taking place from December 9–11, 2024, less than 3 months away. We hope to have a release candidate ready by then.

Now that we're back from DrupalCon and have key milestone dates set, there is a lot to coordinate and plan in the coming weeks, so stay tuned for updates.

Call for contribution

Ambitious? Yes. But achievable if we work together. That's why I'm calling on all of you to get involved with Drupal CMS. Whether it's building recipes, enhancing the Experience Builder, creating AI agents, writing tests, improving documentation, or conducting usability testing – there are countless ways to contribute and make a difference. If you're ready to get involved, visit https://drupal.org/starshot to learn how to get started.

Thank you

This effort has involved so many people that I can't name them all, but I want to give a huge thank you to the Drupal CMS Leadership Team, who I've been working with closely every week: Cristina Chumillas (Lullabot), Gábor Hojtsy (Acquia), Lenny Moskalyk (Drupal Association), Pamela Barone (Technocrat), Suzanne Dergacheva (Evolving Web), and Tim Plunkett (Acquia).

A special shoutout goes to the demo team we assembled for my presentation: Adam Hoenich (Acquia), Amber Matz (Drupalize.me), Ash Sullivan (Acquia), Jamie Abrahams (FreelyGive), Jim Birch (Kanopi), Joe Shindelar (Drupalize.me), John Doyle (Digital Polygon), Lauri Timmanee (Acquia), Marcus Johansson (FreelyGive), Martin Anderson-Clutz (Acquia), Matt Glaman (Acquia), Matthew Grasmick (Acquia), Michael Donovan (Acquia), Tiffany Farriss (Palantir.net), and Tim Lehnen (Drupal Association).

I also want to thank the Drupal CMS track leads and contributors for their development work. Additionally, I'd like to recognize the Drupal Core Committers, Drupal Association staff, Drupal Association Board of Directors, and Certified Drupal Partners for continued support and leadership. There are so many people and organizations whose contributions deserve recognition that I can't list everyone individually, partly to avoid the risk of overlooking anyone. Please know your efforts are deeply appreciated.

Lastly, thank you to everyone who helped make DrupalCon Barcelona a success. It was excellent!

Fourteen new GNU releases in the last month (as of September 30, 2024):

Fourteen new GNU releases in the last month (as of September 30, 2024):

Drupal's form API has been brilliant for many years. Still, recently I found myself wondering why I needed to build a configuration form if I already had a schema for my config. Defining a schema facilitates API-first validation (including some pretty smart constraints), specific typing (e.g. actual booleans or integers instead of '0' or '1' strings), and even translation in Drupal. 

That last part got me thinking; if Drupal automatically provides translation forms for typed configuration, why must I build a form? I started diving into the code and found config_translation_config_schema_info_alter() which maps certain config data types to element classes. The ConfigTranslationFormBase::buildForm() class fetches the schema for each config property from the 'config.typed' service (\Drupal\Core\Config\TypedConfigManager) before building the appropriate elements. So Drupal core automatically provides this translation form - notice the long textarea for the 'body' property:

Screenshot of a config translation form from Drupal core

I had built a block plugin that needed some regex-based validation on a couple of its configuration properties. Validation constraints seemed like a natural fit for these, as an inherent part of the property definitions, rather than just on the form level. Drupal has had good support for validation constraints on configuration since version 10.2. This allows forms to be simpler, and config to be fully validatable, even outside the context of a form (e.g. for setting via APIs or config synchronisation). So I defined my config schema like this:

block.settings.mymodule_myblock: type: block_settings label: 'MyBlock block settings' mapping: svcid: type: string label: 'Service ID' constraints: Regex: pattern: '/^[a-zA-Z0-9_\-]+$/' message: "The %value can only contain simple letters, numbers, underscores or hyphens." default: 'abcde' locked: true envid: type: string label: 'Environment ID' constraints: Regex: pattern: '/^[a-zA-Z0-9_\-]+$/' message: "The %value can only contain simple letters, numbers, underscores or hyphens." default: 'x-j9WsahRe_1An51DhErab-C'

Then I set myself the challenge of building a configuration form 'automatically' from this schema - without using core's config_translation module at all, as this was for a monolingual site. 

I only had two string properties, which meant two textfields, but I wrote the code to use form elements that could be appropriate for other types of property that might get added in the future. The #title of each element could come directly from each label in the schema. (Why do we usually set these in both the schema and form element?!) I added custom default and locked properties to the schema to help encapsulate everything I considered 'inherent' to each part of the config in one place. This meant the configuration form for my block could be fairly simple:

public function blockForm($form, FormStateInterface $form_state) { // Each config property will be returned with its schema from $this->getConfigurables(). foreach ($this->getConfigurables() as $key => $schema_info) { $form[$key] = [ '#type' => match ($schema_info['type']) { 'string', 'label' => 'textfield', 'text' => 'textarea', 'boolean' => 'checkbox', 'integer', 'float' => 'number', 'email' => 'email', }, '#title' => $schema_info['label'], '#default_value' => $this->configuration[$key], '#required' => empty($schema_info['nullable']), '#disabled' => $schema_info['locked'] ?? FALSE, ]; } return $form; }

Hopefully that gives an idea of how simple a config form could be - and this could really be reduced further by refactoring it into a generic trait. The code in core's config_translation module for mapping the type of each property to an element type could be much more useful than the fairly naïve match statement above, if it was refactored out to be available even to monolingual sites.

You can explore my full code at https://gist.github.com/anotherjames/bcb7ba55ec56359240b26d322fe2f5a5. That includes the getConfigurables() method which pulls the schema from the TypedConfigManager.

You'll see that I went a little further and picked up the regex constraints for each config property, for use in #pattern form API properties. This provides quick feedback to admins about what characters are allowed using the HTML5 pattern attribute:

Not all configuration constraints could be built into the form level. It's arguable that since the Regex constraint and HTML pattern attribute support slightly different regular expression features, this particular one shouldn't be included in a generic trait. Then again, the Choice constraint could be especially useful to include, as it could be used to populate #options for select, radios, or checkboxes elements. We've started using backed Enums with labels for fixed sets of options. Can we wire those up to choice constraints together, I wonder?

Whereas my example was for a configurable plugin's form (which I don't believe can use #config_target), Joachim Noreiko (joachim) has submitted a feature request to Drupal core for forms extending ConfigFormBase to get automatically built from schema. This idea of generating form elements from config schema is still in its infancy, so its limits and benefits need to be explored further. Please let us know in a comment here, or in Joachim's feature request, if you have done anything similar, or have ideas or concerns to point out!

When working with strings in Python, you may need to interpolate values into your string and format these values to create new strings dynamically. In modern Python, you have f-strings and the .format() method to approach the tasks of interpolating and formatting strings.

In this tutorial, you’ll learn how to:

• Use f-strings and the .format() method for string interpolation
• Format the interpolated values using replacement fields
• Create custom format specifiers to format your strings

To get the most out of this tutorial, you should know the basics of Python programming and the string data type.

Get Your Code: Click here to download the free sample code that shows you how to use modern string formatting tools in Python.

Take the Quiz: Test your knowledge with our interactive “A Guide to Modern Python String Formatting Tools” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

A Guide to Modern Python String Formatting Tools

You can take this quiz to test your understanding of modern tools for string formatting in Python. These tools include f-strings and the .format() method.

Getting to Know String Interpolation and Formatting in Python

Python has developed different string interpolation and formatting tools over the years. If you’re getting started with Python and looking for a quick way to format your strings, then you should use Python’s f-strings.

Note: To learn more about string interpolation, check out the String Interpolation in Python: Exploring Available Tools tutorial.

If you need to work with older versions of Python or legacy code, then it’s a good idea to learn about the other formatting tools, such as the .format() method.

In this tutorial, you’ll learn how to format your strings using f-strings and the .format() method. You’ll start with f-strings to kick things off, which are quite popular in modern Python code.

Using F-Strings for String Interpolation

Python has a string formatting tool called f-strings, which stands for formatted string literals. F-strings are string literals that you can create by prepending an f or F to the literal. They allow you to do string interpolation and formatting by inserting variables or expressions directly into the literal.

Creating F-String Literals

Here you’ll take a look at how you can create an f-string by prepending the string literal with an f or F:

Python 👇 >>> f"Hello, Pythonista!" 'Hello, Pythonista!' 👇 >>> F"Hello, Pythonista!" 'Hello, Pythonista!' Copied!

Using either f or F has the same effect. However, it’s a more common practice to use a lowercase f to create f-strings.

Just like with regular string literals, you can use single, double, or triple quotes to define an f-string:

Python 👇 >>> f'Single-line f-string with single quotes' 'Single-line f-string with single quotes' 👇 >>> f"Single-line f-string with double quotes" 'Single-line f-string with single quotes' 👇 >>> f'''Multiline triple-quoted f-string ... with single quotes''' 'Multiline triple-quoted f-string\nwith single quotes' 👇 >>> f"""Multiline triple-quoted f-string ... with double quotes""" 'Multiline triple-quoted f-string\nwith double quotes' Copied!

Up to this point, your f-strings look pretty much the same as regular strings. However, if you create f-strings like those in the examples above, you’ll get complaints from your code linter if you have one.

The remarkable feature of f-strings is that you can embed Python variables or expressions directly inside them. To insert the variable or expression, you must use a replacement field, which you create using a pair of curly braces.

Interpolating Variables Into F-Strings

The variable that you insert in a replacement field is evaluated and converted to its string representation. The result is interpolated into the original string at the replacement field’s location:

Python >>> site = "Real Python" 👇 >>> f"Welcome to {site}!" 'Welcome to Real Python!' Copied!

In this example, you’ve interpolated the site variable into your string. Note that Python treats anything outside the curly braces as a regular string.

Read the full article at https://realpython.com/python-formatted-output/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

At DrupalCon Barcelona 2024, Elise West of SystemSeed presented a session on Human-Centered Design (HCD), explaining its growing relevance in Drupal projects. The session highlighted HCD’s role in aligning development with user needs, making it essential for project managers, developers, and product leads. More insights from SystemSeed's experience will follow.

As I upload photos to various services, I generally resize them as required based on portrait or landscape mode. I used to do that for all the photos in a directory and then pick which ones to use. But, I wanted to do it selectively, open the photos in Gnome Nautilus (Files) application and right click and resize the ones I want.

This week I noticed that I can do that with scripts. Those can be in any given language, the selected files will be passed as command line arguments, or full paths will be there in an environment variable NAUTILUS_SCRIPT_SELECTED_FILE_PATHS joined via newline character.

To add any script to the right click menu, you just need to place them in ~/.local/share/nautilus/scripts/ directory. They will show up in the right click menu for scripts.

Below is the script I am using to reduce image sizes:

#!/usr/bin/env python3 import os import sys import subprocess from PIL import Image # paths = os.environ.get("NAUTILUS_SCRIPT_SELECTED_FILE_PATHS", "").split("\n") paths = sys.argv[1:] for fpath in paths: if fpath.endswith(".jpg") or fpath.endswith(".jpeg"): # Assume that is a photo try: img = Image.open(fpath) # basename = os.path.basename(fpath) basename = fpath name, extension = os.path.splitext(basename) new_name = f"{name}_ac{extension}" w, h = img.size # If w > h then it is a landscape photo if w > h: subprocess.check_call(["/usr/bin/magick", basename, "-resize", "1024x686", new_name]) else: # It is a portrait photo subprocess.check_call(["/usr/bin/magick", basename, "-resize", "686x1024", new_name]) except: # Don't care, continue pass

You can see it in action (I selected the photos and right clicked, but the recording missed that part):