Feeds

# # Debian / libvirt # # By default, qemu guests network interfaces are created with the # trustGuestRxFilters attribute set to no. This setting blocks # neighbor solicitation and other ipv6 multicast traffic. To enable # neighbor solicitations, modify your interface definition in your # guest definition XML. # For example if you have a guest named bird0 with an interface # definition that looks like this: $ sudo virsh dumpxml bird0 | grep -A7 "type='direct'" <interface type='direct' trustGuestRxFilters='no'> <mac address='52:54:00:AA:BB:CC'/> <source dev='ens2f0np0' mode='bridge'/> <target dev='macvtap3'/> <model type='virtio'/> <alias name='net4'/> <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/> </interface> # Then use 'virsh edit bird0' change the attribute in the <interface> # element to have the value 'yes' instead: <interface type='direct' trustGuestRxFilters='yes'> <mac address='52:54:00:AA:BB:CC'/> <source dev='ens2f0np0' mode='bridge'/> <target dev='macvtap3'/> <model type='virtio'/> <alias name='net4'/> <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/> </interface> # If the trustGuestRxFilters attribute is missing, it defaults to no, # so be sure that the attribute exists and that the attribute has a # value of 'yes' Tweet

My Monday started with a pleasant surprise that is only possible in cross-timezone collaborations: Ted “tedbow” 1 had made a huge leap forward on #3450957: Prevent modules from being uninstalled if they provide field types used in an Experience Builder (XB) field, where he’s working on the first aspect where XB’s JSON blobs in the database must be queried.
Drupal 11 requires versions of databases that support JSON querying. Ted rather quickly discovered that JSON querying support is not consistent across different databases … but fortunately Brad “bradjones1 is working on #3343634 to add explicit JSON support to Drupal’s database abstraction layer, which is one of the endeavors sponsored by Pitch-Burgh! So: database nerds, unite!

(That means reviewing Brad’s work in that issue is yet another way of contributing to XB — in addition to helping Single Directory Components (SDC) move forward.)

To ensure we are aware of what works on which database from day one, I updated XB’s CI pipeline to run the test suite against MariaDB+MySQL+PostgreSQL+SQLite.

Missed a prior week? See all posts tagged Experience Builder.

Goal: make it possible to follow high-level progress by reading ~5 minutes/week. I hope this empowers more people to contribute when their unique skills can best be put to use!

For more detail, join the #experience-builder Slack channel. Check out the pinned items at the top!

Later on Monday, Ben “bnjmnm” and Jesse landed an MR that starts to connect the UI that you saw two weeks ago to the back end, and brought it alive: there’s now a tree view to provide a sense of place on the canvas, and much more:

Try it yourself locally if you like, but there’s not much you can do yet.
Install the 0.x branch — the “Experience Builder PoC” toolbar item takes you there!

A day later, the first set of unit tests landed: those for StructuredDataPropExpressions (these express where to retrieve values mapped for SDC props).

On Thursday, Harumi “hooromoo”’s MR to provide the foundations for undo/redo support landed, after the most enthusiastic approval I think I’ve ever seen Lee “larowlan” give! :D

That same day, his colleague Griffyn “griffynh” ran the second asynchronous XB meeting in Drupal Slack (archive on d.o: #3453097), this time with 21 participants of the ~250 people in the #experience-builder channel (~100 more compared to 2 weeks ago).
The most heavily debated topics were the data model/persistence layer and Cypress vs vitest for tests.

By Friday, I managed to finish up the work that I already shared a video of a week prior (see week 3), where a PropSource (which itself uses a PropExpression) can be chosen for each component prop and edited using Drupal’s existing Field Widgets. Crucially, it now includes a fully annotated end-to-end test that verifies the demo in detail:

• an article using an XB field can be created
• expected data is stored for the XB field, with a handful of placed components, and all prop source types: static, dynamic and adapter
• each of these prop sources can be evaluated/resolved to a concrete value to pass into the corresponding SDC prop
• each of the placed components renders into the expected HTML

In other words: it’s making the concrete implementation of #3440578: JSON-based data storage proposal for component-based page building tangible and debuggable in a way that required reading dozens of files previously! 2

Thanks to Ted & Lauri for reviewing this!

1. Forever infamous for his shining sunglasses moment on stage during DrupalCon Pittsburgh — captured it in exquisite detail by Mike “mherchel”! ↩︎

2. Hence it’ll serve as a canary for the data model/storage back-end infrastructure in the short–medium term. ↩︎

• Drupal
• Acquia
• Experience Builder

• The XB UI now gained a tree view in the left sidebar that provides a sense of place, and is kept in sync with the canvas.

One of the main data structures you learn about early in your Python learning journey is the dictionary. Dictionaries are the most common and well-known of Python’s mappings. However, there are other mappings in Python’s standard library and third-party modules. Mappings share common characteristics, and understanding these shared traits will help you use them more effectively.

In this tutorial, you’ll learn about:

• Basic characteristics of a mapping
• Operations that are common to most mappings
• Abstract base classes Mapping and MutableMapping
• User-defined mutable and immutable mappings and how to create them

This tutorial assumes that you’re familiar with Python’s built-in data types, especially dictionaries, and with the basics of object-oriented programming.

Get Your Code: Click here to download the free sample code that you’ll use to learn about mappings in Python.

Take the Quiz: Test your knowledge with our interactive “Python Mappings” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

Python Mappings

In this quiz, you'll test your understanding of the basic characteristics and operations of Python mappings. By working through this quiz, you'll revisit the key concepts and techniques of creating a custom mapping.

Understanding the Main Characteristics of Python Mappings

A mapping is a collection that allows you to look up a key and retrieve its value. The keys in mappings can be objects of a broad range of types. However, in most mappings, there are object types that can’t be used as keys, as you’ll learn later in this tutorial.

The previous paragraph described mappings as collections. A collection is an iterable container that has a defined size. However, mappings also have additional features. You’ll explore each of these mapping characteristics with examples from Python’s main mapping types.

The feature that’s most characteristic of mappings is the ability to retrieve a value using a key. You can use a dictionary to demonstrate this operation:

Python >>> points = { ... "Denise": 3, ... "Igor": 2, ... "Sarah": 3, ... "Trevor": 1, ... } >>> points["Sarah"] 3 >>> points["Matt"] Traceback (most recent call last): ... KeyError: 'Matt' Copied!

The dictionary points contains four items, each with a key and a value. You can use the key within the square brackets to fetch the value associated with that key. However, if the key doesn’t exist in the dictionary, the code raises a KeyError.

You can use one of the mappings in the standard-library collections module to assign a default value for keys that aren’t present in the collection. The defaultdict type includes a callable that’s called each time you try to access a key that doesn’t exist. If you want the default value to be zero, you can use a lambda function that returns 0 as the first argument in defaultdict:

Python >>> from collections import defaultdict >>> points_default = defaultdict( ... lambda: 0, ... points, ... ) >>> points_default defaultdict(<function <lambda> at 0x104a95da0>, {'Denise': 3, 'Igor': 2, 'Sarah': 3, 'Trevor': 1}) >>> points_default["Sarah"] 3 >>> points_default["Matt"] 0 >>> points_default defaultdict(<function <lambda> at 0x103e6c700>, {'Denise': 3, 'Igor': 2, 'Sarah': 3, 'Trevor': 1, 'Matt': 0}) Copied!

The defaultdict constructor has two arguments in this example. The first argument is the callable that’s used when a default value is needed. The second argument is the dictionary you created earlier. You can use any valid argument when you call dict() as the second argument in defaultdict() or omit this argument to create an empty defaultdict.

When you access a key that’s missing from the dictionary, the key is added, and the default value is assigned to it. You can also create the same points_default object using the callable int as the first argument since calling int() with no arguments returns 0.

All mappings are also collections, which means they’re iterable containers with a defined length. You can explore these characteristics with another mapping in Python’s standard library, collections.Counter:

Python >>> from collections import Counter >>> letters = Counter("learning python") >>> letters Counter({'n': 3, 'l': 1, 'e': 1, 'a': 1, 'r': 1, 'i': 1, 'g': 1, ' ': 1, 'p': 1, 'y': 1, 't': 1, 'h': 1, 'o': 1}) Copied!

The letters in the string "learning python" are converted into keys in Counter, and the number of occurrences of each letter is used as the value corresponding to each key.

You can confirm that this mapping is iterable, has a defined length, and is a container:

Python >>> for letter in letters: ... print(letter) ... l e a r n i g p y t h o >>> len(letters) 13 >>> "n" in letters True >>> "x" in letters False Copied!

You can use the Counter object letters in a for loop, which confirms it’s iterable. All mappings are iterable. However, the iteration loops through the keys and not the values. You’ll see how to iterate through the values or through both keys and values later in this tutorial.

The built-in len() function returns the number of items in the mapping. This is equal to the number of unique characters in the original string, including the space character. The object is sized since len() returns a value.

You can use the in keyword to confirm which elements are in the mapping. This check alone isn’t sufficient to confirm that the mapping is a container. However, you can also access the object’s .__contains__() special method directly:

Python >>> letters.__contains__("n") True Copied! Read the full article at https://realpython.com/python-mappings/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

2024 Trends: What's New for Drupal Editor Wed, 06/12/2024 - 16:00

People always want to predict the future. They say that is not good, but this doesn't apply to the Drupal 2024 trends. Undoubtedly, Generative Artificial Intelligence will shape tech trends and stimulate further progress in the website development industry.
However, there are other matters worth paying attention to. Mintz, World Wildlife Fund, Chupa Chups, Mattel, and other prominent Drupal websites won’t sit idly by viewing this chaos of innovations and advancements.
Now is the time for unique insights with Golems web development agency about what Drupal 2024 will be like. The more aware you are of prospective game-changing rules, the more up in arms you will act in SEO, marketing, and business growth strategies. Stay tuned to take a sneak peek into the future!

Taking a dive into the deep, dark quagmire of life

Some people have career aspirations from a young age, but for the rest of us, discovering a career-worthy passion, or even learning how to distinguish a job from a career, can take some time. I was nearly ten years into a job in finance (managing insurance/retirement/wealth-building products) when I finally admitted to myself that I couldn’t do it anymore. I felt like I should have a career, not just a job. I wanted to feel proud of the way I spent my days.

Diving in

After some soul-searching, I signed up for a Full-Stack Web Development bootcamp. I was never the kid who tinkered with code or tried to see how my AOL Instant Messenger worked (yes, I was a kid of the 90s) - but I suddenly found myself fascinated with the combination of language, math, and puzzle that was hiding in my browser’s Inspect tool. The bootcamp’s cost made this a risky prospect, but it also made me more likely to commit fully. I can’t say I loved every moment of the grind; working full-time, raising three kids (whom I had adopted just two days into the bootcamp), attending evening classes, keeping up with homework, and maintaining sanity was not always the easiest, but I tend to thrive in chaos. I learned the MERN Stack (MongoDB, Express.js, React, and Node), and after six months I left the bootcamp with one of the highest scores in my class and even higher hopes.

Cue the mass tech layoffs of late 2022, pitting my emerging knowledge against developers with years of experience at tech giants such as the site formerly known as Twitter. My understanding of code was evident, but my ability to explain code was underdeveloped, leaving me flailing in interviews and unable to find anyone willing to take a chance on me. Anyone, that is, until my local Women Who Code chapter posted about Palantir.net’s fellowship program. I applied, spent three months learning Drupal through DrupalEasy's Drupal Career Online program, and started an internship with Palantir.net. That internship turned into a full-time engineering role, and now I’m creating innovative tech and building professional websites from scratch.

Going deep

How could I transition from finance to React/JavaScript to Drupal/PHP in such a short time? Same way I managed to learn French — its proximity to English based on its Latin roots gave me a leg up. Both JavaScript and PHP are Object-Oriented languages, meaning they derive from making real-world objects into code. Object-Oriented Programming, or OOP, is a paradigm that encompasses several coding languages, and since I already had a base in OOP via JavaScript, I mainly needed to learn the syntax of PHP as the concepts were mostly the same. Not only are the languages similar, but the overall folder structure and architecture of both Drupal and React can be set up using the same patterns. I quickly noticed my new Palantiri teammates using the MVC (Model-View-Controller) architecture, which I was already comfortable with.

Another benefit of that MVC architecture is its collaborative nature. By separating sections of the code based on functionality, different people could work simultaneously on building one piece of a site without risking overwriting one another. React most prominently offers collaboration through its separation of interests via Components, delivering an end product to users, while Drupal is set up to not only be collaboratively built, but to allow site builders to delegate content creation, and even site administration, to users.

Surfacing from murky depths

Perhaps the coolest thing I discovered through this journey is the ability to create a site harnessing the powers of both Drupal and React! Drupal’s Content Management System structure creates a powerful backend for React’s simple and extensive front-end library. This just leaves me with the accessibility question - how can we make Drupal a more popular and attainable technology for entry-level developers? According to Statista’s data analytic survey, Drupal ranks 29th among the most-used frameworks among developers worldwide as of 2023. In the same survey, React ranked second only to Node, which is another JavaScript framework.

I think we can find a path for learners of all types, we just need to harness the power of our community to get there. We are not in competition with React or Wordpress or any other framework, Drupal is just another tool that people should have an opportunity to explore at any level. Who knows? Maybe they’ll fall in love like so many others have.

Calm waters

I love the glimmer of awe in people’s eyes now when I tell them I’m a software engineer. I can’t say the path was easy, but I’ve never regretted making the career change, and I never tire of continuing to suck up every ounce of knowledge I can find. I love the curiosity and creativity that I can bring to work with me every day, bouncing from front-end to back-end work, developing brand new technologies, and exploring every little thing that interests me. I’m happy to say I have found my career. 

Culture Drupal People

In this quiz, you’ll test your understanding of the basic characteristics and operations of Python mappings. By working through this quiz, you’ll revisit the key concepts and techniques of creating a custom mapping.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Discover why 1xINTERNET swiftly supported the Drupal Starshot Initiative. In this interview by Kazima Abbas, Baddy Sonja Breidert, CEO and Co-Founder, shares the motivations, goals, and impact behind their involvement. Don't miss these valuable insights from a key player in the Drupal community!

A while back, I wrote about using the SSH agent protocol to satisfy WebAuthn requests. The main problem with this approach is that it required starting the SSH agent with a special argument and also involved being a little too friendly with the implementation - things worked because I could provide an arbitrary public key and the implementation never validated that, but it would be legitimate for it to start doing so and then break everything. And it also only worked for keys stored on tokens that ssh supports - there was no way to extend this to other keystores on the client (such as the Secure Enclave on Macs, or TPM-backed keys on PCs). I wanted a better solution.

It turns out that it was far easier than I expected. The ssh agent protocol is documented here, and the interesting part is the extension support extension mechanism. Basically, you can declare an extension and then just tunnel whatever you want over it. As before, my goto was the go ssh agent package which conveniently implements both the client and server side of this. Implementing the local agent is trivial - look up SSH_AUTH_SOCK, connect to it, create a new agent client that can communicate with that by calling NewClient, and then implement the ExtendedAgent interface, create a new socket, and call ServeAgent against that. Most of the ExtendedAgent functions should simply call through to the original agent, with the exception of Extension(). Just add a case statement against extensionType, define some reasonably namespaced extension, and you're done.

Now you need to use this agent. You probably don't want to use this for arbitrary hosts (agent forwarding should only be enabled for remote systems you trust, not arbitrary machines you connect to - if you enabled agent forwarding for github and github got compromised, github would be able to use any private keys loaded into your agent, and you probably don't want that). So the right approach is to add a Host entry to the ssh config with a ForwardAgent stanza pointing at the socket you created in your new agent. This way the configured subset of remote hosts will automatically talk to this new custom agent, while forwarding for anything else will still be at the user's discretion.

For the remote end things are even easier. Look up SSH_AUTH_SOCK and call NewClient as before, and then simply call client.Extension(). Whatever you stick in the contents argument will simply end up being received at the client end. You now have a communication channel between a the remote system and the local client, and what you do with that is up to you. I'm using it to allow a remote system to obtain auth tokens from Okta and forward WebAuthn challenges that can either be satisfied via a local WebAuthn token or by passing the query off to Mac TouchID, but there's fundamentally no constraints whatsoever on what can be done here.

(If you want to do this on Windows and still have everything work with existing clients you'll need to take this into account - Windows didn't really do Unix sockets until recently so everything there is awful)

comments

Like each month, have a look at the work funded by Freexian’s Debian LTS offering.

Debian LTS contributors

In May, 17 contributors have been paid to work on Debian LTS, their reports are available:

• Adrian Bunk did 34.25h (out of 24.0h assigned and 22.0h from previous period), thus carrying over 11.75h to the next month.
• Bastien Roucariès did 20.0h (out of 20.0h assigned).
• Ben Hutchings did 16.0h (out of 24.0h assigned), thus carrying over 8.0h to the next month.
• Chris Lamb did 18.0h (out of 18.0h assigned).
• Daniel Leidert did 8.0h (out of 10.0h assigned), thus carrying over 2.0h to the next month.
• Emilio Pozuelo Monfort did 35.5h (out of 46.0h assigned), thus carrying over 10.5h to the next month.
• Guilhem Moulin did 13.0h (out of 14.75h assigned and 5.25h from previous period), thus carrying over 7.0h to the next month.
• Lee Garrett did 11.0h (out of 37.25h assigned and 8.75h from previous period), thus carrying over 35.0h to the next month.
• Lucas Kanashiro did 10.0h (out of 20.0h assigned), thus carrying over 10.0h to the next month.
• Markus Koschany did 40.0h (out of 40.0h assigned).
• Ola Lundqvist did 6.5h (out of 22.5h assigned and 1.5h from previous period), thus carrying over 17.5h to the next month.
• Roberto C. Sánchez did 7.75h (out of 11.0h assigned and 1.0h from previous period), thus carrying over 4.25h to the next month.
• Santiago Ruano Rincón did 8.0h (out of 16.0h assigned), thus carrying over 8.0h to the next month.
• Sean Whitton did 5.5h (out of 5.5h assigned and 0.5h from previous period), thus carrying over 0.5h to the next month.
• Sylvain Beucler did 10.5h (out of 0.75h assigned and 45.25h from previous period), thus carrying over 35.5h to the next month.
• Thorsten Alteholz did 14.0h (out of 14.0h assigned).
• Tobias Frost did 7.75h (out of 10.0h assigned and 2.0h from previous period), thus carrying over 4.25h to the next month.

Evolution of the situation

In May, we have released 20 DLAs.

Notable security updates in May included:

• apache2: multiple vulnerabilities which may result in HTTP response splitting, denial of service, or authorization bypass (by Bastien Roucariès, in collaboration with apache2 maintainer Yadd)
• bind9: two vulnerabilities, called KeyTrap and NSEC3, which may result in denial of service (by Santiago Ruano Rincón)
• python-pymysql: potential SQL injection attack (by Chris Lamb)

The aforementioned apache2 was prepared by its Debian maintainer Yadd. This update also involved work on the package test suite in buster, which contributor Bastien Roucariès then forwarded to the apache2 package in unstable. More importantly, a regression in fossil was reported, and Bastien prepared a fix for it. Bastien coordinated the upload of both packages to minimize the introduction of regressions.

Contributor Daniel Leidert also prepared an upload of runc to Debian 11 in order fix a number of CVEs still affecting that package. Finally, contributor Thorsten Alteholz prepared uploads for qtbase-opensource-src, libjwt, and libmicrohttpd in Debian 11. Note that Debian 11 will pass into the LTS phase of support in August and these updates will improve the state and long-term supportability of Debian 11.

Debian 10 is presently in its final month of LTS support (as announced on the debian-lts-announce mailing list, support will end on June 30th), after which no new security updates will be made available on security.debian.org.

However, Freexian and its team of paid Debian contributors will continue to maintain Debian 10 going forward for the customers of the Extended LTS offer. Subscribe right away if you sill have Debian 10 which must be kept secure (and which cannot yet be upgraded).

Thanks to our sponsors

Sponsors that joined recently are in bold.

• Platinum sponsors:
  • TOSHIBA (for 105 months)
  • Civil Infrastructure Platform (CIP) (for 73 months)
  • VyOS Inc (for 37 months)
• Gold sponsors:
  • Roche Diagnostics International AG (for 116 months)
  • Linode (for 110 months)
  • Babiel GmbH (for 99 months)
  • Plat’Home (for 99 months)
  • CINECA (for 73 months)
  • University of Oxford (for 55 months)
  • Deveryware (for 42 months)
  • EDF SA (for 26 months)
  • CERN
• Silver sponsors:
  • Domeneshop AS (for 120 months)
  • Nantes Métropole (for 115 months)
  • Univention GmbH (for 106 months)
  • Université Jean Monnet de St Etienne (for 106 months)
  • Ribbon Communications, Inc. (for 100 months)
  • Exonet B.V. (for 90 months)
  • Leibniz Rechenzentrum (for 84 months)
  • Ministère de l’Europe et des Affaires Étrangères (for 67 months)
  • Cloudways by DigitalOcean (for 57 months)
  • Dinahosting SL (for 55 months)
  • Bauer Xcel Media Deutschland KG (for 49 months)
  • Platform.sh SAS (for 49 months)
  • Moxa Inc. (for 43 months)
  • sipgate GmbH (for 40 months)
  • OVH US LLC (for 38 months)
  • Tilburg University (for 38 months)
  • GSI Helmholtzzentrum für Schwerionenforschung GmbH (for 30 months)
  • Soliton Systems K.K. (for 27 months)
  • THINline s.r.o.
• Bronze sponsors:
  • Evolix (for 121 months)
  • Seznam.cz, a.s. (for 121 months)
  • Intevation GmbH (for 118 months)
  • Linuxhotel GmbH (for 118 months)
  • Daevel SARL (for 116 months)
  • Bitfolk LTD (for 115 months)
  • Megaspace Internet Services GmbH (for 115 months)
  • NUMLOG (for 115 months)
  • Greenbone AG (for 114 months)
  • WinGo AG (for 114 months)
  • Ecole Centrale de Nantes - LHEEA (for 110 months)
  • Entr’ouvert (for 105 months)
  • Adfinis AG (for 102 months)
  • GNI MEDIA (for 97 months)
  • Laboratoire LEGI - UMR 5519 / CNRS (for 97 months)
  • Tesorion (for 97 months)
  • Bearstech (for 88 months)
  • LiHAS (for 88 months)
  • Catalyst IT Ltd (for 83 months)
  • Supagro (for 78 months)
  • Demarcq SAS (for 77 months)
  • Université Grenoble Alpes (for 63 months)
  • TouchWeb SAS (for 55 months)
  • SPiN AG (for 52 months)
  • CoreFiling (for 47 months)
  • Institut des sciences cognitives Marc Jeannerod (for 42 months)
  • Observatoire des Sciences de l’Univers de Grenoble (for 39 months)
  • Tem Innovations GmbH (for 34 months)
  • WordFinder.pro (for 33 months)
  • CNRS DT INSU Résif (for 32 months)
  • Alter Way (for 25 months)
  • Institut Camille Jordan (for 14 months)

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler, “download now”.

This release bumps the long-awaited 3.12 support to a complete level. Now, Nuitka behaves identically to CPython 3.12 for the most part.

In terms of bug fixes, it’s also huge. Especially for Unicode paths and software with Unicode extension module names and Unicode program names, and even non-UTF8 code names, there have been massive amounts of improvements.

Table of Contents

• Bug Fixes

• New Features

• Optimization

• Organizational

• Tests

• Cleanups

• Summary

Bug Fixes

• Standalone: Added support for python-magic-bin package. Fixed in 2.2.1 already.

• Fix: The cache directory creation could fail when multiple compilations started simultaneously. Fixed in 2.2.1 already.

• macOS: For arm64 builds, DLLs can also have an architecture dependent suffix; check that as well. Makes the soundfile dependency scan work. Fixed in 2.2.1 already.

• Fix: Modules where lazy loaders handling adds hard imports when a module is first processed did not affect the current module, potentially causing it not to resolve hidden imports. Fixed in 2.2.1 already.

• macOS: The use of libomp in numba needs to cause the extension module not to be included and not to look elsewhere. Fixed in 2.2.1 already.

• Python3.6+: Fix, added support for keyword arguments of ModuleNotFoundError. Fixed in 2.2.1 already.

• macOS: Detect more versioned DLLs and arm64 specific filenames. Fixed in 2.2.1 already.

• Fix, was not annotating exception exit when converting an import to a hard submodule import. Fixed in 2.2.2 already.

• Fix, branches that became empty can still have traces that need to be merged.

  Otherwise, usages outside the branch will not see propagated assignment statements. As a result, these falsely became unassigned instead. Fixed in 2.2.2 already.

• Windows: Fix, uninstalled self-compiled Python didn’t have proper installation prefix added for DLL scan, resulting in runtime DLLs not picked up from there. Fixed in 2.2.2 already.

• Standalone: Added support for newer PySide6 version 6.7. It needed correction on macOS and has a new data file type. Fixed in 2.2.3 already.

• Standalone: Complete support for pyocd package. Fixed in 2.2.3 already.

• Module: Fix, the created .pyi files were incomplete.

  The list of imported modules created in the finalization step was incomplete, we now go over the actual done modules and mark all non-included modules as dependencies.

• Scons: Fix, need to avoid using Unicode paths towards the linker on Windows. Instead, use a temporary output filename and rename it to the actual filename after Scons has completed.

• Windows: Avoid passing Unicode paths to the dependency walker on Windows, as it cannot handle those. Also, the temporary filenames in the build folder must be in short paths, as it cannot handle them in case that is a Unicode path.

• Scons: For ccache on Windows, the log filename must be a short path too, if the build folder is a Unicode path.

• Windows: Make sure the Scons build executes inside a short path as well, so that a potential Unicode path is visible to the C compiler when resolving the current directory.

• Windows: The encoding of Unicode paths for accelerated mode values of __file__ was not making sure that hex sequences were correctly terminated, so in some cases, it produced ambiguous C literals.

• Windows: Execute binaries created with --windows-uac-admin with and --run options with proper UAC prompt.

• Fix, need to allow for non-UTF8 Unicode in variable names, function names, class names, and method names.

• Python3.10+: Fix, match statements that captured the rest of mapping checks were not working yet.

  match value: case {"key1": 5, **rest}: ... # rest was not assigned here

• Windows: When deleting build folders, make sure the retries leading to a complete deletion always.

• Python2: Fix, could crash with non-unicode program paths on Windows.

• Avoid giving SyntaxWarning from reading source code

  For example, the standard site module of Python 3.12 gives warnings about illegal escape sequences that nobody cares about apparently.

• Fix, the matplotlib warnings by options-nanny were still given even if the no-qt plugin was used, since the variable name referenced there was not actually set yet by that plugin.

• Windows: Fix, when using the uninstalled self-compiled Python, we need python.exe to find DLL dependencies. Otherwise it doesn’t locate the MSVC runtime and Python DLL properly.

• Standalone: Added support for freetype package.

New Features

• Support for Python 3.12 is finally there. We focused on scalability first and because we did things the correct way immediately, rather than rushing to get it working and improving only later.

  As a result, the correctness and performance of Nuitka with previous Python releases are improved as well.

  Some things got delayed, though. We need to do more work to take advantage of other core changes. Concerning exceptions normalized at creation time, the created module code doesn’t yet take advantage. Also, more efficient two-digit long handling is possible with Python 3.12, but not implemented. It will take more time before we have these changes completed.

• Experimental support for Python 3.13 beta 1 is also there, and potentially surprising, but we will try and follow its release cycle closely and aim to support it at the time of release.

  Nuitka has followed all of its core changes so far, and basic tests are passing; the accelerated, module, standalone, and onefile modes all work as expected. The only thing delayed is the uncompiled generator integration, where we need to replicate the exact CPython behavior. We need to have perfect integration only for working with the asyncio loop, so we wait with it until release candidates appear.

• Plugins: Added support to include directories entirely unchanged by adding raw_dir values for data-files section, see Nuitka Package Configuration.

• UI: The new command line option --include-raw-dir was added to allow including directories entirely unchanged.

• Module: Added support for creating modules with Unicode names. Needs a different DLL entry function name and to make use of two-phase initialization for the created extension module.

• Added support for OpenBSD standalone mode.

Optimization

• Python3: Avoid API calls for allocators

  Most effective with Python 3.11 or higher but also many other types like bytes, dict keys, float, and list objects are faster to create with all Python3 versions.

• Python3.5+: Directly use the Python allocator functions for object creation, avoiding the DLL API calls. The coverage is complete with Python3.11 or higher, but many object types like float, dict, list, bytes benefit even before that version.

• Python3: Faster creation of StopIteration objects.

  With Python 3.12, the object is created directly and set as the current exception without normalization checks.

  We also added a new specialized function to create the exception object and populate it directly, avoiding the overhead of calling of the StopIteration type.

• Python3.10+: When accessing freelists, we were not passing for tstate but locally getting the interpreter object, which can be slower by a few percent in some configurations. We now use the free lists more efficient with tuple, list, and dict objects.

• Python3.8+: Call uncompiled functions via vector calls.

  We avoid an API call that ends up being slower than using the same function via the vector call directly.

• Python3.4+: Avoid using _PyObject_LengthHint API calls in list.extend and have our variant that is faster to call.

• Added specialization for os.path.normpath. We might benefit from compile time analysis of it once we want to detect file accesses.

• Avoid using module constants accessor for global constant values

  For example, with (), we used the module-level accessor for no reason, as it is already available as a global value. As a result, constant blobs shrink, and the compiled code becomes slightly smaller , too.

• Anti-Bloat: Avoid using dask from the sparse module. Added in 2.2.2 already.

Organizational

• UI: Major change in console handling.

  Compiled programs on Windows now have a third mode, besides console or not. You can now create GUI applications that attach to an available console and output there.

  The new option --console controls this and allows to enforce console with the force value and disable using it with the disable value, the attach value activates the new behavior.

  Note

  Redirection of outputs to a file in attach mode only works if it is launched correctly, for example, interactively in a shell, but some forms of invocation will not work; prominently, subprocess.call without inheritable outputs will still output to a terminal.

  On macOS, the distinction doesn’t exist anymore; technically it wasn’t valid for a while already; you need to use bundles for non-console applications, though, by default otherwise a console is forced by macOS itself.

• Detect patchelf usage in buggy version 0.18.0 and ask the user to upgrade or downgrade it, as this specific version is known to be broken.

• UI: Make clear that the --nofollow-import-to option accepts patters.

• UI: Added warning for module mode and usage of the options to force outputs as they don’t have any effect.

• UI: Check the success of Scons in creating the expected binary immediately after running it and not only once we reach post-processing.

• UI: Detect empty user package configuration files

• UI: Do not output module ast when a plugin reports an error for the module, for example, a forbidden import.

• Actions: Update from deprecated action versions to the latest versions.

Tests

• Use Nuitka Project Options for the user plugin test rather than passing by environment variables to the test runner.

• Added a new search mode, skip, `` to complement ``resume which resumes right

  after the last test resume stopped on. We can use that while support for a Python version is not complete.

Cleanups

• Solved a TODO about using unified code for setting the StopIteration, coroutines, generators, and asyncgen used to be different.

• Unified how the binary result filename is passed to Scons for modules and executables to use the same result_exe key.

Summary

This release marks a huge step in catching up with compatibility of Python. After being late with 3.12 support, we will now be early with 3.13 support if all goes well.

The many Unicode support related changes also enhanced Nuitka to generate 2 phase loading extension modules, which also will be needed for sub-interpreter support later on.

From here on, we need to re-visit compatibility. A few more obscured 3.10 features are missing, the 3.11 compatibility is not yet complete, and we need to take advantage of the new caching possibilities to enhance performance for example with attribute lookups to where it can be with the core changes there.

For the coming releases until 3.13 is released, we hope to focus on scalability a lot more and get a much needed big improvement there, and complete these other tasks on the side.

Open Source AI needs to require data to be viable

• With many different discussions happening at once, here are the main points:
  • On the issue of training data
    • @mark is concerned with openness of AI not being meaningful if there is not a focus on the training data.” Model weights are the most inscrutable component of current generative AI, and providers that release only [the weights] should not get a free ‘openness’ pass.”
    • @stefano agrees with all of that but questions the criteria used to assign green marks in Mark’s paper, pointing out inconsistencies. They use the example of Pythia-Chat-Base-7, which relies on a dataset from OpenDataHub with potential issues like non-versioned data and stale links, failing to meet stringent requirements required by @juliaferraioli. Similar concerns are raised for other models like OLMo 7B Instruct, which lack specific data versioning details. Maffulli also highlights the case of Pythia-7B, which once may have been compliant but it’s now problematic due to the unavailability of its foundational dataset, the Pile, illustrating the complexities in maintaining an “open source” status over time, if the stringent proposal suggested by @juliaferraioli and the AWS team is adopted.
    • @shujisado adds that while he sympathizes with @juliaferraioli‘s request for datasets, @stefano‘s arguments in support of the concept of “Data information” are aligned with the OSI principles and are reasonable.
    • @spotaws stresses that “data information” alone is insufficient if the data itself is too vague.
    • @juliaferraioli adds that while replicating AI systems like OLMo or Pythia may seem impractical due to costs and statistical nature, the capability is crucial for broader adoption and consistency.  She finds the current definition to be unclear and subjective.
    • @zack recommends to review StarCoder2, recognizing that it would be in the same category of BLOOM: a system with lots of transparency and a dataset made available but released with a restrictive license.
    • @Ezequiel_Lanza joined the conversation in support of the concept of Data information, claiming, with technical arguments that “sharing the dataset is not necessarily required and may not justify the potential risks associated with making it mandatory.”
    • Partially open / restrictive licenses
      • Continuing @marks points regarding restrictive licenses (like the ethical licenses), @stefano has added a link to an article highlighting some reasons why OSI is staying away from these licenses.
      • @pchestek further adds that a partially open license would create even more opportunities for open washing, as “open source AI” could have many meanings.
      • @mark clarified that rather than proposing a variety of meanings, they are seeking to highlight the dimensions of openness in their paper, exploring the broader landscape. 
      • @stefano adds that in the 26 years of OSI, it has contended with numerous organizations claiming varying degrees of openness as “open source. This issue is now mirrored in AI, as companies seek the market value of being labeled Open Source. Open Source is binary: either users have full rights or they don’t, and any system that falls short is not Open Source AI, regardless of how “almost” open it is.
    • Field of use/restriction 
      • @juliaferraioli believes that OSAID should include prohibitions against field-of-use restrictions.
      • @shujisado adds that OSAID specifies four freedoms as requirements for being considered open source and that this should be understood as the same since “freedom” is the same as “non-restricted”. The 10 clauses of the OSD have been replaced by the checklist in draft v0.0.8.
      • @juliaferraioli adds that individual components may be covered by their individual licenses, but the overall system may be subject to additional terms, which is why we need this to be explicit.

Initial Report on Definition Validation

• @Mer has added how far we are regarding our system analysis compared to our current draft definition. Some points that remain incomplete have been highlighted.
• Mistral (Mixtral 8x7B) is considered not in alignment with the OSAID because its data pre-processing code is not released under an OSI-approved license.

Can a derivative of non-open-source AI be considered Open Source AI?

• @tarek_ziade shares his experience fine-tuning a “small” model (200M parameters) for a Firefox feature to describe images, using a base model for image encoding and text decoding. Despite not having 100% traceability of upstream data, Tarek argues that intentional fine-tuning and transparency make the new fine-tuned model open source. Any issues arising from downstream data can be addressed by the project maintainers, maintaining the model’s open source status.

Town hall recording out

• We held our 10th town hall meeting a week and a half ago. You can access the recording here if you missed it.
• A new town hall meeting is scheduled for this Friday, June 14.

After signing up for GitHub Sponsors, I had a nagging feeling that somehow asking for money from other people to support my open source work was inappropriate. But after much reflection, I realized that phrasing the use of GitHub Sponsors as a way to express patronage/support and appreciation for my work instead of sponsorship stopped me feeling bad about it. It also led me to reflect on to what degree people can express thanks to open source maintainers.

⚠️This blog post is entirely from my personal perspective and thus will not necessarily apply to every open source developer out there.Be nice

The absolutely easiest way to show thanks is to simply not be mean. It sounds simple, but plenty of people fail at even this basic level of civility. This isn&apost to say you can&apost say that a project didn&apost work for you or you disagree with something, but there&aposs a massive difference between saying "I tried the project and it didn&apost meet my needs" and "this project is trash".

People failing to support this basic level of civility is what leads to burnout.

Be an advocate

It&aposs rather indirect, but saying nice things about a project is a way of showing thanks. As an example, I have seen various people talk positively about pyproject.toml online, but not directly at me. That still feels nice due to how much effort I put into helping make that file exist and creating the [project] table.

Or put another way, you never know who is reading your public communications.

Produce your own open source

Another indirect way to show thanks is by sharing your own open source code. By maintaining your own code, you&aposll increase the likelihood I myself will become a user of your project. That then becomes a circuitous cycle of open source support between us.

Say thanks

Directly saying "thank you" actually goes a really long way. It takes a lot of positive interactions to counteract a single negative interaction. You might be surprised how much it might brighten someone&aposs day when someone takes the time and effort to reach out and say "thank you", whether that&aposs by DM, email, in-person at a conference, etc.

Fiscal support

As I said in the opening of this post, I set up GitHub Sponsors for myself as a way for people to show fiscal support for my open source work if that&aposs how they prefer to express their thanks (including businesses). Now I&aposm purposefully not saying "sponsor" as to me that implies that giving money leads to some benefit (e.g. getting a shout-out somewhere) which is totally reasonable for people to do. But for me, since every commit is a gift, I&aposm financially secure, and I&aposm not trying to make a living from my volunteer open source work or put in the effort to make sponsorship worth it, I have chosen to treat fiscal support as a way of showing reciprocity for the gift of sharing my code that you&aposve already received. This means I fully support all open source maintainers setting up fiscal support at a minimum, and if they want to put in the effort to go the sponsorship route then they definitely should.

Producing open source also isn&apost financially free. For instance, I pay for:

1. The hosting of this blog via Ghost(Pro)
2. Obsidian Sync to keep my open source notes available on all my devices so when I have an idea I can write it down
3. Obsidian Publish to share my open source notes
4. Computer upgrades (including ergonomic upgrades like keyboards)
5. My personal time away from my wife and child, family and friends (which my open source journal exists to try and point out for those who don&apost realize how much time I put into my volunteer work)

So while open source is "free" for you as the consumer, the producer very likely has concrete financial costs in producing that open source on top of the intangible costs like volunteering their personal time.

But as I listed earlier, there are plenty of other ways to show thanks without having to spend money that can be equally valuable to a maintainer.

I also specifically didn&apost mention contributing. I have said before that contributions are like giving someone a puppy: it seems like a lovely gift at the time, but the recipient is now being "gifted" daily walks involving scooping &#x1F4A9; and vet bills. As such, contributions from others can be a blessing and a curse all at the same time depending on the contribution itself, the attitude of the person making the contribution, etc. So I wouldn&apost always assume my contribution is as welcomed and desired as much as a "thank you" note.

Authored by Nadiia Nykolaichuk, João Paulo Constantino, Ana Carolina, and Gabriel Passarelli.

Authored by Nadiia Nykolaichuk, João Paulo Constantino, Ana Carolina, and Gabriel Passarelli.

#633 – JUNE 11, 2024
View in Browser »

String Interpolation in Python: Exploring Available Tools

In this tutorial, you’ll learn about the different tools that Python provides for performing string interpolation. String interpolation allows you to create new strings by inserting different objects into a string template.
REAL PYTHON

Notebooks for Fundamentals of Music Processing

This is a collection of Python Notebooks for teaching and learning the fundamentals of music processing. Examples include illustrations, sound samples, math, and more.
INTERNATIONAL AUDIO LABS

Upgrade Python Versions Without the Pain

Stop wasting 30% of your team’s sprint on maintaining legacy codebases. Automatically migrate and keep up-to-date on Python versions, so that you can focus on being productive while staying secure, without the risk of breaking changes - Get a code assessment today →
ACTIVESTATE sponsor

Python’s Many Command-Line Utilities

This article describes every command-line tool included with Python, each of which can be run with python -m module_name.
TREY HUNNER

String Interpolation in Python (Quiz)

Take this quiz to test your understanding of the available tools for string interpolation in Python, as well as their strengths and weaknesses. These tools include f-strings, the .format() method, and the modulo operator.
REAL PYTHON

Python 3.12.4 Released

See the full list of changes in this release
CPYTHON DEV BLOG

PEP 712 Rejected

This Python Enhancement Proposal “Adding a ‘converter’ parameter to dataclasses.field” was determined to have an insufficient number of use cases.
PYTHON

Python 3.13.0 Beta 2 Released

CPYTHON DEV BLOG

Articles & Tutorials What Are CRUD Operations?

CRUD operations are the cornerstone of application functionality. Whether you access a database or interact with a REST API, you usually want to create, retrieve, update, and delete data. In this tutorial, you’ll explore how CRUD operations work in practice.
REAL PYTHON

What We Talk About When We Talk About System Design

Mahesh talks about the rules he has encountered when doing research on designing large systems. Guidelines include late-binding on the design, focusing on the problem rather than existing systems, talking about other applications, and more.
MAHESH BALAKRISHNAN

Get Your Own AI Agent to Answer Questions From Your Database

Introducing “Database Mind” - a ready-to-use AI system designed for easy integration into your projects. As part of the “Minds Endpoints” AI platform, it offers a simple plug-and-play API service, enabling developers to effortlessly incorporate advanced AI capabilities into their solutions →
MINDSDB sponsor

Statically Typed Functional Programming With Python 3.12

This detailed article looks at how to use the match statement along with Python’s typing mechanism to write functional programs similar in style to Kotlin.
OSKAR WICKSTROM

How to Annotate a Graph With Matplotlib and Python

The Matplotlib package is great for visualizing data. One of its many features is the ability to annotate points on your graph. This article shows you how.
MIKE DRISCOLL

bytes: The Lesser-Known Python Built-in Sequence

The bytes data type looks a bit like a string, but it isn’t a string. This article explores it and also looks at the main Unicode encoding, UTF-8
STEPHEN GRUPPETTA

Reflecting on One Year of Being an Engineering Manager

“Being a manager is a focus change from code to people, from output to outcomes and from being productive to making most of everyone’s time.” Read more of Victor’s reflecting on his first year as a manager.
VICTOR STOJANOV

Testing With Python: Fake It

This article is on using mock in your Python testing and is part of a larger series on testing in general.
BITECODE

Projects & Code Mesop: Build Web Apps in Python

GITHUB.COM/GOOGLE

WeasyPrint: The Awesome Document Factory

GITHUB.COM/KOZEA

django-axes: Track of Failed Login Attempts in Django

GITHUB.COM/JAZZBAND

Zango: Microservices in Django

GITHUB.COM/HEALTHLANE-TECHNOLOGIES

gloe: Library for Flow-Oriented Code

GITHUB.COM/IDEOS

Events Weekly Real Python Office Hours Q&A (Virtual)

June 12, 2024
REALPYTHON.COM

Wagtail Space NL

June 12 to June 15, 2024
WAGTAIL.SPACE

Django Girls Abraka Workshop 2024

June 13 to June 15, 2024
DJANGOGIRLS.ORG

Python Atlanta

June 13 to June 14, 2024
MEETUP.COM

PyData London 2024

June 14 to June 17, 2024
PYDATA.ORG

PyCamp Leipzig 2024

June 15 to June 17, 2024
BARCAMPS.EU

Wagtail Space US

June 20 to June 23, 2024
WAGTAIL.SPACE

Happy Pythoning!
This was PyCoder’s Weekly Issue #633.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

The Drupal community advanced its Starshot initiative with a key session on June 7, 2024, led by Dries Buytaert and Cristina Chumillas. The session focused on refining the Drupal ecosystem with user-centric improvements and strategic development milestones. Key topics included the mission statement review, concept wireframes, draft milestones for DrupalCon Barcelona, and the Starshot Council.

We’re already on Week 3 of the coding period of Google Summer of Code! As a reminder, I’m adding Python support to a few KDE Frameworks. During the first two weeks I added support for KWidgetsAddons, and it’s now almost finished except for two widgets whose bindings aren’t generated properly and don’t compile.

I also wrote (with the help of my mentor Carl) the necessary CMake code to build the library. That part is probably going to end (hopefully) in extra-cmake-modules so it can be used by anyone easily. The plan is to eventually submit each of the bindings to their repository so it’s easier to keep the C++ libraries in sync with their Python bindings.

This week I will be adding some examples for KWidgetAddons now that it’s on a usable status. If you want to see how it’s going, you can take a look at the code.

When a new major version of Drupal is released, custom code often requires updates to align with API changes, including the removal of deprecated APIs.

Because I keep forgetting certain aspects of this workflow, I decided to document it for future reference.

Tools overview Tool Interface Functionality Target Audience Upgrade Status module UI in Drupal Identifies deprecated code, hosting environment compatibility, and more Site administrators and developers Drupal Check Command-line Identifies deprecated code Developers, especially during coding and continuous integration (CI) Upgrade Status module

The Upgrade Status module assesses a Drupal site's readiness for major version upgrades by checking for deprecated code and other compatibility issues.

Screenshot of a Drupal upgrade status report showing hosting environment compatibility checks.

1. Install the Upgrade Status module like you would install any other Drupal module:

   [code bash]$ ddev composer require –dev drupal/upgrade_status[/code]

   Here, ddev is the tool I prefer for managing my local development environment. composer is a dependency manager for PHP, commonly used to install Drupal modules. The –dev option specifies that the module should be installed as a development requirement, meaning it is necessary for development environments but not installed on production environments.

2. Enable the Upgrade Status module:

   [code bash]$ ddev drush pm-enable upgrade_status[/code]

   drush stands for "Drupal shell" and is a command-line utility for managing Drupal sites. The command pm:enable (where pm stands for "package manager") is used to enable a module in Drupal.

3. After enabling the module, you can access its features by navigating to the Admin > Reports > Upgrade status page at /admin/reports/upgrade-status.

Upgrading PHP and MySQL using DDEV

The Upgrade Status module might recommend updating PHP and MySQL, per Drupal's system requirements.

To update the PHP version of DDEV, use the following command:

[code bash]$ ddev config –-php-version 8.3[/code]

To upgrade the MySQL version of DDEV and migrate your database content, use the following command:

[code bash]$ ddev debug migrate-database mariadb:10.11[/code]

After updating these settings, I restart DDEV and run my PHPUnit tests. Although these tests are integrated into my CI/CD workflow, I also run them locally on my development machine using DDEV for immediate feedback.

Drupal Check

Drupal Check is a command-line tool that scans Drupal projects for deprecated code and compatibility issues.

Output of Drupal Check command indicating no deprecated code was found.

1. Installation:

   [code bash]$ ddev composer require –dev mglaman/drupal-check[/code]

2. Run Drupal Check from the root of your Drupal installation:

   [code bash]$ ./vendor/bin/drupal-check –memory-limit 500M docroot/modules/custom[/code]

   I usually have to increase the memory limit, hence the --memory-limit 500M.

Getting a list of all the files and folders in a directory is a natural first step for many file-related operations in Python. When looking into it, though, you may be surprised to find various ways to go about it.

When you’re faced with many ways of doing something, it can be a good indication that there’s no one-size-fits-all solution to your problems. Most likely, every solution will have its own advantages and trade-offs. This is the case when it comes to getting a list of the contents of a directory in Python.

In this video course, you’ll be focusing on the most general-purpose techniques in the pathlib module to list items in a directory, but you’ll also learn a bit about some alternative tools.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

As is customary Apple announced their latest operating system versions at WWDC yesterday, including macOS 15 Sequoia, named after the national park in the Sierra Nevada mountain range.