Feeds

Black Friday and Cyber Monday are nearly here, so it’s time to do a Python sale! All my books and courses are 35% off until December 4th if you use this code: BF24 at checkout.

You can learn about any of the following topics in my books and courses:

• Basic Python (Python 101)
• PDF Processing (ReportLab book)
• Excel and Python
• Image processing (Pillow book)
• Python Logging (NEW this year!)
• Jupyter Notebook
• JupyterLab 101 (NEW this month!)
• and more

Check out my books and courses today!

Other Python Sales

• Sundeep Agarwal: ~50% off Sundeep’s all book and Python bundles with code FestiveOffer
• Python Jumpstart with Python Morsels: 50% off Trey Hunner’s brand new Python course, an introduction to Python that’s very hands-on ($99 instead of $199)
• Rodrigo 50% off Rodrigo’s all books bundle with code BF24
• The Python Coding Place: 40% off The Python Coding Book and 40% off a lifetime membership to The Python Coding Place with code black2024
• Adam Johnson’s Django-related Deals for Black Friday 2024

The post Black Friday Python Deals 2024 appeared first on Mouse Vs Python.

I’ve just recently launched a self-paced introduction to Python that is extremely hands-on. It’s called Python Jumpstart and it’s based on introductory Python curriculum that I have been iterating on for years.

Learn Python by writing Python code ✍

We do not learn by putting information into the brain. Our brains simply don’t retain knowledge that way.

Learning happens from repeatedly attempting to retrieve information from the brain. When it comes to Python, that means writing Python code.

So unlike most Python courses, Python Jumpstart is not focused on watching videos and rewriting code seen within videos. Instead, this new Python course is based around learning Python by writing Python code.

How Python Jumpstart is structured 🔬

Python Jumpstart includes learning Python by solving 46 short Python exercises.

Before each exercise, you’ll watch a 5 minute video explaining a new Python topic. You’ll then attempt the exercise to put one or more Python topics into practice.

You won’t solve many of the exercises on your first try and that’s okay. These exercises are designed to be revisited a few times over the course of weeks, until you’re satisfied with your solution.

This is not a sprint 📆

This structured path to Python proficiency, includes:

• Carefully crafted exercises that build real understanding
• Short, detailed explanations that won’t waste your time
• A proven teaching approach that focuses on active learning
• Spaced repetition to help concepts stick

This is not a course for impatient or passive learners. Learning takes repetitive effort spaced over many days and this course is structured to embrace that fact.

If you spend 30 minutes on Python Jumpstart each day, I estimate that you’ll complete this course in about 7 weeks. You’ll spend the large majority of that time writing Python code.

This course will take time, but it will be time well-spent.

Launch week special: 50% off until December 2 ⏰

Through Monday December 2, 2024, you can get lifetime access to Python Jumpstart for $99. After this launch week, Python Jumpstart will be $199.

Ready to jumpstart your Python learning journey?

Get Python Jumpstart for $99

Concurrency refers to the ability of a program to manage multiple tasks at once, improving performance and responsiveness. It encompasses different models like threading, asynchronous tasks, and multiprocessing, each offering unique benefits and trade-offs. In Python, threads and asynchronous tasks facilitate concurrency on a single processor, while multiprocessing allows for true parallelism by utilizing multiple CPU cores.

Understanding concurrency is crucial for optimizing programs, especially those that are I/O-bound or CPU-bound. Efficient concurrency management can significantly enhance a program’s performance by reducing wait times and better utilizing system resources.

In this tutorial, you’ll learn how to:

• Understand the different forms of concurrency in Python
• Implement multi-threaded and asynchronous solutions for I/O-bound tasks
• Leverage multiprocessing for CPU-bound tasks to achieve true parallelism
• Choose the appropriate concurrency model based on your program’s needs

To get the most out of this tutorial, you should be familiar with Python basics, including functions and loops. A rudimentary understanding of system processes and CPU operations will also be helpful. You can download the sample code for this tutorial by clicking the link below:

Get Your Code: Click here to download the free sample code that you’ll use to learn about speeding up your Python program with concurrency.

Take the Quiz: Test your knowledge with our interactive “Python Concurrency” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

Python Concurrency

In this quiz, you'll test your understanding of Python concurrency. You'll revisit the different forms of concurrency in Python, how to implement multi-threaded and asynchronous solutions for I/O-bound tasks, and how to achieve true parallelism for CPU-bound tasks.

Exploring Concurrency in Python

In this section, you’ll get familiar with the terminology surrounding concurrency. You’ll also learn that concurrency can take different forms depending on the problem it aims to solve. Finally, you’ll discover how the different concurrency models translate to Python.

What Is Concurrency?

The dictionary definition of concurrency is simultaneous occurrence. In Python, the things that are occurring simultaneously are called by different names, including these:

• Thread
• Task
• Process

At a high level, they all refer to a sequence of instructions that run in order. You can think of them as different trains of thought. Each one can be stopped at certain points, and the CPU or brain that’s processing them can switch to a different one. The state of each train of thought is saved so it can be restored right where it was interrupted.

You might wonder why Python uses different words for the same concept. It turns out that threads, tasks, and processes are only the same if you view them from a high-level perspective. Once you start digging into the details, you’ll find that they all represent slightly different things. You’ll see more of how they’re different as you progress through the examples.

Now, you’ll consider the simultaneous part of that definition. You have to be a little careful because, when you get down to the details, you’ll discover that only multiple system processes can enable Python to run these trains of thought at literally the same time.

In contrast, threads and asynchronous tasks always run on a single processor, which means they can only run one at a time. They just cleverly find ways to take turns to speed up the overall process. Even though they don’t run different trains of thought simultaneously, they still fall under the concept of concurrency.

Note: Threads in most other programming languages often run in parallel. To learn why Python threads can’t, check out What Is the Python Global Interpreter Lock (GIL)?

If you’re curious about even more details, then you can also read about Bypassing the GIL for Parallel Processing in Python or check out the experimental free threading introduced in Python 3.13.

The way the threads, tasks, or processes take turns differs. In a multi-threaded approach, the operating system actually knows about each thread and can interrupt it at any time to start running a different thread. This mechanism is also true for processes. It’s called preemptive multitasking since the operating system can preempt your thread or process to make the switch.

Preemptive multitasking is handy in that the code in the thread doesn’t need to do anything special to make the switch. It can also be difficult because of that at any time phrase. The context switch can happen in the middle of a single Python statement, even a trivial one like x = x + 1. This is because Python statements typically consist of several low-level bytecode instructions.

On the other hand, asynchronous tasks use cooperative multitasking. The tasks must cooperate with each other by announcing when they’re ready to be switched out without the operating system’s involvement. This means that the code in the task has to change slightly to make it happen.

The benefit of doing this extra work upfront is that you always know where your task will be swapped out, making it easier to reason about the flow of execution. A task won’t be swapped out in the middle of a Python statement unless that statement is appropriately marked. You’ll see later how this can simplify parts of your design.

What Is Parallelism?

So far, you’ve looked at concurrency that happens on a single processor. What about all of those CPU cores your cool, new laptop has? How can you make use of them in Python? The answer is to execute separate processes!

A process can be thought of as almost a completely different program, though technically, it’s usually defined as a collection of resources including memory, file handles, and things like that. One way to think about it is that each process runs in its own Python interpreter.

Because they’re different processes, each of your trains of thought in a program leveraging multiprocessing can run on a different CPU core. Running on a different core means that they can actually run at the same time, which is fabulous. There are some complications that arise from doing this, but Python does a pretty good job of smoothing them over most of the time.

Read the full article at https://realpython.com/python-concurrency/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

With the release of Qt 6.8.1 and 6.5.8, we are updating our documentation to clarify Qt’s support for newly released Apple operating system versions. 

Welcome to a new issue of "This Week in KDE Apps"! Every week we cover as much as possible of what's happening in the world of KDE apps.

This week, we are continuing to prepare for the KDE Gear 24.12.0 release, with a focus on bugfixing now that we've entered the feature freeze period.

Meanwhile, and as part of the 2024 end-of-year fundraiser, you can "Adopt an App" in a symbolic effort to support your favorite KDE app. This week, we are particularly grateful to mdPlusPlus and txemaq for supporting Dolphin; mdPlusPlus, Greg Helding and Archie Lamb for Okular; Henning Lammert and Thibault Molleman for Filelight; Nithanim, Dominik Perfler, and Thibault Molleman for Spectacle; Vladimir Solomatin, Akseli Lahtinen, Haakon Johannes Tjelta Meihack, and Nithanim for Kate; Henning Lammert and Marco Ryll for Kasts; GhulDev, Anders Lund Tobias Junghans, and William Wojciechowski for Konsole; Piwix for KWrite; Gabriel Klavans for Tokodon; Matthew Lamont for Kontact; and Gabriel Karlsson for Itinerary.

Any monetary contribution, however small, will help us cover operational costs, salaries, travel expenses for contributors and in general just keep KDE bringing Free Software to the world. So consider donating today!

Getting back to all that's new in the KDE App scene, let's dig in!

Global Changes

The KIO implementation for SFTP used by many KDE applications like Dolphin, Gwenview, and many others, now correctly closes the network connection when if a fatal error occurs, which means it is now possible to reconnect immediately without having to wait a few minutes. (Harald Sitter, 24.12.0. Link)

Many apps received some small bug fixes to ensure they work correctly on Haiku OS. (Luc Schrijvers, Link 1, link 2 and many more)

Dolphin Manage your files

Dolphin now sorts more naturally when comparing filenames by excluding extensions. So now "a.txt" appears before "a 2.txt". (Eren Karakas, 24.12.0. Link)

Haruna Media player

Left-clicking on a video now plays or pauses it by default. (Nate Graham, 25.04.0. Link)

Karp KDE arranger for PDFs

Karp is a new PDF arranger and editor by Tomasz Bojczuk which has just finished the incubation phase.

Tomasz was active this week and added an option to select the PDF version of the resulting PDF (Link) and made it possible to move multiple pages at the same time (Link).

Kate Advanced Text Editor

The build plugin — which allows you to trigger a rebuild from Kate's interface — now supports multiple projects being open at the same time without having to constantly reload the list of targets every time you switch projects. (Waqar Ahmed, 25.04.0. Link)

The ctag indexing doesn't happen anymore on the root and home folder as it makes no sense and just wastes CPU cycles. (Waqar Ahmed, 24.12.0. Link)

Fix getting a PATH when launching Kate outside of the console on macOS. (Waqar Ahmed, 24.12.0. Link)

KMyMoney Personal finance manager based on double-entry bookkeeping

It is no longer possible to apply category filters on the "Net Worth report" of KMyMoney as this was resulting in erroneous results. (Thomas Baumgart, 5.2, Link)

KRDC Connect with RDP or VNC to another computer

Fix loading the gateway server address in the settings. (Fabio Bas, 24.12.0. Link)

Fix a crash when scrolling; the app was previously sending empty scroll events which were rejected by the remote server. (Fabio Bas, 24.12.0. Link)

Konqueror KDE File Manager & Web Browser

Stefano fixed various issues with the Plasma Activities integration inside Konqueror. We now, for example, wait for the Plasma Activity service to be ready, before restoring activities when starting Konqueror. (Stefano Crocco, 24.12.0. Link)

Konsole Use the command line interface

We added the Campbell color scheme from Microsoft. (Mingcong Bai, 25.04.0. Link)

We fixed a few font rendering issues in Konsole. (Matan Ziv-Av, 24.12.0. Link)

Okular View and annotate documents

Changed the default value of the "scroll overlap" feature from 0% to 10%, which means that when you scroll down in a document using Page Down or Space Bar, the bottom 10% of the previous page will remain visible at the top of the view. This helps you retain your spatial awareness when quickly navigating. (David Cerenius, 25.04.0. Link)

Merkuro Calendar Manage your tasks and events with speed and ease

Claudio fixed many issues with the day and month views. Now, clicking on a day in the month view will open the day view on the selected day and not just a random one, the current day will be correctly highlighted, some sizing issues are fixed, and the month view won't appear as disabled anymore in some situations. (Claudio Cambra, 24.12.0. Link 1, link 2 and link 3)

When double-clicking on an empty space in the month view, the incidence editor will use the selected date as its start date. (Claudio Cambra, 24.12.0. Link)

NeoChat Chat on Matrix

We fixed the sed-edit feature in NeoChat, which allows you to type a sed expression like s/foo/bar to edit your previous message. (James Graham, 24.12.0. Link)

On mobile devices, NeoChat won't open the space homepage when trying to just switch the selected space. (James Graham, 24.12.0. Link)

Implemented MSC4228: Search Redirection to harmlessly redirect searches for harmful and potentially illegal content.

OptiImage Image optimizer to reduce the size of images

It is now possible to remove an image from the list of images to optimize. (Soumyadeep Ghosh, Link)

Soumyadeep also fixed an issue where it was possible to add the same image multiple times (Soumyadeep Ghosh, Link)

Skanpage Scan multi-page documents and images

Ported the export dialog in Skanpage to use Kirigami.Dialog, giving it a nicer and more consistent appearance. (Thomas Duckworth, 25.04.0. Link)

Ported Skanpage to use KIO, which allows saving scanned documents to remote folders. (Alexander Stippich, 25.04.0. Link)

Telly Skout A convergent Kirigami TV guide

Fix the list of favorite TV channels being empty after opening a different page. (Plata Hill, 24.12.0. Link)

Tokodon Browse the Fediverse

Joshua added titles to the profile pages, so that it is not empty anymore. (Joshua Goins, 24.12.0. Link)

Quote post are now better detected. (Joshua Goins, 24.12.0. Link)

It is now possible to start a new chat from the conversation page. (Joshua Goins, 24.12.0. Link)

…And Everything Else

This blog only covers the tip of the iceberg! If you’re hungry for more, check out Nate's blog about Plasma and be sure not to miss his This Week in Plasma series, where every Saturday he covers all the work being put into KDE's Plasma desktop environment.

For a complete overview of what's going on, visit KDE's Planet, where you can find all KDE news unfiltered directly from our contributors.

Get Involved

The KDE organization has become important in the world, and your time and contributions have helped us get there. As we grow, we're going to need your support for KDE to become sustainable.

You can help KDE by becoming an active community member and getting involved. Each contributor makes a huge difference in KDE — you are not a number or a cog in a machine! You don’t have to be a programmer either. There are many things you can do: you can help hunt and confirm bugs, even maybe solve them; contribute designs for wallpapers, web pages, icons and app interfaces; translate messages and menu items into your own language; promote KDE in your local community; and a ton more things.

You can also help us by donating. Any monetary contribution, however small, will help us cover operational costs, salaries, travel expenses for contributors and in general just keep KDE bringing Free Software to the world.

To get your application mentioned here, please ping us in invent or in Matrix.

The open-source ecosystem thrives on collaboration, innovation, and inclusivity. Within this space, Drupal has consistently emerged as a beacon of robust, community-driven development. A significant contribution to this success comes from Drupal companies stepping up to sponsor Free and Open Source Software (FOSS) events, ensuring the platform's visibility and fostering its growth.

Some of them are doing more than just sponsoring events—they are cultivating spaces where Drupal can shine amidst the broader open-source community. By anchoring Drupal’s presence at these gatherings, they highlight the platform’s strengths, such as its flexibility, scalability, and vibrant community. Their efforts underscore the idea that Drupal is not just a content management system but a cornerstone of the open-source movement.

Recently, Nico Grienauer wrote an article in the Drop Times about acolono’s presence at ‘Kiss the Globe,’ an interdisciplinary event held in Vienna, Austria. Acolono was one of the event's sponsors, but it did not end there. The company offered discounted tickets to Drupal developers, enabling more contributors to attend. This approach democratizes access to knowledge-sharing opportunities, allowing developers to engage with the broader open-source world while representing Drupal. 

Such measures ensure Drupal's representation and help create a collaborative bridge between diverse technologies and communities. This cross-pollination of ideas enhances the value of open-source events and reflects the cooperative ethos that defines Drupal.

In a world where competition often overshadows collaboration, these efforts remind us of the power of community. Such initiatives deserve appreciation as they propel Drupal forward and uphold the principles of open source—accessibility, inclusivity, and innovation. 

Now, let’s look at the important stories from the past week.

In an interview with Alka Elizabeth, Martin Anderson-Clutz reflects on his experiences presenting at NEDCamp. He delves into his work under the Starshot initiative, which aims to enhance Drupal’s capabilities for event management. He highlights the evolution of Drupal recipes, explores the complexities of event-based tools, and shares developments slated for upcoming releases. He also discusses how collaboration within the Drupal community has shaped tools and practices, offering insights into the future of event management and site-building with Drupal.

For acolono, Open Source is not just a technical solution but a philosophy. It represents collaboration, innovation, and shared responsibility—values that align closely with the ethos of Kiss the Globe. By leveraging Drupal, acolono develops digital tools that are both innovative and environmentally conscious, reinforcing the agency’s belief that technology can be a driver of sustainability. Nico Grienauer shares about acolono's partnership with Kiss the Globe event and their giant push for Drupal.

Bernardo Martinez writes about how DrupalCamp Atlanta regained momentum in 2024 after one of its lead organizers stepped down last year. Earlier in the week, we had published his reflection on this year's Open-Source/DrupalCamp Chattanooga. Here's a report of the event from Bernardo Martinez.

This week, from November 25 to December 1, 2024, features several Drupal events, including the MidCamp 2025 Planning Meeting, the Event Platform Initiative Discussion, and the Dutch Splash Awards. Have a look at the Drop Times’ article on events this week.

Pamela Barone has provided updates on the progress of Drupal CMS version 1 and outlined plans for new work tracks to advance the platform further. With version 1 nearing completion, successful tracks like Project Browser and Workspaces as Content Moderation continue to drive improvements.

DrupalSouth has officially announced its committee for 2024, bringing together a group of dedicated professionals to lead the organization’s activities and initiatives. The newly appointed committee includes individuals with diverse experience, ready to advance the Drupal community in the region.

DrupalCon Singapore 2024 is set to roll within a month. The grand event that brings together Drupal enthusiasts from Asia and beyond is going to be a goldmine of networking opportunities. Have a look at the Social Events at the event.

DrupalNYC invites Drupal enthusiasts to participate in its upcoming "Contribution Day," scheduled for Friday, December 13, 2024, from 9:00 AM to 4:00 PM. The event encourages both new and seasoned contributors to collaborate on advancing the Drupal ecosystem.

DrupalCamp Poland 2025, the country's largest Drupal-focused conference, will occur on June 7 in Warsaw. Organizers have announced that the call for session proposals is now open, inviting speakers to submit their topics for consideration. 

A new podcast series was launched exclusively to deal with topics connected to LibreOffice. The series titled LibreOffice Podcast is being published on PeerTube. The first episode, published on 20 November, discussed the topic "Marketing Free Software".

James Abrahams, Director at FreelyGive. Ltd, recently shared an update on the integration of AI agents into Drupal, sparking a wide-ranging discussion among experts on LinkedIn. His insights focused on the critical role of evaluations in AI systems, as well as the challenges of creating intuitive tools for non-developers within the Drupal ecosystem.

With that let's wind up this week's newsletter.

Thomas Alias K
Sub-editor, The Drop Times

Twig & PHP Templating in Drupal 11 Editor Mon, 11/25/2024 - 10:11

Welcome here! This is your complete guide to Twig and PHP templating in Drupal 11. As you understand,  Twig and PHP are important for the frontend and backend development in Drupal. If you know how they work, you can create beautiful, effective, and easy-to-maintain websites. 
Today, Golems Drupal company explores Twig's smooth template engine and PHP's strong backend logic. Our blog will be helpful for every kind of person, whether you are a skilled Drupal developer, someone who owns a website or business owner, or simply starting your path in this field. We will dive into the details of Twig and PHP in Drupal to help you better understand how they work together so that your digital experiences can be crafted more effectively.

<strong>Topics covered in this episode:</strong><br> <ul> <li><strong><a href="https://talkpython.fm/blog/posts/talk-python-rewritten-in-quart-async-flask/?featured_on=pythonbytes">Talk Python rewritten in Quart</a></strong></li> <li><strong><a href="https://blog.pypi.org/posts/2024-11-14-pypi-now-supports-digital-attestations/?featured_on=pythonbytes">PyPI now supports digital attestations</a></strong></li> <li><strong><a href="https://github.com/LilyFoote/django-rusty-templates?featured_on=pythonbytes">Django Rusty Templates</a></strong></li> <li><a href="https://discuss.python.org/t/pep-639-round-3-improving-license-clarity-with-better-package-metadata/53020/128?featured_on=pythonbytes"><strong>PEP 639 is now supported by PYPI</strong></a></li> <li><strong>Extras</strong></li> <li><strong>Joke</strong></li> </ul><a href='https://www.youtube.com/watch?v=Qk1b0AiZCYo' style='font-weight: bold;'data-umami-event="Livestream-Past" data-umami-event-episode="411">Watch on YouTube</a><br> <p><strong>About the show</strong></p> <p>Sponsored by us! Support our work through:</p> <ul> <li>Our <a href="https://training.talkpython.fm/?featured_on=pythonbytes"><strong>courses at Talk Python Training</strong></a></li> <li><a href="https://courses.pythontest.com/p/the-complete-pytest-course?featured_on=pythonbytes"><strong>The Complete pytest Course</strong></a></li> <li><a href="https://www.patreon.com/pythonbytes"><strong>Patreon Supporters</strong></a></li> </ul> <p><strong>Connect with the hosts</strong></p> <ul> <li>Michael: <a href="https://fosstodon.org/@mkennedy"><strong>@mkennedy@fosstodon.org</strong></a> <strong>/</strong> <a href="https://bsky.app/profile/mkennedy.codes?featured_on=pythonbytes"><strong>@mkennedy.codes</strong></a></li> <li>Brian: <a href="https://fosstodon.org/@brianokken"><strong>@brianokken@fosstodon.org</strong></a> <strong>/</strong> <a href="https://bsky.app/profile/brianokken.bsky.social?featured_on=pythonbytes"><strong>@brianokken.bsky.social</strong></a></li> <li>Show: <a href="https://fosstodon.org/@pythonbytes"><strong>@pythonbytes@fosstodon.org</strong></a> <strong>/</strong> <a href="https://bsky.app/profile/pythonbytes.bsky.social"><strong>@pythonbytes.bsky.social</strong></a></li> </ul> <p>Join us on YouTube at <a href="https://pythonbytes.fm/stream/live"><strong>pythonbytes.fm/live</strong></a> to be part of the audience. Usually <strong>Monday</strong> at 10am PT. Older video versions available there too.</p> <p>Finally, if you want an artisanal, hand-crafted digest of every week of the show notes in email form? Add your name and email to <a href="https://pythonbytes.fm/friends-of-the-show">our friends of the show list</a>, we'll never share it.</p> <p><strong>Michael #1:</strong> <a href="https://talkpython.fm/blog/posts/talk-python-rewritten-in-quart-async-flask/?featured_on=pythonbytes">Talk Python rewritten in Quart</a></p> <ul> <li>Rewrote all of <a href="https://talkpython.fm/?featured_on=pythonbytes">talkpython.fm</a> in <a href="https://quart.palletsprojects.com/en/latest/?featured_on=pythonbytes">Quart</a> (10k lines of code total, 4k changed)</li> <li>Considered <ul> <li>FastAPI</li> <li>Litestar</li> <li>Django</li> <li>Hugo Static Site + Python</li> <li>Flask</li> </ul></li> <li>Discussed the multistage upgrade / conversion process</li> <li>Automating tests for all 1,000 pages</li> </ul> <p><strong>Brian #2:</strong> <a href="https://blog.pypi.org/posts/2024-11-14-pypi-now-supports-digital-attestations/?featured_on=pythonbytes">PyPI now supports digital attestations</a></p> <ul> <li>Dustin Ingram</li> <li>“<strong>Attestations provide a verifiable link to an upstream source repository:</strong> By signing with the identity of the upstream source repository, such as in the case of an upload of a project built with GitHub Actions, PyPI's support for digital attestations defines a strong and verifiable association between a file on PyPI and the source repository, workflow, and even the commit hash that produced and uploaded the file. Additionally, publishing attestations to a transparency log helps mitigate against both compromise of PyPI and compromise of the projects themselves.”</li> <li>For maintainers <ul> <li>If using GH Actions and Trusted Publishing <ul> <li>make sure you use pypa/gh-action-pypi-publish, version v1.11.0 or newer</li> <li>that’s it</li> </ul></li> <li>If not <ul> <li>“Support for automatic attestation generation and publication from other Trusted Publisher environments <a href="https://github.com/pypi/warehouse/issues/17001?featured_on=pythonbytes">is planned</a>.”</li> <li>“While not recommended, maintainers can also <a href="https://docs.pypi.org/attestations/producing-attestations/#the-manual-way">manually</a> <a href="https://docs.pypi.org/attestations/producing-attestations/#the-manual-way">generate and publish attestations</a>.”</li> </ul></li> </ul></li> <li>See also <ul> <li><a href="https://socket.dev/blog/pypi-introduces-digital-attestations?featured_on=pythonbytes">PyPI Introduces Digital Attestations to Strengthen Python Package Security</a> <ul> <li>by Sarah Gooding</li> </ul></li> <li><a href="https://trailofbits.github.io/are-we-pep740-yet/?featured_on=pythonbytes">Are we PEP 740 yet?</a></li> </ul></li> </ul> <p><strong>Michael #3:</strong> <a href="https://github.com/LilyFoote/django-rusty-templates?featured_on=pythonbytes">Django Rusty Templates</a></p> <ul> <li>by Lily Foote</li> <li>An experimental reimplementation of Django's templating language in Rust.</li> <li>Goals <ul> <li>100% compatibility of rendered output.</li> <li>Error reporting that is at least as useful as Django's errors.</li> <li>Improved performance over Django's pure Python implementation.</li> </ul></li> </ul> <p><strong>Brian #4:</strong> <a href="https://discuss.python.org/t/pep-639-round-3-improving-license-clarity-with-better-package-metadata/53020/128?featured_on=pythonbytes"><strong>PEP 639 is now supported by PYPI</strong></a></p> <ul> <li><a href="https://bsky.app/profile/snarky.ca/post/3laudpvnabc27?featured_on=pythonbytes">from Brett Cannon</a></li> <li><a href="https://peps.python.org/pep-0639/?featured_on=pythonbytes">PEP 639 – Improving License Clarity with Better Package Metadata</a></li> <li>For project metadata, use these fields: license and license-files:</li> <li><p>Examples license field</p> <pre><code>[project] license = "MIT" [project] license = "MIT AND (Apache-2.0 OR BSD-2-clause)" [project] license = "MIT OR GPL-2.0-or-later OR (FSFUL AND BSD-2-Clause)" [project] license = "LicenseRef-Proprietary" </code></pre></li> <li><p>Examples of <code>license-files</code>:</p> <pre><code>[project] license-files = ["LICEN[CS]E*", "AUTHORS*"] [project] license-files = ["licenses/LICENSE.MIT", "licenses/LICENSE.CC0"] [project] license-files = ["LICENSE.txt", "licenses/*"] [project] license-files = [] </code></pre></li> </ul> <p>Extras </p> <p>Brian:</p> <ul> <li>Playground Wisdom: Threads Beat Async/Await - interesting read from Armin Ronacher about different language abstractions around concurrency.</li> <li>PythonTest.com Discord community is now live <ul> <li>Launched last week, as of this morning we’ve got 89 members</li> <li>Anyone already a pythontest community member has received an invite</li> <li>Anyone can join through courses.pythontest.com</li> </ul></li> <li>Everything at pythontest.com is 20% off through Dec with code turkeysale2024</li> <li>“Python Testing with pytest” eBook 40% off through Dec 2, use code turkeysale2024</li> </ul> <p>Michael:</p> <ul> <li>Python 3.14.0a2 released</li> <li>Starter packs: <ul> <li>Michael’s Python people: https://bsky.app/starter-pack/mkennedy.codes/3lbdnupl26e2x </li> <li>Directory: https://blueskydirectory.com/starter-packs/all</li> </ul></li> </ul> <p>Joke: <a href="https://www.youtube.com/watch?v=atcqMWqB3hw">curl - heavy metal style</a>!</p>

SSH commands as API microservices 2024-11-25, by Dariusz Suchojad

This is a quick guide on how to turn SSH commands into a REST API service. The use-case may be remote administration of devices or equipment that does not offer a REST interface or making sure that access to SSH commands is restricted to selected external REST-based API clients only.

Python

The first thing needed is code of the service that will connect to SSH servers. Below is a service doing just that - it receives name of the command to execute and host to run in on, translating stdout and stderr of SSH commands into response documents which Zato in turn serializes to JSON.

# -*- coding: utf-8 -*- # stdlib from traceback import format_exc # Zato from zato.server.service import Service class SSHInvoker(Service): """ Accepts an SSH command to run on a remote host and returns its output to caller. """ # A list of elements that we expect on input input = 'host', 'command' # A list of elements that our responses will contain output = 'is_ok', 'cid', '-stdout', '-stderr' def handle(self): # Local aliases host = self.request.input.host command = self.request.input.command # Correlation ID is always returned self.response.payload.cid = self.cid try: # Build the full command full_command = f'ssh {host} {command}' # Run the command and collect output output = self.commands.invoke(full_command) # Assign both stdout and stderr to response self.response.payload.stdout = output.stdout self.response.payload.stderr = output.stderr except Exception: # Catch any exception and log it self.logger.warn('Exception caught (%s), e:`%s', self.cid, format_exc()) # Indicate an error self.response.payload.is_ok = False else: # Everything went fine self.response.payload.is_ok = True Dashboard

In the Zato Dashboard, let's go ahead and create an HTTP Basic Auth definition that a remote API client will authenticate against:

Now, the SSH service can be mounted on a newly created REST channel - note the security definition used and that data format is set to JSON. We can skip all the other details such as caching or rate limiting, for illustration purposes, this is not needed.

Usage

At this point, everything is ready to use. We could make it accessible to external API clients but, for testing purposes, let's simply invoke our SSH API gateway service from the command line:

$ curl "api:password@localhost:11223/api/ssh" -d \ '{"host":"localhost", "command":"uptime"}' { "is_ok": true, "cid": "27406f29c66c2ab6296bc0c0", "stdout": " 09:45:42 up 37 min, 1 user, load average: 0.14, 0.27, 0.18\n"} $ Note that, at this stage, the service should be used in trusted environments only, e.g. it will run any command that it is given on input which means that in the next iteration it could be changed to only allow commands from an allow-list, rejecting anything that is not recognized.

And this completes it - the service is deployed and made accessible via a REST channel that can be invoked using JSON. Any command can be sent to any host and their output will be returned to API callers in JSON responses.

More resources

➤ Python API integration tutorial
➤ What is an integration platform?
➤ Python Integration platform as a Service (iPaaS)
➤ What is an Enterprise Service Bus (ESB)? What is SOA?

More blog posts➤

This minor release reduces Python 3.12+ debugger overhead and improves Python code analysis.

See the change log for details.

Download Wing 10 Now: Wing Pro | Wing Personal | Wing 101 | Compare Products

What's New in Wing 10

AI Assisted Development

Wing Pro 10 takes advantage of recent advances in the capabilities of generative AI to provide powerful AI assisted development, including AI code suggestion, AI driven code refactoring, description-driven development, and AI chat. You can ask Wing to use AI to (1) implement missing code at the current input position, (2) refactor, enhance, or extend existing code by describing the changes that you want to make, (3) write new code from a description of its functionality and design, or (4) chat in order to work through understanding and making changes to code.

Examples of requests you can make include:

"Add a docstring to this method" "Create unit tests for class SearchEngine" "Add a phone number field to the Person class" "Clean up this code" "Convert this into a Python generator" "Create an RPC server that exposes all the public methods in class BuildingManager" "Change this method to wait asynchronously for data and return the result with a callback" "Rewrite this threaded code to instead run asynchronously"

Yes, really!

Your role changes to one of directing an intelligent assistant capable of completing a wide range of programming tasks in relatively short periods of time. Instead of typing out code by hand every step of the way, you are essentially directing someone else to work through the details of manageable steps in the software development process.

Read More

Support for Python 3.12, 3.13, and ARM64 Linux

Wing 10 adds support for Python 3.12 and 3.13, including (1) faster debugging with PEP 669 low impact monitoring API, (2) PEP 695 parameterized classes, functions and methods, (3) PEP 695 type statements, and (4) PEP 701 style f-strings.

Wing 10 also adds support for running Wing on ARM64 Linux systems.

Poetry Package Management

Wing Pro 10 adds support for Poetry package management in the New Project dialog and the Packages tool in the Tools menu. Poetry is an easy-to-use cross-platform dependency and package manager for Python, similar to pipenv.

Ruff Code Warnings & Reformatting

Wing Pro 10 adds support for Ruff as an external code checker in the Code Warnings tool, accessed from the Tools menu. Ruff can also be used as a code reformatter in the Source > Reformatting menu group. Ruff is an incredibly fast Python code checker that can replace or supplement flake8, pylint, pep8, and mypy.

Try Wing 10 Now!

Wing 10 is a ground-breaking new release in Wingware's Python IDE product line. Find out how Wing 10 can turbocharge your Python development by trying it today.

Downloads: Wing Pro | Wing Personal | Wing 101 | Compare Products

See Upgrading for details on upgrading from Wing 9 and earlier, and Migrating from Older Versions for a list of compatibility notes.

I've just released version 1.1.23 of plocate, almost a year after 1.1.22. The changes are mostly around the systemd unit this time, but perhaps more interestingly is that this is the first release where I don't have the majority of patches; in fact, I don't have any patches at all. All of them came from contributors, many of them through the “just do git push to send me a patch email” interface.

I guess this means that I'll need to actually start streamlining my “git am” workflow… it gets me every time. :-)

You can get download numbers for PyPI packages (or projects) from a Google BigQuery dataset. You need a Google account and credentials, and Google gives 1 TiB of free quota per month.

Each month, I have automation to fetch the download numbers for the 8,000 most popular packages over the past 30 days, and make it available as more accessible JSON and CSV files at Top PyPI Packages. This data is widely used for research in academia and industry.

However, as more packages and releases are uploaded to PyPI, and there are more and more downloads logged, the amount of billed data increases too.

This chart shows the amount of data billed per month.

At first, I was only collecting downloads data for 4,000 packages, and it was fetched for two queries: downloads over 365 days and over 30 days. But as time passed, it started using up too much quota to download data for 365 days.

So I ditched the 365-day data, and increased the 30-day data from 4,000 to 5,000 packages. Later, I checked how much quota was being used and increased from 5,000 packages to 8,000 packages.

But then I exceeded the BigQuery monthly quota of 1 TiB fetching data for July 2024.

To fetch the missing data and investigate what's going in, I started Google Cloud's 90-day, $300 (€277.46) free-trial 💸

Here's what I found!

Finding: it costs more to get data for downloads from only pip than from all installers

I use the pypinfo client to help query BigQuery. By default, it only fetches downloads for pip.

Only pip

This command gets one day's download data for the top 10 packages, for pip only:

$ pypinfo --limit 10 --days 1 "" project Served from cache: False Data processed: 58.21 GiB Data billed: 58.21 GiB Estimated cost: $0.29

Results:

project download count boto3 37,251,744 aiobotocore 16,252,824 urllib3 16,243,278 botocore 15,687,125 requests 13,271,314 s3fs 12,865,055 s3transfer 12,014,278 fsspec 11,982,305 charset-normalizer 11,684,740 certifi 11,639,584 Total 158,892,247 All installers

Adding the --all flag gets one day's download data for the top 10 packages, for all installers:

$ pypinfo --all --limit 10 --days 1 "" project Served from cache: False Data processed: 46.63 GiB Data billed: 46.63 GiB Estimated cost: $0.23 project download count boto3 39,495,624 botocore 17,281,187 urllib3 17,225,121 aiobotocore 16,430,826 requests 14,287,965 s3fs 12,958,516 charset-normalizer 12,781,405 certifi 12,647,098 setuptools 12,608,120 idna 12,510,335 Total 168,226,197

So we can see the default pip-only costs an extra 25% data processed and data billed, and costs an extra 25% in dollars.

Unsurprisingly, the actual download counts are higher for all installers. The ranking has changed a bit, but I expect we're still getting more-or-less the same packages in the top thousands of results.

Queries

It sends a query like this to BigQuery for only pip:

SELECT file.project as project, COUNT(*) as download_count, FROM `bigquery-public-data.pypi.file_downloads` WHERE timestamp BETWEEN TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL -2 DAY) AND TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL -1 DAY) AND details.installer.name = "pip" GROUP BY project ORDER BY download_count DESC LIMIT 10

And for all installers:

SELECT file.project as project, COUNT(*) as download_count, FROM `bigquery-public-data.pypi.file_downloads` WHERE timestamp BETWEEN TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL -2 DAY) AND TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL -1 DAY) GROUP BY project ORDER BY download_count DESC LIMIT 10

These queries are the same, except the default has an extra AND details.installer.name = "pip" condition. It seems reasonable it would cost more to do extra filtering work.

Installers

Let's look at the installers:

$ pypinfo --all --limit 100 --days 1 "" installer Served from cache: False Data processed: 29.49 GiB Data billed: 29.49 GiB Estimated cost: $0.15 installer name download count pip 1,121,198,711 uv 117,194,833 requests 29,828,272 poetry 23,009,454 None 8,916,745 bandersnatch 6,171,555 setuptools 1,362,797 Bazel 1,280,271 Browser 1,096,328 Nexus 593,230 Homebrew 510,247 Artifactory 69,063 pdm 62,904 OS 13,108 devpi 9,530 conda 2,272 pex 194 Total 1,311,319,514

pip still by far the most popular, and unsurprising uv is up there too, with about 10% of pip's downloads.

The others are about 25% or less of uv. A lot of them are mirroring services that we wanted to exclude before.

I think given uv's importance, and my expectation that it will continue to take a bigger share of the pie, plus especially the extra cost for filtering by just pip, means that we should switch to fetching data for all downloaders. Plus the others don't account for that much of the pie.

Finding: the number of packages doesn't affect the cost

This was the biggest surprise. Earlier I'd been increasing or decreasing the number to try and remain under quota. But it turns out it makes no difference how many packages you query!

I fetched data for just one day and all installers for different package limits: 1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000. Sample query:

SELECT file.project as project, COUNT(*) as download_count, FROM `bigquery-public-data.pypi.file_downloads` WHERE timestamp BETWEEN TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL -2 DAY) AND TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL -1 DAY) GROUP BY project ORDER BY download_count DESC LIMIT 8000

Result: Interestingly, the cost is the same for all limits (1000-8000): $0.31.

Repeating with one day but filtering for pip only:

Result: Cost increased to $0.39 but again the same for all limits.

Let's repeat with all installers, but for 30 days, and this time query in decreasing limits, in case we were only paying for incremental changes: 8000, 7000, 6000, 5000, 4000, 3000, 2000, 1000:

Result: Again, the cost is the same regardless of package limit: $4.89 per query.

Well then, let's repeat with the limit increasing by powers of ten, up to 1,000,000! This last one fetches data for all 531,022 packages on PyPI:

limit projects count estimated cost bytes billed bytes processed 1 1 0.20 43,447,746,560 43,447,720,943 10 10 0.20 43,447,746,560 43,447,720,943 100 100 0.20 43,447,746,560 43,447,720,943 1000 1,000 0.20 43,447,746,560 43,447,720,943 8000 8,000 0.20 43,447,746,560 43,447,720,943 10000 10,000 0.20 43,447,746,560 43,447,720,943 100000 100,000 0.20 43,447,746,560 43,447,720,943 1000000 531,022 0.20 43,447,746,560 43,447,720,943

Result: Again, same cost, whether for 1 package or 531,022 packages!

Finding: the number of days affects the cost

No surprise. I'd earlier noticed 365 days too took much quota, and I could continue with 30 days.

Here's the estimated cost and bytes billed (for one package, all installers) between one and 30 days (f"pypinfo --all --json --indent 0 --days {days} --limit 1 '' project"), showing a roughly linear increase:

Conclusion

• It doesn't matter how many packages I fetch data for, I might as well fetch all and make it available to everyone, depending on the size of the data file. It will make sense to still offer a smaller file with 8,000 or so packages: often you just need a large-ish yet manageable number.

• It costs more to filter for only downloads from pip, so I've switched to fetching data for all installers.

• The number of days affects the cost, so I will need to decrease this in the future to stay within quota. For example, at some point I may need to switch from 30 to 25 days, and later from 25 to 20 days.

More details from the investigation, the scripts and data files can be found at
hugovk/top-pypi-packages#36.

And let me know if you know any tricks to reduce costs!

Header photo: "The Balancing Rock, Stonehenge, Near Glen Innes, NSW" by the Royal Australian Historical Society, with no known copyright restrictions.

This year it was hard to decide, and we wanted to also show who else got nominated, because they also deserve recognition, so it took a bit longer than we expected.

The Django Software Foundation Board is pleased to announce that the 2024 Malcolm Tredinnick Memorial Prize has been awarded to Rachell Calhoun.

Rachell Calhoun is an influential figure within the Django community, well known for being cheerful and always willing to help others. She consistently empowers folks behind the scenes.

Rachell got her start in the Django community through a Django Girls Seoul event. Being an educator, she started organizing Django Girls Seoul events. Her contributions to Django Girls Seoul and Django Girls Grand Rapids exemplify her commitment to sharing knowledge, spreading Django and lifting others up. Rachell is now a trustee for Django Girls +, contributing to its mission of helping women and other underrepresented groups around the world learn programming with Django.

In 2022, Rachell co-founded Djangonaut Space, an initiative designed to support new contributors to the Django ecosystem, encouraging leadership and growth. Rachell’s willingness to help people achieve their goals and celebrate their achievements has been imprinted in Djangonaut Space’s culture. Rachell and Djangonaut Space have done a stellar job on helping people become contributors and Django community members.

Her commitment to fostering diversity and inclusion extends beyond her organizational work; she has volunteered at multiple DjangoCon US events, bringing her welcoming and inclusive spirit to the community. A long-time volunteer and speaker at DjangoCon US and DjangoCon Europe from 2016 to 2024, she has shared her expertise and insights on various topics related to Django and web development.

Rachell has contributed to Django for many years, she has been instrumental in creating spaces where people of all backgrounds can thrive, making her a beloved and respected member of the global Django ecosystem.

Some quotes from the thirteen people who nominated Rachell had this to say about her:

Rachell advocates for others constantly through sponsorship, inclusivity, and connection. She is extremely empathic and seeks to not only welcome others in, but to actively bring them into the group.

She has been one of the core members of Djangonaut Space which has done a lot for bringing new contributors into the Django community. This program has done a lot to excite and energize the Django community and Rachell is one of the major reasons why. --
Throughout her career she's been involved in Django Girls starting about a decade ago in South Korea. She was a major organizer of the Grand Rapids, MI branch, before moving into the trustee role she occupies now.

Rachell is one of my favorite people and she's been doing an excellent job at growing Django and helping others feel more welcome here. Rachell is an excellent choice for the Malcolm Tredinnick 2024 award!

— Tim Shilling

Rachell is an extremely skillful leader who is always nurturing newcomers into leaders. She has been pivotal to my experience with the Djangonaut Space Program.
I started out as a nervous Djangonaut who didn’t schedule my 1:1s until Rachell checked in with me and made sure I knew the program was a safe space to discuss anything.

When I joined the program organizers as a Navigator Coordinator, I was initially much more of a follower. Rachell knew to step back while continuing to provide her support, so I could step into the leadership role and get my job done.

Rachell shows people that she believes in them. She does this in a friendly, gentle, and encouraging manner. She never forces anyone to make decisions that they don’t feel comfortable with. The community is really lucky to have Rachell.

— Lilian

Rachell Calhoun, one of the organizers and founders of Djangonaut Space, has been an open, supportive, and educational help on my Django journey. Her contributions to the Djangonaut Space program are invaluable—a program I hold quite dearly as a cornerstone of my technical interactions and growth.

Rachell's ideals of nurturing and guiding have shone through the program, for which I am grateful. Encouraging wonderful conversations, organizing and fostering mentorship, and being a great person!

I believe Rachell is an embodiment of the Malcolm Tredinnick spirit and am confident that should she win the prize, she would go on to create more impact for the Django community and the world at large.

— Emmanuel Katchy

Other nominations for this year included:

Anna Makarudze, Fundraising Coordinator at Django Girls+ Foundation, chair of the first DjangoCon Africa, previously served the DSF board as president.

Benjamin Balder Bach, chair of the DSF social media working group, organizer of Django Day Copenhagen for many years.

Black Python Devs, community founded by Jay Miller, to increase diversity and inclusion of typically underrepresented people.

Bhuvnesh Sharma, co-chair of the DSF social media working group, and co-founded and organized Django India.

Carlton Gibson, previously a Django fellow, co-host of Django Chat, volunteers in DjangoCon Europe and provides useful advice in forum and discord.

Christoph Bulter, active helper of the official and unofficial Django Discord.

Django Girls+, a non-profit organization and a community that empowers and helps women to organize free, one-day programming workshops by providing tools, resources and support.

Django Discord moderators and helpers, which are moderating the discord and provide help to keep the place welcoming and inclusive to everyone.

Daniel Moran, active contributor in various open-source projects, including django-tasks-scheduler. He is an administrator of the Django Commons organization.

Ester Beltrami, PyCon Italia and Django London organizer, is also a volunteer and a speaker in events such as EuroPython or DjangoCon Europe.

Felipe de Morais, co-founder of AfroPython, participant of Djangonaut Space program, organized and advised multiple Django Girls workshops across Brazil and Chile.

Jake Howard, speaker and contributor to Django, known for his work around background tasks.

Matt Westcott, frequent speaker and lead the development of Wagtail.

Russel Keith-Magee, python core contributor and previously Django core contributor and also served in the DSF board as President.

Ryan Cheley, django contributor and mentor (navigator) in Djangonaut Space program.

Simon Charette, long-time django contributor, previously member of the Django 5.x steering council

Sheena O’Connell, frequent speaker and DjangoCon Africa organizer.

Tom Carrick, Django Accessibility team creator and member, django contributor for many years and mentor (navigator) in Djangonaut Space program.

Tim Schilling, DEFNA secretary, DjangoCon Us organizer and co-founder of Djangonaut Space.

Will Vincent, former board member of the DSF, co-host of Django Chat and co-writer of Django News.

Each year we receive many nominations, and it is always hard to pick the winner. This year, as always, we received many nominations for the Malcolm Tredinnick Memorial Prize with some being nominated multiple times. Some have been nominated in multiple years. If your nominee didn’t make it this year, you can always nominate them again next year.

Malcolm would be very proud of the legacy he has fostered in our community!

Congratulations Rachell on the well-deserved honor!

A lot has happened with respect to the Hurd since our Childhurds and GNU/Hurd Substitutes post. As long as two years ago some of you have been asking for a progress update and although there have been rumours on a new blog post for over a year, we were kind of waiting for the rumoured x86_64 support.

With all the exciting progress on the Hurd coming available after the recent (last?) merger of core-updates we thought it better not to wait any longer. So here is a short overview of our Hurd work over the past years:

• Update Hurd to 3ff7053, gnumach 1.8+git20220827, and fix build failures,

• A native compilation fix for gcc-boot0,

• Initial rumpdisk support, more on this below, which needed to wait for:

• A libc specific to Hurd, updating gnumach to 1.8+git20221224 and hurd to 0.9.git20230216,

• Some 40 native package build fixes for the Hurd so that all development dependencies of the guix package are now available,

• A hack to use Git source in commencement to update and fix cross build and native build for the Hurd,

• Support for buiding guix natively on the Hurd by splitting the build into more steps for 32-bit hosts

• Even nicer offloading support for Childhurds by introducing Smart Hurdloading so that now both the Bordeaux and Berlin build farms build packages for i586-gnu,

• Locale fixes for wrong glibc-utf8-locales package used on GNU/Hurd,

• More locale fixes to use glibc-utf8-locales/hurd in %standard-patch-inputs,

• And even more locale fixes for using the right locales on GNU/Hurd,

• A new glibc 2.38 allowing us to do (define-public glibc/hurd glibc)—i.e., once again use the same glibc for Linux and Hurd alike, and: Better Hurd support!,

• Creation of hurd-team branch with build fixes, updating gnumach to 1.8+git20230410 and hurd to 0.9.git20231217,

• A constructive meeting with sixteen people during the Guix Days just before FOSDEM '24 with notes that contain some nice ideas,

• Another new glibc 2.39; even better Hurd support, opening the door to x86_64 support,

• Yet another restoring of i586-gnu (32-bit GNU/Hurd) support,

• The installer just learnt about the Hurd! More on this below, and finally,

• Another set of updates: gnumach (1.8+git20240714), mig (1.8+git20231217), hurd (0.9.git20240714), netdde (c0ef248d), rumpkernel (f1ffd640), and initial support for x86_64-gnu, aka the 64bit Hurd.

NetDDE and Rumpdisk support

Back in 2020, Ricardo Wurmus added the NetDDE package that provides Linux 2.6 network drivers. At the time we didn't get to integrate and use it though and meanwhile it bitrotted.

After we resurrected the NetDDE build, and with kind help of the Hurd developers we finally managed to support NetDDE for the Hurd.. This allows the usage of the Intel 82573L Gigabit Ethernet Controller of the Thinkpad X60 (and many other network cards, possibly even WIFI). Instead of using the builtin kernel driver in GNU Mach, it would be running as a userland driver.

What sparked this development was upstream's NetBSD rumpdisk support that would allow using modern hard disks such as SSDs, again running as a userland driver. Hard disk support builtin in GNU Mach was once considered to be a nice hack but it only supported disks up to 128 GiB…

First, we needed to fix the cross build on Guix.

After the initial attempt at rumpdisk support for the Hurd it took (v2) some (v3) work (v4) to finally arrive at rumpdisk support for the Hurd, really, *really* (v5)

Sadly when actually using them, booting hangs:

start: pci.arbiter:

What did not really help is that upstream's rumpkernel archive was ridiculously large. We managed to work with upstream to remove unused bits from the archive. Upstream created a new archive that instead of 1.8 GiB (!) now “only” weighs 670 MiB.

Anyway, after a lot of building, rebuilding, and debugging and some more with kind help from upstream we finally got Rumpdisk and NetDDE to run in a Childhurd.

Initial Guix/Hurd on the Thinkpad X60

Now that the last (!) core-updates merge has finally happened (thanks everyone!), the recipe of installing Guix/Hurd has been much simpfilied. It goes something along these lines.

1. Install Guix/Linux on your X60,

2. Reserve a partition and format it for the Hurd:

   mke2fs -o hurd -L hurd /dev/sdaX

3. In your config.scm, add some code to add GRUB menuentries for booting the Hurd, and mount the Hurd partition under /hurd:

   (use-modules (srfi srfi-26) (ice-9 match) (ice-9 rdelim) (ice-9 regex) (gnu build file-systems)) (define %hurd-menuentry-regex "menuentry \"(GNU with the Hurd[^{\"]*)\".*multiboot ([^ \n]*) +([^\n]*)") (define (text->hurd-menuentry text) (let* ((m (string-match %hurd-menuentry-regex text)) (label (match:substring m 1)) (kernel (match:substring m 2)) (arguments (match:substring m 3)) (arguments (string-split arguments #\space)) (root (find (cute string-prefix? "root=" <>) arguments)) (device-spec (match (string-split root #\=) (("root" device) device))) (device (hurd-device-name->device-name device-spec)) (modules (list-matches "module ([^\n]*)" text)) (modules (map (cute match:substring <> 1) modules)) (modules (map (cute string-split <> #\space) modules))) (menu-entry (label label) (device device) (multiboot-kernel kernel) (multiboot-arguments arguments) (multiboot-modules modules)))) (define %hurd-menuentries-regex "menuentry \"(GNU with the Hurd[^{\"]*)\" \\{([^}]|[^\n]\\})*\n\\}") (define (grub.cfg->hurd-menuentries grub.cfg) (let* ((entries (list-matches %hurd-menuentries-regex grub.cfg)) (entries (map (cute match:substring <> 0) entries))) (map text->hurd-menuentry entries))) (define (hurd-menuentries) (let ((grub.cfg (with-input-from-file "/hurd/boot/grub/grub.cfg" read-string))) (grub.cfg->hurd-menuentries grub.cfg))) ... (operating-system ... (bootloader (bootloader-configuration (bootloader grub-bootloader) (targets '("/dev/sda")) (menu-entries (hurd-menuentries)))) (file-systems (cons* (file-system (device (file-system-label "guix")) (mount-point "/") (type "ext4")) (file-system (device (file-system-label "hurd")) (mount-point "/hurd") (type "ext2")) %base-file-systems)) ...)

4. Create a config.scm for your Hurd system. You can get inspiration from bare-hurd.tmpl and inherit from %hurd-default-operating-system. Use grub-minimal-bootloader and add a static-networking-service-type. Something like:

   (use-modules (srfi srfi-1) (ice-9 match)) (use-modules (gnu) (gnu system hurd)) (operating-system (inherit %hurd-default-operating-system) (bootloader (bootloader-configuration (bootloader grub-minimal-bootloader) (targets '("/dev/sda")))) (kernel-arguments '("noide")) ... (services (cons* (service static-networking-service-type (list %loopback-static-networking (static-networking (addresses (list (network-address (device "eth0") (value "192.168.178.37/24")))) (routes (list (network-route (destination "default") (gateway "192.168.178.1")))) (requirement '()) (provision '(networking)) (name-servers '("192.168.178.1"))))) ...)))

5. Install the Hurd. Assuming you have an ext2 filesystem mounted on /hurd, do something like:

   guix system build --target=i586-pc-gnu vuurvlieg.hurd --verbosity=1 sudo -E guix system init --target=i586-pc-gnu --skip-checks \ vuurvlieg.hurd /hurd sudo -E guix system reconfigure vuurvlieg.scm

6. Reboot and...

Hurray!

We now have Guix/Hurd running on Thinkpad.

Guix/Hurd on Real Iron

While the initial manual install on the X60 was an inspiring milestone, we can do better. As mentioned above, just recently the installer learnt about the Hurd, right after some smaller problems were addressed, like guix system init creating essential devices for the Hurd, not attempting to run a cross-built grub-install to install Grub, soft-coding the hard-coded part:1:device:wd0 root file-system, adding support for booting Guix/Hurd more than once.

To install Guix/Hurd, first, build a 32bit installation image and copy it to a USB stick:

guix system image --image-type=iso9660 --system=i686-linux gnu/system/install.scm dd if=/gnu/store/cabba9e-image.iso of=/dev/sdX status=progress sync

then boot it on a not-too-new machine that has wired internet (although installation over WIFI is possible, there is currently no WIFI support for the installed Hurd to use it). On the new Kernel page:

choose Hurd. Do not choose a desktop environment, that's not available yet. On the Network management page:

choose the new Static networking service. In the final Configuration file step, don't forget to edit:

and fill-in your IP and GATEWAY:

You may want to add some additional packages such as git-minimal from (gnu packages version-control) and sqlite from (gnu packages sqlite).

If you also intend to do Guix development on the Hurd—e.g., debugging and fixing native package builds—then you might want to include all dependencies to build the guix package, see the devel-hurd.tmpl for inspiration on how to do that. Note that any package you add must already have support for cross-building.

Good luck, and let us know if it works for you and on what kind of machine, or why it didn't!

What's next?

In an earlier post we tried to answer the question “Why bother with the Hurd anyway?” An obvious question because it is all too easy to get discouraged, to downplay or underestimate the potential social impact of GNU and the Hurd.

The most pressing problem currently is that the guix-daemon fails when invoking guix authenticate on the Hurd, which means that we cannot easily keep our substitutes up to date. guix pull is known to work but only by pulling from a local branch doing something like:

mkdir -p ~/src/guix cd src/guix git clone https://git.savannah.gnu.org/git/guix.git master guix pull --url=~/src/guix/master

kinda like we did it in the old days. Sometimes it seems that guix does not grok the sqlite store database. This is needs to be resolved but as sqlite does read it this can be easily worked around by doing something like:

mv /var/guix/db/db.sqlite /var/guix/db/db.sqlite.orig sqlite3 /var/guix/db/db.sqlite.orig .dump > /var/guix/db/db.sqlite.dump sqlite3 -init /var/guix/db/db.sqlite.dump /var/guix/db/db.sqlite .quit

Also, the boot process still fails to handle an unclean root file system. Whenever the Hurd has suffered an unclean shutdown, cleaning it must currently be done manually, e.g., by booting from the installer USB stick.

Upstream now has decent support for 64bit (x86_64) albeit with more bugs and fewer packages. As mentioned in the overview we are now working to have that in Guix and have made some progress:

more on that in another post. Other interesting task for Guix include:

• Have guix system reconfigure work on the Hurd,
• Figure out WiFi support with NetDDE (and add it to installer!),
• An isolated build environment (or better wait for, err, contribute to the Guile guix-daemon?),
• An installer running the Hurd, and,
• Packages, packages, packages!

We tried to make Hurd development as easy and as pleasant as we could. As you have seen, things start to work pretty nicely and there is still plenty of work to do in Guix. In a way this is “merely packaging” the amazing work of others. Some of the real work that needs to be done and which is being discussed and is in progress right now includes:

• Audio support (this is sponsored by NLnet, thanks!),
• Rumpnet,
• SMP,
• AArch64.

All these tasks look daunting, and indeed that’s a lot of work ahead. But the development environment is certainly an advantage. Take an example: surely anyone who’s hacked on device drivers or file systems before would have loved to be able to GDB into the code, restart it, add breakpoints and so on—that’s exactly the experience that the Hurd offers. As for Guix, it will make it easy to test changes to the micro-kernel and to the Hurd servers, and that too has the potential to speed up development and make it a very nice experience.

Join #guix and #hurd on libera.chat or the mailing lists and get involved!

The DjangoCon Europe 2026 call for organizers is now over. We’re elated to report we received three viable proposals, a clear improvement over recent years.

We’ll let the successful team decide when and how to make their announcement, but in the meantime – thank you to everyone who took part in this process ❤️ We’re elated to have such a strong community in Europe. And for now, look forward to DjangoCon Europe 2025 in Dublin, Ireland! 🍀

What about 2027?

We’re not ready to plan that yet, but if you’re interested in organizing – take a moment to add your name and email to our DjangoCon Europe 2027 expression of interest form. We’ll make sure to reach out once the time is right.

In Python, the range() function generates a sequence of numbers, often used in loops for iteration. By default, it creates numbers starting from 0 up to but not including a specified stop value. You can also reverse the sequence with reversed(). If you need to count backwards, then you can use a negative step, like range(start, stop, -1), which counts down from start to stop.

The range() function is not just about iterating over numbers. It can also be used in various programming scenarios beyond simple loops. By mastering range(), you can write more efficient and readable Python code. Explore how range() can simplify your code and when alternatives might be more appropriate.

By the end of this tutorial, you’ll understand that:

• A range in Python is an object representing an interval of integers, often used for looping.
• The range() function can be used to generate sequences of numbers that can be converted to lists.
• for i in range(5) is a loop that iterates over the numbers from 0 to 4, inclusive.
• The range parameters start, stop, and step define where the sequence begins, ends, and the interval between numbers.
• Ranges can go backward in Python by using a negative step value and reversed by using reversed().

A range is a Python object that represents an interval of integers. Usually, the numbers are consecutive, but you can also specify that you want to space them out. You can create ranges by calling range() with one, two, or three arguments, as the following examples show:

Python >>> list(range(5)) [0, 1, 2, 3, 4] >>> list(range(1, 7)) [1, 2, 3, 4, 5, 6] >>> list(range(1, 20, 2)) [1, 3, 5, 7, 9, 11, 13, 15, 17, 19] Copied!

In each example, you use list() to explicitly list the individual elements of each range. You’ll study these examples in more detail later on.

A range can be an effective tool. However, throughout this tutorial, you’ll also explore alternatives that may work better in some situations. You can click the link below to download the code that you’ll see in this tutorial:

Get Your Code: Click here to download the free sample code that shows you how to represent numerical ranges in Python.

Construct Numerical Ranges

In Python, range() is built in. This means that you can always call range() without doing any preparations first. Calling range() constructs a range object that you can put to use. Later, you’ll see practical examples of how to use range objects.

You can provide range() with one, two, or three integer arguments. This corresponds to three different use cases:

1. Ranges counting from zero
2. Ranges of consecutive numbers
3. Ranges stepping over numbers

You’ll learn how to use each of these next.

Count From Zero

When you call range() with one argument, you create a range that counts from zero and up to, but not including, the number you provided:

Python >>> range(5) range(0, 5) Copied!

Here, you’ve created a range from zero to five. To see the individual elements in the range, you can use list() to convert the range to a list:

Python >>> list(range(5)) [0, 1, 2, 3, 4] Copied!

Inspecting range(5) shows that it contains the numbers zero, one, two, three, and four. Five itself is not a part of the range. One nice property of these ranges is that the argument, 5 in this case, is the same as the number of elements in the range.

Count From Start to Stop

You can call range() with two arguments. The first value will be the start of the range. As before, the range will count up to, but not include, the second value:

Python >>> range(1, 7) range(1, 7) Copied!

The representation of a range object just shows you the arguments that you provided, so it’s not super helpful in this case. You can use list() to inspect the individual elements:

Python >>> list(range(1, 7)) [1, 2, 3, 4, 5, 6] Copied! Read the full article at https://realpython.com/python-range/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python string concatenation is a fundamental operation that combines multiple strings into a single string. In Python, you can concatenate strings using the + operator or the += operator for appending. For more efficient concatenation of multiple strings, the .join() method is recommended, especially when working with strings in a list. Other techniques include using StringIO for large datasets or the print() function for quick screen outputs.

By the end of this tutorial, you’ll understand that:

• You can concatenate strings in Python using the + operator and the += operator.
• You can use += to append a string to an existing string.
• The .join() method is used to combine strings in a list in Python.
• You can handle a stream of strings efficiently by using StringIO as a container with a file-like interface.

To get the most out of this tutorial, you should have a basic understanding of Python, especially its built-in string data type.

Get Your Code: Click here to download the free sample code that shows you how to efficiently concatenate strings in Python.

Doing String Concatenation With Python’s Plus Operator (+)

String concatenation is a pretty common operation consisting of joining two or more strings together end to end to build a final string. Perhaps the quickest way to achieve concatenation is to take two separate strings and combine them with the plus operator (+), which is known as the concatenation operator in this context:

Python >>> "Hello, " + "Pythonista!" 'Hello, Pythonista!' >>> head = "String Concatenation " >>> tail = "is Fun in Python!" >>> head + tail 'String Concatenation is Fun in Python!' Copied!

Using the concatenation operator to join two strings provides a quick solution for concatenating only a few strings.

For a more realistic example, say you have an output line that will print an informative message based on specific criteria. The beginning of the message might always be the same. However, the end of the message will vary depending on different criteria. In this situation, you can take advantage of the concatenation operator:

Python >>> def age_group(age): ... if 0 <= age <= 9: ... result = "a Child!" ... elif 9 < age <= 18: ... result = "an Adolescent!" ... elif 19 < age <= 65: ... result = "an Adult!" ... else: ... result = "in your Golden Years!" ... print("You are " + result) ... >>> age_group(29) You are an Adult! >>> age_group(14) You are an Adolescent! >>> age_group(68) You are in your Golden Years! Copied!

In the above example, age_group() prints a final message constructed with a common prefix and the string resulting from the conditional statement. In this type of use case, the plus operator is your best option for quick string concatenation in Python.

The concatenation operator has an augmented version that provides a shortcut for concatenating two strings together. The augmented concatenation operator (+=) has the following syntax:

Python string += other_string Copied!

This expression will concatenate the content of string with the content of other_string. It’s equivalent to saying string = string + other_string.

Here’s a short example of how the augmented concatenation operator works in practice:

Python >>> word = "Py" >>> word += "tho" >>> word += "nis" >>> word += "ta" >>> word 'Pythonista' Copied!

In this example, every augmented assignment adds a new syllable to the final word using the += operator. This concatenation technique can be useful when you have several strings in a list or any other iterable and want to concatenate them in a for loop:

Python >>> def concatenate(iterable, sep=" "): ... sentence = iterable[0] ... for word in iterable[1:]: ... sentence += (sep + word) ... return sentence ... >>> concatenate(["Hello,", "World!", "I", "am", "a", "Pythonista!"]) 'Hello, World! I am a Pythonista!' Copied!

Inside the loop, you use the augmented concatenation operator to quickly concatenate several strings in a loop. Later you’ll learn about .join(), which is an even better way to concatenate a list of strings.

Python’s concatenation operators can only concatenate string objects. If you use them with a different data type, then you get a TypeError:

Python >>> "The result is: " + 42 Traceback (most recent call last): ... TypeError: can only concatenate str (not "int") to str >>> "Your favorite fruits are: " + ["apple", "grape"] Traceback (most recent call last): ... TypeError: can only concatenate str (not "list") to str Copied!

The concatenation operators don’t accept operands of different types. They only concatenate strings. A work-around to this issue is to explicitly use the built-in str() function to convert the target object into its string representation before running the actual concatenation:

Python >>> "The result is: " + str(42) 'The result is: 42' Copied!

By calling str() with your integer number as an argument, you’re retrieving the string representation of 42, which you can then concatenate to the initial string because both are now string objects.

Read the full article at https://realpython.com/python-string-concatenation/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

After the criticism in the last post about the limitations of KUserFeedback (KUF) for doing data-driven UX work — let’s get more detailed and constructive:

What insights do we as KDE UX people need to do even better than we are currently doing?

Let us start with what we already get from KUF. We get usage data, like how many people are using Wayland vs. X11. But we only get usage data according to our telemetry policy. So we do not get any deeper insight into how users configure their sessions when using Wayland compared to X11. But this is the kind of information we would need to do proper data-driven UX. What settings are users changing? How many users have icons on their desktop, and which ones? Are people manually mounting network drives? Which System Tray icons are interacted with the most? And so on.

But while this information is already impossible to gather with our current approach, we’re only scratching the surface. We need even deeper UX insights, like understanding where people click. And where they click next (in terms of Markov chains). That way we can understand if people are using Plasma the way we intended when we designed it. Or, how long does it take them to get from point A to point B? Are they taking detours because we’ve laid out paths that users don’t understand in the way we intended?

None of these questions can be answered with our current approach to telemetry.

The basic problem is that we currently send all the raw data to the KDE servers to get the answers we need. And the data we need to collect in order to get the above described desired user insights could of course be used to “identify a specific user” – which is not allowed by our telemetry policy for good reason.

And yet we need even more data. We want to target all users, or only users who exhibit certain behaviors. We want them to fill out questionnaires to better understand why they behave the way they do, to understand their goals and intentions. This would be extremely helpful in understanding bug reports. Or to support our design discussions with relevant data from real users.

All of this can only be achieved with a fundamental change in the way we do telemetry.

Existing alternatives, such as the opt-out Endless OS metrics system, also do not allow enough user insights and share the problem that the data leaves the property of the data owners, the users. That is why we have been working on the privact ecosystem, which allows all the insights described above, while fully preserving users’ privacy. And because of that, we can not only ask for more intimate data, but we can also make participation opt-out and so get data from substantially more people. And why is that? Because with the privact ecosystem, there is no technical possibility that any individual’s personal data can ever be shared remotely. Never. But it would finally enable good user-data-driven UX work. For the sake of KDE and our users.

Please also join the discussion about this issue on invent.kde.org.