Feeds

Review: To Each This World, by Julie E. Czerneda

Publisher: DAW Copyright: November 2022 ISBN: 0-7564-1543-8 Format: Kindle Pages: 676

To Each This World is a standalone science fiction novel.

Henry m'Yama t'Nowak is the Arbiter of New Earth. This is somewhat akin to a president, but only in very specific ways. Henry's job is to deal with the Kmet.

New Earth was settled by slower-than-light colony ship from old Earth, our Earth. It is, so far as they know, the last of humanity in the universe. Origin Earth fell silent hundreds of years previous, before the colonists even landed. New Earth is now a carefully and thoughtfully managed world where humans survived, thrived, and at one point sent out six slower-than-light colony ships of its own. All were feared lost after a rushed launch due to a solar storm.

As this story opens, a probe from one of those ships arrives.

This is cause for rejoicing, but there are two small problems. The first is that the culture of New Earth has changed drastically since the days when they launched the Halcyon colony ships. New Earth is now part of the Duality, a new alliance with aliens painstakingly negotiated after their portal appeared in orbit. The Kmet were peaceful, eager to form an alliance and offer new technology, although they struggled with concepts such as individuality and insisted on interacting only with the Arbiter. Their technological gifts and the apparent loss of the Halcyon colony ships refocused New Earth on safety and caution. This unexpected message is a somewhat tricky political problem, a reminder of the path not taken.

The other small problem is that the reaction of the Kmet to this message is... dramatic.

This book has several problems, but the most serious is that it is simply too long. If you have read any other Czerneda novels, you know that she tends towards sprawling world-building, but usually there are enough twists and turns in the plot to keep the story moving while the protagonists slowly puzzle out the scientific mysteries. To Each This World is not sufficiently twisty for 676 pages. I think you could have cut half the novel without losing any major plot points.

The interesting parts of this book, to me, were figuring out what's going on with the Kmet, some of the political tensions within the New Earth government, and understanding what Henry and Pilot Killian's story had to do with the apparently-unrelated but intriguing interludes following Beth Seeker in a strange place called Doublet. All that stuff is in here, but it's alongside a whole lot of Henry wrestling with lifeboat ethics in situations where he thinks he needs to lie to and manipulate people for their own good. We also get several extended tours of societies that, while vaguely interesting in a science fiction world-building way, have essentially nothing to do with the plot.

We also get a whole lot of Henry's eagerly helpful AI polymorph Flip. I wanted to like this character, and I occasionally managed, but I felt like there was a constant mismatch between, in hindsight, how Czerneda meant for me to see Flip and what I thought she was signaling while I was reading. I wanted Flip to either be a fascinatingly weird companion or to be directly relevant to the plot, but instead there were hundreds of pages of unnerving creepiness mixed with obsequiousness and emotional neediness, all of which I think I read more into than Czerneda had intended. The overall experience was more exhausting than fun.

The core of the plot is solid, and if you like SF novels built around world-building and scientific mysteries, there's a lot here to enjoy. I think Czerneda's Species Imperative series (starting with Survival) is a better execution of some of the same ideas, but I liked that series a lot and was willing to read another take on it. Czerneda is one of the SF writers who takes biology seriously and is willing to write very alien aliens, and that leads to a few satisfying twists. Also, Beth Seeker is a great character (I wish we'd seen more of her), and Killian, while a bit generic, is a serviceable protagonist when Czerneda needs someone to go poke things with a stick.

Henry... I'm not sure what I think of Henry, and your enjoyment of this book may depend on how much you click with him.

Henry is a diplomat and an extrovert. His greatest joy and talent is talking to people, navigating political situations, and negotiating. Science fiction is full of protagonists who should be this character, but they rarely are this character, probably because a lot of writers are introverts. I think Czerneda deserves real credit for making her charismatic politician sufficiently accurate that his thought processes occasionally felt alien. For me, Henry was easiest to appreciate when Killian was the viewpoint protagonist and I could look him through someone else's eyes, but Henry's viewpoint mostly worked as well. There's a lot of competence porn enjoyment in watching him do his thing.

The problem for me is that I thought several of his actions were unforgivably unethical, but no one in the book who matters seems to agree. I can see why he reached those unethical decisions, but they were profound violations of consent. He directly lies to people because he thinks telling the truth would be too risky and not get them to do what he wants them to do, and Czerneda sets up the story to imply that he might be right.

This is not necessarily a bad choice in a novel, but the author has to do some work to bring me along, and Czerneda didn't do enough of that work. I kept wanting there to be some twist or sting or complication that forced Henry to come to terms with what he was doing, but it never happens. He has to pick between two moral principles that I consider rather finely balanced, if not tilted in the opposite direction that he does, and he treats one principle as inviolable and the other as mostly unimportant. The plans he makes on that basis work fine, and those on the other side of that decision are never heard from again. It left a bad taste in my mouth, particularly given how much of the book is built around Henry making tough, tricky decisions under pressure.

I don't know about this book. I have a lot of mixed feelings. Parts of it I quite enjoyed. Parts of it I mostly enjoyed but wish were much less dragged out. Parts of it frustrated or bored me. It's one of those books where the more I thought about it after reading it, the more the parts I disliked annoyed me.

If you like Czerneda's style of world-building and biology, and if you have more tolerance for Henry's decisions than I did, you may well like this, but read Species Imperative first. I should probably also warn that there is a lot of magical technology in this book that blatantly violates some core principles of physics. I have a high tolerance for that sort of thing, but if you don't, you're going to be grumbling.

Rating: 6 out of 10

Me: “Hi everyone, my name’s Matt, and I’m a mediocre programmer.”

Everyone: “Hi, Matt.”

Facilitator: “Are you an alcoholic, Matt?”

Me: “No, not since I stopped reading Twitter.”

Facilitator: “Then I think you’re in the wrong room.”

Yep, that’s my little secret – I’m a mediocre programmer. The definition of the word “hacker” I most closely align with is “someone who makes furniture with an axe”. I write simple, straightforward code because trying to understand complexity makes my head hurt.

Which is why I’ve always avoided the more “academic” languages, like OCaml, Haskell, Clojure, and so on. I know they’re good languages – people far smarter than me are building amazing things with them – but the time I hear the word “endofunctor”, I’ve lost all focus (and most of my will to live). My preferred languages are the ones that come with less intellectual overhead, like C, PHP, Python, and Ruby.

So it’s interesting that I’ve embraced Rust with significant vigour. It’s by far the most “complicated” language that I feel at least vaguely comfortable with using “in anger”. Part of that is that I’ve managed to assemble a set of principles that allow me to almost completely avoid arguing with Rust’s dreaded borrow checker, lifetimes, and all the rest of the dark, scary corners of the language. It’s also, I think, that Rust helps me to write better software, and I can feel it helping me (almost) all of the time.

In the spirit of helping my fellow mediocre programmers to embrace Rust, I present the principles I’ve assembled so far.

Neither a Borrower Nor a Lender Be

If you know anything about Rust, you probably know about the dreaded “borrow checker”. It’s the thing that makes sure you don’t have two pieces of code trying to modify the same data at the same time, or using a value when it’s no longer valid.

While Rust’s borrowing semantics allow excellent performance without compromising safety, for us mediocre programmers it gets very complicated, very quickly. So, the moment the compiler wants to start talking about “explicit lifetimes”, I shut it up by just using “owned” values instead.

It’s not that I never borrow anything; I have some situations that I know are “borrow-safe” for the mediocre programmer (I’ll cover those later). But any time I’m not sure how things will pan out, I’ll go straight for an owned value.

For example, if I need to store some text in a struct or enum, it’s going straight into a String. I’m not going to start thinking about lifetimes and &'a str; I’ll leave that for smarter people. Similarly, if I need a list of things, it’s a Vec<T> every time – no &'b [T] in my structs, thank you very much.

Attack of the Clones

Following on from the above, I’ve come to not be afraid of .clone(). I scatter them around my code like seeds in a field. Life’s too short to spend time trying to figure out who’s borrowing what from whom, if I can just give everyone their own thing.

There are warnings in the Rust book (and everywhere else) about how a clone can be “expensive”. While it’s true that, yes, making clones of data structures consumes CPU cycles and memory, it very rarely matters. CPU cycles are (usually) plentiful and RAM (usually) relatively cheap. Mediocre programmer mental effort is expensive, and not to be spent on premature optimisation. Also, if you’re coming from most any other modern language, Rust is already giving you so much more performance that you’re probably ending up ahead of the game, even if you .clone() everything in sight.

If, by some miracle, something I write gets so popular that the “expense” of all those spurious clones becomes a problem, it might make sense to pay someone much smarter than I to figure out how to make the program a zero-copy masterpiece of efficient code. Until then… clone early and clone often, I say!

Derive Macros are Powerful Magicks

If you start .clone()ing everywhere, pretty quickly you’ll be hit with this error:

error[E0599]: no method named `clone` found for struct `Foo` in the current scope

This is because not everything can be cloned, and so if you want your thing to be cloned, you need to implement the method yourself. Well… sort of.

One of the things that I find absolutely outstanding about Rust is the “derive macro”. These allow you to put a little marker on a struct or enum, and the compiler will write a bunch of code for you! Clone is one of the available so-called “derivable traits”, so you add #[derive(Clone)] to your structs, and poof! you can .clone() to your heart’s content.

But there are other things that are commonly useful, and so I’ve got a set of traits that basically all of my data structures derive:

#[derive(Clone, Debug, Default)] struct Foo { // ... }

Every time I write a struct or enum definition, that line #[derive(Clone, Debug, Default)] goes at the top.

The Debug trait allows you to print a “debug” representation of the data structure, either with the dbg!() macro, or via the {:?} format in the format!() macro (and anywhere else that takes a format string). Being able to say “what exactly is that?” comes in handy so often, not having a Debug implementation is like programming with one arm tied behind your Aeron.

Meanwhile, the Default trait lets you create an “empty” instance of your data structure, with all of the fields set to their own default values. This only works if all the fields themselves implement Default, but a lot of standard types do, so it’s rare that you’ll define a structure that can’t have an auto-derived Default. Enums are easily handled too, you just mark one variant as the default:

#[derive(Clone, Debug, Default)] enum Bar { Something(String), SomethingElse(i32), #[default] // <== mischief managed Nothing, } Borrowing is OK, Sometimes

While I previously said that I like and usually use owned values, there are a few situations where I know I can borrow without angering the borrow checker gods, and so I’m comfortable doing it.

The first is when I need to pass a value into a function that only needs to take a little look at the value to decide what to do. For example, if I want to know whether any values in a Vec<u32> are even, I could pass in a Vec, like this:

fn main() { let numbers = vec![0u32, 1, 2, 3, 4, 5]; if has_evens(numbers) { println!("EVENS!"); } } fn has_evens(numbers: Vec<u32>) -> bool { numbers.iter().any(|n| n % 2 == 0) }

Howver, this gets ugly if I’m going to use numbers later, like this:

fn main() { let numbers = vec![0u32, 1, 2, 3, 4, 5]; if has_evens(numbers) { println!("EVENS!"); } // Compiler complains about "value borrowed here after move" println!("Sum: {}", numbers.iter().sum::<u32>()); } fn has_evens(numbers: Vec<u32>) -> bool { numbers.iter().any(|n| n % 2 == 0) }

Helpfully, the compiler will suggest I use my old standby, .clone(), to fix this problem. But I know that the borrow checker won’t have a problem with lending that Vec<u32> into has_evens() as a borrowed slice, &[u32], like this:

fn main() { let numbers = vec![0u32, 1, 2, 3, 4, 5]; if has_evens(&numbers) { println!("EVENS!"); } } fn has_evens(numbers: &[u32]) -> bool { numbers.iter().any(|n| n % 2 == 0) }

The general rule I’ve got is that if I can take advantage of lifetime elision (a fancy term meaning “the compiler can figure it out”), I’m probably OK. In less fancy terms, as long as the compiler doesn’t tell me to put 'a anywhere, I’m in the green. On the other hand, the moment the compiler starts using the words “explicit lifetime”, I nope the heck out of there and start cloning everything in sight.

Another example of using lifetime elision is when I’m returning the value of a field from a struct or enum. In that case, I can usually get away with returning a borrowed value, knowing that the caller will probably just be taking a peek at that value, and throwing it away before the struct itself goes out of scope. For example:

struct Foo { id: u32, desc: String, } impl Foo { fn description(&self) -> &str { &self.desc } }

Returning a reference from a function is practically always a mortal sin for mediocre programmers, but returning one from a struct method is often OK. In the rare case that the caller does want the reference I return to live for longer, they can always turn it into an owned value themselves, by calling .to_owned().

Avoid the String Tangle

Rust has a couple of different types for representing strings – String and &str being the ones you see most often. There are good reasons for this, however it complicates method signatures when you just want to take some sort of “bunch of text”, and don’t care so much about the messy details.

For example, let’s say we have a function that wants to see if the length of the string is even. Using the logic that since we’re just taking a peek at the value passed in, our function might take a string reference, &str, like this:

fn is_even_length(s: &str) -> bool { s.len() % 2 == 0 }

That seems to work fine, until someone wants to check a formatted string:

fn main() { // The compiler complains about "expected `&str`, found `String`" if is_even_length(format!("my string is {}", std::env::args().next().unwrap())) { println!("Even length string"); } }

Since format! returns an owned string, String, rather than a string reference, &str, we’ve got a problem. Of course, it’s straightforward to turn the String from format!() into a &str (just prefix it with an &). But as mediocre programmers, we can’t be expected to remember which sort of string all our functions take and add & wherever it’s needed, and having to fix everything when the compiler complains is tedious.

The converse can also happen: a method that wants an owned String, and we’ve got a &str (say, because we’re passing in a string literal, like "Hello, world!"). In this case, we need to use one of the plethora of available “turn this into a String” mechanisms (.to_string(), .to_owned(), String::from(), and probably a few others I’ve forgotten), on the value before we pass it in, which gets ugly real fast.

For these reasons, I never take a String or an &str as an argument. Instead, I use the Power of Traits to let callers pass in anything that is, or can be turned into, a string. Let us have some examples.

First off, if I would normally use &str as the type, I instead use impl AsRef<str>:

fn is_even_length(s: impl AsRef<str>) -> bool { s.as_ref().len() % 2 == 0 }

Note that I had to throw in an extra as_ref() call in there, but now I can call this with either a String or a &str and get an answer.

Now, if I want to be given a String (presumably because I plan on taking ownership of the value, say because I’m creating a new instance of a struct with it), I use impl Into<String> as my type:

struct Foo { id: u32, desc: String, } impl Foo { fn new(id: u32, desc: impl Into<String>) -> Self { Self { id, desc: desc.into() } } }

We have to call .into() on our desc argument, which makes the struct building a bit uglier, but I’d argue that’s a small price to pay for being able to call both Foo::new(1, "this is a thing") and Foo::new(2, format!("This is a thing named {name}")) without caring what sort of string is involved.

Always Have an Error Enum

Rust’s error handing mechanism (Results… everywhere), along with the quality-of-life sugar surrounding it (like the short-circuit operator, ?), is a delightfully ergonomic approach to error handling. To make life easy for mediocre programmers, I recommend starting every project with an Error enum, that derives thiserror::Error, and using that in every method and function that returns a Result.

How you structure your Error type from there is less cut-and-dried, but typically I’ll create a separate enum variant for each type of error I want to have a different description. With thiserror, it’s easy to then attach those descriptions:

#[derive(Clone, Debug, thiserror::Error)] enum Error { #[error("{0} caught fire")] Combustion(String), #[error("{0} exploded")] Explosion(String), }

I also implement functions to create each error variant, because that allows me to do the Into<String> trick, and can sometimes come in handy when creating errors from other places with .map_err() (more on that later). For example, the impl for the above Error would probably be:

impl Error { fn combustion(desc: impl Into<String>) -> Self { Self::Combustion(desc.into()) } fn explosion(desc: impl Into<String>) -> Self { Self::Explosion(desc.into()) } }

It’s a tedious bit of boilerplate, and you can use the thiserror-ext crate’s thiserror_ext::Construct derive macro to do the hard work for you, if you like. It, too, knows all about the Into<String> trick.

Banish map_err (well, mostly)

The newer mediocre programmer, who is just dipping their toe in the water of Rust, might write file handling code that looks like this:

fn read_u32_from_file(name: impl AsRef<str>) -> Result<u32, Error> { let mut f = File::open(name.as_ref()) .map_err(|e| Error::FileOpenError(name.as_ref().to_string(), e))?; let mut buf = vec![0u8; 30]; f.read(&mut buf) .map_err(|e| Error::ReadError(e))?; String::from_utf8(buf) .map_err(|e| Error::EncodingError(e))? .parse::<u32>() .map_err(|e| Error::ParseError(e)) }

This works great (or it probably does, I haven’t actually tested it), but there are a lot of .map_err() calls in there. They take up over half the function, in fact. With the power of the From trait and the magic of the ? operator, we can make this a lot tidier.

First off, assume we’ve written boilerplate error creation functions (or used thiserror_ext::Construct to do it for us)). That allows us to simplify the file handling portion of the function a bit:

fn read_u32_from_file(name: impl AsRef<str>) -> Result<u32, Error> { let mut f = File::open(name.as_ref()) // We've dropped the `.to_string()` out of here... .map_err(|e| Error::file_open_error(name.as_ref(), e))?; let mut buf = vec![0u8; 30]; f.read(&mut buf) // ... and the explicit parameter passing out of here .map_err(Error::read_error)?; // ...

If that latter .map_err() call looks weird, without the |e| and such, it’s passing a function-as-closure, which just saves on a few characters typing. Just because we’re mediocre, doesn’t mean we’re not also lazy.

Next, if we implement the From trait for the other two errors, we can make the string-handling lines significantly cleaner. First, the trait impl:

impl From<std::string::FromUtf8Error> for Error { fn from(e: std::string::FromUtf8Error) -> Self { Self::EncodingError(e) } } impl From<std::num::ParseIntError> for Error { fn from(e: std::num::ParseIntError) -> Self { Self::ParseError(e) } }

(Again, this is boilerplate that can be autogenerated, this time by adding a #[from] tag to the variants you want a From impl on, and thiserror will take care of it for you)

In any event, no matter how you get the From impls, once you have them, the string-handling code becomes practically error-handling-free:

Ok( String::from_utf8(buf)? .parse::<u32>()? )

The ? operator will automatically convert the error from the types returned from each method into the return error type, using From. The only tiny downside to this is that the ? at the end strips the Result, and so we’ve got to wrap the returned value in Ok() to turn it back into a Result for returning. But I think that’s a small price to pay for the removal of those .map_err() calls.

In many cases, my coding process involves just putting a ? after every call that returns a Result, and adding a new Error variant whenever the compiler complains about not being able to convert some new error type. It’s practically zero effort – outstanding outcome for the mediocre programmer.

Just Because You’re Mediocre, Doesn’t Mean You Can’t Get Better

To finish off, I’d like to point out that mediocrity doesn’t imply shoddy work, nor does it mean that you shouldn’t keep learning and improving your craft. One book that I’ve recently found extremely helpful is Effective Rust, by David Drysdale. The author has very kindly put it up to read online, but buying a (paper or ebook) copy would no doubt be appreciated.

The thing about this book, for me, is that it is very readable, even by us mediocre programmers. The sections are written in a way that really “clicked” with me. Some aspects of Rust that I’d had trouble understanding for a long time – such as lifetimes and the borrow checker, and particularly lifetime elision – actually made sense after I’d read the appropriate sections.

Finally, a Quick Beg

I’m currently subsisting on the kindness of strangers, so if you found something useful (or entertaining) in this post, why not buy me a refreshing beverage? It helps to know that people like what I’m doing, and helps keep me from having to sell my soul to a private equity firm.

Looking back at my blog, I find lots of mention of Amarok, the KDE audio player, in 2008, 2009, some KDE4-on-OpenSolaris stuff mentions it, and then a long silence until 2021. About a year ago, early 2023, the audio/amarok port was removed from FreeBSD ports. So naturally I was intrigued – maybe even excited – to see Amarok return from vacation with a 3.0 release. And I needed to try it on FreeBSD.

Many of the dependencies are (still) packaged on FreeBSD, so I installed a handful and tried building it on my still-KF5-based X11-based so last-gen Plasma Desktop workstation.

Somewhat to my surprise – I don’t imagine there is FreeBSD CI for Amarok – everything built, ninja install did the right things, and it starts! And toots and parples, clangs, bangs, blips and bloops are reproduced with excellent fidelity.

So, congratulations Amarok folk. It might even return to the FreeBSD ports collection, although – I’m gonna be honest here – I’m not sure it offers me anything in the way of music appreciation that command-line gst-play doesn’t give me. I am just that much more previous-last-gen.

#627 – APRIL 30, 2024
View in Browser »

PEP 686: Make UTF-8 Mode Default

This Python Enhancement Proposal outlines making UTF-8 the default throughout Python. This takes the addition of Unicode introduced in Python 3 to its full extent, applying it to file encoding, pipes, and more. Mechanisms for other encoding are still supported. This PEP is targeted for Python 3.15.
PEPS

What’s Lazy Evaluation in Python?

This tutorial explores lazy evaluation in Python and looks at the advantages and disadvantages of using lazy and eager evaluation methods. By the end of this tutorial, you’ll clearly understand which approach is best for you, depending on your needs.
REAL PYTHON

Build Your Own AI CLI Agent with Open Source by Pieces (OSP)

Unlock the power of Pieces, right in your terminal! Our open-source CLI plugin helps you manage code snippets, chat with your on-device AI copilot, and even auto-generate commit messages. Join our community to refine your Python skills and influence a product used by 1000s of devs. Contribute today →
PIECES sponsor

Serverless Python in 2024

Talk Python interviews Tony Sherman and they discuss the current state of serverless computing in the Python world, including some of the newer tools and best practices.
KENNEDY & SHERMAN podcast

Django Developers Survey 2023 Results

JETBRAINS

Djangonauts Space Session 2 Applications Open!

DJANGONAUTS

PyPy v7.3.16 Release

PYPY

PEP 745: Python 3.14 Release Schedule

PEPS

Quiz: Writing Unit Tests for Your Code With unittest

REAL PYTHON

Discussions High Quality Python Scripts or Small Libraries to Learn From?

HACKER NEWS

Articles & Tutorials Filter Sensitive Contents From Django’s Error Reports

Django has the ability to automatically email admins when a 500 error occurs. These kinds of errors can potentially contain sensitive information though, so there are decorators to hide these values. This post covers those as well as how to filter data when using Sentry.
GONÇALO VALÉRIO

Asyncio Coroutine Object Methods in Python

The async and await keywords that form Python’s coroutine mechanism can be used for class methods as well as the more common case of functions. This article shows you how you can use asyncio with your objects.
JASON BROWNLEE

How to Prevent Data Leakage in pandas & scikit-learn

How you impute missing values in machine learning data sets can effect the quality of your training. This article teaches you what data leakage is and what steps you should take to avoid it.
DATASCHOOL

An Open Letter Regarding the DjangoCon Europe CfP

Putting on a conference is a complex matter and an attempt to clarify how future DjangoCons in Europe would be structured has resulted in push-back. This open letter is by a Django board member explaining the situation and a hope of how to move forward.
DJANGO SOFTWARE FOUNDATION

Python Basics: Lists and Tuples

In this video course, you’ll learn about Python lists and tuples, including how to define and manipulate them in your code. By the end of the course, you’ll be ready to effectively use lists and tuples in your programming projects.
REAL PYTHON course

Don’t Lie in Interviews

This strongly worded opinion piece by Nat is in reaction to common advice given on Reddit and similar boards. Nat counters it all with “don’t lie in interviews”. Strong language warning.
NAT BENNETT

Fake Job Interviews Target Devs With New Python Backdoor

“A new campaign tracked as “Dev Popper” is targeting software developers with fake job interviews in an attempt to trick them into installing a Python remote access trojan (RAT).”
BILL TOULAS

Why You Need a “WTF Notebook”

There’s a very specific reputation Nat wants to have on a team: “Nat helps me solve my problems. Nat get things I care about done.” Keeping a WTF notebook helps him do just that.
NAT BENNETT

Write Unit Tests for Your Python Code With ChatGPT

In this tutorial, you’ll learn how to use ChatGPT to generate tests for your Python code. You’ll use the chat to create doctest, unittest, and pytest tests for your code.
REAL PYTHON

Leibniz Formula for Π in Python, JavaScript, and Ruby

This is a bare-bones, side-by-side comparison of the Leibniz formula for calculating pi in Python, JavaScript, and Ruby, along with performance measurements.
PETER BENGTSSON

Better Test Parametrisation in pytest

This “Things I’ve Learned” post discusses how to take advantage of test parameterisation in pytest.
RODRIGO GIRÃO SERRÃO

Projects & Code All Python 2023 Conference Talks Google Sheet

HH91

zpy: ZSH Helpers for Python Venvs, With Uv or Pip-Tools

GITHUB.COM/ANDYDECLEYRE

django-typescript-routes: Typescript Routes From a URL Conf

GITHUB.COM/BUTTONDOWN

pipxu: Install in Isolated Environments Using UV

GITHUB.COM/BULLETMARK

PyOptInterface: Interface for Mathematical Optimization

GITHUB.COM/METAB0T

Events Weekly Real Python Office Hours Q&A (Virtual)

May 1, 2024
REALPYTHON.COM

Canberra Python Meetup

May 2, 2024
MEETUP.COM

Sydney Python User Group (SyPy)

May 2, 2024
SYPY.ORG

TOUFU

May 4 to May 5, 2024
OHTOUFU.COM

PyDelhi User Group Meetup

May 4, 2024
MEETUP.COM

Melbourne Python Users Group, Australia

May 6, 2024
J.MP

Happy Pythoning!
This was PyCoder’s Weekly Issue #627.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

Enhancements in the Endpoints tool window, extended GitHub gists support for notebooks, and navigation and refactoring across notebooks and scripts – these are just some of the improvements you’ll find in PyCharm 2024.1.1! 

You can download the latest version from our download page or update your current version through our free Toolbox App. 

Key features JetBrains AI Assistant in PyCharm Community Edition

JetBrains AI Assistant is now available in version 2024.1.1 of PyCharm Community Edition! With features ranging from smart suggestions to code generation, now Community Edition users can also enhance their coding journey with AI Assistant.

Improvements to the Endpoints tool window

Search URLs faster and more efficiently with the improved Endpoints tool window in PyCharm 2024.1.1. Use the dedicated Endpoints tab in Search Anywhere to have all your endpoints grouped by application with their routes displayed.

Navigation and refactoring across notebooks and scripts

Enjoy navigation and refactoring between notebooks and Python scripts within a single project in PyCharm. Find declarations or usages easily, use the Rename refactoring, and have our full spectrum of code inspections at your disposal. Changes are synchronized across file types, so if you employ any of these features in a notebook, they will automatically be applied to the related script, and vice versa. 

Learn more Create gists from Jupyter notebooks

You can share Jupyter notebooks seamlessly and quickly now that PyCharm offers full support for GitHub gists. Create a gist for a single notebook or select several files in the Project tool window and create a Git repo with all of them at once.

DataFrame statistics and distribution histograms

Review essential statistics directly within DataFrame headers in both Jupyter notebooks and Python scripts. Gain instant insights into how your data is distributed via the histograms provided in the DataFrame header.

IPython config file in the console

Save time by configuring your IPython console automatically using config files. Eliminate the need to import dependencies manually every time you use the console.

Download PyCharm 2024.1.1

And that’s not all! Please visit our What’s New page to discover other improvements in PyCharm 2024.1.1. You can also check out our full release notes for all the details to ensure you don’t miss out on trying any of the enhancements.

Thank you for your continued support as we strive to improve your PyCharm experience. Please report any bugs through our issue tracker so we can take care of them as soon as possible. Connect with us on X (formerly Twitter) to share your valuable feedback on PyCharm 2024.1.1!

Authored by: Nadiia Nykolaichuk.

Join us for an interactive day devoted to Drupal in healthcare— a relaxed and friendly close to DrupalCon with learning, networking, and discussing. Whether you are in a pharmaceutical company, a state department of health, a non-profit hospital, a public health organization, or anyplace else in the broad healthcare space, there are unique needs in ensuring security, accessibility, compliance, and availability of important information and tools.

Healthcare is a crucial, dynamic space for web development today. Online communication and emerging technologies promise improved access and capabilities for patients and professionals. Crafting inspiring and useful digital experiences, however, must be built on a solid foundation of accessibility, security, and compliance. Come listen to industry leaders share their experience solving these challenges in healthcare.

Get tickets to go to DrupalCon and the Healthcare Summit!

What You Will Get by Attending

• Andy Waldrop will talk about scalable healthcare content in Drupal— with lessons from building WebMD!
• Georgiana Masgras and Jesse Meece will talk about using Drupal to makes it easy to maintain our sites.
• Jon Stewart will talk about how to leverage Drupal for web application development— the Open Web in healthcare!
• Two table talk sessions will give everybody a chance to dive deeply into key topics, including the application of large language models in healthcare and biggest takeways from DrupalCon and the summit. Bring your needs and we will embark on facilitated peer-to-peer problem solving with others who are affected and tech and healthcare industry experts.
• Our sponsors, Evolving Web and Acquia (by way of their partner Phase2), will have special presentations
• We will be closing the day with lightning talks. Invited topics include: Future of Drupal at your company (and how to improve it), multisite platforms, knowledge bases, provider directories, and the healthcare take on layout builder becoming experience builder versus paragraphs.

We will be all wrapped up by 4pm.

Attend DrupalCon and the Healthcare Summit!

Who Should Attend

Everybody interested in hearing and discussing how companies and the community are creating rich digital experiences in the healthcare space. All levels of colleagues in the pharma, medical, clinical, hospital, payers, caregivers, advocates, and healthcare professional space should go to DrupalCon and the Healthcare Summit!

COVID-19 Safer Space

The Oregon Convention Center controls all rooms per ASHREA 62.1 standards, using sensors to monitor occupancy based on CO₂ levels and bring outside air into the space accordingly. They report at the most levels at 800–1000 ppm during normal occupancy, and use MERV-13 filters for any filtration, which is appropriate for COVID-19.

Agaric will have high-quality N95 masks available to anyone who wants them and will have our own CO₂ monitors.

More about the Healthcare Industry Summit

The Healthcare Summit at the 2024 Portland, Oregon DrupalCon is organized by Jeanne Cost, George Matthes, and Pooja Jeeva. I am glad to be playing a part in coordinating this summit as well, especially given Agaric's involvement in and commitment to health and science communities.

Healthcare is a complex, regulation-filled industry that poses great challenges to building vibrant and compliant digital experiences for patients. Whether you are a pharma working towards the next product, a hospital treating someone with cancer, or an insurance company making sure a patient has access to information and proper care, there is much for you to gain from this summit. 

Hear from experts on areas of accessibility, compliant patient experiences, new regulations and standards that can affect your teams and customers, success stories, and much more.

Sponsored by: Acquia and Evolving Web

Sign up to go to the Healthcare Industry Summit at DrupalCon Portland, OR 2024 this coming May 6th through May 9th for the summit

Read more and discuss at agaric.coop.

Matplotlib is one of the most popular data visualization packages for the Python programming language. It allows you to create many different charts and graphs. This tutorial focuses on adding a “watermark” to your graph. If you need to learn the basics, you might want to check out Matplotlib—An Intro to Creating Graphs with Python.

Let’s get started!

Installing Matplotlib

If you don’t have Matplotlib on your computer, you must install it. Fortunately, you can use pip, the Python package manager utility that comes with Python.

Open up your terminal or command prompt and run the following command:

python -m pip install matplotlib

Pip will now install Matplotlib and any dependencies that Matplotlib needs to work properly. Assuming that Matplotlib installs successfully, you are good to go!

Watermarking Your Graph

Adding a watermark to a graph is a fun way to learn how to use Matplotlib. For this example, you will create a simple bar chart and then add some text. The text will be added at an angle across the graph as a watermark.

Open up your favorite Python IDE or text editor and create a new Python file. Then add the following code:

import matplotlib.pyplot as plt def bar_chart(numbers, labels, pos): fig = plt.figure(figsize=(5, 8)) plt.bar(pos, numbers, color="red") # add a watermark fig.text(1, 0.15, "Mouse vs Python", fontsize=45, color="blue", ha="right", va="bottom", alpha=0.4, rotation=25) plt.xticks(ticks=pos, labels=labels) plt.show() if __name__ == "__main__": numbers = [2, 1, 4, 6] labels = ["Electric", "Solar", "Diesel", "Unleaded"] pos = list(range(4)) bar_chart(numbers, labels, pos)

Your bar_chart() function takes in some numbers, labels and a list of positions for where the bars should be placed. You then create a figure to put your plot into. Then you create the bar chart using the list of bar positions and the numbers. You also tell the chart that you want the bars to be colored “red”.

The next step is to add a watermark. To do that, you call fig.text() which lets you add text on top of your plot. Here is a quick listing of the arguments that you need to pass in:

• x, y (the first two arguments are the x/y coordinates for the text)
• fontsize – The size of the font
• color – The color of the text
• ha – Horizontal alignment
• va – Vertical alignment
• alpha – How transparent the text should be
• rotation – How many degrees to rotate the text

The last bit of code in bar_chart() adds the ticks and labels to the bottom of the plot.

When you run this code, you will see something like this:

Isn’t that neat? You now have a simple plot, and you know how to add semi-transparent text to it, too!

Wrapping Up

Proper attribution is important in academics and business. Knowing how to add a watermark to your data visualization can help you do that. You now have that knowledge when using Matplotlib.

The Matplotlib package can do many other types of plots and provides much more customization than what it covered here. Check out its documentation to learn more!

The post How to Watermark a Graph with Matplotlib appeared first on Mouse Vs Python.

A global variable is a variable that you can use from any part of a program, including within functions. Using global variables inside your Python functions can be tricky. You’ll need to differentiate between accessing and changing the values of the target global variable if you want your code to work correctly.

Global variables can play a fundamental role in many software projects because they enable data sharing across an entire program. However, you should use them judiciously to avoid issues.

In this video course, you’ll:

• Understand global variables and how they work in Python
• Access global variables within your Python functions directly
• Modify and create global variables within functions using the global keyword
• Access, create, and modify global variables within your functions with the globals() function
• Explore strategies to avoid using global variables in Python code

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Ron Garret wrote an insightful refutation to 2nd amendment arguments [1].

Interesting article from the UK about British Gas losing a civil suit about bill collecting techniques that are harassment [2]. This should be a criminal offence investigated by the police and prosecuted by the CPS.

David Brin wrote a new version of his essay about dealing with blackmail in the US political system [3].

Cory Doctorow gave an insightful lecture about Enshittification for the Transmediale festival in Berlin [4]. This link has video and a transcript, I read the transcript.

The Cut has an insightful article by a journalist who gave $50k in cash to a scammer and compares the scam to techniques used to extort false confessions [5].

Truth Dig has an informative article about how Nick Bostrom is racist and how his advocacy of eugenics influences Effective Altruism and a lot of Silicon Valley [6].

Bruce Scneier and Nathan Sanders wrote an insightful article about the problems with a frontier flogan for AI development [7].

Brian Krebs wrote an informative article about the links between Chinese APT companies and the Chinese government [8].

• [1] https://tinyurl.com/2cyrnulm
• [2] https://tinyurl.com/24bfy2x7
• [3] https://tinyurl.com/277o2wt3
• [4] https://pluralistic.net/2024/01/30/go-nuts-meine-kerle/
• [5] https://tinyurl.com/27sl85ml
• [6] https://tinyurl.com/25vtw89m
• [7] https://tinyurl.com/26m7h9ro
• [8] https://tinyurl.com/2ajjetba

Related posts:

1. Links March 2024 Bruce Schneier wrote an interesting blog post about his workshop...
2. Links January 2024 Long Now has an insightful article about domestication that considers...
3. Links April 2021 Dr Justin Lehmiller’s blog post comparing his official (academic style)...

Join host George DeMet and Palantir team members who will be presenting sessions at DrupalCon Portland:

• Britany Acre and Cori Neslund on psychological safety and its importance for individuals and teams.
• Tiffany Farriss and Ken Rickard on EditTogether, a new collaborative editing tool that promises to revolutionize the way content is managed within Drupal.
• Tanya Nascimento and Alex Jones on EditTogether's backstory, underlying technology, and the problems it was designed to solve. 

We want to make your project a success.

Let's Chat. Podcast Links EditTogether Open Source Collaborative Editing Tool for Drupal Transcript

George DeMet: 
Hello and welcome to a special episode of Plus Plus, the podcast from Palantir.net where we talk about what’s new and interesting in the world of open source technologies and agile methodologies. I’m your host, George DeMet.

Today, I’ll be talking with the Palantir team members who will be presenting sessions at DrupalCon Portland, which takes place May 6-9 at the Oregon Convention Center.

First, you’ll hear from Britany Acre and Cori Neslund about  psychological safety and its importance for individuals and teams.

Then, Tiffany Farriss and Ken Rickard will talk about EditTogether, a new collaborative editing tool that promises to revolutionize the way content is managed within Drupal.

And finally, Tanya Nascimento and Alex Jones will share the backstory on EditTogether, its underlying technology, and the problems it was designed to solve.

Enjoy!

[Music plays]

George DeMet: 
I'm here with Britany Acre and Cori Neslund to talk about their session that they'll be presenting at DrupalCon, Key to Collaboration: How to Build Psychological Safety with Individuals and Teams. Hi Britany. Hi Cori.

Britany Acre: 
Hi.

Cori Neslund: 
Hi there.

George DeMet: 
So could you both tell us a little bit about your session and why you chose to talk about psychological safety and what your experience is with it in our work environment.

Britany Acre: 
The title of our session as you said, is Key to Collaboration, and I think that kind of zeros in on our why me and Cori are very energized and passionate about better ways of collaborating on our teams and in our workplaces, and that was what kind of led us to work together on this topic building psychological safety.
It's just so foundational to the work that we do here, especially in agile teams, especially in a working environment like ours here at Palantir that's self-organizing lack of psychological safety can be such a blocker, not only to efficiency and productivity in our workspaces, but also just to the human side.

We want to work on teams where people want to work on teams and want to work with each other and psychological safety is so fundamental to the success of that.

Cori Neslund: 
Yeah, I strongly agree with Britany. For me as a program manager who often holds the role of team leads, I've seen over several projects, how high psychological safety can really facilitate the delivery of solid projects for example, in the Continuous Delivery Portfolio, where I spend most of my time, we're very experiment heavy.

So when we notice there is something going on, we have this fundamental basis, psychological safety that allows us to pitch new options, new ideas that leads to experiments that leads to process improvements that overall makes the experience of being in that team very positive and pleasant, right? So, psychological safety is great on this emotional side, but what I think is really interesting is how it drives success.

It drives success for projects, it drives success for companies, and I think we often think of this as like separate. When it's really not, it's a very interconnected topic.

George DeMet: 
For folks who might not be as familiar with the concept of psychological safety, I understand it as this idea that when you're working on a team or working within an organization being psychologically safe means that you feel comfortable speaking up, and saying something if you, see something wrong or if you have an idea for an improvement that can be made and to feel safe doing so knowing that there won't be any repercussions for speaking up in that kind of workplace setting.

Even if you're speaking with someone who might have been at the company longer, have a higher perceived status or something like that. Is that pretty much how you are looking at it in this session? Or are there other dimensions to consider?

Cori Neslund: 
One piece I would also add is that having this psychological safety does allow for healthy conflict, right? It's very natural to struggle with conflict within teams, personally and professionally, but when you have this base on top of being able to have these conversations on top of being able to share your perspective and provide ideas for new ways of working. You also get to the point where when you disagree with someone on your team, everything is fine. It's a very healthy conversation where both sides are sharing and it doesn't end up in this situation of like high tension, which I think a lot of people will be familiar with. So we want to definitely lean into that and say this is also a really solid reason to build this because disagreement is natural and common.

Britany Acre: 
Yeah, that's great. I'd also add risk taking. So psychological safety is when teams feel safe taking risks. And when we do the kind of work that we do, that's so necessary. Sometimes you're really finding kind of out of the box solutions for things. And a lot of that work takes trying things that may seem crazy and risky, but the ability to do that in a way that feels safe.

We're talking about taking risk. We're so highly skilled and experienced people taking these risks, but the ability to take a risk knowing that the company supports you in that risk taking and sees the value of that is also part of psychological safety. There's so many layers and nuances to psychological safety that is even more than the things that the three of us have just mentioned, but I think that those end up being kind of the focus of what we talk about.

George DeMet: 
My next question then is what are some of the hurdles or challenges that teams face when they're trying to build psychological safety and what are some kinds of tools and techniques you can use to overcome them?

Cori Neslund: 
So the main one, the one that I think that comes up the most frequently, or at least initially is getting buy in right to build psychological safety. You have to have the entire team say, “You know what? I'm going to try and be vulnerable here, or I'm going to try and be a little bit more of a risk taker, a little bit more of a speaker”.

And it's really only successful when the full team or the majority of the team, at least initially decides that they want to participate. You gotta get the ball rolling somehow. And we definitely have some stories about that that we can share in our session. But that's very common. There's always this sort of shadow of past experiences, right?

So people come into their workplaces with all the experiences that they had before. Some of them are going to be great, and some of them are going to be terrible. It's very common to say, “Hey, the problems I had at a previous place, I'm going to have them here. So let me be protective of it”. And how do we move past that and say, “No, this team can create a new culture, this team can decide what values they want to embody and how they want to respond to each other.”?

Britany Acre: 
And I think at the end of what Cori just said, she also touched on kind of some of the techniques, right. Sometimes the idea of implementing a culture change can be incredibly daunting. And that's not what we're suggesting. We're saying here's the value of this thing, and then here are some manageable steps to take to achieve it. And you can start with yourself, then with your team. And that kind of energy and motivation, it gets the ball rolling and starts inspiring others. And so, we'll talk about different experiments that you as a team can do to start building that space for yourselves and modeling that for others.

Cori Neslund: 
All of this contributes to organizational change or things outside of your team, and so it can feel a bit daunting to say, “Oh, I want to add psychological safety, but it's such a big task”. But if we were talking about it from the perspective of individuals, so with two people or more on a team, and then with the team as a unit, that's a lot easier to manage. Not easy, right? But it's a lot easier to get the ball rolling in that more limited scenario than it is to say, “Oh, we want this overarching organizational change”.

George DeMet: 
Yeah, I really like that idea, that psychological safety starts with the individual and, really understanding and, taking ownership of, and committing to being part of a more psychologically safe organization, right? It's not something that like, you know, management can come in and, you know, sort of mandate from above to say, “Hey, we're all going to be more psychologically safe now”, though, of course, there are always opportunities within an organization to create the space for that.

Cori, you touched a little bit earlier on those lived experiences that people might have had at past workplaces, or maybe just even in their lives not feeling particularly psychologically safe. I'm curious to hear a little bit more about how that dynamic plays in and, how it can be addressed.

Britany Acre: 
Something that Cori and I have thought about a lot is that psychological safety can look different for everyone. So very different from someone who let's say, like a white man versus a woman versus a person of color. Not only are we bringing our lived experience from previous jobs, but just also just the world that we live in, in the way that we navigate it differently.

And so there's a self awareness involved in psychological safety. And then there's like the awareness of others and their experience as well. And that's not easy, that self work is not easy. Understanding yourself better, understanding the way the world works better is…it's not fun to face some of those realities. But what can we do from our personal positions to make psychological space and safe space for others?And that's certainly part of the conversation.

Cori Neslund: 
Absolutely. A lot of this is also about removing the burden for upkeep from individuals, right? Because historically, people with certain attributes have held more of a responsibility for organizing culture, being sort of the social backbone. And I think it's important as we move into the future to say that that is a shared responsibility that exists throughout the team.
And it's part of dismantling those systems that Britany is talking about. And also, you know, we're in a space where it's a very open-to-anyone sort of industry, right? Like the Drupal community is made up of a lot of people with a lot of varied identities. How do you have a successful team when none of you share similar experiences, right? That's not a blocker in and of itself, but the self-awareness is required for that team to operate, right? And the awareness of others.

George DeMet: 
Right, when er were talking earlier about the kind of self-work that someone has to do with psychological safety, it's not just about saying, “Oh, hey, I'm going to try and speak up more”, but also being aware of intersectionality and what, one person may hear may sound differently to someone based on their experience than another person. So I think that's really important. Is there anything else you'd like to share about your session?

Cori Neslund: 
Well, I'm really hoping people want to come out. I think a lot of the things that Britany and I will be talking about this year will be totally available and accessible for entry level folks, but if you went to either of our sessions from last year, it builds off of a lot of those concepts. And then it's going to be exciting. We have audience participation. You'll get to share your own perspectives, ask questions at the end. So, if you have real scenarios, we invite you to bring them and let's see what we can help you troubleshoot.

Britany Acre: 
Yeah, and the topics that me and Cori choose to speak on, it's because we care deeply about them. And so yes, our session is designed to be approachable and so that you can take something away that you can actually take action on that isn't too overwhelming, but we're also there to have further conversations.

At its root, creating psychological safety on teams is about promoting the idea that diverse ways of thinking and diverse ways of being are valuable both to us and to our clients. And so we would love to talk more whether before or after our talk or at the Palantir.net booth.

George DeMet: 
Thank you so much to both of you, Britany, Cori. So once again if you are going to be attending DrupalCon this is actually going to be a Monday morning session at 9am in the very first session slot. So definitely don't miss it. Grab your coffee and head over to room B117-119. Thank you both so much.

Britany Acre: 
Thanks, George.

Cori Neslund: 
Thank you.

[Music plays]

​​George DeMet: 
So I'm here with Ken Rickard and Tiffany Farriss to talk about the session they'll be presenting at DrupalCon, Collaborative Editing in Drupal Core.

Tiffany Farriss: 
Hi there.

Ken Rickard: 
Good afternoon.

George DeMet: 
So tell us a little bit about your session. I'm really curious to learn more about EditTogether and Some of the challenges you think it could solve within Drupal.

Ken Rickard: 
Sure I'll give you the quick version and then we can dive into some deeper questions. EditTogether provides a framework for multiple users to collaborate on editing a single piece of content at the same time. So it is a open source collaborative editing framework that can be used with any Drupal field., most dynamically or most dramatically with a WYSIWYG editor so that you can have the kind of experience you have with any of the online cloud based editing tools such as Google Docs, where multiple people are editing at the same time, and you can see their cursor, you can see them type in real time. It also adds in elements that allow for commenting, and those comments are tagged directly to the part of the editorial copy that they refer to. So these are again common features in sort of advanced cloud editors. Office 365 offers this, Google Docs offers it, and potentially Drupal offers it.

Tiffany Farriss: 
I think this is really exciting for those who keep Drupal as the center of all of their content management. So, what it does is it reduces the number of handoffs. I think all of us have been, you know, involved in situations where this, Word document or Google Doc. Passes through 5,6,7, 10 hands before it is signed off on to be put up on a website.

And wouldn't it be more efficient if that entire process could transpire in its final location, right? There's just so much. Administrative time and administrative burden that comes when you have to switch systems or switch tools. Heaven forbid someone not be allowed access to Google Drive or somebody doesn't have access to 365. 

This eliminates all of that. It says, okay, this content is destined to live in Drupal. So let's go ahead and author it in Drupal and we can handle all the various steps of approvals and review and suggestions, commenting, editing that normally happen in other spaces. We can do that right within Drupal.

Ken Rickard: 
When we talk about this as a sort of revolutionary change, I've been doing this for 15 years at Palantir. I've been doing it for 18 years overall. And one of my first editorial questions when we're Designing the CMS experience is always, where does first write through happen? Now, this is an old newspaper term that just means, where did you write the first draft?

Very commonly, the answer is, oh, it's in a Google Doc or a Word doc, and then I email it to this other person. And then it gets email chained around and then gets translated into the content management system. So this idea that we can be in, The end to end source of collaboration really does change the dynamic.

George DeMet: 
Yeah, and I really like the idea that it's all in one place. So one of the things I do at Palantir is I work with folks who are writing blog posts and other material for our website. And that content usually gets authored in Google Docs. And so we can have some collaborative back and forth there.

But then when I go and need to take that content and put it. Onto our website and use the WYSIWYG interface we have there. It still requires a fair amount of work to remove some of Google Docs, formatting tags and things like that. So being able to do all of that collaborating with, my colleagues within the Drupal interface, where it's going to be published would be really cool.

Tiffany Farriss: 
When you create and manage content in its final destination, its final form, you don't have to have the multiple passes around. This is the editorial review, and then this is the design review. So you're able to see it, how it's going to actually look, you'll be able to preview it, you'll be able to capture those approval steps. And Drupal does a nice job of audit logging if you need it to and if you have it configured to but it can't capture anything that happened before it hit Drupal. So for a lot of our clients with more complex editorial and auditing kind of needs, this really does start to streamline the number of systems involved, the number of potential forked versions involved, or missed approvals involved. It really streamlines that whole process.

George DeMet: 
So switching gears a little bit, I'm curious to hear a little bit more about some of the key technologies that go into Edit Together and how they play with Drupal.

Ken Rickard: 
Yeah, I mean, the key technologies are all open source and open standard. I think that's really important. At its base, Edit Together is leveraging WebRTC, which is a peer to peer protocol for browsers notifying each other of behavior, basically. So you can make a peer to peer connection and changes that happen in browser A can be reported to browser B through a secure transaction that doesn't expose that data to everyone else on the web, which is kind of fascinating. So that's again, sort of core underlying technology. And then there are pieces, there are open source pieces built on that. The primary one is a service or a system called Yjs. Simply the letter Y and JS for JavaScript. Yjs is a peer to peer JavaScript framework for transmitting the changes between objects from point A to point B.

That's essentially what it does, but it also provides things like revision tracking. It provides the concept of annotating that data, which is how you can do commenting and you can do suggested changes within the document. So it's a very full featured, open source project. It is very mature. It's in use in a lot of other places. That's the core piece without which EditTogether doesn't really work.

George DeMet: 
And just thinking about it from a software open source licensing standpoint, all of these technologies are compatible with Drupal's GPL V2+ licensing, right? So you're not paying any extra fees or anything to kind of get this functionality. It is already there and already available and compatible with the rest of Drupal's licensing.

Tiffany Farriss:
There are parts of collaborative editing that exist today through premium services. You know, Drupal it's had a long partnership with CK Editor, which is a rich text editor. They do offer some premium and paid features that can include rich text. field collaboration. It's a little bit different because that does involve their servers and their services.

The other kind of key differences Ken mentioned is that this applies to any Drupal field. So because EditTogether was written specifically for Drupal, any field, not just rich text fields can have Commenting, suggestions, track changes, user presence, all of those things exist. So it allows you a structured data experience. It's quite collaborative. And I think that ability to manage structured content has always been one of Drupal's kind of killer features, if you will, and the way that it manages that it's one of the reasons that I think it's always been successful in a best of breed stack is because Drupal handles structured data whether it's the source of origin or it's just transforming it really, really well.

What we've not had to date is the ability to collaborate on that structured data. Currently the experiences, you know, you experience the lock. If someone else is working with a node, like you can't go in and edit that piece of content. And that's just not the way our modern workflows work. Imagine being able to work in parallel with everyone else. Our job functions tend to be, you know, we have our fingers in a lot of different pies, if you will. And so, you know, if I'm the person responsible for making sure that the titles and the ledes are solid on a batch of content, it may fall to someone else who's reviewing, the body copy or making sure that the bios are right and we don't have to worry about, “Oh, is this piece of content locked? Oh, I'm gonna have to make a note. I'll come back to that piece later”. And this is a much more modern cloud based solution. If you have a cloud based content management system, shouldn't you all be able to complete whatever it is you need to do, regardless of what everybody else is doing at that time?

George DeMet: 
What is the current state of EditTogetherr? What are the next steps? I know your session is talking about a pathway to get this functionality into Drupal core. What does that look like?

Ken Rickard: 
Well, I think first we have to ask and answer the question of do we want it in Drupal core? I'll be honest, there is a significant technical burden to adding a feature like this into the core system. While Yjs, for example, is an external library, the integrations that are required with it do require a lot of testing and commitment. So that's not a trivial matter.

And that's something we want to I think put out front and center in the presentation, which is, “Hey, this is really a great opportunity. What does it actually mean if we try to pursue it?” That said, you know, where are we now? The big question has to do with what are the principal use cases for things? How do we wire it into a rich text editor? Again, Yjs has native support for that. We have a solution for that that is a feasible route to go. But again, we have to make clear what our long standing commitments are.

Tiffany Farriss: 
Right now EditTogether has been implemented and deployed for the client that built it. There is still some work to be done around, as Ken said, incorporating other testing scenarios and other use case scenarios.

I think that would be really important to do before we release this as a public beta. So right now we're entering into a private beta period where we can continue to build out All the plugins that are necessary for the additional field types. So there's a way that it's architected. There's a plugin framework.

So any type of field in Drupal can have its own plugin for how it's treated. Individual models can do that. So I think there's a lot of documentation, both in terms of user documentation and developer documentation that needs to exist before it goes out in public beta. So that's really what we're focused on now and through the third quarter.

Ken Rickard: 
The other big technical challenge is that WebRTC by itself works fine if you're on a local network, right? And everyone is on the same IP address in the same room. That's fine. For this really to work collaboratively across distance, you have to have what's known as a signaling server. And when we talk about products like Google Docs or Office 365, those are done by cloud hosting providers who provide their own signaling server.

If everyone, again, on the technical side, remember PubSubHubbub and that whole idea of being able to push notifications from place to place, that's what we're talking about, and it turns out there are no longer any public free signaling servers out there. They're all privately branded, they're all privately labeled, or you have to spin up your own.

So, solving that, because it is a barrier to entry right now. If you want this thing, we have to spin up our own. A server that handles it, and that's just to make those socket connections from browser A to browser B, so that you don't have to send your data across the wire, and that's again, both a technical challenge and a killer feature.

Let me repeat. You don't have to send your data across the wire. You're sending browser position, like, literally, my cursor is at this point, space in the browser relative to the document you're looking at, but it doesn't actually send the data of the document. It's really fascinating stuff. It's really smart from an architecture standpoint and becomes a killer feature because it retains compliance to data sovereignty regulations and requirements.

That's where it gets really fascinating. But again, you do have to stand up this sort of extra service. And so that is a big challenge. I think as we go forward with private beta, that's going to be the next big thing we have to tackle is, “Okay, how can we make this a simple and replicable opportunity?” Because you wouldn't want to drop this into Drupal core without having an out of the box solution to this problem.

Tiffany Farriss:
I mean, it's very easy to kick the tires on. There are still organizations that really have complex workflows that may want to stand up their own services. It's actually quite easy for those kinds of enterprises to do on the technical side, right? They still have to go through security review and, you know brokerage review, things like that, which any enterprise would already have processed for, but what's really novel about at it together is it is a very privacy first and security forward kind of collaborative solution for Drupal that I think can be, you know, game changing.

George DeMet: 
So if I'm an organization that's really interested in using EditTogether, or if I'm a developer who wants to help out with it, how can I contribute? What kind of feedback are you looking for? How can someone get involved?

Tiffany Farriss: 
Well, at this point, as I said, we're moving into a private beta. So if you're an organization that is looking to install this yourself you can, reach out to us. We have spun up EditTogether. com. That's where you can get in touch to let us know that you're interested, and we can talk about whether you're a good candidate to be part of that private beta.

Otherwise, when it goes into public beta we'll obviously be looking for those who are willing to add to documentation, create more of those field level plugins or plugins for individual modules, things like that. And if you're really interested in the core work, that may have to happen we do encourage you to come to this session and just reach out and let's start that conversation.

Because I think it has to be more than just Palantir that's interested in this solution for it to become a new killer feature for Drupal, as opposed to just something you can do with Drupal.

George DeMet: 
So for anyone looking to learn more or to get involved once again, that session is Collaborative Editing in Drupal Core.That will be at 3 p. m. on Monday, May 6th, the first day of DrupalCon, and it will be in room A106. Ken, Tiffany, thank you so much for sharing your knowledge and really looking forward to it.

Tiffany Farriss: 
Looking forward to seeing folks there.

Ken Rickard: 
Yeah. Thank you.

[Music plays]

George DeMet: 
I'm here with Alex Jones and Tanya Nascimento and here to talk about their session, EditTogether: Collaborative Editing for Everyone. Welcome, Tanya. Welcome, Alex.

Tanya Nascimento: 
All right, thanks for having us.

Alex Jones: 
Thank you

George DeMet: 
This is the second of the sessions we'll be doing at DrupalCon on EditTogether. We talked earlier with Ken and Tiffany about how it works within the Drupal ecosystem, how it could potentially be added to the core software.

What I was interested in talking with both of you about is really some of the back story. What were the initial client requirements that led to the creation of EditTogether? This is a state government agency within the United States and part of their health department there. What was it that they were looking for that their existing solution was no longer adequate for?

Tanya Nascimento: 
So something that we really dug into in discovery was the administrative burden that this agency is under when they're trying to create content, review that content, and publish that content. Their current workflow is incredibly demanding. They have at least a dozen publications, but within those publications, they have at minimum Three systems that they need to utilize to create that content in, for example, Microsoft Word, then manually handed off to RoboHelp, manually handed off to somebody for review. This is an email, sometimes it's FTP, using SharePoint all along the way and then bringing it back through the system, going through these cycles several times. So they have at least six handoffs for several cycles at least three times per year and when you start to look at that you realize that it's adding up to a lot of potential error in the process as well as just a taxing and exhausting process.  So within that we really wanted to deliver something that was ,ore simplified in a replacement workflow where everything could happen in Drupal, or almost everything, and limiting those handoffs.

George DeMet: 
So they're creating these documents. Are these internally facing documents, externally facing documents? What's the use case there?

Tanya Nascimento: 
A lot of what we're seeing are handbooks, things to help folks enroll in programs like Medicare, where these handbooks include a lot of references to information that's really important for individuals to have when dealing with insurance or when dealing with that enrollment.

And these handbooks require a lot of updates. So let's say you have somebody who's working for this agency who needs to go in and make an update to a policy. They might need to go in and create one small miniscule change with, let's say, the numbering of an item in a very extensive long handbook. In order to do that, they would need to go back to the beginning of this process, go into Word, make that change, hand that off to another person, have that go through this entire system of multiple applications in order to just make one edit.

And it's all based on potential human error. So when we're looking at the version history and the archives of all of these things, it's content that's very important for folks out there in the world to have but it takes a long time in order to just make these seemingly simple updates.

George DeMet: 
It sounds like this is really important information to get right, especially if you're talking about someone needing to fill out a series of forms in order to get health benefits . This is really important stuff that impacts people's lives. 

Tanya Nascimento: 
Definitely. They utilize reviewers from several different adjacent government agencies to ensure that those who are reviewing or creating content are the true subject matter experts for any given publication. Given that, there is a lot of manual handoffs that have to happen in order for folks to ensure that they are directly handing off information to the correct individual. So it isn't something that is super sophisticated right now, but it is something that they can manage especially when they have users who are trusted or on VPN. It's super simple right now, but it's burdensome. There's the administrative burden, but there's also the ongoing cost of utilizing RoboHelp. And because of that, they need something that is streamlined and simple. Right now, what we want to change for them is to create a system where they can create, manage, and publish all in one place. And this allows the agency to have something that's self-sustaining for a long period of time.

George DeMet: 
So as we started to take a look at this challenge looking at all of these different systems then figuring out how to bring them together and streamline them into Drupal what was the process for doing that technical research and analysis?

Alex Jones: 
We essentially wanted to look at their entire content workflow which we did, and we found that most of the pieces of their workflow can be brought into Drupal. Drupal's obviously an incredibly powerful tool for content workflows but the piece that was missing that is present in their current workflow is this collaborative editing piece.

So, currently they create content in a collaborative environment in Microsoft SharePoint. And they expect to be able to do that in Drupal. It's key to their process that they can actually collaborate on content, suggest edits to that content. before that content moves on to the later stages of their workflow cycle.

So, we investigated a couple solutions for that in Drupal. One of those was CKEditor 5, which does have an implementation of collaborative editing. And the reason we couldn't use C Editor 5 was primarily two things: They have a subscription fee that's ongoing and also CKEditor 5 uses a client-server model for collaboration, which means that the state of your content that you're editing is stored in the cloud in a third party service and for our client and a lot of other government clients they have restrictions on where their content can be visible and managed even for a brief period. We also investigated some other tools for building a text editor in Drupal. And what we eventually settled on was building a text editor for Drupal using open source frameworks that are available right now.

So the solution we ended up with was building a text editor using ProseMirror, which is a. JavaScript framework for building rich text editors. And we coupled that with Yjs, which is a JavaScript based shared editing framework that allows you to manage a document state between multiple different client machines. By building it using ProseMirror and Yjs we're trying to expose that last piece of the content workflow for Drupal to the open source space. So ProseMirror is open source. Yjs is open source. They have a lot of plugin functionality and options for extensibility, and we're hoping that by providing a text editor using these open source technologies we're enabling a more rapid development of new features for our clients and also for the community, to further expand and add custom functionality or contribute functionality back to the main module for use by everyone.

George DeMet: 
Which is always the ideal thing you want to see with open source, right? Not just that we're using this code that someone else has made, or groups of folks have made, but also contributing back and improving it. Taking a look at a tool like ProseMirror, it is very modular, we're using it in this case for EditTogether. But it's a tool that could be used. In a lot of other different ways as well.

Alex Jones: 
Absolutely. We've continued forward with ProseMirror because as we've expanded the text editor that EditTogether provides we've realized that ProseMirror can really be used to create a framework for field level collaboration in Drupal. You can build ProseMirror instances that work on rich text, body fields in Drupal but you can also build them for plain text fields and you can build them for any type of field in Drupal that you'd want to see collaboration upon. And that means that ProseMirror coupled with Yjs can provide field level collaboration for all of your fields on your Drupal site.

George DeMet: 
I'm curious to hear a little bit more about some of the challenges that we faced during the development and how we've addressed those.

Alex Jones: 
Yeah, we've encountered technical hurdles on this project in all shapes and sizes. One of the major ones that we've encountered is, finding the documentation on how to do the thing that we're doing. So there's a long history of CKEditor 5 and its integration with Drupal. And it integrates with Drupal using something called the Editor API. Because of that long history in part it is somewhat difficult to find information on how to use that API and how to integrate successfully with Drupal. And so as a byproduct of building this integration with Drupal, we've actually been able to more thoroughly explore the Editor API and a lot of the potentially lesser understood or, lesser documented portions of that integration. And we certainly feel that we have a better hold on how to more quickly and effectively manipulate that API and these integrations for Drupal.

George DeMet: 
I want to go back a little bit to something you touched on earlier, Alex, which was this idea of data sovereignty, right? That particularly in some contexts having control over where, or at least knowing where your data is at every point in the process is so key and so vital.

Alex Jones: 
Data sovereignty is something that's really important to our clients. It's really important to a lot of clients in government agencies, higher ed organizations, hospitals, things like that. Existing collaborative editing implementations are overwhelmingly client-server based models, where clients collaborate, and then their information is sent off to a third party server, and that third party server stores the true state of the content.

That works for these clients only when the system that they're using for that collaboration is approved, like Microsoft SharePoint, if they've approved Microsoft SharePoint or something like that. But it means one that that system has to be approved and then paid for. And two, they can't have that system in Drupal. They have to edit all their content outside of Drupal and then they bring it in.

Instead of using a client server model with WebSockets, we would implement a peer to peer model using WebRTC or Web Real Time Communication. And what that means is that these clients, when they're collaborating over a piece of content, the state of the content is shared only between the collaborating clients and not shared with the server. WebRTC is very secure and it also democratizes collaboration in a way that those client server models don't. And it costs orders of magnitude less to operate than a client server model does.

George DeMet: 
Alex, Tanya, thank you so much.

Tanya Nascimento: 
Thank you.

Alex Jones: 
Thank you for having us.

George DeMet: 
Once again, the session is called EditTogether: Collaborative Editing for Everyone. It'll be on Tuesday, May 7th at 3 p.m. in room C120-122. I will also be part of the conversation so if you are at DrupalCon come and see us and thanks again!
 

Content Strategy Development Drupal Events Open Source Corporate Government Healthcare Higher Education

Recently I became suddenly curious about the sizes of buildings in Southampton, UK, where I live. I don’t know what triggered this sudden curiosity, but I wanted to know what the largest buildings in Southampton are. In this context, I’m using “largest” to mean largest in terms of land area covered – ie. the area of the outline when viewed in plan view. Luckily I know about useful sources of geographic data, and also know how to use GeoPandas, so I could answer this question pretty quickly – in fact, in only five lines of code. I’ll take you through this code below, as an example of a very simple GIS analysis.

First I needed to get hold of the data. I know Ordnance Survey release data on buildings in Great Britain, but to make this even easier we can go to Alastair Rae’s website where he has split the OS data up into Local Authority areas. We need to download the buildings data for Southampton, so we go here and download a GeoPackage file.

Then we need to create a Python environment to do the analysis in. You can do this in various ways – with virtualenvs, conda environments or whatever – but you just need to ensure that Jupyter and GeoPandas are installed. Then create a new notebook and you’re ready to start coding.

First, we import geopandas:

import geopandas as gpd

and then load the buildings data:

buildings = gpd.read_file("Southampton_buildings.gpkg")

The buildings GeoDataFrame has a load of polygon geometries, one for each building. We can calculate the area of a polygon with the .area property – so to create a new ‘area’ column in the GeoDataFrame we can run:

buildings[’area’] = buildings.geometry.area

I’m only interested in the largest buildings, so we can now sort by this new area column, and take the first twenty entries:

top20 = buildings.sort_values(’area’, ascending=False).head(20)

We can then use the lovely explore function to show these buildings on a map. This will load an interactive map in the Jupyter notebook:

top20.explore()

If you’d like to save the interactive map to a standalone HTML file then you can do this instead:

top20.explore().save(“map.html”)

I’ve done that, and uploaded that HTML file to my website – and you can view it here.

So, putting all the code together, we have:

import geopandas as gpd buildings = gpd.read_file("Southampton_buildings.gpkg") buildings[’area’] = buildings.geometry.area top20 = buildings.sort_values(’area’, ascending=False).head(20) top20.explore()

Five lines of code, with a simple analysis, resulting in an interactive map, and all with the power of GeoPandas.

Hopefully in a future post I’ll do a bit more work on this data – I’d like to make a prettier map, and I’d like to try and find some way to test my friends and see if they can work out what buildings they are.

Open Social, a tool within the Drupal community, is actually quite valuable for creating social platforms. It's user-friendly, flexible, and well-supported, making it perfect for setting up social networks, whether they're for internal company use or for the public. 

With features like user profiles, activity feeds, group discussions, events, and messaging, Open Social offers a comprehensive solution for building engaging online communities. 

It's important to spread the word about Open Social so that more people can benefit from its features and capabilities, ultimately enhancing collaboration and communication in various settings.

What is Open Social?

Open Social is a software platform that's open to everyone and is built on top of Drupal, which is a popular system for managing website content. It's designed specifically for creating online communities and social networks.

<strong>Topics covered in this episode:</strong><br> <ul> <li><a href="https://wasmer.io/posts/py2wasm-a-python-to-wasm-compiler"><strong>Announcing py2wasm: A Python to Wasm compiler</strong></a></li> <li><strong>Exploring Python packages with</strong> <a href="https://oven.fming.dev"><strong>Oven</strong></a> <strong>and</strong> <a href="https://pypi-browser.org"><strong>PyPI Browser</strong></a></li> <li><a href="https://www.youtube.com/watch?v=DLBiJ5kYUFg">PyCharm Local LLM</a></li> <li><strong>Google shedding Python devs</strong> <strong>(at</strong> <strong>least in the US).</strong></li> <li><strong>Extras</strong></li> <li><strong>Joke</strong></li> </ul><a href='https://www.youtube.com/watch?v=KlLuQ7UT4t8' style='font-weight: bold;'data-umami-event="Livestream-Past" data-umami-event-episode="381">Watch on YouTube</a><br> <p><strong>About the show</strong></p> <p>Sponsored by ScoutAPM: <a href="https://pythonbytes.fm/scout">pythonbytes.fm/scout</a></p> <p><strong>Connect with the hosts</strong></p> <ul> <li>Michael: <a href="https://fosstodon.org/@mkennedy"><strong>@mkennedy@fosstodon.org</strong></a></li> <li>Brian: <a href="https://fosstodon.org/@brianokken"><strong>@brianokken@fosstodon.org</strong></a></li> <li>Show: <a href="https://fosstodon.org/@pythonbytes"><strong>@pythonbytes@fosstodon.org</strong></a></li> </ul> <p>Join us on YouTube at <a href="https://pythonbytes.fm/stream/live"><strong>pythonbytes.fm/live</strong></a> to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too.</p> <p>Finally, if you want an artisanal, hand-crafted digest of every week of </p> <p>the show notes in email form? Add your name and email to <a href="https://pythonbytes.fm/friends-of-the-show">our friends of the show list</a>, we'll never share it.</p> <p><strong>Michael #1:</strong> <a href="https://wasmer.io/posts/py2wasm-a-python-to-wasm-compiler"><strong>Announcing py2wasm: A Python to Wasm compiler</strong></a></p> <ul> <li>py2wasm converts your Python programs to WebAssembly, running them at 3x faster speeds</li> <li>thanks to <a href="https://nuitka.net/">Nuitka</a></li> </ul> <p><strong>Brian #2:</strong> <strong>Exploring Python packages with</strong> <a href="https://oven.fming.dev"><strong>Oven</strong></a> <strong>and</strong> <a href="https://pypi-browser.org"><strong>PyPI Browser</strong></a></p> <ul> <li><a href="https://pypi.org">pypi.org</a> is great, but there are some handy alternatives</li> <li><a href="https://oven.fming.dev"><strong>Oven</strong></a> <ul> <li>Shows how to install stuff with pip, pdm, rye, and poetry</li> <li>Similar meta and description as PyPI</li> <li>Includes README.md view (no tables yet, though)</li> <li>Nice listing of versions</li> <li>Ability to look at what files are in wheels and tarballs (very cool) </li> <li>Can deploy yourself. Node/Remix app.</li> <li>Really slick.</li> </ul></li> <li><a href="https://pypi-browser.org">PyPI Browser</a> <ul> <li>View versions</li> <li>View wheel and tarball contents.</li> <li>Metadata and contents.</li> <li>No README view</li> <li>Is a Starlette app that you can deploy on your on with a private registry. So that’s cool.</li> </ul></li> </ul> <p><strong>Michael #3:</strong> <a href="https://www.youtube.com/watch?v=DLBiJ5kYUFg">PyCharm Local LLM</a></p> <ul> <li>Pretty awesome full line completer based on a local LLM for PyCharm</li> <li>Requires PyCharm Professional</li> <li>An example, given this partial function in Flask: <pre><code>@blueprint.get('/listing') def listing(): videos = video_service.all_videos() </code></pre></li> </ul> <p>Typing ret →</p> <p><img src="https://python-bytes-static.nyc3.digitaloceanspaces.com/llm-complete.png" alt="img" /></p> <p>That is, typing ret autocompletes to:</p> <pre><code>return flask.render_template('home/listing.html', videos=videos) </code></pre> <p>Which is pretty miraculous, and correct.</p> <p><strong>Brian #4:</strong> <strong>Google shedding Python devs</strong> <strong>(at</strong> <strong>least in the US).</strong></p> <ul> <li><a href="https://techcrunch.com/2024/04/29/google-lays-off-staff-from-flutter-dart-python-weeks-before-its-developer-conference/">Google lays off staff from Flutter, Dart and Python teams weeks before its developer conference</a> - techcrunch</li> <li><a href="https://www.theregister.com/2024/04/29/google_python_flutter_layoffs/">Python, Flutter teams latest on the Google chopping block</a> - The Register <ul> <li>“Despite Alphabet last week <a href="https://www.theregister.com/2024/04/26/register_kettle_ai/">reporting</a> a 57 percent year-on-year jump in net profit to $23.66 billion for calendar Q1, more roles are being expunged as the mega-corp cracks down on costs.”</li> <li>“As for the Python team, the current positions have <a href="https://social.coop/@Yhg1s/112332127058328855">reportedly</a> been "reduced" in favor of a new team based in Munich.”</li> </ul></li> <li>MK: Related and timely: <a href="https://www.wheresyoured.at/the-men-who-killed-google/">How one power-hungry leader destroyed Google search</a></li> </ul> <p><strong>Extras</strong> </p> <p>Brian:</p> <ul> <li><a href="https://andrewwegner.com/python-gotcha-strip-functions-unexpected-behavior.html">Python Gotcha: strip, lstrip, rstrip can remove more than expected</a> <ul> <li>Reminder: You probably want .removesuffix() and .removeprefix() </li> </ul></li> </ul> <p>Michael:</p> <ul> <li><a href="https://lmstudio.ai/blog/llama-3">Using Llama3</a> in <a href="https://lmstudio.ai">LMStudio</a></li> </ul> <p><strong>Joke:</strong> <a href="https://devhumor.com/media/broken-system">Broken System</a></p>

53rd issue of Andrei Neagoie's must-read monthly Python Newsletter: Whitehouse Recommends Python, Memory Footprint, Let's Talk About Devin, and much more. Read the full newsletter to get up-to-date with everything you need to know from last month.

I’m assuming you are reading this because you are already convinced that migrating your Drupal 7 site to Drupal 10 is not just a proactive measure but a strategic move for your organization. But to anyone else on your team it looks like an unnecessary big project to rebuild the website when it’ll look and feel the same (we always recommend redesigns or additional features during a D7 to D10 Migration). We get it. For folks not in the know, this seems like a waste of funds. However, with Drupal 7 coming to an end in about 9 months (January 5th, 2025), the urgency to transition becomes increasingly stressful. We know you know, but you still have your team members (or your boss!) left to convince. Let’s make it easier for you with this article. Understanding the impact of Drupal 7 End of Life Before you talk to your team about why you need to migrate to Drupal 10, let’s examine some of the implications of persisting with Drupal 7.  No more security updates or advisories for core, contributed modules and themes. The Drupal Security Team may publicly post moderate to less critical issues affecting Drupal 7 in the public issue queue for resolution, provided they are not widely exploitable. Unsupported Drupal 7 contributed modules or themes won't be eligible for new maintainership or reclassification as supported. If you’re currently using them, it is a good idea to take ownership or be a maintainer of those projects. PHP version 5.5 and below will now be supported. This lack of support could lead to compatibility issues, security vulnerabilities, and potential performance drawbacks. If your Drupal 7 website is running on Windows, you will no longer receive security fixes for Windows-related issues. It is recommended to move to a different operating system. You will no longer receive assistance for tasks associated with Drupal 7, such as documentation navigation, automated testing, packaging, and other related activities. Making the case for Drupal 7 to 10 migration to your team Let’s give you some powerful pointers to discuss with your team to get buy-in on the Drupal 7 to 10 migration. Remarkably Enhanced User Experience for Content Editors and Site Builders in D10 There are 3 things that matter a lot to content editors and site builders :   User-friendly admin interface - It should allow for efficient content creation, editing, and site management without requiring extensive technical knowledge. Customization - This includes options for customizing layouts, adding new features, and integrating third-party tools and services. Media management - To upload, organize, and embed images, videos, and other multimedia content within articles. Claro admin theme Drupal 10’s new Claro admin theme (a part of core) offers a clean, modern and user-friendly interface to help organize and find what you need easily. Olivero is the new default front-end theme now and it comes with a modern look and feel. The theme integrates seamlessly with all of Drupal’s features and is the most accessible theme (WCAG level AA compliant) till now. The flexible Layout builder module is now in core and it is now easier to create pages and customize layouts the way you want. The modern and functional media management system makes it simpler to upload, reuse, and manage media assets on your Drupal site. Optimized Website Performance and SEO Improvements With every new release, Drupal is getting better at delivering performance. With Drupal 10’s new and improved caching mechanisms, BigPipe technology, optimized codebase, and effective content delivery mechanisms, your website can now load faster and offer a great user experience. It incorporates various enhancements to boost performance in content rendering and HTTP responses. With Drupal 10, you can implement lazy loading for embedded content and responsive images, significantly enhancing load times. Additionally, the introduction of the new JS minification feature dynamically reduces code and markup, thereby further improving performance. The new Single Directory Component (SDC) approach of theming your website is a revolutionary step towards frontend development which also greatly improves website performance by groupong together files necessary to render components (Twig, CSS, JS). And don’t forget, better website performance also means a better SEO ranking on search engines. Managing Content is Easier Now that you've settled into using Drupal 7 for a while, you might feel like managing content is pretty straightforward. But hold on – let me tell you about Drupal 10, where things get even smoother and more user-friendly. With Drupal 10, organizing your content consistently becomes much simpler. You can reuse existing fields easily and create new ones more smoothly, all in one place. Editing content is smoother too, with text fields that ensure your text looks just right. Plus, managing older versions of your content, whether it's in blocks or pages, is a breeze with the new unified editing experience. The new CKEditor 5 version offers an enhanced content editing experience. Its features like drag-and-drop image insertion, real-time collaboration, and seamless integration with Drupal's content management system make creating and editing content very simple. Its customizable toolbar allows you to tailor the editing experience to suit your specific needs. You also easily copy and paste content from Word/Google Docs to the editor without worrying about formatting as it automatically removes any markup. Improved Security And no, we’re not just talking about the lack of security support for Drupal 7 after Jan 2025. Because of the way it has been built and due to its many modern dependencies, Drupal 10 is now more secure than it has ever been. As you may already be aware, Drupal has been aligning its release cycles with its dependencies, including PHP and Symfony, since version 8. This means that as PHP versions continue to evolve, older versions like Drupal 7 may become incompatible with the latest PHP releases. This lack of compatibility can leave your Drupal 7 site vulnerable to security risks and other issues. Drupal 10 relies on the latest versions of Symfony (6) and PHP (8.1), making it more secure and better performing.  Twig, Drupal 10's default template engine, not only simplifies the development process but also enhances security by preventing direct database interactions within the code. This prevents vulnerabilities such as cross-site scripting and code injections. By default, Drupal 10 strengthens website security by suggesting users choose stronger passwords, minimizing unauthorized access risks through parameters like minimum length and complexity. The Time is Now! The urgency for you to address the Drupal 7 migration depends on the complexity of your website. The more extensive your site's content and features, the longer the migration process will likely take. It’s going to take even longer if you have many custom modules and features. But you don’t want to rush the process. We have seen (and fixed) a lot of bad migrations (like a lot!), most of them done in haste or without proper planning. With a 9-month window to Drupal 7 end-of-life starting now, we believe this is the optimal time to initiate your migration process. A Drupal 7 to 10 migration is going to be a complete rebuild (which is why it takes time) but once you’re on Drupal 10, future upgrades are going to be very, very easy.  Don’t forget to check out this article that features our Drupal experts discussing what’s new in Drupal 10 in detail. You can even catch up with the video of this panel discussion. Final Thoughts If you’re thinking this migration (Drupal 7 to Drupal 10) is going to be your last big transition, you are absolutely right. Because even though Drupal continues to innovate, progress and release further versions, your website will now only need effortless and straightforward upgrades. Yes, upgrades will remain easy forever. So what next? Start looking for a Drupal certified migration partner (like Specbee) and get a site audit (we’ll do it for you for free!) so you know how much time you have to start the migration process.

I was looking at some code that sets file permissions – file modes – by calling chmod(2). The command-line chmod(1) has a bunch of possibilities for setting permissions, but the underlying system-call needs an int, and the C headers for it are annoying to use. So I fired up some C++ machinery to “make it nicer to use”.

tl;dr: scroll to the bottom for a compile-time, compact way of writing file permissions so that chmod(filename, "rwx--x--x"_mode) does something plausible, with good error messages in case of typo’s.

The manpage for chmod(1) is fairly extensive. There are two ways to specify a file mode, either in octal or symbolically. Octal is just a number, symbolically is much more complicated.

On the C API side, the manpage for chmod(2) shows you need a pathname and a mode, and there are a whole bunch of symbolic constants to use, like S_IRUSR.

How I think about file modes

It turns out I nearly always think about file modes in octal. I have somehow internalized things like “755 for executables” and “666 for crap” and “600 for files in ~/.ssh” but I don’t really have names for these things. If I think about it, I can use the symbolic manipulations like ugo+rw for crap. But I don’t see permissions in this symbolic form, and the octal form is persnickety in C source, probably because I don’t expect everyone to know “leading 0 means octal”.

But there is a form in which I see file modes every day: the output from ls -l, where permissions are shown with 10 characters, the first 10 on this line:

-rw-r--r-- 1 adridg users 0 Apr 30 11:46 example.txt

The first - says something about the type of file and is - for regular files, d for directories, l for symbolic links, and there are others, too. That’s not really the mode, though, while the next 9 characters are: each group of three shows r, w, and x in that order, or a - in each position, indicating the read, write, or execute bit for the corresponding class of logins. The first three are the current user, next three are group, the last three for others.

The C code to call chmod with this mode looks like

chmod("example.txt", 0644); chmod("example.txt", S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH);

One is octal-arcane and the other is just arcane and hard-to-read.

Turning readable modes into numbers

So I thought to myself: I can write a C++ function that turns the 9-characters of the mode in text form, into an actual mode_t value (that’s an integer).

That’s a trivial exercise, really, although it gets a bit persnickety in error handling. I would just throw on an incorrect length, or any incorrect character, and leave it at that.

From there, though, I went on to: I can write a consteval C++ function that does the computation at compile-time (guaranteed to be only at compile-time, because consteval). This is a function that can only be called with a compile-time string literal, so the function signature is potentially:

consteval mode_t from_readable_mode(const char (&permission_string)[10])

The array-reference is to 10 characters because there are 9 mode characters, and then a trailing NUL byte (zero) in a string literal. This function can be called (in the context, sill, of chmod()) like so:

chmod("example.txt", from_readable_mode("rw-r--r--"));

and at compile-time that is already computed to be 420 (that’s 0644 octal).

The last step of make-it-cool (for an appropriate value of “cool”) is to turn the whole thing into a user-defined literal. In my source code I can now write

chmod("example.txt", "rw-r--r--"_mode);

Which really satisfies my own desire for “readable, compact, compile-time”.

Don’t get me started on Qt

For some reason, the values of QFileDevice::Permission are in hexadecimal, but written as hex, look like the corresponding octal values. So in Qt code if you don’t use the symbolic representation of the flags, and just go ram an integer into there, you get 0x644 meaning the same as 0660 in the call to chmod() and everything becomes just that much more confusing and fraught.

Meaningful error messages

C++ and “helpful, friendly, easy-to-read error messages” go together like peaches and .. battery acid? Like things that don’t go well together at all. In recent GCC versions there has been a marked improvement in the look of error messages.

With some judicious use of templates and naming of types, existing error messages can be improved. You still get a veritable torrent of messages, but if, somewhere in the middle, the error message (here, from Clang 17) says:

mode.h:26:9: note: subexpression not valid in a constant expression 26 | throw invalid_permission_character{}; mode.h:41:9: note: in call to 'expected_character_at_position(&"rwxbadbug"[0])' 41 | detail::expected_character_at_position<3, 'r'>(s) |

Then it’s a lot easier to decide that the character at position 3 – the letter b – is not expected, and maybe an r was expected there instead.

User-defined file mode literals

Here is the definition of my _mode literal. String-based literals get a pointer and a size, which is inconvenient because they don’t turn back into fixed-size arrays.

consteval mode_t operator""_mode(const char *s, size_t len) { if (len != 9) { throw detail::invalid_permission_string_length{}; } return detail::from_readable_mode_string(s); }

Anyway, if the string is the wrong size then a meaningful exception is thrown, which isn’t constexpr, so you get a meaningful error message:

mode.h:65:9: note: subexpression not valid in a constant expression 65 | throw detail::invalid_permission_string_length{}; main.cc:44:24: note: in call to 'operator""_mode(&"birb"[0], 4)' 44 | std::cout << "birb"_mode;

Here us the implementation of the function that does the actual work, turning the string into a mode_t:

consteval mode_t from_readable_mode_string(const char * const s) { return detail::expected_character_at_position<0, 'r'>(s) | detail::expected_character_at_position<1, 'w'>(s) | detail::expected_character_at_position<2, 'x'>(s) | detail::expected_character_at_position<3, 'r'>(s) | detail::expected_character_at_position<4, 'w'>(s) | detail::expected_character_at_position<5, 'x'>(s) | detail::expected_character_at_position<6, 'r'>(s) | detail::expected_character_at_position<7, 'w'>(s) | detail::expected_character_at_position<8, 'x'>(s); }

It’s really wordy, which is unfortunate, but by writing it like this, the error message – at least with Clang 17 – repeats the template parameters and mentions the specific subexpression that is problematic.

Here is a Clang 17 error message when using an inappropriate permission character:

mode.h:26:9: note: subexpression not valid in a constant expression 26 | throw invalid_permission_character{}; mode.h:44:9: note: in call to 'expected_character_at_position(&"------uwu"[0])' 44 | detail::expected_character_at_position<6, 'r'>(s) | mode.h:67:12: note: in call to 'from_readable_mode_string(&"------uwu"[0])' 67 | return detail::from_readable_mode_string(s);

Huh, I guess you can’t give uWu permission to others. The same error message from GCC 13 looks similar:

main.cc:44:18: in 'constexpr' expansion of 'operator""_mode(((const char*)"------uwu"), 9)' mode.h:67:45: in 'constexpr' expansion of 'detail::from_readable_mode_string(s)' mode.h:44:55: in 'constexpr' expansion of 'detail::expected_character_at_position<6, 'r'>(((const char*)s))' mode.h:26:9: error: expression '<throw-expression>' is not a constant expression 26 | throw invalid_permission_character{};

And here’s the implementation that turns a single character into a bit in a file mode value:

template<int position, char accept> consteval mode_t expected_character_at_position(const char * const permission_string) { const char c = permission_string[position]; if(c == accept) { return 1 << (8-position); } if(c == '-') { return 0; } throw invalid_permission_character{}; }

This is a bit wordy, but it ensures that position and the acceptable (expected) char are front-and-center in error messages, and that the expected character and - are the only characters for which this is a constant expression – everything else will fail because exceptions aren’t constexpr.

So there you have it, a compact constexpr representation of file modes with meaningful error messages. You can find the code in my personal GitHub repository playground, in the subdirectory mode/ . I won’t link it here because, frankly, it is time I get my ass in gear and migrate to some other forge.

I'm happy to have been able to attend my first in-person KDE event, the Automation & Systematization Sprint in Berlin. Previously, my contributions to KDE have consisted of submitting and triaging bug reports. During this weekend, I was able to meet some of the KDE team in person, and become more involved. I've started working with the Bugzilla Bot code, and plan to start digging into the automated test code.

The Bugzilla product list had fallen out of date, so first I updated that (yay, my first accepted MR!). I also started working on using the GitLab API to automate these updates. In the near future, I'll be tackling some requested improvements to the Bugzilla Bot. This will lessen the amount of boring manual bug chores and free people up to do more valuable work.

Thanks to the KDE team for being so friendly and willing to help me learn the development environment. I'm happy to have found more ways to contribute that I enjoy, and will be valuable to the project.