Planet Python

Lockdown Mode for Apple devices About • Blog • Newsletter • Links Lockdown Mode for Apple devices

Published 2024-07-02 by Seth Larson
Reading time: minutes

Back in September 2023 the libwebp vulnerability (also known as BLASTPASS) was being actively exploited to target a journalist's mobile device. After reading the report from Citizen Lab I learned about an iOS feature called "Lockdown Mode" for Apple devices.

I've been running Lockdown Mode for almost a year now, and at the time I promised a write-up of my experience with the feature, so here it is!

How does Lockdown Mode keep your phone more secure?

Lockdown Mode prevents some methods of sending or injecting data into your phone without your active engagement (such as preloading data, injecting data into unsecured connections, etc). Data that's processed by your phone automatically, such as images, can exploit flaws in image format parser in order to escape and begin executing code.

BLASTPASS exploited memory safety issues in the libwebp library which processes WebP images. The malicious WebP image was delivered to the target's device via a PassKit attachment which can be sent in a text message.

What does Lockdown Mode disable?

Here's the full list of disabled or degraded features when Lockdown Mode is enabled, quoted from Apple's docs on the feature:

• Most message attachment types are blocked. Some features such as links and link previews are unavailable.
• Certain complex web technologies are blocked. (ie JavaScript JIT)
• FaceTime calls from unknown contacts are blocked. SharePlay and Live Photos are unavailable.
• Photo location information is excluded. Shared Albums are removed and disabled.
• Wi-Fi must be secure for device to connect to a network. 2G cellular support is disabled.
• Mobile Device Management and Configuration Profiles are disabled.

What are the impacts?

The biggest impacts for day-to-day usage is two-fold: Message Links and Search.

With Lockdown Mode enabled, links will not highlight like they typically do, and they won't show the fancy preloaded image that gives you a preview of the content on the other side of a click.

Not having links and link previews in messages is a real inconvenience. The fastest work-around to extract a link in the middle of a text message is to either copy the whole message into your own message box and then copy the URL or to screenshot the message and use Live Text to copy-and-paste directly from your screenshot.

If you're able to persuade your partner to send links in a separate message, that also speeds up the copy-and-paste process by copying the whole message. Persuading your partner is left as an exercise to the reader :)

The other major impact is not being able to search through my messages. This feature is super helpful when you're trying to recall something from years ago, but not something you're using every day usually. This feature being disabled has never been such a problem that I've had a memorable negative outcome, but it definitely is frustrating when you know the answer is somewhere in your messages.

The only other time Lockdown Mode has introduced friction is during Trina and I's wedding. The wedding party was sharing pictures and videos via a Shared Album which aren't available when Lockdown Mode is enabled. Fortunately, I could disable Lockdown Mode for a short time after the wedding was over, copy all the photos that I wanted, and then re-enable Lockdown Mode to work-around this.

Beyond this, some image formats don't load in any context (likely WebP?) and I haven't noticed any slowdown from not having a JavaScript JIT.

Would I recommend Lockdown Mode?

For most people: no. If you have a decent reason to expect you'd be the target of a cyberattack, then you should definitely consider it.

There is a non-zero amount of extra friction to using your phone, but as someone who's trying to actively reduce my phone usage anyway it wasn't a big issue over the year that I've had it enabled.

Bonus tip: Quick one-time disabling of biometric authentication

Privacy gated by biometrics (ie, "Face ID" or fingerprint scanners) doesn't have the same legal protections as a password. Biometrics are quite convenient, especially if you've configured a relatively short amount of time that your phone will lock itself after a lack of use.

So how can one have the benefits of biometrics while maintaining the ability to disable biometrics if needed?

By holding down the volume up and side button on your iPhone you'll bring up the screen that offers to shut down your phone or enter "SOS mode". If you select cancel on this screen your phone will become locked again but will require non-biometric authentication for the next phone unlock.

Give it a try on your phone, so you understand what to do ahead of time.

Because this process is fast (takes less than a second of holding the two buttons) it's great to have in your back pocket in case you need it. It's also useful for one-time activities when you're separated from your device such as crossing a security checkpoint.

Thanks for reading! ♡ Did you find this article helpful and want more content like it? Get notified of new posts by subscribing to the RSS feed or the email newsletter.

This work is licensed under CC BY-SA 4.0

An overview of the different options available for working with sparse arrays in Python

This tutorial looks at how to add server-side components to our client-side setup with Django.

In this tutorial, we are going to explain how to install Python on Ubuntu 24.04 OS. Python is a high-level ...

Read More

The post How to Install Python on Ubuntu 24.04 appeared first on RoseHosting.

Python has many built-in functions that you can use directly without importing anything. These functions cover a wide variety of common programming tasks that include performing math operations, working with built-in data types, processing iterables of data, handling input and output in your programs, working with scopes, and more.

In this tutorial, you’ll:

• Get to know Python’s built-in functions
• Learn about common use cases of Python’s built-in functions
• Use these functions to solve practical problems

To get the most out of this tutorial, you’ll need to be familiar with Python programming, including topics like working with built-in data types, functions, classes, decorators, scopes, and the import system.

Get Your Code: Click here to download the free sample code that shows you how to use Python’s built-in functions.

Take the Quiz: Test your knowledge with our interactive “Python's Built-in Functions: A Complete Exploration” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

Python's Built-in Functions: A Complete Exploration

Take this quiz to test your knowledge of the available built-in functions in Python. By taking this quiz, you'll deepen your understanding of how to use these functions and the common programming problems they cover, from mathematical computations to Python-specific features.

Built-in Functions in Python

Python has several functions available for you to use directly from anywhere in your code. These functions are known as built-in functions and they cover many common programming problems, from mathematical computations to Python-specific features.

Note: Many of Python’s built-in functions are classes with function-style names. Good examples are str, tuple, list, and dict, which are classes that define built-in data types. These classes are listed in the Python documentation as built-in functions and you’ll find them in this tutorial.

In this tutorial, you’ll learn the basics of Python’s built-in functions. By the end, you’ll know what their use cases are and how they work. To kick things off, you’ll start with those built-in functions related to math computations.

Using Math-Related Built-in Functions

In Python, you’ll find a few built-in functions that take care of common math operations, like computing the absolute value of a number, calculating powers, and more. Here’s a summary of the math-related built-in functions in Python:

Function Description abs() Calculates the absolute value of a number divmod() Computes the quotient and remainder of integer division max() Finds the largest of the given arguments or items in an iterable min() Finds the smallest of the given arguments or items in an iterable pow() Raises a number to a power round() Rounds a floating-point value sum() Sums the values in an iterable

In the following sections, you’ll learn how these functions work and how to use them in your Python code.

Getting the Absolute Value of a Number: abs()

The absolute value or modulus of a real number is its non-negative value. In other words, the absolute value is the number without its sign. For example, the absolute value of -5 is 5, and the absolute value of 5 is also 5.

Note: To learn more about abs(), check out the How to Find an Absolute Value in Python tutorial.

Python’s built-in abs() function allows you to quickly compute the absolute value or modulus of a number:

Python >>> from decimal import Decimal >>> from fractions import Fraction >>> abs(-42) 42 >>> abs(42) 42 >>> abs(-42.42) 42.42 >>> abs(42.42) 42.42 >>> abs(complex("-2+3j")) 3.605551275463989 >>> abs(complex("2+3j")) 3.605551275463989 >>> abs(Fraction("-1/2")) Fraction(1, 2) >>> abs(Fraction("1/2")) Fraction(1, 2) >>> abs(Decimal("-0.5")) Decimal('0.5') >>> abs(Decimal("0.5")) Decimal('0.5') Copied!

In these examples, you compute the absolute value of different numeric types using the abs() function. First, you use integer numbers, then floating-point and complex numbers, and finally, fractional and decimal numbers. In all cases, when you call the function with a negative value, the final result removes the sign.

For a practical example, say that you need to compute the total profits and losses of your company from a month’s transactions:

Python >>> transactions = [-200, 300, -100, 500] >>> incomes = sum(income for income in transactions if income > 0) >>> expenses = abs( ... sum(expense for expense in transactions if expense < 0) ... ) >>> print(f"Total incomes: ${incomes}") Total incomes: $800 >>> print(f"Total expenses: ${expenses}") Total expenses: $300 >>> print(f"Total profit: ${incomes - expenses}") Total profit: $500 Copied!

In this example, to compute the expenses, you use the abs() function to get the absolute value of the expenses, which results in a positive value.

Finding the Quotient and Remainder in Division: divmod()

Python provides a built-in function called divmod() that takes two numbers as arguments and returns a tuple with the quotient and remainder that result from the integer division of the input numbers:

Read the full article at https://realpython.com/python-built-in-functions/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

I've gathered a group of Python experts who have been thinking deeply about where Python is going and who have lived through where it has been. This episode is all about near-term Python trends and things we each believe will be important to focus on as Python continues to grow. Our panelists are Jodie Burchell, Carol Willing, and Paul Everett.<br/> <br/> <strong>Episode sponsors</strong><br/> <br/> <a href='https://talkpython.fm/code-comments'>Code Comments</a><br> <a href='https://talkpython.fm/posit'>Posit</a><br> <a href='https://talkpython.fm/training'>Talk Python Courses</a><br/> <br/> <strong>Links from the show</strong><br/> <br/> <div><b>Shiny course at Talk Python</b>: <a href="https://talkpython.fm/shiny" target="_blank" rel="noopener">talkpython.fm/shiny</a><br/> <br/> <b>Jodie Burchell</b>: <a href="https://twitter.com/t_redactyl" target="_blank" rel="noopener">@t_redactyl</a><br/> <b>Carol on Mastodon</b>: <a href="https://hachyderm.io/@willingc" target="_blank" rel="noopener">@willingc@hachyderm.io</a><br/> <b>Paul Everitt</b>: <a href="https://twitter.com/paulweveritt" target="_blank" rel="noopener">@paulweveritt</a><br/> <b>Watch this episode on YouTube</b>: <a href="https://www.youtube.com/watch?v=7398lV5edDw" target="_blank" rel="noopener">youtube.com</a><br/> <b>Episode transcripts</b>: <a href="https://talkpython.fm/episodes/transcript/468/python-trends-episode-2024" target="_blank" rel="noopener">talkpython.fm</a><br/> <br/> <b>--- Stay in touch with us ---</b><br/> <b>Subscribe to us on YouTube</b>: <a href="https://talkpython.fm/youtube" target="_blank" rel="noopener">youtube.com</a><br/> <b>Follow Talk Python on Mastodon</b>: <a href="https://fosstodon.org/web/@talkpython" target="_blank" rel="noopener"><i class="fa-brands fa-mastodon"></i>talkpython</a><br/> <b>Follow Michael on Mastodon</b>: <a href="https://fosstodon.org/web/@mkennedy" target="_blank" rel="noopener"><i class="fa-brands fa-mastodon"></i>mkennedy</a><br/></div>

Introduction

eGenix PyRun™ is our open source, one file, no installation version of Python, making the distribution of a Python interpreter to run Python based scripts and applications to Unix based systems simple and efficient.

eGenix PyRun's executable only needs 4-6MB on disk, but still supports most Python applications and scripts.

Compared to a regular Python installation of typically 100MB on disk, eGenix PyRun is ideal for applications and scripts that need to be distributed to containers, VMs, clusters, client installations, customers or end-users.

It makes "installing" Python on a Unix based system as simple as copying a single file.

eGenix has been using eGenix PyRun as run-time for the Linux version of mxODBC Connect Server product since 2008 with great success and decided to make it available as a stand-alone open-source product.

We provide the source archive to build your own eGenix PyRun on Github, as well as a few binary distributions to get you started on Linux x86_64. In the future, we will set up automated builds for several other platforms.

Please see the product page for more details:

    >>> eGenix PyRun - One file Python Runtime

News

This major release of eGenix PyRun comes with the following enhancements:

Enhancements / Changes

• Added support for Python 3.8 - 3.11
• Removed support for Python 3.5-3.7
• Modernized the directory setup and build
• Changed the license to the Apache2 license
• Extracted the code from our internal mono-repo to put on Github
• Relaunched the project on Github

For a complete list of changes, please see the eGenix PyRun Changelog. Downloads

Please visit the eGenix PyRun product page for downloads, instructions on installation and documentation of the product.

Support

Commercial support for this product is available directly from eGenix.com.

Please see the support section of our website for details.

More Information

For more information on eGenix PyRun, licensing and download instructions, please write to sales@egenix.com.

Enjoy !

Marc-Andre Lemburg, eGenix.com

EuroPython 2024 is set to be an electrifying event for Python enthusiasts, taking place in the vibrant city of Prague from July 8th to 14th. Whether you&aposre a first-timer or a seasoned attendee, here are some tips to help you squeeze every bit of goodness out of the conference.

Plan Your Schedule in Advance &#x1F5D3;️

Review the EuroPython schedule to spot sessions, workshops, and events that spark your interest. The week is loaded with activities: Monday and Tuesday are for tutorials and free workshops (C API Summit, HumbleData, DjangoGirls and WASM Summit); Wednesday to Friday are full of keynotes and talks, social events, PyLadies events and panel discussions. Finally, the weekend is reserved for the Open Source Sprints.

Mix and match your experience by attending a variety of sessions! From technical talks and tutorials to keynote speeches, there&aposs a smorgasbord of content for all levels. We have talks for beginners and advanced Pythonistas including tracks like Python internals, LLMs, Ethics & Philosophy, Web Technologies, Education & Community, DevOps, and so much more.

Pro tip: Keynote speakers always drop some serious knowledge bombs &#x1F609;Engage in Workshops & Summits &#x1F6E0;️

Dive into workshops for a hands-on learning adventure. Whether you&aposre a newbie learning to build websites with Django Girls or exploring data science with HumbleData, there&aposs something for everyone. Discuss the state of the C API at the C API Summit or get the latest on WebAssembly at the WASM Summit.

Some of these events have limited slots, while others are first-come, first-served. Make sure to register on the website https://ep2024.europython.eu/programme and arrive early on the day of the event!

Network Actively &#x1F91D;

Sessions are packed with information, but they might not cover your particular problem. Don&apost hesitate to ask the speakers about anything they didn&apost cover. Make sure to use Discord to reach out if you cannot find them in the corridors.

All participants should join our Discord server as this is the main communication platform during the conference. The link invite will be sent via email on the following days.

Take advantage of breaks to mingle and tap into the wealth of knowledge around you. And don&apost forget the Pac-Man rule: always leave a spot open so others can easily join the conversation.

Engage in the Hallway & Harness the power of networking!

Chat with fellow attendees, speakers, and sponsors during breaks, social events, and open spaces. Networking can lead to new collaborations, job opportunities, and friendships within the Python community. Here are some tips to help you start a conversation

• Look for lanyards with the "speaker" tag and ask them about their topic.
• Approach someone with a simple "Hi, what do you think of the event so far?" or "What sessions have you found interesting?"
• Pay attention to the stickers on people&aposs badges indicating their preferred level of contact: "no contact," "happy to hug," or "handshake only." Always respect these preferences.

Remember it&aposs normal to feel shy, many people do, but the Python community is known for being extra nice :) Don&apost miss out on the chance to connect with others, this is the perfect place to try and reach out.

Visit the Sponsors Hall &#x1F6CD;️

Stroll through the Sponsor&aposs Hall to check out the latest products and services from exhibitors. It&aposs swag central and a goldmine for discovering new tools and technologies to boost your Python projects. A lot of our sponsors are also actively hiring in case you are on the lookout.

If you are attending remotely, Sponsors will have channels where you can chat and find out more. Remember to also keep an eye on the Virtual Swag and Job board on our website for extra gifts.

Attend Social Events &#x1F389;

Don&apost miss the social events like the PyLadies lunch and Social Event, Prague Boat Trip, Speakers’ Dinner, and the big Social Event where you are invited to bring your board games and instruments!

These events are perfect for connecting & sharing ideas with like-minded individuals in a relaxed setting. Plus, they are a whole lot of fun! &#x1F389;

Take Care of Yourself &#x1F9D8;

Conferences are a marathon, not a sprint. Stay hydrated, eat well, and get plenty of rest. We will help by serving light lunches, snacks and coffee during the breaks throughout the day. Take breaks to recharge and keep your energy levels high.

Feeling overwhelmed? We have a Quiet Room and a low-stimulation room at the venue so you can wind down.

Explore Prague &#x1F3F0;

Seize the chance to explore the enchanting city of Prague. Dive into the local culture, savour the cuisine, and visit the landmarks. The Prague Congress Centre is conveniently located, making it easy to venture out.

Our volunteers wrote a page with tips and things to do: https://ep2024.europython.eu/explore

We hope it helps you and your companions to have a great time in Prague &#x1F603;

Share Your Experience &#x1F4E2;

After the conference, share your newfound wisdom with your community. Write a blog post, give a presentation, or organize a meetup to spread the knowledge. It reinforces what you&aposve learned and helps those who couldn&apost attend.

Make sure to tag us on socials @EuroPython or use #EuroPython2024. We will be reposting the coolest community posts during the event!

Conclusion

EuroPython 2024 promises to be an enriching experience brimming with learning, networking, and fun. By planning ahead, engaging actively, and taking care of yourself, you&aposll make the most of this incredible event.

See you in Prague! &#x1F31F;

For more details, visit the EuroPython 2024 official website.

This tutorial looks at how to build client-side UI components in Django with Stimulus and Tailwind.

I am pleased to announce the release of scikit-survival 0.23.0.

This release adds support for scikit-learn 1.4 and 1.5, which includes missing value support for RandomSurvivalForest. For more details on missing values support, see the section in the release announcement for 0.23.0.

Moreover, this release fixes critical bugs. When fitting SurvivalTree, the sample_weight is now correctly considered when computing the log-rank statistic for each split. This change also affects RandomSurvivalForest and ExtraSurvivalTrees which pass sample_weight to the individual trees in the ensemble. Therefore, the outputs produced by SurvivalTree, RandomSurvivalForest, and ExtraSurvivalTrees will differ from previous releases.

This release fixes a bug in ComponentwiseGradientBoostingSurvivalAnalysis and GradientBoostingSurvivalAnalysis when dropout is used. Previously, dropout was only applied starting with the third iteration, now dropout is applied in the second iteration too.

Finally, this release adds compatibility with numpy 2.0 and drops support for Python 3.8.

Install

scikit-survival is available for Linux, macOS, and Windows and can be installed either

via pip:

pip install scikit-survival

or via conda

conda install -c conda-forge scikit-survival

Hello! Today I have released new digital updates to my PyQt5, PyQt6, PySide2 and PySide6 book Create GUI Applications with Python & Qt.

This update brings all versions up to date with the latest developments in Qt, As well as corrections and additions to existing chapters, there are new sections dealing with form layouts, built-in dialogs and developing Qt applications using a Model View Controller (MVC) architecture.

As always, if you've previously bought a copy of the book you get these updates for free! Just go to the downloads page and enter the email you used for the purchase.

You can buy the latest editions below --

• PyQt6 Book 5th Edition, Create GUI Applications with Python & Qt6
• PySide6 Book, 5th Edition, Create GUI Applications with Python & Qt6
• PyQt5 Book 5th Edition, Create GUI Applications with Python & Qt5
• PySide2 Book 5th Edition, Create GUI Applications with Python & Qt5

If you bought the book elsewhere (in paperback or digital) you can register to get these updates too. Email your receipt to register@pythonguis.com

Enjoy!

55th issue of Andrei Neagoie's must-read monthly Python Newsletter

aider: Console-based LLM pair programming tool, to edit code in your local git repository.

Links:

• Repo
• Home

napar: A fast, interactive, multi-dimensional image viewer for annotation and analysis of large images.

Links:

• Repo
• Home
• Fund

Plane: Modern, self-hostable issue and product roadmap tracker. An alternative to JIRA, Linear, and Asana.

Links:

• Repo
• Home

Codex: Self-hostable comic archive browser and reader.

Links:

• Repo
• Demo

What techniques go into synthesizing a guitar sound in Python? What higher-level programming and Python concepts can you practice while building advanced projects? This week on the show, we talk with Real Python author and core team member Bartosz Zaczyński about his recent step-by-step project, Build a Guitar Synthesizer: Play Musical Tablature in Python.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

For months now I’ve been collecting a load of links saying that I’ll get round to blogging them "soon". Well, I’m currently babysitting for a friend’s daughter (who is sleeping peacefully upstairs), so I’ve finally found time to write them up.

So, here are a load of links – a lot of them are geospatial- or data-related, but I hope you might find something here you like even if those aren’t your specialisms:

• QGIS Visual Changelog for v3.36 – QGIS is great GIS software that I use a lot in my work. The reason this link is here though is because of their ‘visual changelog’ that they write for every new version – it is a really good way of showing what has changed in a GUI application, and does a great job of advertising the new features.
• List of unusual units of measurement – a fascinating list from Wikipedia. I think my favourite are the barn (and outhouse/shed) and the shake – you’ll have to read the article to see what these are!
• List of humourous units of measurement – linked from the above is this list of humourous units of measurement, including the ‘beard-second’, and the Smoot.
• lonboard – a relatively new package for very efficiently creating interactive webmaps of geospatial data from Python. It is so much faster than geopandas.explore(), and is developing at a rapid pace. I’m using it a lot these days.
  
• A new metric for measuring learning – an interesting post from Greg Wilson about the time it takes learners to realise that something was worth learning. I wonder what the values are for some things that I do a lot of – remote sensing, GIS, Python, cloud computing, cloud-native geospatial etc.
• The first global 30-m land-cover dynamic monitoring product with fine classification system from 1985 to 2022 – an interesting dataset that I haven’t had chance to investigate in detail yet, but purports to give 30m global land cover every 5 years from 1985 and every year from 2000.
• CyberChef – from GCHQ (yes, the UK signals intelligence agency) comes this very useful web-based tool for converting data formats, extracting data, doing basic image processing, extracting files and more. You can even build multiple tools into a ‘recipe’ or pipeline.
• envreport – a simple Python script to report lots of information about the Python environment in a diffable form – I can see this being very useful
• Antarctic glaciers named after satellites – seven Antarctic glaciers have been named after satellites, reflecting (ha!) the importance of satellites in monitoring Antarctica
• MoMAColors and MetBrewer – these are color palettes derived from artwork at the Museum of Modern Art and the Metropolitan Museum of Art in New York. There are some beautiful sets of colours in here (see below) which I want to use in some future data visualisations.

• geospatial-cli – a list of geospatial command-line tools. Lots of them I knew, but there were some gems I hadn’t come across before here too.
• map-gl-tools – a nice Javascript package to add a load of syntactic sugar to the MapLibreJS library. I’ve recently started using this library and found things quite clunky (coming from a Leaflet background) so this really helped
• CoolWalks – a nice research paper looking at creating walking routes on the shaded side of the street for various cities
• Writing efficient code for GeoPandas and Shapely in 2023 – a very useful notebook showing how to do things efficiently in GeoPandas in 2023. There are a load of old ways of doing things which are no longer the most efficient!
• Inside PostGIS: Calculating Distance – a post explaining how PostGIS’s distance calculation algorithms work
• quackosm – a Python and CLI tool for reading OpenStreetMap data using DuckDB – either for future analysis in DuckDB or for export to GeoParquet
• Comparing odc.stac.load and stackstac for raster composite workflow – fairly self-explanatory, but it’s interesting to see the differences between two widely-used tools in the STAC ecosystem
• Towards Flexible Data Schemas – this article shows how the flexibility of data schemes in specifications like STAC really help their adoption and use for diverse purposes

An extremely common problem in programming is that multiple parts of a program need to be kept in sync – they need to do exactly the same thing or behave in a consistent way. It is in response to this problem that we have mantras like “DRY” (Don’t Repeat Yourself), or, as I prefer it, OAOO, “Each and every declaration of behaviour should appear Once And Only Once”.

For both of these mantras, if you are faced with possible duplication of any kind, the answer is simply “just say no”. However, since programming mantras are to be understood as proverbs, not absolute laws, there are times that obeying this mantra can hurt more than it helps, so in this post I’m going to discuss other approaches.

Most of what I say is fairly language agnostic I think, but I’ve got specific tips for Python and web development.

Contents

• The essential problem

• The ideal solution: derive

• Alternative solution: test

• Examples

  • Example 1 - external data sources

  • Example 2 - defining UI behaviour for domain objects

  • Example 3 - external polymorphism and static typing

  • Example 4 - generated code

• Conclusion

The essential problem

To step back for a second, the essential problem that we are addressing here is that if making a change to a certain behaviour requires changing more than one place in the code, we have the risk than one will be forgotten. This results in bugs, which can be of various degrees of seriousness depending on the code in question.

To pick a concrete example, suppose we have a rule that says that items in a deleted folder get stored for 30 days, then expunged. We’re going to need some code that does the actual expunging after 30 days, but we’re also going to need to tell the user about the limit somewhere in the user interface. “Once And Only Once” says that the 30 days limit needs to be defined in a single place somewhere, and then reused.

There is a second kind of motivating example, which I think often crops up when people quote “Don’t Repeat Yourself”, and it’s really about avoiding tedious things from a developer perspective. Suppose you need to add an item to a menu, and you find out that first you’ve got to edit the MENU_ITEMS file to add an entry, then you’ve got to edit the MAIN_MENU constant to refer to the new entry, then you’ve got to define a keyboard shortcut in the MENU_SHORTCUTS file, then a menu icon somewhere else etc. All of these different places are in some way repeating things about how menus work. I think this is less important in general, but it is certainly life-draining as a developer if code is structured in this way, especially if it is difficult to discover or remember all the things that have to be done.

The ideal solution: derive

OAOO and DRY say that we aim to have a single place that defines the rule or logic, and any other place should be derived from this.

Regarding the simple example of a time limit displayed in the UI and used in the backend, this might be as simple as defining a constant e.g. in Python:

from datetime import timedelta EXPUNGE_TIME_LIMIT = timedelta(days=30)

We then import and use this constant in both our UI and backend.

An important part of this approach is that the “deriving” process should be entirely automatic, not something that you can forget to do. In the case of a Python import statement, that is very easy to achieve, and relatively hard to get wrong – if you change the constant where it is defined in one module, any other code that uses it will pick up the change the next time the Python process is restarted.

Alternative solution: test

By “test”, I mean ideally an automated test, but manual tests may also work if they are properly scripted. The idea is that you write a test that checks the behaviour or code is synced. Often, it may be that for one (or more) instances that need the behaviour will define it using some constant as above, let’s say the “backend” code. Then, for one instance, e.g. the UI, you would hard code “30 days” without using the constant, but have a test that uses the backend constant to build a string, and checks the UI for that string.

Examples

In the example above, it might be hard to see why you want to use the fundamentally less reliable, less automatic method I’m suggesting. So I now have to show some motivating examples where the “derive” method ends up losing to the cruder, simpler alternative of “test”.

Example 1 - external data sources

My first example comes from the project I’m currently working on, which involves creating CAM files from input data. Most of the logic for that is driven using code, but there are some dimensions that are specified as data tables by the engineers of the physical product.

These data tables look something like below. The details here aren’t important, and I’ve changed them – it’s enough to know that we’ve are creating some physical “widgets” which need to have specific dimensions specified:

Widgets have length 150mm unless specified below

Widget id

Location

Length (mm)

A

start

100

A

end

120

F

start

105

F

end

110

These tables are supplied at design-time rather than run-time i.e. they are bundled with the software and can’t be changed after the code is shipped. But it is still convenient to read them in automatically rather than simply duplicate the tables in my code by some process. So, for the body of the table, that’s exactly what my code does on startup – it reads the bundled XLSX/CSV files.

So we are obeying “derive” here — there is a single, canonical source of data, and anywhere that needs it derives it by an entirely automatic process.

But what about that “150mm” default value specified in the header of that table?

It would be possible to “derive” it by having a parser. Writing such a parser is not hard to do – for this kind of thing in Python I like parsy, and it is as simple as:

import parsy as P default_length_parser = ( P.string("Widgets have length ") >> P.regex(r"\d+").map(int) << P.string("mm unless specified below") )

In fact I do something similar in some cases. But in reality, the “parser” here is pretty simplistic – it can’t deal with the real variety of English text that might be put into the sentence, and to claim I’m “deriving” it from the table is a bit of a stretch – I’m just matching a specific, known pattern. In addition, it’s probably not the case that any value for the default length would work – most likely if it was 10 times larger, there would be some other problem, and I’d want to do some manual checking.

So, let’s admit that we are really just checking for something expected, using the “test” approach. You can still define a constant that you use in most of the code:

DEFAULT_LENGTH_MM = 150

And then you test it is what you expect when you load the data file:

assert worksheets[0].cell(1, 1).value == f"Widgets have length {DEFAULT_LENGTH_MM}mm unless specified below"

So, I’ve achieved my aim: a guard against the original problem of having multiple sources of information that could potentially be out of sync. But I’ve done it using a simple test, rather than a more complex and fragile “derive” that wouldn’t have worked well anyway.

By the way, for this specific project – we’re looking for another contract developer! It’s a very worthwhile project, and one I’m really enjoying – a small flexible team, with plenty of problem solving and fun challenges, so if you’re a talented developer and interested give me a shout.

Example 2 - defining UI behaviour for domain objects

Suppose you have a database that stores information about some kind of entity, like customers say, and you have different types of customer, represented using an enum of some kind, perhaps a string enum like this in Python:

from enum import StrEnum class CustomerType(StrEnum): ENTERPRISE = "Enterprise" SMALL_FRY = "Small fry" # Let’s be honest! Try not to let the name leak… LEGACY = "Legacy"

We need to a way edit the different customer types, and they are sufficiently different that we want quite different interfaces. So, we might have a dictionary mapping the customer type to a function or class that defines the UI. If this were a Django project, it might be a different Form class for each type:

CUSTOMER_EDIT_FORMS = { CustomerType.ENTERPRISE: EnterpriseCustomerForm, CustomerType.SMALL_FRY: SmallFryCustomerForm, CustomerType.LEGACY: LegacyCustomerForm, }

Now, the DRY instinct kicks in and we notice that we now have two things we have to remember to keep in sync — any addition to the customer enum requires a corresponding addition to the UI definition dictionary. Maybe there are multiple dictionaries like this.

We could attempt to solve this by “deriving”, or some “correct by construction” mechanism that puts the creation of a new customer type all in one place.

For example, maybe we’ll have a base Customer class with get_edit_form_class() as an abstractmethod, which means it is required to be implemented. If I fail to implement it in a subclass, I can’t even construct an instance of the new customer subclass – it will throw an error.

from abc import abstractmethod class Customer: @abstractmethod def get_edit_form_class(self): pass class EnterpriseCustomer(Customer): def get_edit_form_class(self): return EnterpriseCustomerForm class LegacyCustomer(Customer): ... # etc.

I still need my enum value, or at least a list of valid values that I can use for my database field. Maybe I could derive that automatically by looking at all the sublclasses?

CUSTOMER_TYPES = [ cls.__name__.upper().replace("CUSTOMER", "") for cls in Customer.__subclasses__() ]

Or maybe an __init_subclass__ trick, and I can perhaps also set up the various mappings I’ll need that way?

It’s at this point you should stop and think. In addition to requiring you to mix UI concerns into the Customer class definitions, it’s getting complex and magical.

The alternative I’m suggesting is this: require manual syncing of the two parts of the code base, but add a test to ensure that you did it. All you need is a few lines after your CUSTOMER_EDIT_FORMS definition:

CUSTOMER_EDIT_FORMS = { # etc as before } for c_type in CustomerType: assert ( c_type in CUSTOMER_EDIT_FORMS ), f"You've defined a new customer type {c_type}, you need to add an entry in CUSTOMER_EDIT_FORMS"

You could do this as a more traditional unit test in a separate file, but for simple things like this, I think an assertion right next to the code works much better. It really helps local reasoning to be able to look and immediately conclude “yes, I can see that this dictionary must be exhaustive because the assertion tells me so.” Plus you get really early failure – as soon as you import the code.

This kind of thing crops up a lot – if you create a class here, you’ve got to create another one over there, or add a dictionary entry etc. In these cases, I’m finding simple tests and assertions have a ton of advantages when compared to clever architectural contortions (or other things like advanced static typing gymnastics):

• they are massively simpler to create and understand.

• you can write your own error message in the assertion. If you make a habit of using really clear error messages, like the one above, your code base will literally tell you how to maintain it.

• you can easily add things like exceptions. “Every Customer type needs an edit UI defined, except Legacy because they are read only” is an easy, small change to the above.

  • This contrasts with cleverer mechanisms, which might require relaxing other constraints to the point where you defeat the whole point of the mechanism, or create more difficulties for yourself.

• the rule about how the code works is very explicit, rather than implicit in some complicated code structure, and typically needs no comment other than what you write in the assertion message.

• you express and enforce the rule, with any complexities it gains, in just one place. Ironically, if you try to enforce this kind of constraint using type systems or hierarchies to eliminate repetition or the need for any kind of code syncing, you may find that when you come to change the constraint it actually requires touching far more places.

• temporarily silencing the assertion while developing is easy and doesn’t have far reaching consequences.

Of course, there are many times when being able to automatically derive things at the code level, including some complex relationships between parts of the code, can be a win, and it’s the kind of thing you can do in Python with its many powerful techniques.

But my point is that you should remember the alternative: “synchronise manually, and have a test to check you did it.” Being able to add any kind of executable code at module level – the same level as class/function/constant definitions – is a Python super-power that you should use.

Example 3 - external polymorphism and static typing

A variant of the above problem is when, instead of an enum defining different types, I’ve got a set of classes that all need some behaviour defined.

Often we just use polymorphism where a base class defines the methods or interfaces needed and sub-classes provide the implementation. However, as in the previous case, this can involve mixing concerns e.g. user interface code, possibly of several types, is mixed up with the base domain objects. It also imposes constraints on class hierarchies.

Recently for these kind of cases, I’m more likely to prefer external polymorphism to avoid these problems. To give an example, in my current project I’m using the Command pattern or plan-execute pattern extensively, and it involves manipulating CAM objects using a series of command objects that look something like this:

@dataclass class DeleteFeature: feature_name: str @dataclass class SetParameter: param_name: str value: float @dataclass class SetTextSegment: text_name: str segment: int value: str Command: TypeAlias = DeleteFeature | SetParameter | SetTextSegment

Note that none of them share a base class, but I do have a union type that gives me the complete set.

It’s much more convenient to define the behaviour associated with these separately from these definitions, and so I have multiple other places that deal with Command, such as the place that executes these commands and several others. One example that requires very little code to show is where I’m generating user-presentable tables that show groups of commands. I convert each of these Command objects into key-value pairs that are used for column headings and values:

def get_command_display(command: Command) -> tuple[str, str | float | bool]: match command: case DeleteFeature(feature_name=feature_name): return (f"Delete {feature_name}", True) case SetParameter(param_name=param_name, value=value): return (param_name, value) case SetTextSegment(text_name=text_name, segment=segment, value=value): return (f"{text_name}[{segment}]", value)

This is giving me a similar problem to the one I had before I had before: if I add a new Command, I have to remember to add the new branch to get_command_display.

I could split out get_command_display into a dictionary of functions, and apply the same technique as in the previous example, but it’s more work, a less natural fit for the problem and potentially less flexible.

Instead, all I need to do is add exhaustiveness checking with one more branch:

match command: ... # etc case _: assert_never(command)

Now, pyright will check that I didn’t forget to add branches here for any new Command. The error message is not controllable, in contrast to hand-written asserts, but it is clear enough.

The theme here is that additions in one part of the code require synchronised additions in other parts of the code, rather than being automatically correct “by construction”, but you have something that tests you didn’t forget.

Example 4 - generated code

In web development, ensuring consistent design and keeping different things in sync is a significant problem. There are many approaches, but let’s start with the simple case of using a single CSS stylesheet to define all the styles.

We may want a bunch of components to have a consistent border colour, and a first attempt might look like this (ignoring the many issues of naming conventions here):

.card-component, .bordered-heading { border-color: #800; }

This often becomes impractical when we want to organise by component, rather than by property, which introduces duplication:

.card-component { border-color: #800; } /* somewhere far away … */ .bordered-heading { border-color: #800; }

Thankfully, CSS has variables, so the first application of “derive” is straightforward – we define a variable which we can use in multiple places:

:root { --primary-border-color: #800; } /* elsewhere */ .bordered-heading { border-bottom: 1px solid var(--primary-border-color); }

However, as the project grows, we may find that we want to use the same variables in different contexts where CSS isn’t applicable. So the next step at this point is typically to move to Design Tokens.

Practically speaking, this might mean that we now have our variables defined in a separate JSON file. Maybe something like this (using a W3C draft spec):

{ "primary-border-color": { "$value": "#800000", "$type": "color" } "primary-hightlight-color": { "$value": "#FBC100", "$type": "color" } }

From this, we can automatically generate CSS fragments that contain the same variables quite easily – for simple cases, this isn’t more than a 50 line Python script.

However, we’ve got some choices when it comes to how we put everything together. I think the general assumption in web development world is that a fully automatic “derive” is the only acceptable answer. This typically means you have to put your own CSS in a separate file, and then you have a build tool that watches for changes, and compiles your CSS plus the generated CSS into the final output that gets sent to the browser.

In addition, once you’ve bought into these kind of tools you’ll find they want to do extensive changes to the output, and define more and more extensions to the underlying languages. For example, postcss-design-tokens wants you to write things like:

.foo { color: design-token('color.background.primary'); }

And instead of using CSS variables in the output, it puts the value of the token right in to every place in your code that uses it.

This approach has various problems, in particular that you become more and more dependent on the build process, and the output gets further from your input. You can no longer use the Dev Tools built in to your browser to do editing – the flow of using Dev Tools to experiment with changing a single spacing or colour CSS variable for global changes is broken, you need your build tool. And you can’t easily copy changes from Dev Tools back into the source, because of the transformation step, and debugging can be similarly difficult. And then, you’ll probably want special IDE support for the special CSS extensions, rather than being able to lean on your editor simply understanding CSS, and any other tools that want to look at your CSS now need support etc.

It’s also a lot of extra infrastructure and complexity to solve this one problem, especially when our design tokens JSON file is probably not going to change that often, or is going to have long periods of high stability. There are good reasons to want to be essentially build free. The current state of the art in this space is that to get your build tool to compile your CSS you add import './styles.css' in your entry point Javascript file! What if I don’t even have a Javascript file? I think I understand how this sort of thing came about, but don’t try to tell me that it’s anything less than completely bonkers.

Do we have an alternative to the fully automatic derive?

Using the “test” approach, we do. We can even stick with our single CSS file – we just write it like this:

/* DESIGN TOKENS START */ /* auto-created block - do not edit */ :root { --primary-border-color: #800000; --primary-highlight-color: #FBC100; } /* DESIGN TOKENS END */ /* the rest of our CSS here */

The contents of this block will be almost certainly auto-generated. We won’t have a process that fully automatically updates it, however, because this is the same file where we are putting our custom CSS, and we don’t want any possibility of lost work due to the file being overwritten as we are editing it.

On the other hand we don’t want things to get out of sync, so we’ll add a test that checks whether the current styles.css contains the block of design tokens that we expect to be there, based on the JSON. For actually updating the block, we’ll need some kind of manual step – maybe a script that can find and update the DESIGN TOKEN START block, maybe cog – which is a perfect little tool for this use case — or we could just copy-paste.

There are also slightly simpler solutions in this case, like using a CSS import if you don’t mind having multiple CSS files.

Conclusion

For all the examples above, the solutions I’ve presented might not work perfectly for your context. You might also want to draw the line at different place to me. But my main point is that we don’t have to go all the way with a fully automatic derive solution to eliminate any manual syncing. Having some manual work plus a mechanism to test that two things are in sync is a perfectly legitimate solution, and it can avoid some of the large costs that come with structuring everything around “derive”.

 

I'm pleased to announce the release of Python 3.13 beta 3.

https://www.python.org/downloads/release/python-3130b3/

 

This is a beta preview of Python 3.13

Python 3.13 is still in development. This release, 3.13.0b3, is the third of four beta release previews of 3.13.

Beta release previews are intended to give the wider community the opportunity to test new features and bug fixes and to prepare their projects to support the new feature release.

We strongly encourage maintainers of third-party Python projects to test with 3.13 during the beta phase and report issues found to the Python bug tracker as soon as possible. While the release is planned to be feature complete entering the beta phase, it is possible that features may be modified or, in rare cases, deleted up until the start of the release candidate phase (Tuesday 2024-07-30). Our goal is to have no ABI changes after beta 4 and as few code changes as possible after 3.13.0rc1, the first release candidate. To achieve that, it will be extremely important to get as much exposure for 3.13 as possible during the beta phase.

 

Please keep in mind that this is a preview release and its use is not recommended for production environments.

 Major new features of the 3.13 series, compared to 3.12

Some of the new major new features and changes in Python 3.13 are:

New features

• A new and improved interactive interpreter , based on PyPy’s, featuring multi-line editing and color support, as well as colorized exception tracebacks .
• An experimental free-threaded build mode , which disables the Global Interpreter Lock, allowing threads to run more concurrently. The build mode is available as an experimental feature in the Windows and macOS installers as well.
• A preliminary, experimental JIT , providing the ground work for significant performance improvements.
• The (cyclic) garbage collector is now incremental , which should mean shorter pauses for collection in programs with a lot of objects.
• A modified version of mimalloc is now included, optional but enabled by default if supported by the platform, and required for the free-threaded build mode.
• Docstrings now have their leading indentation stripped, reducing memory use and the size of .pyc files. (Most tools handling docstrings already strip leading indentation.)
• The dbm module has a new dbm.sqlite3 backend that is used by default when creating new files.
• The minimum supported macOS version was changed from 10.9 to 10.13 (High Sierra). Older macOS versions will not be supported going forward.
  

Typing

• Support for type defaults in type parameters.
• A new type narrowing annotation , typing.TypeIs.
• A new annotation for read-only items in TypeDicts .

Removals and new deprecations

• PEP 594 (Removing dead batteries from the standard library) scheduled removals of many deprecated modules: aifc, audioop, chunk, cgi, cgitb, crypt, imghdr, mailcap, msilib, nis, nntplib, ossaudiodev, pipes, sndhdr, spwd, sunau, telnetlib, uu, xdrlib, lib2to3.
• Many other removals of deprecated classes, functions and methods in various standard library modules.
• C API removals and deprecations. (Some removals present in alpha 1 were reverted in alpha 2, as the removals were deemed too disruptive at this time.)
• New deprecations, most of which are scheduled for removal from Python 3.15 or 3.16.

(Hey, fellow core developer, if a feature you find important is missing from this list, let Thomas know.)

For more details on the changes to Python 3.13, see What’s new in Python 3.13 . The next pre-release of Python 3.13 will be 3.13.0b4, currently scheduled for 2024-07-16.

 More resources

• Online Documentation
• PEP 719 , 3.13 Release Schedule
• Report bugs at Issues · python/cpython · GitHub .
• Help fund Python directly (or via GitHub Sponsors), and support the Python community.

 Enjoy the new releases

Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation.

Your release team,
Thomas Wouters
Łukasz Langa
Ned Deily
Steve Dower