Planet Python

Subscribe to Planet Python feed
Planet Python -
Updated: 4 hours 1 min ago

Test and Code: 221: How to get pytest to import your code under test

Mon, 2024-06-03 14:23

We've got some code we want to test, and some tests.
The tests need to be able to import the code under test, or at least the API to it, in order to run tests against it.
How do we do that? How do we set things up so that our tests can import our code?

In this episode, we discuss two options:

  • Installing the code under test as a pip installable package with `pip install -e /path/to/local/package`.
  • Using the pythonpath pytest setting.

Sponsored by

  • An Email Delivery Platform that developers love. 
  • An email-sending solution with industry-best analytics, SMTP, and email API, SDKs for major programming languages, and 24/7 human support. 
  • Try for Free at MAILTRAP.IO

Sponsored by The Complete pytest Course

  • For the fastest way to learn pytest, go to
  • Whether your new to testing or pytest, or just want to maximize your efficiency and effectiveness when testing.
<p>We've got some code we want to test, and some tests.<br>The tests need to be able to import the code under test, or at least the API to it, in order to run tests against it.<br>How do we do that? How do we set things up so that our tests can import our code?</p><p>In this episode, we discuss two options:</p><ul><li>Installing the code under test as a pip installable package with `pip install -e /path/to/local/package`.</li><li>Using the <a href="">pythonpath pytest setting</a>.</li></ul> <br><p><strong>Sponsored by</strong></p><ul><li>An Email Delivery Platform that developers love. </li><li>An email-sending solution with industry-best analytics, SMTP, and email API, SDKs for major programming languages, and 24/7 human support. </li><li>Try for Free at <a href="">MAILTRAP.IO</a></li></ul><p><strong>Sponsored by</strong> <strong>The Complete pytest Course</strong></p><ul><li>For the fastest way to learn pytest, go to <a href=""></a></li><li>Whether your new to testing or pytest, or just want to maximize your efficiency and effectiveness when testing.</li></ul>
Categories: FLOSS Project Planets

Python Morsels: Python's many command-line utilities

Mon, 2024-06-03 13:16

Every command-line tool included with Python. These can be run with python -m module_name.

Table of contents

  1. How -m works
  2. General-purpose CLI tools
  3. Especially handy on Windows machines
  4. Working with Python code
  5. Analyzing Python code
  6. Just for fun
  7. Other Python-related tools
  8. Oddly meta tools
  9. Less useful tools
  10. Every command-line tool in Python

How -m works

Running Python with the -m command-line argument tells Python to run a given Python module as if it were a Python script.

Some modules do something at import time. For example the antigravity module will open up a web browser for an XKCD comic. Running this module from the command-line would do the same thing as importing it:

$ python -m antigravity

This is called an "import side effect" and most modules avoid import side effects. Fun Easter egg modules like antigravity and this are the exception.

Modules that avoid import side effects need a different mechanism to change their behavior when run as a command-line script or when imported as a module. Python uses a __name__ variable to distinguish between importing a module and running a module as a script.

When Python runs a module as a script, it sets the module's name to the string "__main__" (normally __name__ would contain the module's actual name). See more in defining a main function in Python.

For packages, Python also looks for a file to run (there's one in the zipfile package for example).

This distinction between module versus script allows for some really nifty command-line tools.

General-purpose CLI tools

The first tools we'll look …

Read the full article:
Categories: FLOSS Project Planets

Real Python: String Interpolation in Python: Exploring Available Tools

Mon, 2024-06-03 10:00

String interpolation allows you to create strings by inserting objects into specific places in a target string template. Python has several tools for string interpolation, including f-strings, the str.format() method, and the modulo operator (%). Python’s string module also provides the Template class, which you can use for string interpolation.

In this tutorial, you’ll:

  • Learn how to use f-strings for eager string interpolation
  • Perform lazy string interpolation using the str.format() method
  • Learn the basics of using the modulo operator (%) for string interpolation
  • Decide whether to use f-strings or the str.format() method for interpolation
  • Create templates for string interpolation with string.Template

To get the most out of this tutorial, you should be familiar with Python strings, which are represented by the str class.

Get Your Code: Click here to download the free sample code you’ll use to explore string interpolation tools in Python.

Take the Quiz: Test your knowledge with our interactive “String Interpolation in Python: Exploring Available Tools” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

String Interpolation in Python: Exploring Available Tools

Take this quiz to test your understanding of the available tools for string interpolation in Python, as well as their strengths and weaknesses. These tools include f-strings, the .format() method, and the modulo operator.

String Interpolation in Python

Sometimes, when working with strings, you’d make up strings by using multiple different string values. Initially, you could use the plus operator (+) to concatenate strings in Python. However, this approach results in code with many quotes and pluses:

Python >>> name = "Pythonista" >>> day = "Friday" # Of course 😃 >>> "Hello, " + name + "! Today is " + day + "." 'Hello, Pythonista! Today is Friday.' Copied!

In this example, you build a string using some text and a couple of variables that hold string values. The many plus signs make the code hard to read and write. Python must have a better and cleaner way.

Note: To learn more about string concatenation in Python, check out the Efficient String Concatenation in Python tutorial.

The modulo operator (%) came to make the syntax a bit better:

Python >>> "Hello, %s! Today is %s." % (name, day) 'Hello, Pythonista! Today is Friday.' Copied!

In this example, you use the modulo operator to insert the name and day variables into the string literals. The process of creating strings by inserting other strings into them, as you did here, is known as string interpolation.

Note: Formatting with the modulo operator is inspired by printf() formatting used in C and many other programming languages.

The %s combination of characters is known as a conversion specifier. They work as replacement fields. The % operator marks the start of the specifier, while the s letter is the conversion type and tells the operator that you want to convert the input object into a string. You’ll learn more about conversion specifiers in the section about the modulo operator.

Note: In this tutorial, you’ll learn about two different types of string interpolation:

  1. Eager interpolation
  2. Lazy interpolation

In eager interpolation, Python inserts the values into the string at execution time in the same place where you define the string. In lazy interpolation, Python delays the insertion until the string is actually needed. In this latter case, you create string templates at one point in your code and fill the template with values at another point.

But the story doesn’t end with the modulo operator. Later, Python introduced the str.format() method:

Python >>> "Hello, {}! Today is {}.".format(name, day) 'Hello, Pythonista! Today is Friday.' Copied!

The method interpolates its arguments into the target string using replacement fields limited by curly brackets. Even though this method can produce hard-to-read code, it represents a significant advance over the modulo operator: it supports the string formatting mini-language.

Note: String formatting is a fundamental topic in Python, and sometimes, people think that formatting and interpolation are the same. However, they’re not. In this tutorial, you’ll only learn about interpolation. To learn about string formatting and the formatting mini-language, check out the Python’s Format Mini-Language for Tidy Strings tutorial.

Python continues to evolve, and every new version brings new, exciting features. Python 3.6 introduced formatted string literals, or f-strings for short:

Python >>> f"Hello, {name}! Today is {day}." 'Hello, Pythonista! Today is Friday.' Copied!

F-strings offer a more readable and clean way to create strings that include other strings. To make an f-string, you must prefix it with an f or F. Again, curly brackets delimit the replacement fields.

Note: To learn more about f-strings, check out the Python’s F-String for String Interpolation and Formatting tutorial.

Read the full article at »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Categories: FLOSS Project Planets

Mike Driscoll: Python Logging Book Released!

Mon, 2024-06-03 09:09

The latest Python book from Michael Driscoll is now out. You can get Python Logging today on any of your favorite platforms!  The Kindle version of the book is only 99 cents for a limited time!

What does every new developer do when they are first learning to program? They print out strings to their terminal. It’s how we learn! But printing out to the terminal isn’t what you do with most professional applications.

In those cases, you log in. Sometimes, you log into multiple locations at once. These logs may serve as an audit trail for compliance purposes or help the engineers debug what went wrong.

Python Logging teaches you how to log in the Python programming language. Python is one of the most popular programming languages in the world. Python comes with a logging module that makes logging easy.

What You’ll Learn

In this book, you will learn how about the following:

  • Logger objects
  • Log levels
  • Log handlers
  • Formatting your logs
  • Log configuration
  • Logging decorators
  • Rotating logs
  • Logging and concurrency
  • and more!
Where to Purchase

You can get Python Logging at the following websites:

Download the Code

You can get the code for the book from GitHub:


The post Python Logging Book Released! appeared first on Mouse Vs Python.

Categories: FLOSS Project Planets

PyCharm: The State of Django 2024

Mon, 2024-06-03 08:19

Are you curious to discover the latest trends in Django development?

In collaboration with the Django Foundation, PyCharm surveyed more than 4,000 Django developers from around the globe and analyzed the trends in framework usage based on their answers.

In this blog post, we share the following key findings with you:

  • Every third Django developer also uses Flask or FastAPI.
  • Most developers use Django for both full-stack and API development.
  • 61% of Django developers use asynchronous technologies.
  • And many more insights!

Dive in to learn about these findings in more detail and discover other trends in Django development, while also benefiting from illustrative infographics.

Backend: Every third Django developer also uses Flask or FastAPI

Django remains the go-to framework for 74% of developers, though this does represent a minor dip from last year when this figure stood at 83%. FastAPI has managed to maintain its popularity, as 25% of respondents reported using it. Meanwhile, Flask saw a slight decline in popularity (29% in 2022 to 26% in 2023).

33% of web developers who work primarily with Django also use Flask or FastAPI, showing diverse backend skills.

Taking into account that the majority of fully employed developers (49%) report working on several projects at the same time, this may indicate that they choose different tools for different purposes:

  • Django – for larger, more complex web apps due to its “batteries included” approach.
  • Flask – for simpler applications (especially static sites) or microservices.
  • FastAPI – to create API endpoints, especially if your application includes a lot of IO calls (especially for real-time web applications).

The fact that only 11% of all Django developers use all three frameworks might mean that most of them use Flask and FastAPI for similar purposes, shifting to FastAPI due to its async capabilities.

Want to learn how Django is different from Flask and FastAPI? Check out our detailed comparisons between Django and Flask, and Django and FastAPI to see which framework will best suit your needs.

Developing APIs: Most developers use Django for both full-stack and API development

This year’s survey revealed Django is popular for both full-stack (74%) and API development (60%), with a trend towards API work among fully employed devs. Fully employed developers are more likely to use Django for REST API development (65% vs. 60% on average), but less likely to use it for full-stack development (68% vs. 74% on average).

With the rising popularity of htmx, the trend might change in favor of using Django more for full-stack development. 

Interestingly, while DRF has retained the pole position among third-party packages, its popularity dipped as Django Ninja, known for its speed and typing capabilities, continues to gain ground. Django Ninja offers high performance and asynchronous capabilities, similar to another very popular choice for creating APIs, FastAPI, but within the Django ecosystem, which makes the learning curve shorter.

Work with APIs? Read this tutorial to learn how to build APIs with the Django REST framework.

Async: 61% of Django developers use async

There’s a clear transition towards using asynchronous technologies among Django developers, with 61% now incorporating async in their projects (up from 53% last year).

FastAPI, which was built with async programming in mind, is now used by 21% of all Django developers who use async technologies. Django async views are also being used by more respondents (14%), though FastAPI is still more popular for async tasks. With more async support planned for upcoming Django 5 releases, the interest in using async in Django might increase even more.

Frontend: a shift in Django developers’ preferences towards htmx, Alpine.js, and Tailwind CSS

When it comes to frontend, JavaScript is still the most popular language, with 68% of developers stating that they use it. That said, it’s gradually conceding its comfortable lead (75% in 2021 and 2022) to TypeScript, which has grown significantly from 19% in 2021 to 28% in 2023. ​​This increase in popularity is probably attributable to its static typing features, which help catch errors early in the development process and thus make code more robust and maintainable.

Professional Django developers still clearly favor JavaScript frameworks over their competitors, with usage rates of 26% for Vue, 35% for jQuery, and 42% for React, though their overall use is declining year over year.

Newer frameworks like htmx (which grew from 16% in 2022 to 23% in 2023) and Alpine.js (which grew from 6% to 10%) are rapidly gaining traction, suggesting a shift towards simpler tools for modern user interfaces. There is a dedicated django-htmx package developed by Adam Johnson.

Dennis Ivy on htmx:

“Happy to see HTMX here. I’m not the biggest fan (I default to React) but I think it’s suitable for many Django projects that need a little flexibility without having to move to a full fledged JS framework/library.”

We continue to see a downward trend for Bootstrap and significant growth for Tailwind CSS, whose popularity has doubled in the last two years. The increasing preference for Tailwind CSS over Bootstrap suggests a desire for more customizable and less prescriptive approaches to styling in web projects. Read this article from package creator Tim Kamanin for a guided introduction to using Tailwind CSS in Django

Dennis Ivy on TailwindCSS:

“Love to see the rise of Tailwind-Django usage. Hope to see more native integration and educational content in this area.” Databases: 75% of Django developers favor PostgreSQL, and 50% rely on Redis for caching

In the Django ecosystem, PostgreSQL leads the pack as the primary database choice (76%) among developers, highlighting the preference for robust, SQL-based systems for web applications. The interest in NoSQL databases like MariaDB (10%) and MongoDB (8%) is also notable, reflecting a diversifying database landscape.

The inclusion of the schema-less and scalable MongoDB among the top database choices, despite its lack of official Django support, reflects developers’ willingness to integrate more flexible, document-oriented databases.

Do you work with MongoDB? Read this step-by-step guide on how to connect Django with MongoDB.

Dennis Ivy on MongoDB:

“This number surprises me considering the different approach Mongo takes and the incompatibility with Django. Curious to know the experience level of those 8%. I would suspect that those are newer devs and / or devs running experimental projects. 

After speaking to several MongoDB team members I was able to conclude my thoughts on this integration in an article.”

On the caching front, Redis remains the go-to solution (54%) for enhancing web app responsiveness, with Memcached (20%) also gaining traction.

Orchestration: More than 50% of Django developers use container orchestration 

Among the favored services, Amazon ECS/Fargate (19%) leads by virtue of its ease of use and integration with AWS, making it a natural choice for developers in the AWS ecosystem. 

Self-managed Kubernetes (14%) appeals to those seeking flexibility and control over their infrastructure, as well as the ability to easily migrate and share between private and public clouds. The popularity of Amazon EKS (12%) and Docker Swarm (12%) might be attributable to the balance they offer between manageability and scalability, catering to various deployment needs.

Deal with Kubernetes infrastructure? Read this guide on how to deploy Django apps in Kubernetes.

CI systems: GitHub Actions is leading the industry

GitHub Actions’ growth (45% in 2023 compared to 35% in 2021) in the CI field highlights its convenience for developers who already use GitHub for source code management. Its simplicity, leveraging straightforward YAML files for pipeline management, makes it an accessible and efficient tool for automating software workflows directly within GitHub’s ecosystem. Moreover, it offers the flexibility of using custom hardware configurations that meet your needs with enough processing power or memory to run larger jobs.

IaC: Infrastructure as code is used by 39% of Django developers

The use of infrastructure as code solutions by 39% of respondents underscores a growing trend towards automation and infrastructure management through code. For larger projects, IaC can ensure more reliable, repeatable, and scalable infrastructure setups. Terraform is the most commonly used infrastructure as code provisioning engine, with 20% of all respondents preferring it over the other options.

Interestingly, an open-source solution, Pulumi, was chosen by 5% of the respondents. The reason for this could be the fact that, right from its inception, Pulumi has offered the flexibility to use any programming language for managing infrastructure. This makes Pulumi widely accessible to developers and DevOps engineers from any background. Terraform began offering a similar option via a CDK in 2022.

Insights based on Django developers’ jobs and experience Django learning resources

Fully employed developers watch less YouTube to learn Django (32% vs. 39% on average) and use fewer AI tools for this purpose (22% vs. 25% on average).

Among team leads, the Django News newsletter, HackerNews, Reddit, and even X (formerly Twitter) are more popular methods of staying up to date with Django developments. They even report learning more often from their friends (16% vs. 11% on average). 

Junior professionals use YouTube and StackOverflow much more often for both educational purposes and to keep abreast of developments in the Django ecosystem. When it comes to their preferred learning method, they tend to use new AI tools more often than their senior colleagues (38% vs. 25% on average).

  • Fully employed Django developers are more likely to use Django only for work (23% vs. 17% on average). 
  • The core component that team leads and fully employed developers as a group favor most is the possibility of migration, with their less preferred components including authentication, templates, and even class-based views.
  • As their main editor, team leads tend to prefer PyCharm (31% vs. 29% on average) and Vim (12% vs. 7% on average) to VS Code (31% vs. 29% on average).
Start developing Django apps with PyCharm

Do you work with Django? PyCharm is the best-in-class IDE for Django. Code faster with Django-specific code insights, code completion, and highlighting. Navigate across your project easily. Connect to your database in a single click, and work on TypeScript, JavaScript, and other frontend frameworks. PyCharm also supports Flask and FastAPI out of the box.

Try PyCharm for free

Survey demographics

After filtering out duplicate and unreliable responses, the data set includes around 4,000 responses collected between September and October 2023.

Regional distribution

44% of respondents are based in Europe, 19% in North America, and 17% in Asia.

Age distribution

Most of the respondents are in the 21–49 age range. 38% of all respondents are aged 31–39, while 30% are aged 21–29.

Professional coding experience

The majority of respondents have been coding professionally for 11+ years. 24% of the survey participants have 3–5 years of professional coding experience, and 19% have worked as professional developers for 6–10 years. Those who have been developing professionally for less than two years represent another 25% of the survey respondents.

Job roles

79% of respondents stated that their job roles include development/programming or software engineering. 16% of respondents are team leads. 10% of the respondents stated that their job includes data analysis, data engineering, or data science. 

The data set includes responses only from official Django Software Foundation channels. The responses were collected through the promotion of the survey on official Django channels, such as and the DSF’s X (formerly Twitter) account, without the involvement of any JetBrains channels. In order to prevent the survey from being slanted in favor of any specific tool or technology, no product-, service-, or vendor-related channels were used to collect responses.

Want to learn more? Explore the Django Developers Survey 2023 to see the full survey data.

Categories: FLOSS Project Planets

Real Python: Quiz: String Interpolation in Python: Exploring Available Tools

Mon, 2024-06-03 08:00

Test your understanding of Python’s tools for string interpolation, including f-strings, the .format() method, and the modulo operator.

Take this quiz after reading our String Interpolation in Python: Exploring Available Tools tutorial.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Categories: FLOSS Project Planets

Robin Wilson: Introducing pyAURN – a Python package for accessing UK air quality data

Mon, 2024-06-03 06:04

I realised recently that I’d never actually blogged about my pyAURN package – so it’s about time that I did.

When doing some freelance work on air quality a while back, I wanted an easy way to access UK air quality from the Automatic Urban and Rural Network (AURN). Unfortunately, there isn’t a nice API for accessing the data. Strangely, though, they do provide the data in a series of RData files for use by the openair R package. I wanted to use the data from Python though – but conveniently there is a Python package for reading RData files.

So, I put all these together into a simple Python package called pyAURN. It is strongly based upon openair, but is a lot more limited in functionality – it basically only covers importing the data, and doesn’t have many plotting or analysis functions.

Here’s an example of how to use it:

from pyaurn import importAURN, importMeta, timeAverage # Download metadata of site IDs, names, locations etc metadata = importMeta() # Download 4 years of data for the Marylebone Road site # (MY1 is the site ID for this site) # Note: range(2016, 2022) will produce a list of six years: 2016, 2017, 2018, 2019, 2020, and 2021. # Alternatively define a list of years to use eg. [2016,2017,2018,2019,2020,2021] data = importAURN("MY1", range(2016, 2022)) # Group the DataFrame by a frequency of monthly, and the statistic mean(). data_monthly = timeAverage(data,avg_time="month",statistic="mean")

I found this really useful for my air quality analysis work – and I hope you do too. The package is on PyPI, so you can run pip install pyaurn or view the project on Github.

Categories: FLOSS Project Planets

Zato Blog: New API Integration Tutorial in Python

Sun, 2024-06-02 04:00
New API Integration Tutorial in Python 2024-06-02, by Dariusz Suchojad

Do you know what airports, telecom operators, defense forces and health care organizations have in common?

They all rely heavily on deep-backend software systems which are integrated and automated using principled methodologies, innovative techniques and well-defined implementation frameworks.

If you'd like to learn how to integrate and automate such complex systems correctly, head over to the new API integration tutorial that will show you how to do it in Python too.

# -*- coding: utf-8 -*- # Zato from zato.server.service import Service # ############################################################################## class MyService(Service): """ Returns user details by the person's name. """ name = '' # I/O definition input = '-name' output = 'user_type', 'account_no', 'account_balance' def handle(self): # For later use name = or 'partner' # REST connections crm_conn =['CRM'].conn billing_conn =['Billing'].conn # Prepare requests crm_request = {'UserName':name} billing_params = {'USER':name} # Get data from CRM crm_data = crm_conn.get(self.cid, crm_request).data # Get data from Billing billing_data =, params=billing_params).data # Extract the business information from both systems user_type = crm_data['UserType'] account_no = crm_data['AccountNumber'] account_balance = billing_data['ACC_BALANCE']'cid:{self.cid} Returning user details for {name}') # Now, produce the response for our caller self.response.payload = { 'user_type': user_type, 'account_no': account_no, 'account_balance': account_balance, } # ##############################################################################

API programming screenshots
➤ Here's the API integration tutorial again
➤ More API programming examples in Python

More blog posts
Categories: FLOSS Project Planets

Tryton News: Newsletter May 2024

Sat, 2024-06-01 02:00

During the last month we focused on fixing bugs, improving the behaviour of things, speeding-up performance issues - building on the changes from our last release. We also added some new features which we would like to introduce to you in this newsletter.

For an in depth overview of the Tryton issues please take a look at our issue tracker or see the issues and merge requests filtered by label.

Changes for the User CRM, Sales, Purchases and Projects

We’ve moved the language field on the party form to the header.

Accounting, Invoicing and Payments

Tryton now ensures the payment amount is greater than zero.

Stock, Production and Shipments

We’ve added a code with a sequence to the production bill of materials (BOM) to clearly distinguish similarly named BOMs.

We’ve also added a shipped state to the customer shipments.
The shipped-state becomes handy when the delivery process takes a long time.
After being packed now the delivery can become shipped or done (similar to internal and drop-shipments). The transition packed → shipped will delete the staging outgoing moves.

As stock lots are not associated with a company, we now ensure that sequences used to generate their numbers are not be linked to a specific company to avoid access right errors.

User Interface

We updated the TinyMCE in Sao to version 7 (see external changelog).

In all of our Tryton clients the user can now close tabs by clicking on them using the middle mouse button.

To avoid accidentally changing configuration values for external services, we now raise a confirmation warning when changing the credential settings in the following modules:

  • account_payment_braintree
  • account_payment_stripe
more… (click for more details) System Data and Configuration

We’ve standardized the format of pictures used in Tryton. Product and avatar images are converted to RGB with a resolution of 300 DPI, so we can rely on a consistent printed size.

We added a description field to the product image which can be used as the alt property on websites.

Now reports in Tryton can be printed with a company logo in PNG-format (for transparency support).

New Documentation

We added section anchors to each option of the configuration documentation. To refer to the docs of an option, you can now use e.g. Email settings <config-email>.

New Releases

We released bug fixes for the currently maintained long term support series
7.0 and 6.0, and for the penultimate series 6.8.

Changes for the System Administrator

The logging messages generated by modules and the bus have been unified.

The CSV import now handles the :lang= suffix when it is appended to a translatable field name.

Changes for Implementers and Developers

We’ve added partial support for the Unique and Exclude constraints in the SQLite backend with UNIQUE INDEX as long as they do not use parameters.

A python-format flag is now added to the PO-files.

Authors: @dave @pokoli @udono

1 post - 1 participant

Read full topic

Categories: FLOSS Project Planets

Real Python: The Real Python Podcast – Episode #206: Building Python Unit Tests &amp; Exploring a Data Visualization Gallery

Fri, 2024-05-31 08:00

How do you start adding unit tests to your Python code? Can the built-in unittest framework cover most or all of your needs? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Categories: FLOSS Project Planets

Zero to Mastery: Python Monthly Newsletter 💻🐍

Fri, 2024-05-31 06:00
54th issue of Andrei Neagoie's must-read monthly Python Newsletter: Python Security Best Practices, State of Python 2024, Prompt Engineering, and much more. Read the full newsletter to get up-to-date with everything you need to know from last month.
Categories: FLOSS Project Planets

Ian Ozsvald: What I&#8217;ve been up to since 2022

Thu, 2024-05-30 08:25

This has been terribly quiet since July 2022, oops. It turns out that having an infant totally sucks your time! In the meantime I’ve continued to build up:

  • Training courses – I’ve just listed my new Fast Pandas course plus the existing Successful Data Science Projects and Software Engineering for Data Scientists with runs of all three for July and September
  • My NotANumber newsletter, it goes out every month or so, carries Python data science jobs and talks on my strategic work, RebelAI leadership community and Higher Performance Python book updates
  • RebelAI – my private data science leadership community (there’s no web presence, just get in touch) for “excellent data scientists turned leaders” – this is having a set of very nice impacts for members
  • High Performance Python O’Reilly book – we’re working on the 3rd edition
  • PyDataLondon 2024 has a great schedule and if you’re coming – do find me and say hi!
Ian is a Chief Interim Data Scientist via his Mor Consulting. Sign-up for Data Science tutorials in London and to hear about his data science thoughts and jobs. He lives in London, is walked by his high energy Springer Spaniel and is a consumer of fine coffees.
Categories: FLOSS Project Planets

EuroPython: How EuroPython Proposals Are Selected: An Inside Look

Thu, 2024-05-30 06:24

With the number of Python-related conferences around the world, many people might wonder how the selection process is configured and performed. For the largest and oldest European Python conference, EuroPython, we wanted to share how this process works.

The Programme team for each EuroPython conference changes every year. There are mechanisms in place to carry on some of the processes and traditions when dealing with proposals for the next event, although you can still find some differences each year. These differences are not large enough to make a significant impact.

The 2024 Process

In this post, we highlight how the 2024 process was conducted and your role in it, whether as a submitter or potential contributor in future versions.

Opening the Call for Proposals

This year, the Call for Proposals (CfP) configuration was based on the 2023 version, with minor modifications. For example, two new tracks were added to enable more people to categorise their proposals:

  • PyData: Research & Applications
  • PyData: LLMs

This change was motivated by the popularity of these topics at other global conferences. In addition, other tracks were merged or removed to keep the number manageable.

For many people, having a configuration with both an Abstract and a Description is confusing. Not everyone knows what to write in each field. To address this, we decided to be clearer: we dropped the Description field and asked explicitly for an Outline section. The intention was that to submit a proposal, one would require an Abstract and an Outline for their session.

Reviewers from the Community

We opened a form to get help from the community for reviewing the talks and decided to accept most (if not all) of them to nurture our review results as much as possible. We had more than 20 people helping with reviewing proposals on Pretalx.

We created a few “track groups” containing related categories that the reviewers could choose from. This way, there was no pressure to have an opinion on a topic one might not be familiar with.

We had an average of six reviews per proposal, which greatly helped us make a final decision.

Community Voting

Another way to receive input is through Community Voting, which allows participants who have attended any of the EuroPythons since 2012 to vote on the upcoming programme.

Using a separate simple web application, people participated by voting for the proposals they wanted to see at EuroPython 2024. We were fortunate to have enough people voting to get a good estimate of preferences.

Fun fact: Around 11 people were able to review all of the nearly 640 proposals we received this year.

We are very grateful to everyone who participated!

My reaction when I saw a good proposal to be voted at EP 2024.Programme Committee

This year the programme committee was mostly formed by a group of new people to the conference, helped by a few people familiar with the process from last year. In total, around 11 people were actively participating.

Like most Programme teams, we did our best to get people from different areas to have a more diverse general mindset, including skills from Data, Core, DevOps, and Web technologies.

It was important for us to have local people on the team, and we are very happy to have had two members from the local Czech community helping, while the rest were spread across Europe.

Selection Process

Based on the reviewers&apos results from Pretalx and Community Voting, we generated a master sheet that was used to perform the selection process.

Track by track, the Programme team went through each proposal that had good Pretalx and Community Voting results and voted (again) for the talks they believed were good material for the conference.

During the selection process, we felt that we did not have enough expertise in a specific area. Therefore, we are very thankful that we could add four more members to the selection team to remedy that.

After three calls, each lasting around 2 hours, the Programme team had the first batch of accepted proposals. The speakers for these proposals were notified as soon as the decision was made. Following a similar process, we did the same for the second (and final) batch of accepted and rejected proposals.

To ensure the acceptance of proposals from most tracks and topics, many plots and statistical analyses were created to visualise the ratio of accepted proposals to submitted ones, the variety of topics, and the diversity of speakers.

Plots from pretalx visualising Proposals by Submission date, Session type, Track & State

Even though it sounds cliché, there were many good proposals we couldn&apost accept immediately since the high volume and quality of proposals made it challenging to make instant decisions. We kept debating whether to place them on the waiting list.

Ultimately, we created another category for proposals that "could be accepted" allowing us to manage and organise high-quality proposals that required further deliberation.

Programme team trying to figure which talk to choose from the waiting listWhat about sponsored talks?

Each year, the conference offers sponsors with certain packages the perk of hosting a sponsored talk, meaning that some of the talk slots had to be saved for that purpose. Slots not taken were filled by proposals on the waiting list.

Is selecting the talks the end of the story?

No. After proposals are accepted/confirmed, special requirements emerge, mainly about "I’m sorry, I cannot be at the conference, can I do it online?" Which, in our opinion, is unfortunate news—not because we don’t like it, but because we have learned that remote talks are not as popular with attendees.

Even though there are some special cases that we fully understand, we noticed a few cases not being convincing enough. In those cases, we had to encourage people to give up their slot for other in-person proposals. This is a tricky process as we are limited in the total amount of remote talks possible, the specific reasons for the change, and the overall scenario for the conference.

What is needed to get accepted?

Most rejected proposals are rejected because they have a weak abstract.

We have tried many means to encourage people to ask questions and seek feedback about their proposals, and we have hosted calls providing the details of good proposals. Still, every year we get proposals that have a poorly structured, incomplete abstract, etc.

For us, a good abstract contains the following:

  • Context of your talk or problem
  • Definition of the problem
  • Why is it important to find a solution to that problem?
  • What will be discussed and what will attendees learn?
  • Previous requirements or additional comments on your talk

You can also imagine a proposal like an elevator pitch. You need to describe it in a way that’s striking and motivates people to attend.

Don’t Forget About the Outline!

This year, we introduced an “outline” field for you to paste the outline of your talk, including the time you will spend on each item. This is essential to get an idea of how much you will be talking about each topic. (Hint: add up the expected times.)

The outline might sound like an obvious topic to you, but many people failed to provide a detailed one. Some even copied the abstract here, so you might understand the importance of this field as well.

Why Does It Feel Like the Same People Are Speakers Every Year?

The main reason for this is that those people followed the proper abstract structure and provided a descriptive outline. Having experience being rejected certainly helps. So we hope that after giving you detailed selection process standards, you know how to crack the selection process.

What about AI?

We discussed a few proposals that “felt AI written” and even used external tools to assess them. In the end, we didn’t have a strict ruling against people using Artificial Intelligence tools to improve their proposals.

When a proposal felt like it was AI-generated, we went deeper into the proposal and the speaker&aposs background. For example, by analysing the bio from the speaker and checking if the person was giving talks somewhere else. Most importantly, if the “speaker” was a real person.

Independently of how the Programme team feels towards AI tools, we cannot completely ignore how these tools are helping some people with structure and grammar, as well as overall assisting them in their writing process. This might change in the future, but currently, we have not written regulations against the usage of AI tools.

The 2025 Process and Final Words

As described before, the team and process can change a bit next year, but we expect the same critical aspects of a good abstract and outline to be essential to the process.

We encourage you to ask for feedback, participate in sessions teaching how to write good proposals, participate on our Speaker&aposs Mentorship programme. These can truly help you to get accepted into the conference.

Having said all this, each conference has a different selection process. Maybe the reason your proposal was not selected is due to a better proposal on the same topic, or too many similar proposals in the same track, or your proposal just did not fit this year&aposs Zeitgeist (i.e. Community Voting).

Please don’t be discouraged! We highly recommend you keep working on your proposal, tweak it, write a new one, and most importantly, try again.

Submitting Several Proposals Doesn’t Help!

We value quality over quantity and will compare your proposals against each other. This is extra work and might even give you less of a chance because of a split vote between your proposals. So submitting more than 10 proposals to get accepted is the wrong approach.

The Call for Proposals will likely be open earlier next year. We hope you can follow the recommendations in this post and get your proposal to accepted for EuroPython 2025.

And remember: Don’t be afraid to ask for feedback!

Thanks for reading! This community post is written by Cristián on behalf of EuroPython 2024 Programme team
Categories: FLOSS Project Planets

Matt Layman: About, FAQ, and Home Page - Building SaaS with Python and Django #192

Wed, 2024-05-29 20:00
In this episode, we worked on some core pages to round out the JourneyInbox user interface. This led us to work updating UI layout, writing copy, and doing other fundamentals for making templated pages.
Categories: FLOSS Project Planets

Python⇒Speed: Let’s optimize! Running 15× faster with a situation-specific algorithm

Wed, 2024-05-29 20:00
pre { white-space: pre; overflow-x: auto; font-size: 80%; }

Let’s speed up some software! Our motivation: we have an image, a photo of some text from a book. We want to turn it into a 1-bit image, with just black and white, extracting the text so we can easily read it.

We’ll use an example image from scikit-image, an excellent image processing library:

from import page import numpy as np IMAGE = page() assert IMAGE.dtype == np.uint8

Here’s what it looks like (it’s licensed under this license):

Median-based local thresholding

The task we’re trying to do—turning darker areas into black, and lighter areas into white—is called thresholding. Since the image is different in different regions, with some darker and some lighter, we’ll get the best results if we use local thresholding, where the threshold is calculated from the pixel’s neighborhood.

Simplifying somewhat, for each pixel in the image we will:

  1. Calculate the median of the surrounding neighborhood.
  2. Subtract a magic constant from the calculated median to calculate our local threshold.
  3. If the pixel’s value is bigger than the threshold, the result is white, otherwise it’s black.

scikit-image includes an implementation of this algorithm. Here’s how we use it:

from skimage.filters import threshold_local def skimage_median_local_threshold(img, neighborhood_size, offset): threshold = threshold_local( img, block_size=neighborhood_size, method="median", offset=offset ) result = (img > threshold).astype(np.uint8) result *= 255 return result # The neighborhood size and offset value were determined "empirically", i.e. # they're manually tuning the algorithm to work well with our specific # example image. SKIMAGE_RESULT = skimage_median_local_threshold(IMAGE, 11, 10)

And here’s what the results look like:

Let’s see if we can make this faster!

Step 1. Reimplement our own version

We’re going to be using the Numba compiler, which lets us compile Python code to machine code at runtime. Here’s an initial implementation of the algorithm; it’s not quite identical to the original, for example the way edge pixels are handled, but it’s close enough for our purposes:

from numba import jit @jit def median_local_threshold1(img, neighborhood_size, offset): # Neighborhood size must be an odd number: assert neighborhood_size % 2 == 1 radius = (neighborhood_size - 1) // 2 result = np.empty(img.shape, dtype=np.uint8) # For every pixel: for i in range(img.shape[0]): # Calculate the Y borders of the neighborhood: min_y = max(i - radius, 0) max_y = min(i + radius + 1, img.shape[0]) for j in range(img.shape[1]): # Calculate the X borders of the neighborhood: min_x = max(j - radius, 0) max_x = min(j + radius + 1, img.shape[1]) # Calculate the median: median = np.median(img[min_y:max_y, min_x:max_x]) # Set the image to black or white, depending how it relates to # the threshold: if img[i, j] > median - offset: # White: result[i, j] = 255 else: # Black: result[i, j] = 0 return result NUMBA_RESULT1 = median_local_threshold1(IMAGE, 11, 10)

Here’s the resulting image; it looks similar enough that for our purposes:

Now we can compare the performance of the two implementations:

Code Elapsed milliseconds skimage_median_local_threshold(IMAGE, 11, 10) 76 median_local_threshold1(IMAGE, 11, 10) 87

It’s slower. But that’s OK, we’re just getting started.

Step 2: A faster implementation of the median algorithm

Calculating a median is pretty expensive, and we’re doing it for every single pixel, so let’s see if we can speed it up.

The generic median implementation Numba provides is likely to be fairly generic, since it needs to work in a wide variety of circumstances. We can hypothesize that it’s not optimized for our particular case. And even if it is, having our own implementation will allow for a second round of optimization, as we’ll see in the next step.

We’re going to implement a histogram-based median, based on the fact we’re using 8-bit images that only have a limited range of potential values. The median is the value where 50% of the pixels’ values are smaller, and 50% are bigger.

Here’s the basic algorithm for a histogram-based median:

  • Each pixel’s value will go into a different bucket in the histogram; since we know our image is 8-bit, we only need 256 buckets.
  • Then, we add up the size of each bucket in the histogram, from smallest to largest, until we hit 50% of the pixels we inspected.
@jit def median_local_threshold2(img, neighborhood_size, offset): assert neighborhood_size % 2 == 1 radius = (neighborhood_size - 1) // 2 result = np.empty(img.shape, dtype=np.uint8) # 😎 A histogram with a bucket for each of the 8-bit values possible in # the image. We allocate this once and reuse it. histogram = np.empty((256,), dtype=np.uint32) for i in range(img.shape[0]): min_y = max(i - radius, 0) max_y = min(i + radius + 1, img.shape[0]) for j in range(img.shape[1]): min_x = max(j - radius, 0) max_x = min(j + radius + 1, img.shape[1]) # Reset the histogram to zero: histogram[:] = 0 # Populate the histogram, counting how many of each value are in # the neighborhood we're inspecting: neighborhood = img[min_y:max_y, min_x:max_x].ravel() for k in range(len(neighborhood)): histogram[neighborhood[k]] += 1 # Use the histogram to find the median; keep adding buckets until # we've hit 50% of the pixels. The corresponding bucket is the # median. half_neighborhood_size = len(neighborhood) // 2 for l in range(256): half_neighborhood_size -= histogram[l] if half_neighborhood_size < 0: break median = l if img[i, j] > median - offset: result[i, j] = 255 else: result[i, j] = 0 return result NUMBA_RESULT2 = median_local_threshold2(IMAGE, 11, 10)

Here’s the resulting image:

And here’s the performance of our new implementation:

Code Elapsed milliseconds median_local_threshold1(IMAGE, 11, 10) 86 median_local_threshold2(IMAGE, 11, 10) 18

That’s better!

Step 3: Stop recalculating the histogram from scratch

Our algorithm uses a rolling neighborhood or window over the image, calculating the median for a window around each pixel. And the neighborhood for one pixel has a significant overlap for the neighborhood of the next pixel. For example, let’s say we’re looking at a neighborhood size of 3. We might calculate the median of this area:

...... .\\\.. .\\\.. .\\\.. ...... ......

And then when process the next pixel we’ll calculate the median of this area:

...... ..///. ..///. ..///. ...... ......

If we superimpose them, we can see there’s an overlap, the X:

...... .\XX/. .\XX/. .\XX/. ...... ......

Given the histogram for the first pixel, if we remove the values marked with \ and add the ones marked with /, we’ve calculated the exact histogram for the second pixel. So for a 3×3 neighborhood, instead of processing 3 columns we process 2, a minor improvement. For a 11×11 neighborhood, we will go from processing 11 columns to 2 columns, a much more significant improvement.

Here’s what the code looks like:

@jit def median_local_threshold3(img, neighborhood_size, offset): assert neighborhood_size % 2 == 1 radius = (neighborhood_size - 1) // 2 result = np.empty(img.shape, dtype=np.uint8) histogram = np.empty((256,), dtype=np.uint32) for i in range(img.shape[0]): min_y = max(i - radius, 0) max_y = min(i + radius + 1, img.shape[0]) # Populate histogram as if we started one pixel to the left: histogram[:] = 0 initial_neighborhood = img[min_y:max_y, 0:radius].ravel() for k in range(len(initial_neighborhood)): histogram[initial_neighborhood[k]] += 1 for j in range(img.shape[1]): min_x = max(j - radius, 0) max_x = min(j + radius + 1, img.shape[1]) # 😎 Instead of recalculating histogram from scratch, re-use the # previous pixel's histogram. # Substract left-most column we don't want anymore: if min_x > 0: for y in range(min_y, max_y): histogram[img[y, min_x - 1]] -= 1 # Add new right-most column: if max_x < img.shape[1]: for y in range(min_y, max_y): histogram[img[y, max_x - 1]] += 1 # Find the the median from the updated histogram: half_neighborhood_size = ((max_y - min_y) * (max_x - min_x)) // 2 for l in range(256): half_neighborhood_size -= histogram[l] if half_neighborhood_size < 0: break median = l if img[i, j] > median - offset: result[i, j] = 255 else: result[i, j] = 0 return result NUMBA_RESULT3 = median_local_threshold3(IMAGE, 11, 10)

Here’s the resulting image:

And here’s the performance of our latest code:

Code Elapsed microseconds median_local_threshold2(IMAGE, 11, 10) 17,066 median_local_threshold3(IMAGE, 11, 10) 6,386 Step #4: Adapative heuristics

Notice that a median’s definition is symmetrical:

  1. The first value that is smaller than the highest 50% values.
  2. Or, the first value that is larger than the lowest 50% values. We used this definition in our code above, adding up buckets from the smallest to the largest.

Depending on the distribution of values, one approach to adding up buckets to find the median may be faster than the other. For example, given a 0-255 range, if the median is going to be 10 we want to start from the smallest bucket to minimize additions. But if the median is going to be 200, we want to start from the largest bucket.

So which side we should start from? One reasonable heuristic is to look at the previous median we calculated, which most of the time will be quite similar to the new median. If the previous median was small, start from the smallest buckets; if it was large, start from the largest buckets.

@jit def median_local_threshold4(img, neighborhood_size, offset): assert neighborhood_size % 2 == 1 radius = (neighborhood_size - 1) // 2 result = np.empty(img.shape, dtype=np.uint8) histogram = np.empty((256,), dtype=np.uint32) median = 0 for i in range(img.shape[0]): min_y = max(i - radius, 0) max_y = min(i + radius + 1, img.shape[0]) histogram[:] = 0 initial_neighborhood = img[min_y:max_y, 0:radius].ravel() for k in range(len(initial_neighborhood)): histogram[initial_neighborhood[k]] += 1 for j in range(img.shape[1]): min_x = max(j - radius, 0) max_x = min(j + radius + 1, img.shape[1]) if min_x > 0: for y in range(min_y, max_y): histogram[img[y, min_x - 1]] -= 1 if max_x < img.shape[1]: for y in range(min_y, max_y): histogram[img[y, max_x - 1]] += 1 half_neighborhood_size = ((max_y - min_y) * (max_x - min_x)) // 2 # 😎 Find the the median from the updated histogram, choosing # the starting side based on the previous median; we can go from # the leftmost bucket to the rightmost bucket, or in reverse: the_range = range(256) if median < 127 else range(255, -1, -1) for l in the_range: half_neighborhood_size -= histogram[l] if half_neighborhood_size < 0: median = l break if img[i, j] > median - offset: result[i, j] = 255 else: result[i, j] = 0 return result NUMBA_RESULT4 = median_local_threshold4(IMAGE, 11, 10)

The end result is 25% faster. Since the heuristic is tied to the image contents, the performance impact will depend on the image.

Code Elapsed microseconds median_local_threshold3(IMAGE, 11, 10) 6,381 median_local_threshold4(IMAGE, 11, 10) 4,920 The big picture

Here’s a performance comparison of all the versions of the code:

Code Elapsed microseconds skimage_median_local_threshold(IMAGE, 11, 10) 76,213 median_local_threshold1(IMAGE, 11, 10) 86,494 median_local_threshold2(IMAGE, 11, 10) 17,145 median_local_threshold3(IMAGE, 11, 10) 6,398 median_local_threshold4(IMAGE, 11, 10) 4,925

Let’s go over the steps we went through:

  1. Switch to a compiled language: this gives us more control.
  2. Reimplement the algorithm taking advantage of constrained requirements: our median only needed to handle uint8, so a histogram was a reasonable solution.
  3. Reuse previous calculations to prevent repetition: our histogram for the neighborhood of a pixel is quite similar to that of the previous pixel. This means we can reuse some of the calculations.
  4. Adaptively tweak the algorithm at runtime: as we run on an actual image, we use what we’ve learned up to this point to hopefully run faster later on. The decision from which side of the histogram to start is arbirary in general. But in this specific algorithm, the overlapping pixel neighborhoods mean we can make a reasonable guess.

This process demonstrates part of why generic libraries may be slower than custom code you write for your particular use case and your particular data.

Next steps

What else can you do to speed up this algorithm? Here are some ideas:

  • There may be a faster alternative to histogram-based medians.
  • We’re not fully taking advantage of histogram overlap; there’s also overlap between rows.
  • The cumulative sum in the histogram doesn’t benefit from instruction-level parallelism or SIMD. It’s possible that using one of those would result in faster results even if it uses more instructions.
  • So far the code has only used a single CPU. Given each row is calculated independently, parallelism would probably work well if done in horizontal stripes, probably taller than one pixel so as to maximize utilization of memory caches.

Want to learn more about optimizing compiled code for Python data processing? This article is an extract from a book I’m working on; test readers are currently going through initial drafts. Aimed at Python developers, data scientists, and scientists, the book covers topics like instruction-level parallelism, memory caches, and other performance optimization techniques. Learn more and sign up to get updates here.

Categories: FLOSS Project Planets

Anarcat: Playing with fonts again

Wed, 2024-05-29 17:38

I am getting increasingly frustrated by Fira Mono's lack of italic support so I am looking at alternative fonts again.

Commit Mono

This time I seem to be settling on either Commit Mono or Space Mono. For now I'm using Commit Mono because it's a little more compressed than Fira and does have a italic version. I don't like how Space Mono's parenthesis (()) is "squarish", it feels visually ambiguous with the square brackets ([]), a big no-no for my primary use case (code).

So here I am using a new font, again. It required changing a bunch of configuration files in my home directory (which is in a private repository, sorry) and Emacs configuration (thankfully that's public!).

One gotcha is I realized I didn't actually have a global font configuration in Emacs, as some Faces define their own font family, which overrides the frame defaults.

This is what it looks like, before:

Fira Mono


Commit Mono

(Notice how those screenshots are not sharp? I'm surprised too. The originals look sharp on my display, I suspect this is something to do with the Wayland transition. I've tried with both grim and flameshot, for what its worth.)

They are pretty similar! Commit Mono feels a bit more vertically compressed maybe too much so, actually -- the line height feels too low. But it's heavily customizable so that's something that's relatively easy to fix, if it's really a problem. Its weight is also a little heavier and wider than Fira which I find a little distracting right now, but maybe I'll get used to it.

All characters seem properly distinguishable, although, if I'd really want to nitpick I'd say the © and ® are too different, with the latter (REGISTERED SIGN) being way too small, basically unreadable here. Since I see this sign approximately never, it probably doesn't matter at all.

I like how the ampersand (&) is more traditional, although I'll miss the exotic one Fira produced... I like how the back quotes (`, GRAVE ACCENT) drop down low, nicely aligned with the apostrophe. As I mentioned before, I like how the bar on the "f" aligns with the other top of letters, something in Fira mono that really annoys me now that I've noticed it (it's not aligned!).

A UTF-8 test file

Here's the test sheet I've made up to test various characters. I could have sworn I had a good one like this lying around somewhere but couldn't find it so here it is, I guess.

US keyboard coverage: abcdefghijklmnopqrstuvwxyz`1234567890-=[]\;',./ ABCDEFGHIJKLMNOPQRSTUVWXYZ~!@#$%^&*()_+{}|:"<>? latin1 coverage: ¡¢£¤¥¦§¨©ª«¬­®¯°±²³´µ¶·¸¹º»¼½¾¿ EURO SIGN, TRADE MARK SIGN: €™ ambiguity test: e¢coC0ODQ iI71lL!|¦ b6G&0B83 [](){}/\.…·• zs$S52Z% ´`'"‘’“”«» all characters in a sentence, uppercase: the quick fox jumps over the lazy dog THE QUICK FOX JUMPS OVER THE LAZY DOG same, in french: voix ambiguë d'un cœur qui, au zéphyr, préfère les jattes de kiwis. VOIX AMBIGUË D'UN CŒUR QUI, AU ZÉPHYR, PRÉFÈRE LES JATTES DE KIWIS. Ligatures test: -<< -< -<- <-- <--- <<- <- -> ->> --> ---> ->- >- >>- =<< =< =<= <== <=== <<= <= => =>> ==> ===> =>= >= >>= <-> <--> <---> <----> <=> <==> <===> <====> :: ::: __ <~~ </ </> /> ~~> == != /= ~= <> === !== !=== =/= =!= <: := *= *+ <* <*> *> <| <|> |> <. <.> .> +* =* =: :> (* *) /* */ [| |] {| |} ++ +++ \/ /\ |- -| <!-- <!--- Box drawing alignment tests: █ ╔══╦══╗ ┌──┬──┐ ╭──┬──╮ ╭──┬──╮ ┏━━┳━━┓ ┎┒┏┑ ╷ ╻ ┏┯┓ ┌┰┐ ▉ ╱╲╱╲╳╳╳ ║┌─╨─┐║ │╔═╧═╗│ │╒═╪═╕│ │╓─╁─╖│ ┃┌─╂─┐┃ ┗╃╄┙ ╶┼╴╺╋╸┠┼┨ ┝╋┥ ▊ ╲╱╲╱╳╳╳ ║│╲ ╱│║ │║ ║│ ││ │ ││ │║ ┃ ║│ ┃│ ╿ │┃ ┍╅╆┓ ╵ ╹ ┗┷┛ └┸┘ ▋ ╱╲╱╲╳╳╳ ╠╡ ╳ ╞╣ ├╢ ╟┤ ├┼─┼─┼┤ ├╫─╂─╫┤ ┣┿╾┼╼┿┫ ┕┛┖┚ ┌┄┄┐ ╎ ┏┅┅┓ ┋ ▌ ╲╱╲╱╳╳╳ ║│╱ ╲│║ │║ ║│ ││ │ ││ │║ ┃ ║│ ┃│ ╽ │┃ ░░▒▒▓▓██ ┊ ┆ ╎ ╏ ┇ ┋ ▍ ║└─╥─┘║ │╚═╤═╝│ │╘═╪═╛│ │╙─╀─╜│ ┃└─╂─┘┃ ░░▒▒▓▓██ ┊ ┆ ╎ ╏ ┇ ┋ ▎ ╚══╩══╝ └──┴──┘ ╰──┴──╯ ╰──┴──╯ ┗━━┻━━┛ └╌╌┘ ╎ ┗╍╍┛ ┋ ▏▁▂▃▄▅▆▇█ Dashes alignment test: HYPHEN-MINUS, MINUS SIGN, EN, EM DASH, HORIZONTAL BAR, LOW LINE -------------------------------------------------- −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− –––––––––––––––––––––––––––––––––––––––––––––––––– —————————————————————————————————————————————————— ―――――――――――――――――――――――――――――――――――――――――――――――――― __________________________________________________

So there you have it, got completely nerd swiped by typography again. Now I can go back to writing a too-long proposal again.

Sources and inspiration for the above:

  • the unicode(1) command, to lookup individual characters to disambiguate, for example, - (U+002D HYPHEN-MINUS, the minus sign next to zero on US keyboards) and − (U+2212 MINUS SIGN, a math symbol)

  • searchable list of characters and their names - roughly equivalent to the unicode(1) command, but in one page, amazingly the /usr/share/unicode database doesn't have any one file like this

  • bits/UTF-8-Unicode-Test-Documents - full list of UTF-8 characters

  • UTF-8 encoded plain text file - nice examples of edge cases, curly quotes example and box drawing alignment test which, incidentally, showed me I needed specific faces customisation in Emacs to get the Markdown code areas to display properly, also the idea of comparing various dashes

  • sample sentences in many languages - unused, "Sentences that contain all letters commonly used in a language"

  • UTF-8 sampler - unused, similar

Other fonts

In my previous blog post about fonts, I had a list of alternative fonts, but it seems people are not digging through this, so I figured I would redo the list here to preempt "but have you tried Jetbrains mono" kind of comments.

My requirements are:

  • no ligatures: yes, in the previous post, I wanted ligatures but I have changed my mind. after testing this, I find them distracting, confusing, and they often break the monospace nature of the display
  • monospace: this is to display code
  • italics: often used when writing Markdown, where I do make use of italics... Emacs falls back to underlining text when lacking italics which is hard to read
  • free-ish, ultimately should be packaged in Debian

Here is the list of alternatives I have considered in the past and why I'm not using them:

  • agave: recommended by tarzeau, not sure I like the lowercase a, a bit too exotic, packaged as fonts-agave

  • Cascadia code: optional ligatures, multilingual, not liking the alignment, ambiguous parenthesis (look too much like square brackets), new default for Windows Terminal and Visual Studio, packaged as fonts-cascadia-code

  • Fira Code: ligatures, was using Fira Mono from which it is derived, lacking italics except for forks, interestingly, Fira Code succeeds the alignment test but Fira Mono fails to show the X signs properly! packaged as fonts-firacode

  • Hack: no ligatures, very similar to Fira, italics, good alternative, fails the X test in box alignment, packaged as fonts-hack

  • Hermit: no ligatures, smaller, alignment issues in box drawing and dashes, packaged as fonts-hermit somehow part of cool-retro-term

  • IBM Plex: irritating website, replaces Helvetica as the IBM corporate font, no ligatures by default, italics, proportional alternatives, serifs and sans, multiple languages, partial failure in box alignment test (X signs), fancy curly braces contrast perhaps too much with the rest of the font, packaged in Debian as fonts-ibm-plex

  • Intel One Mono: nice legibility, no ligatures, alignment issues in box drawing, not packaged in Debian

  • Iosevka: optional ligatures, italics, multilingual, good legibility, has a proportional option, serifs and sans, line height issue in box drawing, fails dash test, not in Debian

  • Jetbrains Mono: (mandatory?) ligatures, good coverage, originally rumored to be not DFSG-free (Debian Free Software Guidelines) but ultimately packaged in Debian as fonts-jetbrains-mono

  • Monoid: optional ligatures, feels much "thinner" than Jetbrains, not liking alignment or spacing on that one, ambiguous 2Z, problems rendering box drawing, packaged as fonts-monoid

  • Mononoki: no ligatures, looks good, good alternative, suggested by the Debian fonts team as part of fonts-recommended, problems rendering box drawing, em dash bigger than en dash, packaged as fonts-mononoki

  • Source Code Pro: italics, looks good, but dash metrics look whacky, not in Debian

  • spleen: bitmap font, old school, spacing issue in box drawing test, packaged as fonts-spleen

  • sudo: personal project, no ligatures, zero originally not dotted, relied on metrics for legibility, spacing issue in box drawing, not in Debian

So, if I get tired of Commit Mono, I might probably try, in order:

  1. Hack
  2. Jetbrains Mono
  3. IBM Plex Mono

Iosevka, Monoki and Intel One Mono are also good options, but have alignment problems. Iosevka is particularly disappointing as the EM DASH metrics are just completely wrong (much too wide).

This was tested using the Programming fonts site which has all the above fonts, which cannot be said of Font Squirrel or Google Fonts, amazingly. Other such tools:

Categories: FLOSS Project Planets

Mike Driscoll: Episode 42 – Harlequin – The SQL IDE for Your Terminal

Wed, 2024-05-29 17:06

This episode focuses on the Harlequin application, a Python SQL IDE for your terminal written using the amazing Textual package.

I was honored to have Ted Conbeer, the creator of Harlequin, on the show to discuss his creation and the other things he does with Python.

Specifically, we focused on the following topics:

  • Favorite Python packages
  • Origins of Harlequin
  • Why program for the terminal versus a GUI
  • Lessons learned in creating the tool
  • Asyncio
  • and more!

The post Episode 42 – Harlequin – The SQL IDE for Your Terminal appeared first on Mouse Vs Python.

Categories: FLOSS Project Planets

Django Weblog: Django Enhancement Proposal 14: Background Workers

Wed, 2024-05-29 15:04

As of today, DEP-14 has been approved 🛫

The DEP was written and stewarded by Jake Howard. A very enthusiastic community has been active with feedback and encouragement, while the Django Steering Council gave the final inputs before its formal acceptance. The implementation of DEP-14 is expected to be a major leap forward for the “batteries included” philosophy of Django.

Whilst Django is a web framework, there's more to web applications than just the request-response lifecycle. Sending emails, communicating with external services or running complex actions should all be done outside the request-response cycle.

Django doesn't have a first-party solution for long-running tasks, however the ecosystem is filled with incredibly popular frameworks, all of which interact with Django in slightly different ways. Other frameworks such as Laravel have background workers built-in, allowing them to push tasks into the background to be processed at a later date, without requiring the end user to wait for them to occur.

Library maintainers must implement support for any possible task backend separately, should they wish to offload functionality to the background. This includes smaller libraries, but also larger meta-frameworks with their own package ecosystem such as Wagtail.

This proposal sets out to provide an interface and base implementation for long-running background tasks in Django.

Future work

The DEP will now move on to the Implementation phase before being merged into Django itself.

If you would like to help or try it out, go have a look at django-tasks, a separate reference implementation by Jake Howard, the author of the DEP.

Jake will also be speaking about the DEP in his talk at DjangoCon Europe at DjangoCon Europe 2024 in Vigo next week.

Categories: FLOSS Project Planets

PyCharm: PyCharm 2024.1.2: What’s New!

Wed, 2024-05-29 14:54

PyCharm 2024.1.2 is here with features designed to enhance your productivity and streamline your development workflow. This update includes support for DRF viewsets and routers in the Endpoints tool window, code assistance for TypedDict and Unpack, and improved debugger performance when handling large collections.

You can download the latest version from our download page or update your current version through our free Toolbox App

For more details, please visit our What’s New page.

Download PyCharm 2024.1.2

Key features Support for DRF viewsets and routers in the Endpoints tool window

When working with the Django REST Framework in PyCharm, not only can you specify function-based or class-based views in the path, but you can now also specify viewsets and see the results in the Endpoints tool window. Additionally, you can map HTTP methods to viewset methods, and PyCharm will display the HTTP methods next to the relevant route, including for custom methods. Routes without @actions decorators are now displayed with the related viewset methods.

Learn more Code assistance for TypedDict and Unpack

PEP 692 made it possible to add type information for keyword arguments of different types by using TypedDict and Unpack. PyCharm allows you to use this feature confidently by providing parameter info, type checking, and code completion.

Improved debugger performance for large collections

PyCharm’s debugger now offers a smoother experience, even when very large collections are involved. You can now work on your data science projects without having to put up with high CPU loads and UI freezes.

Download PyCharm 2024.1.2

Be sure to check out our release notes to learn all of the details and ensure you don’t miss out on any new features.

We appreciate your support as we work to improve your PyCharm experience. Please report any bugs via our issue tracker so we can resolve them promptly. Connect with us on X (formerly Twitter) to share your feedback on PyCharm 2024.1.2!

Categories: FLOSS Project Planets

The Python Show: 42 - Harlequin - The SQL IDE for Your Terminal

Wed, 2024-05-29 10:14

This episode focuses on the Harlequin application, a Python SQL IDE for your terminal written using the amazing Textual package.

I was honored to have Ted Conbeer, the creator of Harlequin, on the show to discuss his creation and the other things he does with Python.

Specifically, we focused on the following topics:

  • Favorite Python packages

  • Origins of Harlequin

  • Why program for the terminal versus a GUI

  • Lessons learned in creating the tool

  • Asyncio

  • and more!

Categories: FLOSS Project Planets