Planet Python

We’re thrilled to announce Azure Developers – Python Day! Join us on December 5th for a full day of online training and discover the latest services and features in Azure designed specifically for Python developers. You’ll learn cutting-edge cloud development techniques that can save you time and money while providing your customers with the best experience possible.

December 5, 2024 from 9:30 am – 4:00 pm (Pacific Time) / 17:30 – 00:00 (UTC)

Select “Notify Me” on the YouTube Video to ensure you don’t miss the event!

During the event, you’ll hear directly from the experts behind the latest features in Azure designed for Python developers, techniques to save time and money, and a special session on our recently announced AI Toolkit for VS Code.

Whether you’re a beginner or an experienced Python developer, this event is for you. We’ll cover six main topic areas: Application Development, Artificial Intelligence, Cloud Native, Data Services, Security, Serverless, and Developer Productivity.

Agenda Session Title Theme Speaker Time( PT/ UTC) Welcome to Azure Developers – Python Day Dawn Wages, Senior Program Manager

Jay Gordon, Principal Product Manager

9:50 AM / 16:50 Dev Containers and Codespaces for quick skilling and deployments Developer Productivity Sarah Kaiser, Senior Cloud Developer Advocate 10:00 AM / 17:00 Cloudy with a Chance of Jupyter – Install JupyterHub on Azure in 30 mins Data Services Dharhas Pothina, CTO, Quansight 10:30 AM / 17:30 Langchain on Azure SQL to enlighten AI with your own data AI Davide Mauri, Principal Program Manager 11:00 AM / 18:00 Integrating AI into your Python apps with App Service Sidecars AI, App Development Byron Tardif, Principal Mgr, Product Manager

Tulika Chaudharie, Principal Product Manager

Jeff Martinez, Product Manager

11:30 AM / 18:30 Getting started with Python on Azure Cosmos DB App Development Theo Van Kraay, Principal Program Manager 12:00 PM / 19:00 Transforming AI development in VS Code AI, App Development, Developer Productivity Rong Lu, Principal Mgr, Product Manager

Zhidi Shang, Principal Program Manager Lead

12:30 PM / 19:30 Building Scalable GenAI Apps with Azure Cosmos DB & LangChain AI, App Development James Codella, Principal Product Mgr 1:00 PM / 20:00 Python + Azure for Absolute Beginners App Development Rohit Ganguly, Product Manager II 1:30 PM / 20:30 Deploying Python apps with GitHub Copilot for @azure AI, Developer Productivity Pamela Fox, Principal Cloud Developer Advocate 2:00 PM / 21:00 Securing Python Applications Security, Cloud Native Joylynn Kirui, Senior Security Cloud Advocate 2:30 PM / 21:30 Your First Full Stack Python Web Application App Development Renee Noble, Senior Cloud Developer Advocate 3:00 PM / 22:00 Deploying a scalable Django app with Microsoft Azure App Development Velda Kiara, Senior Software Engineer, Python MVP

Abigail Gbadago, Senior Software Engineer, Python MVP

3:30 PM / 22:30 Closing Remarks Dawn Wages, Senior Program Manager

Jay Gordon, Principal Product Manager

4:00 PM / 23:00

Don’t miss this opportunity to build the best applications with Python. Join us on December 5th on the Azure Developers YouTube and Twitch channels. See you there!

The post Announcing: Azure Developers – Python Day appeared first on Python.

For a program to be useful, it often needs to communicate with the outside world. In Python, the input() function allows you to capture user input from the keyboard, while you can use the print() function to display output to the console.

These built-in functions allow for basic user interaction in Python scripts, enabling you to gather data and provide feedback. If you want to go beyond the basics, then you can even use them to develop applications that are not only functional but also user-friendly and responsive.

By the end of this tutorial, you’ll know how to:

• Take user input from the keyboard with input()
• Display output to the console with print()
• Use readline to improve the user experience when collecting input on UNIX-like systems
• Format output using the sep and end keyword arguments of print()

To get the most out of this tutorial, you should have a basic understanding of Python syntax and familiarity with using the Python interpreter and running Python scripts.

Get Your Code: Click here to download the free sample code that you’ll use to learn about basic input and output in Python.

Take the Quiz: Test your knowledge with our interactive “Basic Input and Output in Python” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

Basic Input and Output in Python

In this quiz, you'll test your understanding of Python's built-in functions for user interaction, namely input() and print(). These functions allow you to capture user input from the keyboard and display output to the console, respectively.

Reading Input From the Keyboard

Programs often need to obtain data from users, typically through keyboard input. In Python, one way to collect user input from the keyboard is by calling the input() function:

The input() function pauses program execution to allow you to type in a line of input from the keyboard. Once you press the Enter key, all characters typed are read and returned as a string, excluding the newline character generated by pressing Enter.

If you add text in between the parentheses, effectively passing a value to the optional prompt argument, then input() displays the text you entered as a prompt:

Python >>> name = input("Please enter your name: ") Please enter your name: John Doe >>> name 'John Doe' Copied!

Adding a meaningful prompt will assist your user in understanding what they’re supposed to input, which makes for a better user experience.

The input() function always reads the user’s input as a string. Even if you type characters that resemble numbers, Python will still treat them as a string:

Python 1>>> number = input("Enter a number: ") 2Enter a number: 50 3 4>>> type(number) 5<class 'str'> 6 7>>> number + 100 8Traceback (most recent call last): 9 File "<python-input-1>", line 1, in <module> 10 number + 100 11 ~~~~~~~^~~~~ 12TypeError: can only concatenate str (not "int") to str Copied!

In the example above, you wanted to add 100 to the number entered by the user. However, the expression number + 100 on line 7 doesn’t work because number is a string ("50") and 100 is an integer. In Python, you can’t combine a string and an integer using the plus (+) operator.

You wanted to perform a mathematical operation using two integers, but because input() always returns a string, you need a way to read user input as a numeric type. So, you’ll need to convert the string to the appropriate type:

Python >>> number = int(input("Enter a number: ")) Enter a number: 50 >>> type(number) <class 'int'> >>> number + 100 150 Copied!

In this updated code snippet, you use int() to convert the user input to an integer right after collecting it. Then, you assign the converted value to the name number. That way, the calculation number + 100 has two integers to add. The calculation succeeds and Python returns the correct sum.

Note: When you convert user input to a numeric type using functions like int() in a real-world scenario, it’s crucial to handle potential exceptions to prevent your program from crashing due to invalid input.

The input() function lets you collect information from your users. But once your program has calculated a result, how do you display it back to them? Up to this point, you’ve seen results displayed automatically as output in the interactive Python interpreter session.

However, if you ran the same code from a file instead, then Python would still calculate the values, but you wouldn’t see the results. To display output in the console, you can use Python’s print() function, which lets you show text and data to your users.

Writing Output to the Console

In addition to obtaining data from the user, a program will often need to present data back to the user. In Python, you can display data to the console with the print() function.

Read the full article at https://realpython.com/python-input-output/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Generative AI and LLMs have been hot topics this year, but are they affecting trends in data science and machine learning? What new trends in data science are worth following? Every year, JetBrains collaborates with the Python Software Foundation to carry out the Python Developer Survey, which can offer some useful insight into these questions.

The results from the latest iteration of the survey, collected between November 2023 and February 2024, included a new Data Science section. This allowed us to get a more complete picture of data science trends over the past year and highlighted how important Python remains in this domain. 

While 48% of Python developers are involved in data exploration and processing, the percentage of respondents using Python for data analysis dropped from 51% in 2022 to 44% in 2023. The percentage of respondents using Python for machine learning dropped from 36% in 2022 to 34% in 2023. At the same time, 27% of respondents use Python for data engineering, and 8% use it for MLOps – two new categories that were added to the survey in 2023. 

Let’s take a closer look at the trends in the survey results to put these numbers into context and get a better sense of what they mean. Read on to learn about the latest developments in the fields of data science and machine learning to prepare yourself for 2025.

Data processing: pandas remains the top choice, but Polars is gaining ground

Data processing is an essential part of data science. pandas, a project that is 15 years old, is still at the top of the list of the most commonly used data processing tools. It is used by 77% of respondents who do data exploration and processing. As a mature project, its API is stable, and many working examples can be found on the internet. It’s no surprise that pandas is still the obvious choice. As a NumFOCUS sponsored project, pandas has proven to the community that it is sustainable and its governance model has gained user trust. It is a great choice for beginners who may still be learning the ropes of data processing, as it’s a stable project that does not undergo rapid changes.

On the other hand, Polars, which pitches itself as DataFrames for the new era, has been in the spotlight quite a bit both last year and this year, thanks to the advantages it provides in terms of speed and parallel processing. In 2023, a company led by the creator of Polars, Ritchie Vink, was formed to support the development of the project. This ensures Polars will be able to maintain its rapid pace of development. In July of 2024, version 1.0 of Polars was released. Later, Polars expanded its compatibility with other popular data science tools like Hugging Face and NVIDIA RAPIDS. It also provides a lightweight plotting backend, just like pandas.

So, for working professionals in data science, there is an advantage to switching to Polars. As the project matures, it can become a load-bearing tool in your data science workflow and can be used to process more data faster. In the 2023 survey, 10% of respondents said that they are using Polars as their data processing tool. It is not hard to imagine this figure being higher in this year’s survey.

Whether you are a working professional or just starting to process your first dataset, it is important to have an efficient tool at hand that can make your work more enjoyable. With PyCharm, you can inspect your data as interactive tables, which you can scroll, sort, filter, convert to plots, or use to generate heat maps. Moreover, you can get analytics for each column and use AI assistance to explain DataFrames or create visualizations. Apart from pandas and Polars, PyCharm provides this functionality for Hugging Face datasets, NumPy, PyTorch, and TensorFlow.

Try PyCharm for free An interactive table in PyCharm 2024.2.2 Pro provides tools for inspecting pandas and Polars DataFrames

The popularity of Polars has led to the creation of a new project called Narwhals. Independent from pandas and Polars, Narwhals aims to unite the APIs of both tools (and many others). Since it is a very young project (started in February 2024), it hasn’t yet shown up on our list of the most popular data processing tools, but we suspect it may get there in the next few years.

Also worth mentioning are Spark (16%) and Dask (7%), which are useful for processing large quantities of data thanks to their parallel processes. These tools require a bit more engineering capability to set up. However, as the amount of data that projects depend on increasingly exceeds what a traditional Python program can handle, these tools will become more important and we may see these figures go up.

Data visualization: Will HoloViz Panel surpass Plotly Dash and Streamlit within the next year?

Data scientists have to be able to create reports and explain their findings to businesses. Various interactive visualization dashboard tools have been developed for working with Python. According to the survey results, the most popular of them is Plotly Dash.

Plotly is most known in the data science community for the ggplot2 library, which is a highly popular visualization library for users of the R language. Ever since Python became popular for data science, Plotly has also provided a Python library, which gives you a similar experience to ggplot2 in Python. In recent years, Dash, a Python framework for building reactive web apps developed by Plotly, has become an obvious choice for those who are used to Plotly and need to build an interactive dashboard. However, Dash’s API requires some basic understanding of the elements used in HTML when designing the layout of an app. For users who have little to no frontend experience, this could be a hurdle they need to overcome before making effective use of Dash.

Second place for “best visualization dashboard” goes to Streamlit, which has now joined forces with Snowflake. It doesn’t have as long of a history as Plotly, but it has been gaining a lot of momentum over the past few years because it’s easy to use and comes packaged with a command line tool. Although Streamlit is not as customizable as Plotly, building the layout of the dashboard is quite straightforward, and it supports multipage apps, making it possible to build more complex applications.

However, in the 2024 results these numbers may change a little. There are up-and-coming tools that could catch up to – or even surpass – these apps in popularity. One of them is HoloViz Panel. As one of the libraries in the HoloViz ecosystem, it is sponsored by NumFocus and is gaining traction among the PyData community. Panel lets users generate reports in the HTML format and also works very well with Jupyter Notebook. It offers templates to help new users get started, as well as a great deal of customization options for expert users who want to fine-tune their dashboards.

ML models: scikit-learn is still prominent, while PyTorch is the most popular for deep learning

Because generative AI and LLMs have been such hot topics in recent years, you might expect deep learning frameworks and libraries to have completely taken over. However, this isn’t entirely true. There is still a lot of insight that can be extracted from data using traditional statistics-based methods offered by scikit-learn, a well-known machine learning library mostly maintained by researchers. Sponsored by NumFocus since 2020, it remains the most important library in machine learning and data science. SciPy, another Python library that provides support for scientific calculations, is also one of the most used libraries in data science.

Having said that, we cannot ignore the impact of deep learning and the increase in popularity of deep learning frameworks. PyTorch, a machine learning library created by Meta, is now under the governance of the Linux Foundation. In light of this change, we can expect PyTorch to continue being a load-bearing library in the open-source ecosystem and to maintain its level of active community involvement. As the most used deep learning framework, it is loved by Python users – especially those who are familiar with numpy, since “tensors”, the basic data structures in PyTorch, are very similar to numpy arrays. 

You can inspect Pytorch tensors in PyCharm 2024.2.2 Pro just like you inspect Numpy arrays

Unlike TensorFlow, which uses a static computational graph, PyTorch uses a dynamic one – and this makes profiling in Python a blast. To top it all off, PyTorch also provides a profiling API, making it a good choice for research and experimentation. However, if your deep learning project needs to be scalable in deployment and needs to support multiple programming languages, TensorFlow may be a better choice, as it is compatible with many languages, including C++, JavaScript, Python, C#, Ruby, and Swift. Keras is a tool that makes TensorFlow more accessible and is also popular for deep learning frameworks.

Another framework we cannot ignore for deep learning is Hugging Face Transformers. Hugging Face is a hub that provides many state-of-the-art pre-trained deep learning models that are popular in the data science and machine learning community, which you can download and train further yourself. Transformers is a library maintained by Hugging Face and the community for state-of-the-art machine learning with PyTorch, TensorFlow, and JAX. We can expect Hugging Face Transformers will gain more users in 2024 due to the popularity of LLMs. 

With PyCharm you can identify and manage Hugging Face models in a dedicated tool window. PyCharm can also help you to choose the right model for your use case from the large variety of Hugging Face models directly in the IDE. 

One new library that is worth paying attention to in 2024 is Scikit-LLM, which allows you to tap into Open AI models like ChatGPT and integrate them with scikit-learn. This is very handy when text analysis is needed, and you can perform analysis using models from scikit-learn with the power of modern LLM models.

MLOps: The future of data science projects

One aspect of data science projects that is essential but frequently overlooked is MLOps (machine learning operations). In the workflow of a data science project, data scientists need to manage data, retrain the model, and have version control for all the data and models used. Sometimes, when a machine learning application is deployed in production, performance and usage also need to be observed and monitored.

In recent years, MLOps tools designed for data science projects have emerged. One of the issues that has been bothering data scientists and data engineers is versioning the data, which is crucial when your pipeline constantly has data flowing in. 

Data scientists and engineers also need to track their experiments. Since the machine learning model will be retrained with new data and hyperparameters will be fine-tuned, it’s important to keep track of model training and experiment results. Right now, the most popular tool is TensorBoard. However, this may be changing soon. TensorBoard.dev has been deprecated, which means users are now forced to deploy their own TensorBoard installations locally or share results using the TensorBoard integration with Google Colab. As a result, we may see a drop in the usage of TensorBoard and an uptick in that of other tools like MLflow and PyTorch.

Another MLOps step that is necessary for ensuring that data projects run smoothly is shipping the development environment for production. The use of Docker containers, a common development practice among software engineers, seems to have been adopted by the data science community. This ensures that the development environment and the production environment remain consistent, which is important for data science projects involving machine learning models that need to be deployed as applications. We can see that Docker is a popular tool among Python users who need to deploy services to the cloud.

This year, Docker containers is slightly ahead of Anaconda in the “Python installation and upgrade” category.

2023 survey results 2022 survey results Big data: How much is enough?

One common misconception is that we will need more data to train better, more complex models in order to improve prediction. However, this is not the case. Since models can be overfitted, more is not always better in machine learning. Different tools and approaches will be required depending on the use case, the model, and how much data is being handled at the same time.

The challenge of handling a huge amount of data in Python is that most Python libraries rely on the data being stored in the memory. We could just deploy cloud computing resources with huge amounts of memory, but even this approach has its limitations and would sometimes be slow and costly.

When handling huge amounts of data that are hard to fit in memory, a common solution is to use distributed computing resources. Computation tasks and data are distributed over a cluster to be performed and handled in parallel. This approach makes data science and machine learning operations scalable, and the most popular engine for this is Apache Spark. Spark can be used with PySpark, the Python API library for it.

As of Spark 2.0, anyone using Spark RDD API is encouraged to switch to Spark SQL, which provides better performance. Spark SQL also makes it easier for data scientists to handle data because it enables SQL queries to be executed. We can expect PySpark to remain the most popular choice in 2024.

Another popular tool for managing data in clusters is Databricks. If you are using Databricks to work with your data in clusters, now you can benefit from the powerful integration of Databricks and PyCharm. You can write code for your pipelines and jobs in PyCharm, then deploy, test, and run it in real time on your Databricks cluster without any additional configuration.

Communities: Events shifting focus toward data science

Many newcomers to Python are using it for data science, and thus more Python libraries have been catering to data science use cases. In that same vein, Python events like PyCon and EuroPython are beginning to include more tracks, talks, and workshops that focus on data science, while events that are specific to data science, like PyData and SciPy, remain popular, as well.

Final thoughts

Data science and machine learning are becoming increasingly active, and together with the popularity of AI and LLMs, more and more new open source tools have become available for use in data science. The landscape of data science continues to change rapidly, and we are excited to see what becomes most popular in the 2024 survey results.

Enhance your data science experience with PyCharm

Modern data science demands skills for a wide range of tasks, including data processing and visualization, coding, model deployment, and managing large datasets. As an integrated development environment (IDE), PyCharm helps you efficiently build this skill set. It provides intelligent coding assistance, top-tier debugging, version control, integrated database management, and seamless Docker integration. For data science, PyCharm supports Jupyter notebooks, as well as key scientific and machine learning libraries, and it integrates with tools like the Hugging Face models library, Anaconda, and Databricks.

Start using PyCharm for your data science projects today and enjoy its latest improvements, including features for inspecting pandas and Polars DataFrames, and for the layer by layer inspection of PyTorch tensors, which is handy when exploring data and building deep learning models.

Try PyCharm for free

String formatting is essential in Python for creating dynamic and well-structured text by inserting values into strings. This tutorial covers various methods, including f-strings, the .format() method, and the modulo operator (%). Each method has unique features and benefits for different use cases. The string formatting mini-language provides additional control over the output format, allowing for aligned text, numeric formatting, and more.

F-strings provide an intuitive and efficient way to embed expressions inside string literals. The .format() method offers flexibility for lazy interpolation and is compatible with Python’s formatting mini-language. The modulo operator is an older technique still found in legacy code. Understanding these methods will help you choose the best option for your specific string formatting needs.

By the end of this tutorial, you’ll understand that:

• String formatting in Python involves inserting and formatting values within strings using interpolation.
• Python supports different types of string formatting, including f-strings, the .format() method, and the modulo operator (%).
• F-strings are generally the most readable and efficient option for eager interpolation in Python.
• Python’s string formatting mini-language offers features like alignment, type conversion, and numeric formatting.
• While f-strings are more readable and efficient compared to .format() and the % operator, the .format() method supports lazy evaluation.

To get the most out of this tutorial, you should be familiar with Python’s string data type and the available string interpolation tools. Having a basic knowledge of the string formatting mini-language is also a plus.

Get Your Code: Click here to download the free sample code you’ll use to learn about Python’s string formatting tools.

Take the Quiz: Test your knowledge with our interactive “Python String Formatting: Available Tools and Their Features” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

Python String Formatting: Available Tools and Their Features

You can take this quiz to test your understanding of the available tools for string formatting in Python, as well as their strengths and weaknesses. These tools include f-strings, the .format() method, and the modulo operator.

Interpolating and Formatting Strings in Python

String interpolation involves generating strings by inserting other strings or objects into specific places in a base string or template. For example, here’s how you can do some string interpolation using an f-string:

Python >>> name = "Bob" >>> f"Hello, {name}!" 'Hello, Bob!' Copied!

In this quick example, you first have a Python variable containing a string object, "Bob". Then, you create a new string using an f-string. In this string, you insert the content of your name variable using a replacement field. When you run this last line of code, Python builds a final string, 'Hello, Bob!'. The insertion of name into the f-string is an interpolation.

Note: To dive deeper into string interpolation, check out the String Interpolation in Python: Exploring Available Tools tutorial.

When you do string interpolation, you may need to format the interpolated values to produce a well-formatted final string. To do this, you can use different string interpolation tools that support string formatting. In Python, you have these three tools:

1. F-strings
2. The str.format() method
3. The modulo operator (%)

The first two tools support the string formatting mini-language, a feature that allows you to fine-tune your strings. The third tool is a bit old and has fewer formatting options. However, you can use it to do some minimal formatting.

Note: The built-in format() function is yet another tool that supports the format specification mini-language. This function is typically used for date and number formatting, but you won’t cover it in this tutorial.

In the following sections, you’ll start by learning a bit about the string formatting mini-language. Then, you’ll dive into using this language, f-strings, and the .format() method to format your strings. Finally, you’ll learn about the formatting capabilities of the modulo operator.

Using F-Strings to Format Strings

Python 3.6 added a string interpolation and formatting tool called formatted string literals, or f-strings for short. As you’ve already learned, f-strings let you embed Python objects and expressions inside your strings. To create an f-string, you must prefix the string with an f or F and insert replacement fields in the string literal. Each replacement field must contain a variable, object, or expression:

Python >>> f"The number is {42}" 'The number is 42' >>> a = 5 >>> b = 10 >>> f"{a} plus {b} is {a + b}" '5 plus 10 is 15' Copied!

In the first example, you define an f-string that embeds the number 42 directly into the resulting string. In the second example, you insert two variables and an expression into the string.

Formatted string literals are a Python parser feature that converts f-strings into a series of string constants and expressions. These are then joined up to build the final string.

Using the Formatting Mini-Language With F-Strings

When you use f-strings to create strings through interpolation, you need to use replacement fields. In f-strings, you can define a replacement field using curly brackets ({}) as in the examples below:

Python >>> debit = 300.00 >>> credit = 450.00 >>> f"Debit: ${debit}, Credit: ${credit}, Balance: ${credit - debit}" 'Debit: $300, Credit: $450.0, Balance: $150.0' Copied! Read the full article at https://realpython.com/python-string-formatting/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

To check if a string contains another string in Python, use the in membership operator. This is the recommended method for confirming the presence of a substring within a string. The in operator is intuitive and readable, making it a straightforward way to evaluate substring existence.

Additionally, you can use string methods like .count() and .index() to gather more detailed information about substrings, such as their frequency and position. For more complex substring searches, use regular expressions with the re module. When you’re dealing with tabular data, then pandas provides efficient tools for searching for substrings within DataFrame columns.

By the end of this tutorial, you’ll understand that:

• The in membership operator is the recommended way to check if a Python string contains a substring.
• Converting input text to lowercase generalizes substring checks by removing case sensitivity.
• The .count() method counts occurrences of a substring, while .index() finds the first occurrence’s position.
• Regular expressions in the re module allow for advanced substring searches based on complex conditions.
• The .str.contains() method in pandas identifies which DataFrame entries contain a specific substring.

Understanding these methods and tools enables you to effectively check for substrings in Python strings, catering to various needs from simple checks to complex data analysis.

Get Your Code: Click here to download the free sample code that you’ll use to check if a string contains a substring.

How to Confirm That a Python String Contains Another String

If you need to check whether a string contains a substring, use Python’s membership operator in. In Python, this is the recommended way to confirm the existence of a substring in a string:

Python >>> raw_file_content = """Hi there and welcome. ... This is a special hidden file with a SECRET secret. ... I don't want to tell you The Secret, ... but I do want to secretly tell you that I have one.""" >>> "secret" in raw_file_content True Copied!

The in membership operator gives you a quick and readable way to check whether a substring is present in a string. You may notice that the line of code almost reads like English.

Note: If you want to check whether the substring is not in the string, then you can use not in:

Python >>> "secret" not in raw_file_content False Copied!

Because the substring "secret" is present in raw_file_content, the not in operator returns False.

When you use in, the expression returns a Boolean value:

• True if Python found the substring
• False if Python didn’t find the substring

You can use this intuitive syntax in conditional statements to make decisions in your code:

Python >>> if "secret" in raw_file_content: ... print("Found!") ... Found! Copied!

In this code snippet, you use the membership operator to check whether "secret" is a substring of raw_file_content. If it is, then you’ll print a message to the terminal. Any indented code will only execute if the Python string that you’re checking contains the substring that you provide.

Note: Python considers empty strings always as a substring of any other string, so checking for the empty string in a string returns True:

Python >>> "" in "secret" True Copied!

This may be surprising because Python considers emtpy strings as false, but it’s an edge case that is helpful to keep in mind.

The membership operator in is your best friend if you just need to check whether a Python string contains a substring.

However, what if you want to know more about the substring? If you read through the text stored in raw_file_content, then you’ll notice that the substring occurs more than once, and even in different variations!

Which of these occurrences did Python find? Does capitalization make a difference? How often does the substring show up in the text? And what’s the location of these substrings? If you need the answer to any of these questions, then keep on reading.

Generalize Your Check by Removing Case Sensitivity

Python strings are case sensitive. If the substring that you provide uses different capitalization than the same word in your text, then Python won’t find it. For example, if you check for the lowercase word "secret" on a title-case version of the original text, the membership operator check returns False:

Python >>> title_cased_file_content = """Hi There And Welcome. ... This Is A Special Hidden File With A Secret Secret. ... I Don't Want To Tell You The Secret, ... But I Do Want To Secretly Tell You That I Have One.""" >>> "secret" in title_cased_file_content False Copied!

Despite the fact that the word secret appears multiple times in the title-case text title_cased_file_content, it never shows up in all lowercase. That’s why the check that you perform with the membership operator returns False. Python can’t find the all-lowercase string "secret" in the provided text.

Humans have a different approach to language than computers do. This is why you’ll often want to disregard capitalization when you check whether a string contains a substring in Python.

Read the full article at https://realpython.com/python-string-contains-substring/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python exceptions provide a mechanism for handling errors that occur during the execution of a program. Unlike syntax errors, which are detected by the parser, Python raises exceptions when an error occurs in syntactically correct code. Knowing how to raise, catch, and handle exceptions effectively helps to ensure your program behaves as expected, even when encountering errors.

In Python, you handle exceptions using a try … except block. This structure allows you to execute code normally while responding to any exceptions that may arise. You can also use else to run code if no exceptions occur, and the finally clause to execute code regardless of whether an exception was raised.

By the end of this tutorial, you’ll understand that:

• Exceptions in Python occur when syntactically correct code results in an error.
• You can handle exceptions using the try, except, else, and finally keywords.
• The try … except block lets you execute code and handle exceptions that arise.
• Python 3 introduced more built-in exceptions compared to Python 2, making error handling more granular.
• It’s bad practice to catch all exceptions at once using except Exception or the bare except clause.
• Combining try, except, and pass allows your program to continue silently without handling the exception.
• Using try … except is not inherently bad, but you should use it judiciously to handle only known issues appropriately.

In this tutorial, you’ll get to know Python exceptions and all relevant keywords for exception handling by walking through a practical example of handling a platform-related exception. Finally, you’ll also learn how to create your own custom Python exceptions.

Get Your Code: Click here to download the free sample code that shows you how exceptions work in Python.

Take the Quiz: Test your knowledge with our interactive “Python Exceptions: An Introduction” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

Python Exceptions: An Introduction

In this quiz, you'll test your understanding of Python exceptions. You'll cover the difference between syntax errors and exceptions and learn how to raise exceptions, make assertions, and use the try and except block.

Understanding Exceptions and Syntax Errors

Syntax errors occur when the parser detects an incorrect statement. Observe the following example:

Python Traceback >>> print(0 / 0)) File "<stdin>", line 1 print(0 / 0)) ^ SyntaxError: unmatched ')' Copied!

The arrow indicates where the parser ran into the syntax error. Additionally, the error message gives you a hint about what went wrong. In this example, there was one bracket too many. Remove it and run your code again:

Python >>> print(0 / 0) Traceback (most recent call last): File "<stdin>", line 1, in <module> ZeroDivisionError: division by zero Copied!

This time, you ran into an exception error. This type of error occurs whenever syntactically correct Python code results in an error. The last line of the message indicates what type of exception error you ran into.

Instead of just writing exception error, Python details what type of exception error it encountered. In this case, it was a ZeroDivisionError. Python comes with various built-in exceptions as well as the possibility to create user-defined exceptions.

Raising an Exception in Python

There are scenarios where you might want to stop your program by raising an exception if a condition occurs. You can do this with the raise keyword:

You can even complement the statement with a custom message. Assume that you’re writing a tiny toy program that expects only numbers up to 5. You can raise an error when an unwanted condition occurs:

Python low.py number = 10 if number > 5: raise Exception(f"The number should not exceed 5. ({number=})") print(number) Copied!

In this example, you raised an Exception object and passed it an informative custom message. You built the message using an f-string and a self-documenting expression.

When you run low.py, you’ll get the following output:

Python Traceback Traceback (most recent call last): File "./low.py", line 3, in <module> raise Exception(f"The number should not exceed 5. ({number=})") Exception: The number should not exceed 5. (number=10) Copied!

The program comes to a halt and displays the exception to your terminal or REPL, offering you helpful clues about what went wrong. Note that the final call to print() never executed, because Python raised the exception before it got to that line of code.

With the raise keyword, you can raise any exception object in Python and stop your program when an unwanted condition occurs.

Debugging During Development With assert Read the full article at https://realpython.com/python-exceptions/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

There has been a lot of changes in the low-level Python space these days. The biggest has to be how many projects have rewritten core performance-intensive sections in Rust. Or even the wholesale adoption of Rust for newer projects such as uv and ruff. On this episode, we dive into the tools and workflow needed to build these portions of Python apps in Rust with David Seddon and Samuel Colvin.<br/> <br/> <strong>Episode sponsors</strong><br/> <br/> <a href='https://talkpython.fm/posit'>Posit</a><br> <a href='https://talkpython.fm/citizens'>Data Citizens Podcast</a><br> <a href='https://talkpython.fm/training'>Talk Python Courses</a><br/> <br/> <strong>Links from the show</strong><br/> <br/> <div><b>Samuel Colvin</b>: <a href="https://github.com/samuelcolvin?featured_on=talkpython" target="_blank" >github.com/samuelcolvin</a><br/> <b>David Seddon</b>: <a href="https://github.com/seddonym?featured_on=talkpython" target="_blank" >github.com/seddonym</a><br/> <br/> <b>Pydantic</b>: <a href="https://pydantic.dev?featured_on=talkpython" target="_blank" >pydantic.dev</a><br/> <b>PEP 0759</b>: <a href="https://peps.python.org/pep-0759/?featured_on=talkpython" target="_blank" >peps.python.org</a><br/> <b>TypeShed</b>: <a href="https://github.com/python/typeshed?featured_on=talkpython" target="_blank" >github.com</a><br/> <b>Maturin</b>: <a href="https://www.maturin.rs/?featured_on=talkpython" target="_blank" >maturin.rs</a><br/> <b>rloop</b>: <a href="https://github.com/gi0baro/rloop?featured_on=talkpython" target="_blank" >github.com</a><br/> <b>Grimp</b>: <a href="https://github.com/seddonym/grimp?featured_on=talkpython" target="_blank" >github.com</a><br/> <b>Grimp Workflows</b>: <a href="https://github.com/seddonym/grimp/blob/master/.github/workflows/main.yml?featured_on=talkpython" target="_blank" >github.com</a><br/> <b>White House recommends memory safe languages</b>: <a href="https://www.whitehouse.gov/oncd/briefing-room/2024/02/26/press-release-technical-report/?featured_on=talkpython" target="_blank" >whitehouse.gov</a><br/> <b>Installing Rust</b>: <a href="https://www.rust-lang.org/tools/install?featured_on=talkpython" target="_blank" >rust-lang.org</a><br/> <b>jiter</b>: <a href="https://github.com/pydantic/jiter?featured_on=talkpython" target="_blank" >github.com</a><br/> <b>import-linter</b>: <a href="https://github.com/seddonym/import-linter/?featured_on=talkpython" target="_blank" >github.com</a><br/> <b>Logfire</b>: <a href="https://pydantic.dev/logfire?featured_on=talkpython" target="_blank" >pydantic.dev</a><br/> <b>Crabs in Snakes, David Seddon, Pycon Italia</b>: <a href="https://www.youtube.com/watch?v=L844NAJ24QI" target="_blank" >youtube.com</a><br/> <b>Watch this episode on YouTube</b>: <a href="https://www.youtube.com/watch?v=xP2GZZ3ylG4" target="_blank" >youtube.com</a><br/> <b>Episode transcripts</b>: <a href="https://talkpython.fm/episodes/transcript/487/building-rust-extensions-for-python" target="_blank" >talkpython.fm</a><br/> <br/> <b>--- Stay in touch with us ---</b><br/> <b>Subscribe to us on YouTube</b>: <a href="https://talkpython.fm/youtube" target="_blank" >youtube.com</a><br/> <b>Follow Talk Python on Mastodon</b>: <a href="https://fosstodon.org/web/@talkpython" target="_blank" ><i class="fa-brands fa-mastodon"></i>talkpython</a><br/> <b>Follow Michael on Mastodon</b>: <a href="https://fosstodon.org/web/@mkennedy" target="_blank" ><i class="fa-brands fa-mastodon"></i>mkennedy</a><br/></div>

During the last month we focused on fixing bugs, improving the behaviour of things, speeding-up performance issues - building on the changes from our last Tryton Release 7.4. We also added some new features which we would like to introduce to you in this newsletter.

For an in depth overview of the Tryton issues please take a look at our issue tracker or see the issues and merge requests filtered by label.

Changes for the User Accounting, Invoicing and Payments

Now we compute the maturity date of grouped account move lines per debit/credit.

New Releases

We released bug fixes for the currently maintained long term support series
7.0 and 6.0, and for the penultimate series 7.4 and 7.2.

Changes for the Translators

Now we support plural translations for report and ir.message. The other translatable strings are labels that have no count. The plural rule is a Python expression which returns an integer following the gettext design. For the report, it is possible to use the Genshi i18n:choose, i18n:singular, i18n:plural and for OpenDocument the gettext and ngettext methods are added to the evaluation context.

Changes for the System Administrator

Now we can set the decimal precision of default context from the environment variable TRYTOND_DECIMAL_PREC.

Changes for Implementers and Developers

To provide better feedback to the user, now we also add equivalent domains of SQL constraints.

Now we extract the timesheet cost computation from the _get_cost method to allow customization e.g. to use a different composition of the costs.
Also we round the work total with currency digits in get_total method now, to allow extending the computation methods, without rounding too early.

A trytond.model.Model is defined by a python class:

from trytond.model import ModelSQL, fields class Party(ModelSQL): """Party""" __name__ = 'party.party'

It was needed to define three different identifiers:

1. Python class name: Party
2. class doc-string: Party
3. trytond.model.Model.__name__: party.party

The translated text string of the model which is shown in the user interface was extracted from the __doc__ (doc-string) of the class definition.

Now we replaced the class doc-string extraction by parsing the trytond.model.Model.__name__ attribute and try to make it human readable. Changing an existing __name__ attribute usually implies a database migration.
To work-around existing not so useful __name__ values we introduced the new class attribute trytond.model.Model.__string__ which let you define the model string directly.

We also take the chance and clean-up the name-space by renaming

• ir.model.name field into ir.model.string,
• ir.model.model field into ir.model.name and
• ir.model.field.field_description into ir.model.field.string

Finally we removed most of the former doc-strings from all the classes, only in some rare cases we add the __string__ attribute.

Authors: @dave @pokoli @udono

1 post - 1 participant

Read full topic

Python f-strings offer a concise and efficient way to interpolate variables, objects, and expressions directly into strings. By prefixing a string with f or F, you can embed expressions within curly braces ({}), which are evaluated at runtime.

This makes f-strings faster and more readable compared to older approaches like the modulo (%) operator or the string .format() method. Additionally, f-strings support advanced string formatting using Python’s string format mini-language.

By the end of this tutorial, you’ll understand that:

• An f-string in Python is a string literal prefixed with f or F, allowing for the embedding of expressions within curly braces {}.
• To include dynamic content in an f-string, place your expression or variable inside the braces to interpolate its value into the string.
• An f-string error in Python often occurs due to syntax issues, such as unmatched braces or invalid expressions within the string.
• F-string calculation in Python involves writing expressions within the curly braces of an f-string, which Python evaluates at runtime.
• Python 3.12 improved f-strings by allowing nested expressions and the use of backslashes.

This tutorial will guide you through the features and advantages of f-strings, including interpolation and formatting. By familiarizing yourself with these features, you’ll be able to effectively use f-strings in your Python projects.

Get Your Code: Click here to download the free sample code that shows you how to do string interpolation and formatting with Python’s f-strings.

Take the Quiz: Test your knowledge with our interactive “Python F-Strings” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

Python F-Strings

In this quiz, you'll test your knowledge of Python f-strings. With this knowledge, you'll be able to include all sorts of Python expressions inside your strings.

Interpolating and Formatting Strings Before Python 3.6

Before Python 3.6, you had two main tools for interpolating values, variables, and expressions inside string literals:

1. The string interpolation operator (%), or modulo operator
2. The str.format() method

You’ll get a refresher on these two string interpolation tools in the following sections. You’ll also learn about the string formatting capabilities that these tools offer in Python.

The Modulo Operator (%)

The modulo operator (%) was the first tool for string interpolation and formatting in Python and has been in the language since the beginning. Here’s what using this operator looks like in practice:

Python >>> name = "Jane" >>> "Hello, %s!" % name 'Hello, Jane!' Copied!

In this quick example, you use the % operator to interpolate the value of your name variable into a string literal. The interpolation operator takes two operands:

• A string literal containing one or more conversion specifiers
• The object or objects that you’re interpolating into the string literal

The conversion specifiers work as replacement fields. In the above example, you use the %s combination of characters as a conversion specifier. The % symbol marks the start of the specifier, while the s letter is the conversion type and tells the operator that you want to convert the input object into a string.

If you want to insert more than one object into your target string, then you can use a tuple. Note that the number of objects in the tuple must match the number of format specifiers in the string:

Python >>> name = "Jane" >>> age = 25 >>> "Hello, %s! You're %s years old." % (name, age) 'Hello, Jane! You're 25 years old.' Copied!

In this example, you use a tuple of values as the right-hand operand to %. Note that you’ve used a string and an integer. Because you use the %s specifier, Python converts both objects to strings.

You can also use dictionaries as the right-hand operand in your interpolation expressions. To do this, you need to create conversion specifiers that enclose key names in parentheses:

Python >>> "Hello, %(name)s! You're %(age)s years old." % {"name": "Jane", "age": 25} "Hello, Jane! You're 25 years old." Copied!

This syntax provides a readable approach to string interpolation with the % operator. You can use descriptive key names instead of relying on the positional order of values.

When you use the % operator for string interpolation, you can use conversion specifiers. They provide some string formatting capabilities that take advantage of conversion types, conversion flags, and some characters like the period (.) and the asterisk (*). Consider the following example:

Python >>> "Balance: $%.2f" % 5425.9292 'Balance: $5425.93' >>> print("Name: %s\nAge: %5s" % ("John", 35)) Name: John Age: 35 Copied! Read the full article at https://realpython.com/python-f-strings/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The if __name__ == "__main__" idiom is a Python construct that helps control code execution in scripts. It’s a conditional statement that allows you to define code that runs only when the file is executed as a script, not when it’s imported as a module.

When you run a Python script, the interpreter assigns the value "__main__" to the __name__ variable. If Python imports the code as a module, then it sets __name__ to the module’s name instead. By encapsulating code within if __name__ == "__main__", you can ensure that it only runs in the intended context.

By the end of this tutorial, you’ll understand that:

• Python’s if __name__ == "__main__" idiom allows code to run only when the script is executed, not when it’s imported.
• The idiom checks if the __name__ variable equals "__main__", confirming that the script is the top-level module.
• Using this idiom helps prevent unintended code execution during module imports.
• It’s useful for adding script-specific logic, such as user input or test cases, without affecting module imports.
• Best practices suggest using this idiom minimally and placing it at the bottom of the script for clarity.

You’ve likely encountered Python’s if __name__ == "__main__" idiom when reading other people’s code. No wonder—it’s widespread! Understanding Python’s if __name__ == "__main__" idiom will help you to manage script execution and module imports effectively. In this tutorial you’ll explore its mechanics, appropriate usage, and best practices.

Take the Quiz: Test your knowledge with our interactive “Python Name-Main Idiom” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

Python Name-Main Idiom

Test your knowledge of Python's if __name__ == "__main__" idiom by answering a series of questions! You've probably encountered the name-main idiom and might have even used it in your own scripts. But did you use it correctly?

Get Your Code: Click here to download the free sample code that you’ll use to learn about the name-main idiom.

In Short: It Allows You to Execute Code When the File Runs as a Script, but Not When It’s Imported as a Module

For most practical purposes, you can think of the conditional block that you open with if __name__ == "__main__" as a way to store code that should only run when your file is executed as a script.

You’ll see what that means in a moment. For now, say you have the following file:

Python echo.py 1def echo(text: str, repetitions: int = 3) -> str: 2 """Imitate a real-world echo.""" 3 echoes = [text[-i:].lower() for i in range(repetitions, 0, -1)] 4 return "\n".join(echoes + ["."]) 5 6if __name__ == "__main__": 7 text = input("Yell something at a mountain: ") 8 print(echo(text)) Copied!

In this example, you define a function, echo(), that mimics a real-world echo by gradually printing fewer and fewer of the final letters of the input text.

Below that, in lines 6 to 8, you use the if __name__ == "__main__" idiom. This code starts with the conditional statement if __name__ == "__main__" in line 6. In the indented lines, 7 and 8, you then collect user input and call echo() with that input. These two lines will execute when you run echo.py as a script from your command line:

Shell $ python echo.py Yell something at a mountain: HELLOOOO ECHOOOOOOOOOO ooo oo o . Copied!

When you run the file as a script by passing the file object to your Python interpreter, the expression __name__ == "__main__" returns True. The code block under if then runs, so Python collects user input and calls echo().

Try it out yourself! You can download all the code files that you’ll use in this tutorial from the link below:

Get Your Code: Click here to download the free sample code that you’ll use to learn about the name-main idiom.

At the same time, if you import echo() in another module or a console session, then the nested code won’t run:

Python >>> from echo import echo >>> print(echo("Please help me I'm stuck on a mountain")) ain in n . Copied!

In this case, you want to use echo() in the context of another script or interpreter session, so you won’t need to collect user input. Running input() would mess with your code by producing a side effect when importing echo.

When you nest the code that’s specific to the script usage of your file under the if __name__ == "__main__" idiom, then you avoid running code that’s irrelevant for imported modules.

Nesting code under if __name__ == "__main__" allows you to cater to different use cases:

• Script: When run as a script, your code prompts the user for input, calls echo(), and prints the result.
• Module: When you import echo as a module, then echo() gets defined, but no code executes. You provide echo() to the main code session without any side effects.

By implementing the if __name__ == "__main__" idiom in your code, you set up an additional entry point that allows you to use echo() right from the command line.

There you go! You’ve now covered the most important information about this topic. Still, there’s more to find out, and there are some subtleties that can help you build a deeper understanding of this code specifically and Python more generally.

Read the full article at https://realpython.com/if-name-main-python/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Logging in Python lets you record important information about your program’s execution. You use the built-in logging module to capture logs, which provide insights into application flow, errors, and usage patterns. With Python logging, you can create and configure loggers, set log levels, and format log messages without installing additional packages. You can also generate log files to store records for later analysis.

Using a logging library instead of print() calls gives you better control over log management, formatting, and output destinations. The logging module also allows you to set severity levels, simplifying the management and filtering of log data.

By the end of this tutorial, you’ll understand that:

• Logging in a computer involves recording program execution information for analysis.
• You can use logging for debugging, performance analysis, and monitoring usage patterns.
• No installation is needed for Python logging as the logging module is part of Python’s standard library.
• Logging in Python works by configuring loggers and setting log levels.
• Using a logging library provides structured logging and control over log output.
• You should prefer logging over print() because it decreases the maintainance burden and allows you to manage log levels.
• You can generate a log file in Python by setting filename and encoding through basicConfig().

You’ll do the coding for this tutorial in the Python standard REPL. If you prefer Python files, then you’ll find a full logging example as a script in the materials of this tutorial. You can download this script by clicking the link below:

Get Your Code: Click here to download the free sample code that you’ll use to learn about logging in Python.

Take the Quiz: Test your knowledge with our interactive “Logging in Python” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

Logging in Python

In this quiz, you'll test your understanding of Python's logging module. With this knowledge, you'll be able to add logging to your applications, which can help you debug errors and analyze performance.

If you commonly use Python’s print() function to get information about the flow of your programs, then logging is the natural next step for you. This tutorial will guide you through creating your first logs and show you how to make logging grow with your projects.

Starting With Python’s Logging Module

The logging module in Python’s standard library is a ready-to-use, powerful module that’s designed to meet the needs of beginners as well as enterprise teams.

Note: Since logs offer a variety of insights, the logging module is often used by other third-party Python libraries, too. Once you’re more advanced in the practice of logging, you can integrate your log messages with the ones from those libraries to produce a homogeneous log for your application.

To leverage this versatility, it’s a good idea to get a better understanding of how the logging module works under the hood. For example, you could take a stroll through the logging module’s source code

The main component of the logging module is something called the logger. You can think of the logger as a reporter in your code that decides what to record, at what level of detail, and where to store or send these records.

Exploring the Root Logger

To get a first impression of how the logging module and a logger work, open the Python standard REPL and enter the code below:

Python >>> import logging >>> logging.warning("Remain calm!") WARNING:root:Remain calm! Copied!

The output shows the severity level before each message along with root, which is the name the logging module gives to its default logger. This output shows the default format that can be configured to include things like a timestamp or other details.

In the example above, you’re sending a message on the root logger. The log level of the message is WARNING. Log levels are an important aspect of logging. By default, there are five standard levels indicating the severity of events. Each has a corresponding function that can be used to log events at that level of severity.

Note: There’s also a NOTSET log level, which you’ll encounter later in this tutorial when you learn about custom logging handlers.

Here are the five default log levels, in order of increasing severity:

Log Level Function Description DEBUG logging.debug() Provides detailed information that’s valuable to you as a developer. INFO logging.info() Provides general information about what’s going on with your program. WARNING logging.warning() Indicates that there’s something you should look into. ERROR logging.error() Alerts you to an unexpected problem that’s occured in your program. CRITICAL logging.critical() Tells you that a serious error has occurred and may have crashed your app.

The logging module provides you with a default logger that allows you to get started with logging without needing to do much configuration. However, the logging functions listed in the table above reveal a quirk that you may not expect:

Python >>> logging.debug("This is a debug message") >>> logging.info("This is an info message") >>> logging.warning("This is a warning message") WARNING:root:This is a warning message >>> logging.error("This is an error message") ERROR:root:This is an error message >>> logging.critical("This is a critical message") CRITICAL:root:This is a critical message Copied!

Notice that the debug() and info() messages didn’t get logged. This is because, by default, the logging module logs the messages with a severity level of WARNING or above. You can change that by configuring the logging module to log events of all levels.

Adjusting the Log Level

To set up your basic logging configuration and adjust the log level, the logging module comes with a basicConfig() function. As a Python developer, this camel-cased function name may look unusual to you as it doesn’t follow the PEP 8 naming conventions:

Read the full article at https://realpython.com/python-logging/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Hi everyone! A warm welcome to the newly elected EuroPython Society Fellows in the year 2024.

• Laís Carvalho
• Cyril Bitterich

EuroPython Society Fellows

EuroPython Society Fellows have contributed significantly towards our mission, the EuroPython conference and the Society as an organisation. They are eligible for a lifetime free attendance of the EuroPython conference and will be listed on our EuroPython Society Fellow Grant page in recognition of their work.‌

Laís has been volunteering with the EPS since 2020 and is currently serving as a board member for 2023–2024. Theofanis wrote in nominating her for the EuroPython Society fellowship.

Lais is the "superhero" of this year&aposs EuroPython Organization. She&aposs constantly trying to improve our processes, our way of operating, to push for diversity and inclusion, and to make things happen. Apart from that, she&aposs always passionate about the EuroPython Society Mission, actioning on community outreach, expanding EPS communication channels and helping, in action, EPS to support the local python communities. I have no doubt that with her key contribution, 2024 will be one of the most impactful years for the mission of EPS.

Cyril started volunteering for EuroPython since 2022, He started helping with Ops and Programme teams initially and his firefighting skills proved invaluable as he supported other teams and the general organisation, making him a de facto member of the 2024 core organisers team. Theofanis Petkos put it best in his email, nominating Cyril for the EuroPython Society fellowship

Cyril is one of the most dedicated volunteers I&aposve ever met. You&aposll argue with him, he&aposll care, you&aposll ask for his help. He will have input, he will review everything on time. Constantly trying to help, to include more people and always passionate about documenting all of our processes. Cyril is one of the people I&aposm very proud I have worked with and I&aposm sure with his contribution EPS will make a lot of steps forward.

The EuroPython Society Board would like to congratulate and thank all the above new Fellows for their tireless work towards our mission! If you want to send in your nomination, check out our Fellowship page and get in touch!

Many thanks,
EuroPython Society
https://www.europython-society.org/

Since the very inception of my journey in Python and PyLadies, I have always thought of having a PyLadies Conference, a celebration of PyLadies. There were conversations here and there, but nothing was fruitful then. In 2023, Mariatta, Cheuk, Maria Jose, and many more PyLadies volunteers around the globe made this dream come true, and we had our first ever PyLadiesCon.
I submitted a talk for the first-ever PyLadiesCon (how come I didn&apost?), and it was rejected. In 2024, I missed the CFP deadline. I was sad. Will I never be able to participate in PyLadiesCon?

On October 10th, 2024, I had my talk at PyCon NL. I woke up early to practice. I saw an email from PyLadiesCon, titled "Invitation to be a Keynote Speaker at PyLadiesCon". The panic call went to Kushal Das. "Check if there is any attack in the Python server? I got a spamming email about PyLadiesCon and the address is correct. "No, nothing.", replied Kushal after checking. Wait then "WHAT???". PyLadiesCon wants me to give the keynote. THE KEYNOTE in PyLadiesCon.

Thank you Audrey for conceptualizing and creating PyLadies, our home.

And here I am now. I will give the keynote on 7 December 2024 at PyLadiesCon on how PyLadies gave me purpose. See you all there.

Dreams do come true.

What are common issues with using notebooks for Python development? How do you know the current state, share reproducible results, or create interactive applications? This week on the show, we speak with Akshay Agrawal about the open-source reactive marimo notebook for Python.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

2 Weeks ago I was down with fever, but as I became better after a few days I managed to attend Amnesty International's Stockholm chapter's event on activism. I had to travel to a different part of Stockholm, which was fun.

The day started with welcome talk and then a new "Palestine Activist Group" presented their work (in both English and Swedish). Then the main opening talk (in Swedish) from a very senior Swedish journalist about history and important dates related to Palestine. I managed to understand 50-60% of the words. Still a lot of learn.

During the break I had longer chats about privacy, data and our lives. It was good to find folks who knows/uses Python/Fedora/Debian/Ubuntu and other Open Source tools in their daily lives, sometimes even without knowing.

After the morning break, I attended a workshop about being an activist. It was filled with discussions, and of course a lot of new words in Swedish for me. But, the delivery of the workshop was good, people talked about what they think about being an individual activist and things to consider etc.

Next day I had a workshop for a selected group, about boycotting large services and how to providing least amount data to preserve our privacy. The group consisted people from various parts of Swedish society, people working in other government agencies and large multinational companies.

We talked about metadata and phones provide both metadata and real data to the various services via apps. Explaining people to imagine that every time they get out of home, there are many humans walking with them to every destination, keeping notes of every place visit, and every chats they are having with another person or about groups, and then later selling that information to other businesses. It is one thing to talk about these, but it is complete opposite to show. So, I showed them live how much our phones/computers talk. I had my old Pixel4a configured with GrapheneOS, and one of the participant connected their regular Pixel phone to the same VPN. And people seemed to be offended by seeing the amount of data flowing, even when the phone is not use. The below is a screenshot I took today morning from the same demo setup.

We also talked about various chatting platforms, I already built another system to teach kids about how various social chatting platforms work. It was a perfect tool for this demo. We discussed about various other technologies in our daily lives and how they affect privacy.

Whether learning a new natural language or just browsing foreign websites, you sometimes come across text that you want to read but is written in a language you don't fully understand. To translate one natural language into your native language you can use a translator tool, like Google Translate.

In this tutorial, we'll build a desktop translator application to translate natural language using the Google Translate APIs. The UI will be built using the Tkinter GUI library from the Python standard library.

Table of Contents

• Demo: A Translator App with Tkinter
• Installing the Required Packages
• Building the Window
• Creating the GUI
• Implementing the Translation Functionality
• Using the Translator App
• Conclusion

Demo: A Translator App with Tkinter

We'll work through the process of building the app step by step. Once we finish the tutorial, the app will look and work like the following:

You can select the source and destination languages. Then, you can paste or type some text in the left-hand area and hit the Translate button to get the translated text in the right-hand area.

Installing the Required Packages

Our Translator uses the googletrans library to perform the actual translation via Google. Tkinter is already available in the standard library, so we just need to install googletrans-py and a library called httpcore to deal with connection errors.

The first task will be to set up a Python virtual environment. Open the terminal, and run the following commands:

• Windows
• macOS/Linux

bat > mkdir translator > cd translator > python -m venv venv > .\venv\Scripts\activate > python -m pip install googletrans-py httpcore sh $ mkdir translator $ cd translator/ $ python -m venv venv $ source venv/bin/activate (venv) $ python -m pip install googletrans-py httpcore

Working through these instructions, first we create a root directory for the Translator app. Next we create and activate a Python virtual environment for the project. Finally, we install googletrans and httpcore packages.

Now, create a file named translator.py in the root of your project. Additionally, create a folder called images/ where you'll store the icons for the application. The folder structure should look like this:

python translator/ &boxv &boxvr&boxh&boxh images/ &boxv &boxvr&boxh&boxh arrow.png &boxv &boxur&boxh&boxh logo.png &boxv &boxur&boxh&boxh translator.py

The images for this project can be downloaded here.

The images/ folder contains the two icons that you'll use for the application. The translator.py is the app's source file.

Building the Window

Open the translator.py file with your favorite Python code editor. We'll start by creating our main window:

python import tkinter as tk class TranslatorApp(tk.Tk): def __init__(self): super().__init__() self.title("Language Translator") self.resizable(width=False, height=False) if __name__ == "__main__": app = TranslatorApp() app.mainloop()

This code imports Tkinter and then defines the application's main class, which we have called TranslatorApp. This class will hold the application's main window and allow us to run the main loop.

Importing tkinter under the alias tk is a common convention in Tkinter code.

Inside the class we define the __init__() method, which handles initialization of the class. In this method we first call the initializer __init__() of the parent class, tk.Tk, to initialize the app's window. Then, we set the window's title using the title() method. To make the window unresizable, we use the resizable() method with width and height set to False.

At the bottom of the code, we have the if __name__ == "__main__" idiom to check whether the file is being run directly as an executable program. Inside the condition block we first create an instance of TranslatorApp and then run the application's main loop or event loop.

If you run this code, you'll get an empty Tkinter window on your desktop:

python python translator.py

The empty Tkinter window

Creating the GUI

Now that the main window is set up, let's start adding widgets to build the GUI. To do this, we'll create a method called setup_ui(), as shown below:

python import tkinter as tk class TranslatorApp(tk.Tk): def __init__(self): super().__init__() self.title("Language Translator") self.resizable(width=False, height=False) self.setup_ui() def setup_ui(self): frame = tk.Frame(self) frame.pack(padx=10, pady=10) if __name__ == "__main__": app = TranslatorApp() app.mainloop()

The setup_ui() method will define the application's GUI. In this method, we first create a frame widget using the tk.Frame class whose master argument is set to self (the application's main window). Next, we position the frame inside the main window using the pack() geometry manager, using padx and pady arguments to set some padding around the frame.

Finally, we add the call to self.setup_ui() to the __init__() method.

We'll continue to develop the UI by adding code to the setup_ui() method.

Net we'll add the app's logo. In the setup_ui() method add the following code below the frame definition:

python import tkinter as tk class TranslatorApp(tk.Tk): def __init__(self): super().__init__() self.title("Language Translator") self.resizable(width=False, height=False) self.setup_ui() def setup_ui(self): frame = tk.Frame(self) frame.pack(padx=10, pady=10) self.logo = tk.PhotoImage(file="images/logo.png").subsample(5, 5) tk.Label(frame, image=self.logo).grid(row=0, column=0, sticky="w") if __name__ == "__main__": app = TranslatorApp() app.mainloop()

This code loads the logo using the tk.PhotoImage class. To resize it, we use the subsample() method. Then, we add the logo to the frame using a tk.Label widget. The label takes the frame and the logo as arguments. Finally, to position the logo, we use the grid() geometry manager with appropriate values for the row, column, and sticky arguments.

The sticky argument determines which side of a cell the widget should align -- North (top), South (bottom), East (right) or West (left). Here we're aligning it on the Wiest or left of the cell with "w":

Tkinter window with the Google Translate logo in it

Let's start adding some inputs to the UIs. First, we'll create the language selection drop-down boxes:

python import tkinter as tk import tkinter.ttk as ttk from googletrans import LANGUAGES DEFAULT_SOURCE = "English" DEFAULT_DEST = "Dutch" class TranslatorApp(tk.Tk): def __init__(self): super().__init__() self.title("Language Translator") self.resizable(width=False, height=False) self.setup_ui() def setup_ui(self): frame = tk.Frame(self) frame.pack(padx=10, pady=10) self.logo = tk.PhotoImage(file="images/logo.png").subsample(5, 5) tk.Label(frame, image=self.logo).grid(row=0, column=0, sticky="w") # Source language combobox languages = [lang.title() for lang in LANGUAGES.values()] self.from_language = ttk.Combobox(frame, values=languages) self.from_language.current(languages.index(DEFAULT_SOURCE)) self.from_language.grid(row=1, column=0, sticky="we") # Arrow icon self.arrows = tk.PhotoImage(file="images/arrow.png").subsample(15, 15) tk.Label(frame, image=self.arrows).grid(row=1, column=1) # Destination language combobox self.to_language = ttk.Combobox(frame, values=languages) self.to_language.current(languages.index(DEFAULT_DEST)) self.to_language.grid(row=1, column=2, sticky="we") if __name__ == "__main__": app = TranslatorApp() app.mainloop()

We can get a list of languages from the googletrans module. We also define the default languages for when the application starts up, using constants DEFAULT_SOURCE and DEFAULT_DEST.

To build the language list for the combo boxes we take the LANGUAGES dictionary imported from googletrans and convert it into a list of strings called languages. We use a list comprehension to make all the names start with an uppercase letter, using title().

Next we create two combo boxes, to hold the list of source and destination langauges. The combo boxes are created using the ttk.Combobox class. One to the left and another to the right. Between the combo boxes, we've also added an arrow icon loaded using the tk.PhotoImage class. Again, we've added the icon to the app's window using ttk.Label.

Both combo boxes take frame and values as arguments. The values argument populates the combo boxes with languages. To specify the default language, we use the current() method, looking up the position of our default languages in the languages list with index().

To position the combo boxes inside the frame, we use the grid() geometry manager with the appropriate arguments. Run the application, and you will see the following window:

Source and destination languages

If you click on the sources combo box on the left, then you get the following:

Source language combo box

Similarly, if you click on the destination combo box on the right, you get the following:

Destination combo box

With the source and destination combo boxes in place, let's add three more widgets: two scrollable text widgets and a button. The scrollable text on the left will hold the source text, while the scrollable text on the right will hold the translated text. The button will allow us to run the actual translation.

Get back to the code editor and update the setup_ui() method as follows. Note that we also need to import the ScrollText class:

python import tkinter as tk import tkinter.ttk as ttk from tkinter.scrolledtext import ScrolledText from googletrans import LANGUAGES DEFAULT_SOURCE = "English" DEFAULT_DEST = "Dutch" class TranslatorApp(tk.Tk): def __init__(self): super().__init__() self.title("Language Translator") self.resizable(width=False, height=False) self.setup_ui() def setup_ui(self): frame = tk.Frame(self) frame.pack(padx=10, pady=10) self.logo = tk.PhotoImage(file="images/logo.png").subsample(5, 5) tk.Label(frame, image=self.logo).grid(row=0, column=0, sticky="w") # Source language combobox languages = [lang.title() for lang in LANGUAGES.values()] self.from_language = ttk.Combobox(frame, values=languages) self.from_language.current(languages.index(DEFAULT_SOURCE)) self.from_language.grid(row=1, column=0, sticky="we") # Arrow icon self.arrows = tk.PhotoImage(file="images/arrow.png").subsample(15, 15) tk.Label(frame, image=self.arrows).grid(row=1, column=1) # Destination language combobox self.to_language = ttk.Combobox(frame, values=languages) self.to_language.current(languages.index(DEFAULT_DEST)) self.to_language.grid(row=1, column=2, sticky="we") # Source text self.from_text = ScrolledText( frame, font=("Dotum", 16), width=50, height=20, ) self.from_text.grid(row=2, column=0) # Translated text self.to_text = ScrolledText( frame, font=("Dotum", 16), width=50, height=20, state="disabled", ) self.to_text.grid(row=2, column=2) # Translate button self.translate_button = ttk.Button( frame, text="Translate", command=self.translate, ) self.translate_button.grid(row=3, column=0, columnspan=3, pady=10) def translate(self): pass if __name__ == "__main__": app = TranslatorApp() app.mainloop()

In the code snippet, we use the ScrolledText class to create the two scrolled text areas. Both text areas take frame, font, width, and height as arguments. The second text area also takes state as an additional argument. Setting state to "disabled" allows us to create a read-only text area.

Then, we use the ttk.Button class to create a button with frame, text, and command as arguments. The command argument allows us to bind the button's click event to the self.translate() method, which we will define in a moment. For now, we've added a placeholder.

To position all these widgets on the app's window, we use the grid() geometry manager. Now, the app will look like the following:

Translator app's GUI

Our translation app's GUI is ready! Finally, we can start adding functionality to the application.

Implementing the Translation Functionality

We'll implement the language translation functionality in the translate() method. This gets the current values from the UI and then uses googletrans to perform the translation. We need a few more imports, and to create the translator instance at the top of the application:

python import tkinter as tk import tkinter.ttk as ttk from tkinter.messagebox import showerror from tkinter.scrolledtext import ScrolledText import httpcore from googletrans import LANGUAGES, Translator DEFAULT_SOURCE = "English" DEFAULT_DEST = "Dutch" translator = Translator()

Here we've imported the showerror helper for displaying error boxes in our application. We've imported httpcore which we'll use to handle HTTP errors when accessing the API. Finally, we've added an import for the Translator class from googletrans. This is what handles the actual translation.

To use it, we create an instance of the class as translator.

We'll continue by implementing the translate method. Below we're just showing the function itself:

python class TranslatorApp(tk.Tk): # ... def translate(self): source_language = self.from_language.get() destination_language = self.to_language.get() text = self.from_text.get(1.0, tk.END).strip() if not source_language or not destination_language: showerror( title="Error", message="Make sure to set the source and destination language", ) return if not text: showerror( title="Error", message="Make sure to enter some text to translate", ) return try: translation = self.translator.translate( text, src=source_language, dest=destination_language, ) except httpcore.ConnectError: showerror( title="Error", message="Make sure you have an internet connection", ) return except Exception as e: showerror( title="Error", message=f"An unexpected error occurred: {e}", ) return self.to_text.config(state="normal") self.to_text.delete(1.0, tk.END) self.to_text.insert(tk.END, translation.text) self.to_text.config(state="disabled")

The translate() method handles the entire translation process. It starts by retrieving the source and destination languages from the corresponding combo boxes. If either language is not defined, then we display a message to inform the user about the problem. To do this, we use a showerror dialog.

Next, we try to get the input text from the source scrolled area on the left. If the user doesn't provide any text, then we display an error with the appropriate message.

Once we have the source and destination language and some text to translate, we can perform the actual translation. To run this task, we use the translate() method of the self.translator object, which is an instance of googletrans.Translator.

If the call to translate() finds a connection error, then we tell the user to check their internet connection. To handle any other exceptions, we catch the generic Exception class and display an error message with the exception details.

If the translation is successful, then we enable the destination scrolled area, display the translated text, and disable the area again so it remains read-only.

The complete final code is shown below:

python import tkinter as tk import tkinter.ttk as ttk from tkinter.messagebox import showerror from tkinter.scrolledtext import ScrolledText import httpcore from googletrans import LANGUAGES, Translator DEFAULT_SOURCE = "English" DEFAULT_DEST = "Dutch" translator = Translator() class TranslatorApp(tk.Tk): def __init__(self): super().__init__() self.title("Language Translator") self.resizable(width=False, height=False) self.setup_ui() def setup_ui(self): languages = [lang.title() for lang in LANGUAGES.values()] frame = tk.Frame(self) frame.pack(padx=10, pady=10) self.logo = tk.PhotoImage(file="images/logo.png").subsample(5, 5) tk.Label(frame, image=self.logo).grid(row=0, column=0, sticky="w") # Source language combobox self.from_language = ttk.Combobox(frame, values=languages) self.from_language.current(languages.index(DEFAULT_SOURCE)) self.from_language.grid(row=1, column=0, sticky="we") # Arrow icon self.arrows = tk.PhotoImage(file="images/arrow.png").subsample(15, 15) tk.Label(frame, image=self.arrows).grid(row=1, column=1) # Destination language combobox self.to_language = ttk.Combobox(frame, values=languages) self.to_language.current(languages.index(DEFAULT_DEST)) self.to_language.grid(row=1, column=2, sticky="we") # Source text self.from_text = ScrolledText( frame, font=("Dotum", 16), width=50, height=20, ) self.from_text.grid(row=2, column=0) # Translated text self.to_text = ScrolledText( frame, font=("Dotum", 16), width=50, height=20, state="disabled", ) self.to_text.grid(row=2, column=2) # Translate button self.translate_button = ttk.Button( frame, text="Translate", command=self.translate, ) self.translate_button.grid(row=3, column=0, columnspan=3, pady=10) def translate(self): source_language = self.from_language.get() destination_language = self.to_language.get() if not source_language or not destination_language: showerror( title="Error", message="Make sure to set the source and destination language", ) return text = self.from_text.get(1.0, tk.END).strip() if not text: showerror( title="Error", message="Make sure to enter some text to translate", ) return try: translation = translator.translate( text, src=source_language, dest=destination_language, ) except httpcore.ConnectError: showerror( title="Error", message="Make sure you have an internet connection", ) return except Exception as e: showerror( title="Error", message=f"An unexpected error occurred: {e}", ) return self.to_text.config(state="normal") self.to_text.delete(1.0, tk.END) self.to_text.insert(tk.END, translation.text) self.to_text.config(state="disabled") if __name__ == "__main__": app = TranslatorApp() app.mainloop() Using the Translator App

The video below demonstrates how we can use our app to translate some text from one natural language to another:

Great! You have successfully built a language translator using Python, Tkinter, and the googletrans package.

Conclusion

In this tutorial we built a Translator application using the Tkinter GUI library from the Python standard library. We worked step by step through building the UI using a grid layout, and then implemented the language translation functionality with googletrans.

Try and take what you've learnt in this tutorial & applying it to your own projects!

Spyder 6 includes a brand new remote development platform, building the foundations for enabling Spyder to easily leverage the power and data of remote desktops, servers, clusters and the cloud from anywhere! Hendrik Louzada, the lead developer behind this new backend, shares the details on how it all works, how he implemented it and his experiences doing so, and what's next for remote development in Spyder 6.1!

Ready to boost your workflows with pre-trained ML models? 

PyCharm‘s integration with Hugging Face is designed to bring a new level of productivity to your machine learning and development workflows. 

This integration helps you seamlessly find and use the best-fit model from the Hugging Face library, access model documentation, and manage models – all in your IDE.

The integration is available starting with PyCharm 2024.2.

Read the blog post to learn more about the integration and how to use it.

What is Hugging Face?

Hugging Face is a platform where machine learning and data science developers share pre-trained AI models. The platform provides tools to build, deploy, and train machine learning models.

Key features of the Hugging Face integration

PyCharm’s integration with Hugging Face is designed to streamline workflows for developers, enabling seamless access to pre-trained machine learning models from the IDE. This integration allows you to:

• Simplify model selection and usage.
• Import models as easily as importing any other library, allowing you to stay focused on your code. 
• Eliminate distractions caused by switching between tools and browsers. 
• Maintain control over your machine storage.

Lysandre Debut, the Chief Open-Source Officer at Hugging Face, shares:

“As a Hugging Face user, even more so than as a Hugging Face team member, the HF integration in PyCharm has been instrumental in speeding up the machine learning workflows I’ve worked on. As a result of this feature, I find myself keeping my focus in the IDE, with much less context switching during development.” Easily find the best model for your task

Looking to use a Hugging Face model but unsure which one fits your needs? Whether you’re building a text classifier, working on image recognition, or exploring other areas, PyCharm simplifies the process for you.

With the Hugging Face integration, you can quickly access a vast library of models tailored to various tasks. Right-click in the editor, select Insert HF Model, and explore models categorized by task type. To make your search even easier, you can filter models by likes, licensing, or specific tags.

For each model, PyCharm provides a detailed model card, including essential information and sample code that you can use. Found the perfect fit? Just click Use Model to insert the necessary code snippet directly into your project and start building immediately.

Seamlessly access model documentation

When working with machine learning models, quick access to documentation is handy – and PyCharm ensures you stay focused. With the Hugging Face integration, you can instantly view detailed model information without leaving your IDE or interrupting your workflow.

Simply hover over a model name in your code, and PyCharm will display its full model card, including the tasks the model is designed for, date of its last update, its origin, licensing details, and other details.

No more jumping between your browser and IDE – everything you need is right next to your code.

Manage models stored on your machine

Hugging Face models can take up significant storage space, but PyCharm makes it easy to stay in control. With this integration, you can view the models you’ve downloaded, identify those you no longer need, and declutter with ease. 

Just head to the Hugging Face tool window to see and manage your models.

PyCharm helps you to keep your system optimized while ensuring you retain access to the tools you actually need – all from the comfort of your IDE! 

Get started with PyCharm integrated with Hugging Face

PyCharm is a powerful IDE designed for machine learning, data science, and web development. It has everything you need for productive coding, including intelligent coding assistance (both local and AI-powered), smart navigation, and project-wide refactorings. 

It also provides a full suite of integrations tailored to streamline machine learning and data science workflows, including support for Jupyter Notebooks, Databricks, and popular scientific libraries like pandas, Polars, NumPy, scikit-learn, and more.

Use the PyCharm4HF code on the redeem page below to get started with PyCharm for free.

Get a 3-month PyCharm subscription for free

Need more guidance? Head to the documentation for step-by-step instructions on using Hugging Face with PyCharm.

We’d love to hear your thoughts! Share your experience with the integration in the comments below.

In this tutorial, we will learn about Python lists (creating lists, changing list items, removing items, and other list operations) with the help of examples.

If you’ve worked with statistical data, you’ve likely encountered z-scores. A z-score measures how far a data point is from the mean, expressed in terms of standard deviations. It helps identify outliers and compare data distributions, making it a vital tool in data science.

In this guide, we’ll show you how to calculate z-scores in Python using a custom function and built-in libraries like SciPy. You’ll also learn to visualize z-scores for better insights.

1. What is a z-score?

A z-score measures how many standard deviations a data point is from the mean. The formula for calculating the z-score of a data point X is:

\[Z_{X} = \frac{X - \overline{X}}{S}\]

Where:

• \(Z_{X}\) is the z score of the point \(X\);
• \(X\) is the value for which we want to calculate the Z score;
• \(\overline{X}\) is the mean of the sample;
• \(S\) is the standard deviation of the sample.

2. Python z score using a custom function

A custom function allows you to implement the z-score formula directly. Here’s how to define and use it in Python:

content_copy Copy

def calculate_z(X, X_mean, X_sd): return (X - X_mean) / X_sd

The function takes three arguments:

• a vector X of values for which you want to calculate the z-scores, like a pandas dataframe column, for example;
• the mean of the values in X;
• the standard deviation of the values in X.

Finally, in the return clause, we apply the z-score formula explained above.

To test our function, we will use data from Playfair (1821). He collected data regarding the price of wheat and the typical weekly wage for a “good mechanic” in England from 1565 to 1821. His objective was to show how well-off working men were in the 19th century. This dataset is available in the HistData R package and also on the webpage of Professor Vincent Arel-Bundock, a great source of datasets. It consists of 3 variables: year, price of wheat (in Shillings) and weekly wages (in Shillings).

We will be calculating the z-scores for the weekly wages. First we load the dataset directly from the website, as indicated in the code below.

content_copy Copy

import pandas as pd data = pd.read_csv("https://vincentarelbundock.github.io/Rdatasets/csv/HistData/Wheat.csv") print(data['Wages'].mean()) print(data['Wages'].std()) data["z-score_wages"] = calculate_z(data["Wages"], data["Wages"].mean(), data["Wages"].std())

The average weekly wage during the period was 11.58 Shillings, with a standard deviation of 7.34. With this information, we can calculate the Z score for each observation in the dataset. This is done and stored in a new column called “z-score_wages”.

If you check the first row of the data frame, you will find out that in 1565 the z score was around -0.9, that is, the wages were 0.9 standard deviations below the mean of the values for the whole period.

3. Python z score using SciPy

A second option to calculate z-scores in Python is to use the zscore method of the SciPy library as shown below. Ensure you set a policy for handling missing values if your dataset is incomplete.

In the code below, we calculate the z-scores for Wheat prices. If you look at the z-score summary statistics, you will see that the price of wheat varied between -1.13 and 3.65 standard deviations away from the mean in the observed period.

content_copy Copy

from scipy import stats data["z-score_wheat"] = stats.zscore(data["Wheat"], nan_policy="omit") data["z-score_wheat"].describe()

3. Visualising z scores

Below you can better visualize the basic idea of z scores: to measure how far away a data point is from the mean in terms of standard deviations. This visualization was created in D3, a JavaScript library for interactive data visualization. Click “See average wage” to see the averave wage for the whole period. Then check out how far from the mean each data point is and finally note that the z-score consists of this distance in terms of standard deviation.

1. See Average Wage 2. See Distance to the Mean 3. See Z-Scores Reset

4. Visualizing z scores with Matplotlib

The code below plots the wage z scores over time and shows them as the distance from the point to the mean, as demonstrated in the D3 visualization above. Please consult the lesson ‘Storytelling with Matplotlib - Visualizing historical data’ to learn more about Matplotlib visualizations.

content_copy Copy

# Calculate mean wage mean_wage = data["z-score_wages"].mean() # Create the plot fig, ax = plt.subplots(figsize=(10, 6)) # Scatter plot of wages over years ax.plot(data["Year"], data["z-score_wages"], 'o', color='#FF6885', label="Wage Z-scores", markeredgewidth=0.5) # Add a horizontal line for the mean wage ax.axhline(y=mean_wage, color='gray', linestyle='dashed', label=f"Mean Z-score = {mean_wage:.2f}") # Add gray lines connecting points to the mean for year, wage in zip(data["Year"], data["z-score_wages"]): ax.plot([year, year], [mean_wage, wage], color='gray', linestyle='dotted', linewidth=1) # Customize the plot ax.set_xlabel("Year") ax.set_ylabel("Z-scores") ax.set_title("Z-scores Over Time") ax.legend() # Show the plot plt.show()

Have questions or insights? Leave a comment below, and I’ll be happy to help.

Happy coding!

Conclusions

• A z score is a measure of how many standard deviations a data point is away from the mean. It can be easily calculated in Python;
• You can visualize z-scores using traditional python libraries like Matplotlib or Seaborn.