Planet Python
Real Python: Python's Magic Methods: Leverage Their Power in Your Classes
As a Python developer who wants to harness the power of object-oriented programming, you’ll love to learn how to customize your classes using special methods, also known as magic methods or dunder methods. A special method is a method whose name starts and ends with a double underscore. These methods have special meanings for Python.
Python automatically calls magic methods as a response to certain operations, such as instantiation, sequence indexing, attribute managing, and much more. Magic methods support core object-oriented features in Python, so learning about them is fundamental for you as a Python programmer.
In this tutorial, you’ll:
- Learn what Python’s special or magic methods are
- Understand the magic behind magic methods in Python
- Customize different behaviors of your custom classes with special methods
To get the most out of this tutorial, you should be familiar with general Python programming. More importantly, you should know the basics of object-oriented programming and classes in Python.
Get Your Code: Click here to download the free sample code that shows you how to use Python’s magic methods in your classes.
Getting to Know Python’s Magic or Special MethodsIn Python, special methods are also called magic methods, or dunder methods. This latter terminology, dunder, refers to a particular naming convention that Python uses to name its special methods and attributes. The convention is to use double leading and trailing underscores in the name at hand, so it looks like .__method__().
Note: In this tutorial, you’ll find the terms special methods, dunder methods, and magic methods used interchangeably.
The double underscores flag these methods as core to some Python features. They help avoid name collisions with your own methods and attributes. Some popular and well-known magic methods include the following:
Special Method Description .__init__() Provides an initializer in Python classes .__str__() and .__repr__() Provide string representations for objects .__call__() Makes the instances of a class callable .__len__() Supports the len() functionThis is just a tiny sample of all the special methods that Python has. All these methods support specific features that are core to Python and its object-oriented infrastructure.
Note: For the complete list of magic methods, refer to the special method section on the data model page of Python’s official documentation.
The Python documentation organizes the methods into several distinct groups:
- Basic customization
- Customizing attribute access
- Customizing class creation
- Customizing instance and subclass checks
- Emulating generic types
- Emulating callable objects
- Emulating container types
- Emulating numeric types
- with statement context managers
- Customizing positional arguments in class pattern matching
- Emulating buffer types
- Special method lookup
Take a look at the documentation for more details on how the methods work and how to use them according to your specific needs.
Here’s how the Python documentation defines the term special methods:
A method that is called implicitly by Python to execute a certain operation on a type, such as addition. Such methods have names starting and ending with double underscores. (Source)
There’s an important detail to highlight in this definition. Python implicitly calls special methods to execute certain operations in your code. For example, when you run the addition 5 + 2 in a REPL session, Python internally runs the following code under the hood:
Python >>> (5).__add__(2) 7 Copied!The .__add__() special method of integer numbers supports the addition that you typically run as 5 + 2.
Reading between the lines, you’ll realize that even though you can directly call special methods, they’re not intended for direct use. You shouldn’t call them directly in your code. Instead, you should rely on Python to call them automatically in response to a given operation.
Note: Even though special methods are also called magic methods, some people in the Python community may not like this latter terminology. The only magic around these methods is that Python calls them implicitly under the hood. So, the official documentation refers to them as special methods instead.
Magic methods exist for many purposes. All the available magic methods support built-in features and play specific roles in the language. For example, built-in types such as lists, strings, and dictionaries implement most of their core functionality using magic methods. In your custom classes, you can use magic methods to make callable objects, define how objects are compared, tweak how you create objects, and more.
Note that because magic methods have special meaning for Python itself, you should avoid naming custom methods using leading and trailing double underscores. Your custom method won’t trigger any Python action if its name doesn’t match any official special method names, but it’ll certainly confuse other programmers. New dunder names may also be introduced in future versions of Python.
Magic methods are core to Python’s data model and are a fundamental part of object-oriented programming in Python. In the following sections, you’ll learn about some of the most commonly used special methods. They’ll help you write better object-oriented code in your day-to-day programming adventure.
Controlling the Object Creation ProcessWhen creating custom classes in Python, probably the first and most common method that you implement is .__init__(). This method works as an initializer because it allows you to provide initial values to any instance attributes that you define in your classes.
Read the full article at https://realpython.com/python-magic-methods/ »[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
PyCharm: How To Learn Django: A Comprehensive Guide for Beginners
Brett Cannon: An experimental pip subcommand for the Python Launcher for Unix
There are a couple of things I always want to be true when I install Python packages for a project:
- I have a virtual environment
- Pip is up-to-date
For virtual environments, you would like them to be created as fast as possible and (usually) with the newest version of Python. For keeping pip up-to-date, it would be nice to not have to do that for every single virtual environment you have.
To help make all of this true for myself, I created an experimental Python Launcher for Unix "subcommand": py-pip. The CLI app does the following:
- Makes sure there is a globally cached copy of pip, and updates it if necessary
- Uses the Python Launcher for Unix to create a virtual environment where it finds a pyproject.toml file
- Runs pip using the virtual environment&aposs interpreter
This is all done via a py-pip.pyz file (which you can rename to just py-pip if you want). The py-pip.pyz file available from a release of py-pip can be made executable (e.g. chmod a+x py-pip.pyz). The shebang of the file is already set to #!/usr/bin/env py so it&aposs ready to use the newest version of Python you have installed. Stick that on your PATH and you can then use that instead of py -m pip to run pip itself.
To keep pip up-to-date, the easiest way to do that is to have only a single copy of pip to worry about. Thanks to the pip team releasing a self-contained pip.pyz along with pip always working with supported, it means if we just cache a copy of pip.pyz and keep that up-to-date then we can have that one copy to worry about.
Having a single copy of pip also means we don&apost need to install pip for each virtual environment. That lets us use microvenv and skip the overhead of installing pip in each virtual environment.
Now, this is an experiment. Much like the Python Launcher for Unix, py-pip is somewhat optimized for my own workflow. I am also keeping an eye on PEP 723 and PEP 735 as a way to only install packages that have been written down somewhere instead of ever installing a package à la carte as I think that&aposs a better practice to follow and might actually trump all of this. But since I have seen others have frustration from both forgetting the virtual environment and having to keep pip up-to-date, I decided to open source the code.
PyCoder’s Weekly: Issue #610 (Jan. 2, 2024)
#610 – JANUARY 2, 2024
View in Browser »
In this tutorial, you’ll explore the process of creating a boilerplate for a Flask web project. It’s a great starting point for any scalable Flask web app that you wish to develop in the future, from basic web pages to complex web applications.
REAL PYTHON
Slides related to the upcoming JIT commit for Python 3.13. Note, GitHub paginates it if you don’t download it, so click the “More Pages” button to keep reading.
GITHUB.COM/BRANDTBUCHER
Although 2023 was full of AI news in computer science, it wasn’t the only news. This article summarizes the breakthroughs in 2023.
BILL ANDREWS
Cosine similarity is a check to see if two vectors point in the same direction, regardless of magnitude. This test is frequently used in some machine learning algorithms. This article details the various steps in speeding up the code, starting with vanilla Python and going all the way down to hand tuned assembly language.
ASH VARDANIAN
It’s been a fascinating year for the Python language and community. PyCoder’s Weekly included over 1,500 links to articles, blog posts, tutorials, and projects in 2023. Christopher Trudeau is back on the show this week to help wrap up everything by sharing some highlights and Python trends from across the year.
REAL PYTHON podcast
Python includes soft keywords: tokens that are important to the parser but can also be used as variable names. This article shows you what a soft keyword is and how to find them in Python 3.12 (both the easy and hard way).
RODRIGO GIRÃO SERRÃO
This is a cross-language developer survey of tools used in the industry. It includes questions about AI adoption, cloud tools, and more. 54% of respondents use Python as their most frequent language.
JETBRAINS
This articles shows you how to take advantage of some the newer async mechanisms in Django to build a messaging app. Things that used to require a third party library are now part of the framework.
TOM DEKAN • Shared by Tom Dekan
The Boyer-Moore majority vote algorithm looks for an element that appears more than n/2 times in a sequence using O(n) time. This article shows you how it works using Python code.
GITHUB.COM/NAUGHTYCONSTRICTOR • Shared by Mohammed Younes ELFRAIHI
Many developers dread the idea of becoming a manager, but there are some things you can only learn by doing. This article outlines why management might be the right thing for you.
CHARITY MAJORS
Redowan has strong opinions on reserving dataclasses for data-class purposes only: their methods should have no data modification side-effects. This article outlines why.
REDOWAN DELOWAR
Django 5 was recently released and this in-depth article covers what changed, how to upgrade from an earlier version, and how the Django version numbering system works.
ERIC MATTHES
“Figuring out how much parallelism your program can use is surprisingly tricky.” This article shows you why it is complicated and what you can determine.
ITAMAR TURNER-TRAURING
A quick tip on how to set an environment variable so that pip refuses to install a package unless in an active virtual environment.
DANIEL ROY GREENFIELD
This post talks about how to store configuration for your script and how and when to load the information into your program.
ROBERT RODE
Knowing when to raise the right exception is important, but often you don’t have to: Python might do it for you.
JAMES BENNETT
January 3, 2024
REALPYTHON.COM
January 4, 2024
MEETUP.COM
January 4, 2024
SYPY.ORG
January 9, 2024
PITERPY.COM
January 9, 2024
MEETUP.COM
January 10 to January 11, 2024
NOKIDBEHIND.ORG
Happy Pythoning!
This was PyCoder’s Weekly Issue #610.
View in Browser »
[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]
Hynek Schlawack: How to Ditch Codecov for Python Projects
Codecov’s unreliability breaking CI on my open source projects has been a constant source of frustration for me for years. I have found a way to enforce coverage over a whole GitHub Actions build matrix that doesn’t rely on third-party services.
Real Python: HTTP Requests With Python's urllib.request
If you need to perform HTTP requests using Python, then the widely used Requests library is often the way to go. However, if you prefer to use only standard-library Python and minimize dependencies, then you can turn to urllib.request instead.
In this video course, you’ll:
- Learn the essentials of making basic HTTP requests with urllib.request
- Explore the inner workings of an HTTP message and how urllib.request represents it
- Grasp the concept of handling character encodings in HTTP messages
- Understand common hiccups when using urllib.request and learn how to resolve them
If you’re already familiar with HTTP requests such as GET and POST, then you’re well prepared for this video course. Additionally, you should have prior experience using Python to read and write files, ideally using a context manager.
In the end, you’ll discover that making HTTP requests doesn’t have to be a frustrating experience, despite its reputation. Many of the challenges people face in this process stem from the inherent complexity of the Internet. The good news is that the urllib.request module can help demystify much of this complexity.
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
Django Weblog: Django bugfix releases issued: 4.2.9 and 5.0.1
Today we've issued 5.0.1 and 4.2.9 bugfix releases.
The release package and checksums are available from our downloads page, as well as from the Python Package Index. The PGP key ID used for this release is Mariusz Felisiak: 2EF56372BA48CD1B.
Talk Python to Me: #444: The Young Coder's Blueprint to Success
Django Weblog: DjangoCon Europe 2024 CFP Now Open
It's a new year. What better way to start it than submitting your talk or workshop for DjangoCon Europe 2024, in beautiful Vigo, Spain?
The Call for Proposals (CFP) is open now, and will be until midnight on February 29th. That's two whole months, but you don't have to leave it to the last minute:
DjangoCon Europe 2024, Vigo CFP
We're looking for a range of talks on technical and non-technical topics. We're looking for talks accessible to all skill levels, and we're looking for submissions from new and seasoned speakers.
If you're asking, can I do this? The answer is yes. If you've got a topic that interest you, then it interests us.
If you've got half an idea, or aren't sure in any way, and want to chat, you can jump on the DjangoCon Europe Slack, and find us there.
Don't be shy, we want to hear from you!
DjangoCon Europe 2024, Vigo CFP
We'll see you in Vigo!
Real Python: Python's Array: Working With Numeric Data Efficiently
When you start your programming adventure, one of the most fundamental concepts that you encounter early on is the array. If you’ve recently switched to Python from another programming language, then you might be surprised that arrays are nowhere to be found as a built-in syntactical construct in Python. Instead of arrays, you typically use lists, which are slightly different and more flexible than classic arrays.
That said, Python ships with the lesser-known array module in its standard library, providing a specialized sequence type that can help you process binary data. Because it’s not as widely used or well documented as other sequences, there are many misconceptions surrounding the use of the array module. After reading this tutorial, you’ll have a clear idea of when to use Python’s array module and the corresponding data type that it provides.
In this tutorial, you’ll learn how to:
- Create homogeneous arrays of numbers in Python
- Modify numeric arrays just like any other sequence
- Convert between arrays and other data types
- Choose the right type code for Python arrays
- Emulate nonstandard types in arrays
- Pass a Python array’s pointer to a C function
Before you dive in, you may want to brush up on your knowledge of manipulating Python sequences like lists and tuples, defining custom classes and data classes, and working with files. Ideally, you should be familiar with bitwise operators and be able to handle binary data in Python.
You can download the complete source code and other resources mentioned in this tutorial by clicking the link below:
Get Your Code: Click here to download the free source code that shows you how to use Python’s array with your numeric data.
Understanding Arrays in ProgrammingSome developers treat arrays and Python’s lists as synonymous. Others argue that Python doesn’t have traditional arrays, as seen in languages like C, C++, or Java. In this brief section, you’ll try to answer whether Python has arrays.
Arrays in Computer ScienceTo understand arrays better, it helps to zoom out a bit and look at them through the lens of theory. This will clarify some baseline terminology, including:
- Abstract data types
- Data structures
- Data types
Computer science models collections of data as abstract data types (ADTs) that support certain operations like insertion or deletion of elements. These operations must satisfy additional constraints that describe the abstract data type’s unique behaviors.
The word abstract in this context means these data types leave the implementation details up to you, only defining the expected semantics or the set of available operations that an ADT must support. As a result, you can often represent one abstract data type using a few alternative data structures, which are concrete implementations of the same conceptual approach to organizing data.
Programming languages usually provide a few data structures in the form of built-in data types as a convenience so that you don’t have to implement them yourself. This means you can focus on solving more abstract problems instead of starting from scratch every time. For example, the Python dict data type is a hash table data structure that implements the dictionary abstract data type.
To reiterate the meaning of these terms, abstract data types define the desired semantics, data structures implement them, and data types represent data structures in programming languages as built-in syntactic constructs.
Some of the most common examples of abstract data types include these:
In some cases, you can build more specific kinds of abstract data types on top of existing ADTs by incorporating additional constraints. For instance, you can build a stack by modifying the queue or the other way around.
As you can see, the list of ADTs doesn’t include arrays. That’s because the array is a specific data structure representing the list abstract data type. The list ADT dictates what operations the array must support and which behaviors it should exhibit. If you’ve worked with the Python list, then you should already have a pretty good idea of what the list in computer science is all about.
Note: Don’t confuse the list abstract data type in computer science with the list data type in Python, which represents the former. Similarly, it’s easy to mistake the theoretical array data structure for a specific array data type, which many programming languages provide as a convenient primitive type built into their syntax.
The list abstract data type is a linear collection of values forming an ordered sequence of elements. These elements follow a specific arrangement, meaning that each element has a position relative to the others, identified by a numeric index that usually starts at zero. The list has a variable but finite length. It may or may not contain values of different types, as well as duplicates.
The interface of the list abstract data type resembles Python’s list, typically including the following operations:
List ADT Python’s list Get an element by an index fruits[0] Set an element at a given index fruits[0] = "banana" Insert an element at a given index fruits.insert(0, "banana") Delete an element by an index fruits.pop(0), del fruits[0] Delete an element by a value fruits.remove("banana") Delete all elements fruits.clear() Find the index of a given element fruits.index("banana") Append an element at the right end fruits.append("banana") Merge with another list fruits.extend(veggies), fruits + veggies Sort elements fruits.sort() Get the number of elements len(fruits) Iterate over the elements iter(fruits) Check if an element is present "banana" in fruitsNow that you understand where the array data structure fits into the bigger picture, it’s time to take a closer look at it.
Read the full article at https://realpython.com/python-array/ »[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
Tryton News: Newsletter January 2024
During the last month we mainly focused on fixing bugs, adjusting how things work, improving performance and adding new features.
Changes for the User Sales, Purchases and ProjectsRelated shipments and moves are now reset and cancelled when a purchase request is cancelled.
When creating a sale from an opportunity the sale now has default addresses and payment terms, if they are not defined on the opportunity.
The blanket agreement record names now contain the reference and will fallback to the id if there isn’t a number or a reference available, like we do for sales and purchases.
We added some UTM parameters to emails sent by the marketing modules. The following parameters were added:
- utm_campaign
- utm_medium
- utm_source
The create purchase wizard on purchase requests now opens the newly created purchases.
Since Tryton defaults to the most used currency for new purchases, we’ve now updated it to also do the same for purchase requests.
Accounting, Invoicing and PaymentsWhen using sale advance payments Tryton no longer sets the invoice date if the advance payment condition has no invoice delay. If you’d like the invoice date to default to today’s date, set a delay of 0.
Stock, Production and ShipmentsOn an unaltered system modification of product locations is now restricted to the Stock Administrator access group.
The progress of a move is now rounded to four digits.
Tryton now only checks if a lot is required when a move is going to the done state.
The stock location code is now included in the record name of a location.
We added a confirmation dialogue to the cancel- buttons on shipments.
When moving consumable products the default to-location is now preset with the default product location.
User InterfaceThe URL button is now hidden, when a URL field is empty, as disabling it did not prevent the user from clicking on it.
Each button in a list-view is now rendered read-only when the appropriate record is also read-only.
We improved the behaviour of button clicks. Now clicking rapidly on a button only launches the action once. This same behaviour has also been implemented for widget buttons.
Now labels are aligned to the start of the line on extra small screens.
On small screens we now hide the search widget on list views. A toolbar button shows the search widget on demand.
The workflow graphs for models no longer overlap and incorrectly share states.
More (click for more details) DocumentationWe reworked parts of the Tryton documentation.
In validation error messages the record name is now prefixed with the word “record” in order to make the message clearer.
New ReleasesWe released bug fixes for the currently maintained long term support series
7.0 and 6.0, and for the penultimate series 6.8.
For the Tryton desktop client we now support the arm64 darwin architecture allowing it to be built on Apple Silicon.
Changes for Implementers and DevelopersThe order of keys is now retained when changing a fields.Dictionary through an on_change method.
For selection and multiselection fields we now use the string version of the value in error messages.
1 post - 1 participant
Doug Hellmann: imapautofiler 1.14.0 - sort-by-year action
Seth Michael Larson: 2023 year in review
Published 2023-12-31 by Seth Larson
Reading time: minutes
2023 was a great year! So much happened, but a few things in particular stood out to be when putting together this post.
I got married to my wife, Trina after 9 years of being together. We met in college and fell in love with each others' passion for adventure, food, and life. Our ceremonies included both of our cultures and we had family and friends from all over the globe together to celebrate with us.
"Vu Quy" or "tea ceremony" with family. Photo credit: Summer Street Photography
We were married near where we first met on the University of Minnesota east bank campus and the Mississippi river. The photo below is my favorite of the whole day:
Stone Arch Bridge with the Minneapolis downtown in the background. Photo credit: Summer Street Photography
I traveled to many new places this year and got to see friends everywhere I went. This was my first year traveling to New York, Florida, Texas, Rhode Island, and northern Nevada. Places I'm looking forward to exploring in 2024 include Japan, Seattle, and Pittsburgh (see you at PyCon US!)
Elastic had its first Engineering All Hands in-person since COVID to start off 2023 where I got to meet my long-time friend and colleague Quentin Pradet. Quentin and I have been working on open source together for over 5 years and this was our first opportunity to meet in person.
Quentin and I together at Elastic EAH 2023
After 3 great years at Elastic I was hired by the Python Software Foundation to be the Security Developer-in-Residence. I still have days when I think I'm dreaming, I'm so grateful I have the opportunity to work full-time serving a community I love.
Banner included in OpenSSF's announcement of my hiring.
This blog saw a huge burst of activity thanks to my new position where I publish weekly reports on what I've been working on. There were 34 new publications to the blog in 2023 (up from 12 in 2022), of those 24 were related to the Security Developer-in-Residence role.
The top posts by readership for this year were:
- Python 3.12.0 from a supply-chain perspective
- Quirks of Python package versioning (PEP 440)
- Visualizing the CPython release process
- Patching the libwebp vulnerability across the Python ecosystem
- Python Security Response Team handling of CVE-2023-40217
If I had to pick a favorite post outside of this list it would be “For You” is not for me discussing my current outlook on internet consumption. Look forward to more posts on the blog, hopefully continuing the trend that I'm on of shorter but more frequent publications.
Outside of software I plan to spend more time playing games (beat Pikmin 4 and Super Mario RPG is in-progress) and learning some hardware-hacking for retro gaming like Gameboy and GameCube modding.
Hope you all had a lovely 2023, looking forward to what we can do in 2024! 🥳
Thanks for reading! ♡ Did you find this article helpful and want more content like it? Get notified of new posts by subscribing to the RSS feed or the email newsletter.
This work is licensed under CC BY-SA 4.0
Paolo Melchiorre: My 2023 in review
The review of my 2023, trying to remember all the things done in this year, in which more than anyone I met many fantastic people and visited new countries.
Zero to Mastery: Python Monthly Newsletter 💻🐍
Matt Layman: Python, Markdown, and Tailwind: Best Buds!
PyPy: PyPy has moved to Git, GitHub
PyPy has moved its canonical repo and issue tracker from https://foss.heptapod.net/pypy/pypy to https://github.com/pypy/pypy. Obviously, this means development will now be tracked in Git rather than Mercurial.
MotivationWe still feel Mercurial is a better version control system. The named branch model and user interface are superior. But
-
foss.heptapod.net is not well indexed in google/bing/duckduckgo search, so people find it harder to search for issues in the project.
-
Since Heptapod has tightened its spam control, we get reports that users create issues only to have them flagged as spam.
-
Open Source has become synonymous with GitHub, and we are too small to change that.
-
Much of the current development comes as a reaction to fixing issues. Tracking interlocking issues is easier if all the code is on the same platform.
-
The FAQ presents two arguments against the move. Github notes solves much of point (1): the difficulty of discovering provenance of commits, although not entirely. But the main problem is point (2), it turns out that not moving to GitHub is an impediment to contribution and issue reporting.
-
People who wish to continue to use Mercurial can use the same method below to push to GitHub.
-
GitHub is more resource rich than foss.heptapod.net. We could add CI jobs to replace some of our aging buildbot infrastructure.
The migration required two parts: migrating the code and then migrating the issues and merge requests.
Code migration 1: code and notesI used a fork of git-remote-hg to create a local Git repo with all the changesets. Then I wanted to add a Git note to each commit with the branch it came from. So I prepared a file with two columns: the Git commit hash, and the corresponding branch from Mercurial. Mercurial can describe each commit in two ways: either the commit hash or by a number index. I used hg log to convert an index i to the Mercurial hash, and then git-hg-helper from git-remote-hg to convert the Mercurial hash to a Git hash:
$(cd pypy-git; git-hg-helper git-rev $(cd ../pypy-hg; hg log -r $i -T"{node}\n"))Then I used hg log again to print the Mercurial branch for the index i:
$(cd pypy-hg; hg log -r $i -T'{branch}\n')Putting these two together, I could loop over all the commits by their numerical index to prepare the file. Then I iterated over each line in the file, and added the Git note. Since the git note add command works on the current HEAD, I needed to checkout each commit in turn and then add the note:
git checkout -q <hash> && git notes --ref refs/notes/branch add -m branch:<branch>I could then use git push --all to push to GitHub.
Code migration 2: prepare the branchesPyPy has almost 500 open branches. The code migration created all the branch HEADs, but git push --all did not push them. I needed to check them out and push each one. So I created a file with all the branch names
cd pypy-hg; hg branches | cut -f1 -d" " > branches.txtand then push each one to the GitHub repo
while read branch; do git checkout branches/$branch && git push origin branches/$branch; done < branches.txtNote that the branches were named branches/XXX by the migration, not branch/XXX. This confuses the merge request migration, more about that later.
Issue and merge request migrationI used the solution from node-gitlab-2-github which worked almost perfectly. It is important to do the conversion on a private repo otherwise every mention of a sucessfully mapped user name notifies the user about the transfer. This can be quite annoying for a repo the size of PyPy with 600 merge requests and over 4000 issues. Issues transfered without a problem: the script properly retained the issue numbers. However the script does not convert the Mercurial hashes to Git hashes, so the bare hashes in comments show up without a link to the commit. Merge requests are more of a problem:
- The Mercurial named branch "disappears" once it is merged, so a merge request to a merged branch does not find the target branch name in Git. The conversion creates an issue instead with the label gitlab merge request.
- For some reason, the branches created by git-remote-hg are called branches/XXX and not branch/XXX as expected by GitLab. This messes up the merge request/PR conversion. For some of the branches (open PRs and main target branches) I manually created additional branches without the es. The net result is that open merge requests became open PRs, merged merge requests became issues, and closed-not-merged merge requests were not migrated.
PyPy already migrated once from Bitbucket to Heptapod. Many of the issues reflect the multiple transitions: they have lines like "Created originally on Bitbucket by XXX" from the first transition, and an additional line "In Heptapod" from this transition.
CreditsWe would like to express our gratitude to the Octobus team who support Heptapod. The transition from Bitbucket was quite an effort, and they have generously hosted our developement since then. We wish them all the best, and still believe that Mercurial should have "won".
Next stepsWhile the repo at GitHub is live, there are still a few more things we need to do:
- Documentation needs an update for the new repo and the build automation from readthedocs must be adjusted.
- The wiki should be copied from Heptapod.
- buildbot.pypy.org should also look at the new repo. I hope the code is up to the task of interacting with a Git repo.
- speed.pypy.org tracks changes, it too needs to reference the new location
- To keep tracking branches with Git notes on new commits, I activated a github action by Julian to add a Git branch note to each commit. Please see the README there for directions on using Git notes.
- Some of the merge requests were not migrated. If someone wants to, they could migrate those once they figure out the branch naming problems.
Additionally, now is the time for all of you to prove the move is worthwhile:
- Star the repo, let others know how to find it,
- Help fix some of the open issues or file new ones,
- Take advantage of the more familiar workflow to get involved in the project,
- Suggest ways to improve the migration: are there things I missed or could have done better?
Heptapod did not allow personal forks, so we were generous with a commit bit to the main repo. Additionally, we (well, me) have been using a commit-directly-to-main workflow. We will now be adopting a more structured workflow. Please fork the repo and submit a pull request for any changes. We can now add some pre-merge CI to check that the PR at least passes the first stage of translation. The live and active branches will be:
- main: what was "default" in Mercurial, it is the Python2.7 interpreter and the base of the RPython interpreter,
- py3.9: the Python3.9 interpreter, which also includes all RPython changes from main. This is exactly like on Mercurial, and
- py3.10: the Python3.10 interpreter, which also includes all RPython changes from main and all bugfixes from py3.9. This is exactly like on Mercurial.
If you want to figure out how a Mercurial commit relates to a Git commit, you can use git-hg-helper. You run it in the Git repo. It takes the full long hash from one repo and gives you the corresponding hash of the other repo:
$ git-hg-helper git-rev d64027c4c2b903403ceeef2c301f5132454491df 4527e62ad94b0e940a5b0f9f20d29428672f93f7 $ git-hg-helper hg-rev 4527e62ad94b0e940a5b0f9f20d29428672f93f7 d64027c4c2b903403ceeef2c301f5132454491df Finding branchesBranches migrated from Mercurial will have a branches prefix, not branch. While GitLab uses branch for its prefix, the git-remote-hg script uses branches. New work should be in a PR targeting main, py3.9 or py3.10.
Thanks for helping to make PyPy better.
Matti
Real Python: The Real Python Podcast – Episode #185: 2023 Real Python Tutorial & Video Course Wrap-Up
Three members of the Real Python team are joining us this week: Kate Finegan, Tappan Moore, and Philipp Acsany. We wanted to share a year-end wrap-up with tutorials, step-by-step projects, code conversations, and video courses that showcase what our team created this year.
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
The Three of Wands: attrs iv: Zero-overhead Frozen attrs Classes
Here&aposs a quick and easy one.
attrs supports frozen classes. Frozen classes are cool for a multitude of reasons, but they&aposre a tiny bit slower to instantiate compared to non-frozen classes, because they need to perform some additional checks and avoiding these checks in the __init__ uses a tiny bit of time. (We&aposre talking slotted classes here; which are awesome and the default in attrs nowadays.)
But there&aposs a way to avoid this overhead and make frozen classes be the exact same speed as ordinary classes. The only caveat is: you have to use type-checking.
Create a file somewhere in your code base and put this in it:
from functools import partial from typing import TYPE_CHECKING if TYPE_CHECKING: from attrs import frozen else: from attrs import define frozen = partial(define, unsafe_hash=True)Now import frozen from this module and use it just like you&aposd use attrs.define or attrs.frozen. You&aposre done!
This technique should also work for dataclasses, with a slight adjustment left as an exercize to the reader.
The eagle-eyed reader might notice that we&aposre actually bamboozling the type-checker: your classes won&apost actually be frozen at runtime. The kicker is: they don&apost actually need to be.
As long as you&aposre running one on your codebase, the typechecker is the actual thing that&aposll prevent you from mutating your instances. The unsafe_hash=True will make the classes hashable, and it&aposs only unsafe if you mutate them after construction, which you won&apost. I guess you&aposll have to be careful when using a REPL or a different context where a typechecker might not hold sway, but I think that&aposs not too big of an issue.
If you&aposre still unconvinced, I&aposll leave you with two final thoughts: the memory in your computer is, ultimately, mutable. What makes immutable data structures in other languages immutable is just the amount of hoops you have to jump through to apply a mutation. This also demonstrates a basic technique statically-compiled languages use to be fast: move part of the work out of runtime into a separate, pre-runtime step. Which is exactly what we&aposve done here.
Happy New Year!
Talk Python to Me: #443: Python Bytes Crossover 2023
Pages
Recent Publications
- Managing Hidden Dependencies in OO Software: a Study Based on Open Source Projects
- Open Source Communities as Liminal Ecosystems
- Investigating developers' email discussions during decision-making in Python language evolution
- Developers, Quality Control and Download Volume in Open Source Software (OSS) Projects
More...FLOSS Project Planets
- Reproducible Builds (diffoscope): diffoscope 265 released
- Drupalize.Me: Tuning Drupalize.Me Search Results with Solr Query Re-Ranking and Search API
- Drupal Association blog: New Things Happening At DrupalCon Portland
- Jonathan McDowell: Sorting out backup internet #2: 5G modem
- Gábor Hojtsy: New Upgrade Status 4.2.0 is beautiful in Gin, improves continuous integration compatibility and more
FLOSS Research
- Open Source AI Definition – Weekly update April 15
- Submit your proposal for All Things Open – Doing Business with Open Source
- Compelling responses to NTIA’s AI Open Model Weights RFC
- Open Source AI Definition – Weekly update April 8
- OSI participates in Columbia Convening on openness and AI; first readouts available