Planet Python

Subscribe to Planet Python feed
Planet Python - http://planetpython.org/
Updated: 1 day 59 min ago

Real Python: HTTP Requests With Python's urllib.request

Tue, 2024-01-02 09:00

If you need to perform HTTP requests using Python, then the widely used Requests library is often the way to go. However, if you prefer to use only standard-library Python and minimize dependencies, then you can turn to urllib.request instead.

In this video course, you’ll:

  • Learn the essentials of making basic HTTP requests with urllib.request
  • Explore the inner workings of an HTTP message and how urllib.request represents it
  • Grasp the concept of handling character encodings in HTTP messages
  • Understand common hiccups when using urllib.request and learn how to resolve them

If you’re already familiar with HTTP requests such as GET and POST, then you’re well prepared for this video course. Additionally, you should have prior experience using Python to read and write files, ideally using a context manager.

In the end, you’ll discover that making HTTP requests doesn’t have to be a frustrating experience, despite its reputation. Many of the challenges people face in this process stem from the inherent complexity of the Internet. The good news is that the urllib.request module can help demystify much of this complexity.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Categories: FLOSS Project Planets

Django Weblog: Django bugfix releases issued: 4.2.9 and 5.0.1

Tue, 2024-01-02 04:03

Today we've issued 5.0.1 and 4.2.9 bugfix releases.

The release package and checksums are available from our downloads page, as well as from the Python Package Index. The PGP key ID used for this release is Mariusz Felisiak: 2EF56372BA48CD1B.

Categories: FLOSS Project Planets

Talk Python to Me: #444: The Young Coder's Blueprint to Success

Tue, 2024-01-02 03:00
Are you early in your software dev or data science career? Maybe it hasn't even really started yet and you're still in school. On this episode we have Sydney Runkle who has had a ton of success in the Python space and she hasn't even graduated yet. We sit down to talk about what she's done and might do differently again to achieve that success. It's "The Young Coder's Blueprint to Success" on episode 444 of Talk Python To Me.<br/> <br/> <strong>Links from the show</strong><br/> <br/> <div><b>Sydney Runkle</b>: <a href="https://www.linkedin.com/in/sydney-runkle-105a35190/" target="_blank" rel="noopener">linkedin.com</a><br/> <b>Pydantic</b>: <a href="https://pydantic.dev" target="_blank" rel="noopener">pydantic.dev</a><br/> <b>Code Combat</b>: <a href="https://codecombat.com/play" target="_blank" rel="noopener">codecombat.com</a><br/> <b>Humanitarian Toolbox</b>: <a href="http://www.htbox.org" target="_blank" rel="noopener">www.htbox.org</a><br/> <b>PyCon 2024</b>: <a href="https://us.pycon.org/2024/" target="_blank" rel="noopener">pycon.org</a><br/> <b>Good first issue example</b>: <a href="https://github.com/pydantic/pydantic/labels/good%20first%20issue" target="_blank" rel="noopener">github.com</a><br/> <b>Watch this episode on YouTube</b>: <a href="https://www.youtube.com/watch?v=LtEYowIazVQ" target="_blank" rel="noopener">youtube.com</a><br/> <b>Episode transcripts</b>: <a href="https://talkpython.fm/episodes/transcript/444/the-young-coders-blueprint-to-success" target="_blank" rel="noopener">talkpython.fm</a><br/> <br/> <b>--- Stay in touch with us ---</b><br/> <b>Subscribe to us on YouTube</b>: <a href="https://talkpython.fm/youtube" target="_blank" rel="noopener">youtube.com</a><br/> <b>Follow Talk Python on Mastodon</b>: <a href="https://fosstodon.org/web/@talkpython" target="_blank" rel="noopener"><i class="fa-brands fa-mastodon"></i>talkpython</a><br/> <b>Follow Michael on Mastodon</b>: <a href="https://fosstodon.org/web/@mkennedy" target="_blank" rel="noopener"><i class="fa-brands fa-mastodon"></i>mkennedy</a><br/></div><br/> <strong>--- Episode sponsors ---</strong><br/> <a href='https://talkpython.fm/training'>Talk Python Training</a>
Categories: FLOSS Project Planets

Django Weblog: DjangoCon Europe 2024 CFP Now Open

Mon, 2024-01-01 13:00

It's a new year. What better way to start it than submitting your talk or workshop for DjangoCon Europe 2024, in beautiful Vigo, Spain?

The Call for Proposals (CFP) is open now, and will be until midnight on February 29th. That's two whole months, but you don't have to leave it to the last minute:

DjangoCon Europe 2024, Vigo CFP

We're looking for a range of talks on technical and non-technical topics. We're looking for talks accessible to all skill levels, and we're looking for submissions from new and seasoned speakers.

If you're asking, can I do this? The answer is yes. If you've got a topic that interest you, then it interests us.

If you've got half an idea, or aren't sure in any way, and want to chat, you can jump on the DjangoCon Europe Slack, and find us there.

Don't be shy, we want to hear from you!

DjangoCon Europe 2024, Vigo CFP

We'll see you in Vigo!

Categories: FLOSS Project Planets

Real Python: Python's Array: Working With Numeric Data Efficiently

Mon, 2024-01-01 09:00

When you start your programming adventure, one of the most fundamental concepts that you encounter early on is the array. If you’ve recently switched to Python from another programming language, then you might be surprised that arrays are nowhere to be found as a built-in syntactical construct in Python. Instead of arrays, you typically use lists, which are slightly different and more flexible than classic arrays.

That said, Python ships with the lesser-known array module in its standard library, providing a specialized sequence type that can help you process binary data. Because it’s not as widely used or well documented as other sequences, there are many misconceptions surrounding the use of the array module. After reading this tutorial, you’ll have a clear idea of when to use Python’s array module and the corresponding data type that it provides.

In this tutorial, you’ll learn how to:

  • Create homogeneous arrays of numbers in Python
  • Modify numeric arrays just like any other sequence
  • Convert between arrays and other data types
  • Choose the right type code for Python arrays
  • Emulate nonstandard types in arrays
  • Pass a Python array’s pointer to a C function

Before you dive in, you may want to brush up on your knowledge of manipulating Python sequences like lists and tuples, defining custom classes and data classes, and working with files. Ideally, you should be familiar with bitwise operators and be able to handle binary data in Python.

You can download the complete source code and other resources mentioned in this tutorial by clicking the link below:

Get Your Code: Click here to download the free source code that shows you how to use Python’s array with your numeric data.

Understanding Arrays in Programming

Some developers treat arrays and Python’s lists as synonymous. Others argue that Python doesn’t have traditional arrays, as seen in languages like C, C++, or Java. In this brief section, you’ll try to answer whether Python has arrays.

Arrays in Computer Science

To understand arrays better, it helps to zoom out a bit and look at them through the lens of theory. This will clarify some baseline terminology, including:

  • Abstract data types
  • Data structures
  • Data types

Computer science models collections of data as abstract data types (ADTs) that support certain operations like insertion or deletion of elements. These operations must satisfy additional constraints that describe the abstract data type’s unique behaviors.

The word abstract in this context means these data types leave the implementation details up to you, only defining the expected semantics or the set of available operations that an ADT must support. As a result, you can often represent one abstract data type using a few alternative data structures, which are concrete implementations of the same conceptual approach to organizing data.

Programming languages usually provide a few data structures in the form of built-in data types as a convenience so that you don’t have to implement them yourself. This means you can focus on solving more abstract problems instead of starting from scratch every time. For example, the Python dict data type is a hash table data structure that implements the dictionary abstract data type.

To reiterate the meaning of these terms, abstract data types define the desired semantics, data structures implement them, and data types represent data structures in programming languages as built-in syntactic constructs.

Some of the most common examples of abstract data types include these:

In some cases, you can build more specific kinds of abstract data types on top of existing ADTs by incorporating additional constraints. For instance, you can build a stack by modifying the queue or the other way around.

As you can see, the list of ADTs doesn’t include arrays. That’s because the array is a specific data structure representing the list abstract data type. The list ADT dictates what operations the array must support and which behaviors it should exhibit. If you’ve worked with the Python list, then you should already have a pretty good idea of what the list in computer science is all about.

Note: Don’t confuse the list abstract data type in computer science with the list data type in Python, which represents the former. Similarly, it’s easy to mistake the theoretical array data structure for a specific array data type, which many programming languages provide as a convenient primitive type built into their syntax.

The list abstract data type is a linear collection of values forming an ordered sequence of elements. These elements follow a specific arrangement, meaning that each element has a position relative to the others, identified by a numeric index that usually starts at zero. The list has a variable but finite length. It may or may not contain values of different types, as well as duplicates.

The interface of the list abstract data type resembles Python’s list, typically including the following operations:

List ADT Python’s list Get an element by an index fruits[0] Set an element at a given index fruits[0] = "banana" Insert an element at a given index fruits.insert(0, "banana") Delete an element by an index fruits.pop(0), del fruits[0] Delete an element by a value fruits.remove("banana") Delete all elements fruits.clear() Find the index of a given element fruits.index("banana") Append an element at the right end fruits.append("banana") Merge with another list fruits.extend(veggies), fruits + veggies Sort elements fruits.sort() Get the number of elements len(fruits) Iterate over the elements iter(fruits) Check if an element is present "banana" in fruits

Now that you understand where the array data structure fits into the bigger picture, it’s time to take a closer look at it.

Read the full article at https://realpython.com/python-array/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Categories: FLOSS Project Planets

Tryton News: Newsletter January 2024

Mon, 2024-01-01 02:00

During the last month we mainly focused on fixing bugs, adjusting how things work, improving performance and adding new features.

Changes for the User Sales, Purchases and Projects

Related shipments and moves are now reset and cancelled when a purchase request is cancelled.

When creating a sale from an opportunity the sale now has default addresses and payment terms, if they are not defined on the opportunity.

The blanket agreement record names now contain the reference and will fallback to the id if there isn’t a number or a reference available, like we do for sales and purchases.

We added some UTM parameters to emails sent by the marketing modules. The following parameters were added:

  • utm_campaign
  • utm_medium
  • utm_source

The create purchase wizard on purchase requests now opens the newly created purchases.

Since Tryton defaults to the most used currency for new purchases, we’ve now updated it to also do the same for purchase requests.

Accounting, Invoicing and Payments

When using sale advance payments Tryton no longer sets the invoice date if the advance payment condition has no invoice delay. If you’d like the invoice date to default to today’s date, set a delay of 0.

Stock, Production and Shipments

On an unaltered system modification of product locations is now restricted to the Stock Administrator access group.

The progress of a move is now rounded to four digits.

Tryton now only checks if a lot is required when a move is going to the done state.

The stock location code is now included in the record name of a location.

We added a confirmation dialogue to the cancel- buttons on shipments.

When moving consumable products the default to-location is now preset with the default product location.

User Interface

The URL button is now hidden, when a URL field is empty, as disabling it did not prevent the user from clicking on it.

Each button in a list-view is now rendered read-only when the appropriate record is also read-only.

We improved the behaviour of button clicks. Now clicking rapidly on a button only launches the action once. This same behaviour has also been implemented for widget buttons.

Now labels are aligned to the start of the line on extra small screens.

On small screens we now hide the search widget on list views. A toolbar button shows the search widget on demand.


The workflow graphs for models no longer overlap and incorrectly share states.

More (click for more details) Documentation

We reworked parts of the Tryton documentation.

In validation error messages the record name is now prefixed with the word “record” in order to make the message clearer.

New Releases

We released bug fixes for the currently maintained long term support series
7.0 and 6.0, and for the penultimate series 6.8.

Changes for the System Administrator

For the Tryton desktop client we now support the arm64 darwin architecture allowing it to be built on Apple Silicon.

Changes for Implementers and Developers

The order of keys is now retained when changing a fields.Dictionary through an on_change method.

For selection and multiselection fields we now use the string version of the value in error messages.

Authors: @dave @pokoli @udono

1 post - 1 participant

Read full topic

Categories: FLOSS Project Planets

Doug Hellmann: imapautofiler 1.14.0 - sort-by-year action

Sun, 2023-12-31 10:52
What’s new in 1.14.0? add python 3.12 to test matrix add sort-by-year action
Categories: FLOSS Project Planets

Seth Michael Larson: 2023 year in review

Sat, 2023-12-30 19:00
2023 year in review AboutBlogNewsletterLinks 2023 year in review

Published 2023-12-31 by Seth Larson
Reading time: minutes

2023 was a great year! So much happened, but a few things in particular stood out to be when putting together this post.

I got married to my wife, Trina after 9 years of being together. We met in college and fell in love with each others' passion for adventure, food, and life. Our ceremonies included both of our cultures and we had family and friends from all over the globe together to celebrate with us.


"Vu Quy" or "tea ceremony" with family. Photo credit: Summer Street Photography

We were married near where we first met on the University of Minnesota east bank campus and the Mississippi river. The photo below is my favorite of the whole day:


Stone Arch Bridge with the Minneapolis downtown in the background. Photo credit: Summer Street Photography

I traveled to many new places this year and got to see friends everywhere I went. This was my first year traveling to New York, Florida, Texas, Rhode Island, and northern Nevada. Places I'm looking forward to exploring in 2024 include Japan, Seattle, and Pittsburgh (see you at PyCon US!)

Elastic had its first Engineering All Hands in-person since COVID to start off 2023 where I got to meet my long-time friend and colleague Quentin Pradet. Quentin and I have been working on open source together for over 5 years and this was our first opportunity to meet in person.


Quentin and I together at Elastic EAH 2023

After 3 great years at Elastic I was hired by the Python Software Foundation to be the Security Developer-in-Residence. I still have days when I think I'm dreaming, I'm so grateful I have the opportunity to work full-time serving a community I love.


Banner included in OpenSSF's announcement of my hiring.

This blog saw a huge burst of activity thanks to my new position where I publish weekly reports on what I've been working on. There were 34 new publications to the blog in 2023 (up from 12 in 2022), of those 24 were related to the Security Developer-in-Residence role.

The top posts by readership for this year were:

If I had to pick a favorite post outside of this list it would be “For You” is not for me discussing my current outlook on internet consumption. Look forward to more posts on the blog, hopefully continuing the trend that I'm on of shorter but more frequent publications.

Outside of software I plan to spend more time playing games (beat Pikmin 4 and Super Mario RPG is in-progress) and learning some hardware-hacking for retro gaming like Gameboy and GameCube modding.

Hope you all had a lovely 2023, looking forward to what we can do in 2024! 🥳

Thanks for reading! ♡ Did you find this article helpful and want more content like it? Get notified of new posts by subscribing to the RSS feed or the email newsletter.

This work is licensed under CC BY-SA 4.0

Categories: FLOSS Project Planets

Paolo Melchiorre: My 2023 in review

Sat, 2023-12-30 18:00

The review of my 2023, trying to remember all the things done in this year, in which more than anyone I met many fantastic people and visited new countries.

Categories: FLOSS Project Planets

Zero to Mastery: Python Monthly Newsletter 💻🐍

Sat, 2023-12-30 11:42
49th issue of Andrei Neagoie's must-read monthly Python Newsletter: Python Errors, Tools in Python Land, Architecting Monorepos, and much more. Read the full newsletter to get up-to-date with everything you need to know from last month.
Categories: FLOSS Project Planets

Matt Layman: Python, Markdown, and Tailwind: Best Buds!

Fri, 2023-12-29 19:00
You are rendering content with Python and want to show some Markdown, but you style your pages with Tailwind. With Tailwind’s built-in reset, how can you style the tags of your rendered HTML that come from Markdown? This article shows how that can be done. Specifically, I am assuming that you are working with Python’s Markdown package. I recently worked on a project where I needed to render some Markdown descriptions.
Categories: FLOSS Project Planets

PyPy: PyPy has moved to Git, GitHub

Fri, 2023-12-29 09:19

PyPy has moved its canonical repo and issue tracker from https://foss.heptapod.net/pypy/pypy to https://github.com/pypy/pypy. Obviously, this means development will now be tracked in Git rather than Mercurial.

Motivation

We still feel Mercurial is a better version control system. The named branch model and user interface are superior. But

  • foss.heptapod.net is not well indexed in google/bing/duckduckgo search, so people find it harder to search for issues in the project.

  • Since Heptapod has tightened its spam control, we get reports that users create issues only to have them flagged as spam.

  • Open Source has become synonymous with GitHub, and we are too small to change that.

  • Much of the current development comes as a reaction to fixing issues. Tracking interlocking issues is easier if all the code is on the same platform.

  • The FAQ presents two arguments against the move. Github notes solves much of point (1): the difficulty of discovering provenance of commits, although not entirely. But the main problem is point (2), it turns out that not moving to GitHub is an impediment to contribution and issue reporting.

  • People who wish to continue to use Mercurial can use the same method below to push to GitHub.

  • GitHub is more resource rich than foss.heptapod.net. We could add CI jobs to replace some of our aging buildbot infrastructure.

Method

The migration required two parts: migrating the code and then migrating the issues and merge requests.

Code migration 1: code and notes

I used a fork of git-remote-hg to create a local Git repo with all the changesets. Then I wanted to add a Git note to each commit with the branch it came from. So I prepared a file with two columns: the Git commit hash, and the corresponding branch from Mercurial. Mercurial can describe each commit in two ways: either the commit hash or by a number index. I used hg log to convert an index i to the Mercurial hash, and then git-hg-helper from git-remote-hg to convert the Mercurial hash to a Git hash:

$(cd pypy-git; git-hg-helper git-rev $(cd ../pypy-hg; hg log -r $i -T"{node}\n"))

Then I used hg log again to print the Mercurial branch for the index i:

$(cd pypy-hg; hg log -r $i -T'{branch}\n')

Putting these two together, I could loop over all the commits by their numerical index to prepare the file. Then I iterated over each line in the file, and added the Git note. Since the git note add command works on the current HEAD, I needed to checkout each commit in turn and then add the note:

git checkout -q <hash> && git notes --ref refs/notes/branch add -m branch:<branch>

I could then use git push --all to push to GitHub.

Code migration 2: prepare the branches

PyPy has almost 500 open branches. The code migration created all the branch HEADs, but git push --all did not push them. I needed to check them out and push each one. So I created a file with all the branch names

cd pypy-hg; hg branches | cut -f1 -d" " > branches.txt

and then push each one to the GitHub repo

while read branch; do git checkout branches/$branch && git push origin branches/$branch; done < branches.txt

Note that the branches were named branches/XXX by the migration, not branch/XXX. This confuses the merge request migration, more about that later.

Issue and merge request migration

I used the solution from node-gitlab-2-github which worked almost perfectly. It is important to do the conversion on a private repo otherwise every mention of a sucessfully mapped user name notifies the user about the transfer. This can be quite annoying for a repo the size of PyPy with 600 merge requests and over 4000 issues. Issues transfered without a problem: the script properly retained the issue numbers. However the script does not convert the Mercurial hashes to Git hashes, so the bare hashes in comments show up without a link to the commit. Merge requests are more of a problem:

  • The Mercurial named branch "disappears" once it is merged, so a merge request to a merged branch does not find the target branch name in Git. The conversion creates an issue instead with the label gitlab merge request.
  • For some reason, the branches created by git-remote-hg are called branches/XXX and not branch/XXX as expected by GitLab. This messes up the merge request/PR conversion. For some of the branches (open PRs and main target branches) I manually created additional branches without the es. The net result is that open merge requests became open PRs, merged merge requests became issues, and closed-not-merged merge requests were not migrated.
Layered conversions

PyPy already migrated once from Bitbucket to Heptapod. Many of the issues reflect the multiple transitions: they have lines like "Created originally on Bitbucket by XXX" from the first transition, and an additional line "In Heptapod" from this transition.

Credits

We would like to express our gratitude to the Octobus team who support Heptapod. The transition from Bitbucket was quite an effort, and they have generously hosted our developement since then. We wish them all the best, and still believe that Mercurial should have "won".

Next steps

While the repo at GitHub is live, there are still a few more things we need to do:

  • Documentation needs an update for the new repo and the build automation from readthedocs must be adjusted.
  • The wiki should be copied from Heptapod.
  • buildbot.pypy.org should also look at the new repo. I hope the code is up to the task of interacting with a Git repo.
  • speed.pypy.org tracks changes, it too needs to reference the new location
  • To keep tracking branches with Git notes on new commits, I activated a github action by Julian to add a Git branch note to each commit. Please see the README there for directions on using Git notes.
  • Some of the merge requests were not migrated. If someone wants to, they could migrate those once they figure out the branch naming problems.

Additionally, now is the time for all of you to prove the move is worthwhile:

  • Star the repo, let others know how to find it,
  • Help fix some of the open issues or file new ones,
  • Take advantage of the more familiar workflow to get involved in the project,
  • Suggest ways to improve the migration: are there things I missed or could have done better?
How will development change?

Heptapod did not allow personal forks, so we were generous with a commit bit to the main repo. Additionally, we (well, me) have been using a commit-directly-to-main workflow. We will now be adopting a more structured workflow. Please fork the repo and submit a pull request for any changes. We can now add some pre-merge CI to check that the PR at least passes the first stage of translation. The live and active branches will be:

  • main: what was "default" in Mercurial, it is the Python2.7 interpreter and the base of the RPython interpreter,
  • py3.9: the Python3.9 interpreter, which also includes all RPython changes from main. This is exactly like on Mercurial, and
  • py3.10: the Python3.10 interpreter, which also includes all RPython changes from main and all bugfixes from py3.9. This is exactly like on Mercurial.
Working between the repos Finding commits

If you want to figure out how a Mercurial commit relates to a Git commit, you can use git-hg-helper. You run it in the Git repo. It takes the full long hash from one repo and gives you the corresponding hash of the other repo:

$ git-hg-helper git-rev d64027c4c2b903403ceeef2c301f5132454491df 4527e62ad94b0e940a5b0f9f20d29428672f93f7 $ git-hg-helper hg-rev 4527e62ad94b0e940a5b0f9f20d29428672f93f7 d64027c4c2b903403ceeef2c301f5132454491df Finding branches

Branches migrated from Mercurial will have a branches prefix, not branch. While GitLab uses branch for its prefix, the git-remote-hg script uses branches. New work should be in a PR targeting main, py3.9 or py3.10.

Thanks for helping to make PyPy better.

Matti

Categories: FLOSS Project Planets

Real Python: The Real Python Podcast – Episode #185: 2023 Real Python Tutorial &amp; Video Course Wrap-Up

Fri, 2023-12-29 07:00

Three members of the Real Python team are joining us this week: Kate Finegan, Tappan Moore, and Philipp Acsany. We wanted to share a year-end wrap-up with tutorials, step-by-step projects, code conversations, and video courses that showcase what our team created this year.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Categories: FLOSS Project Planets

The Three of Wands: attrs iv: Zero-overhead Frozen attrs Classes

Fri, 2023-12-29 05:00

Here&aposs a quick and easy one.

attrs supports frozen classes. Frozen classes are cool for a multitude of reasons, but they&aposre a tiny bit slower to instantiate compared to non-frozen classes, because they need to perform some additional checks and avoiding these checks in the __init__ uses a tiny bit of time. (We&aposre talking slotted classes here; which are awesome and the default in attrs nowadays.)

But there&aposs a way to avoid this overhead and make frozen classes be the exact same speed as ordinary classes. The only caveat is: you have to use type-checking.

Create a file somewhere in your code base and put this in it:

from functools import partial from typing import TYPE_CHECKING if TYPE_CHECKING: from attrs import frozen else: from attrs import define frozen = partial(define, unsafe_hash=True)

Now import frozen from this module and use it just like you&aposd use attrs.define or attrs.frozen. You&aposre done!

This technique should also work for dataclasses, with a slight adjustment left as an exercize to the reader.

The eagle-eyed reader might notice that we&aposre actually bamboozling the type-checker: your classes won&apost actually be frozen at runtime. The kicker is: they don&apost actually need to be.

As long as you&aposre running one on your codebase, the typechecker is the actual thing that&aposll prevent you from mutating your instances. The unsafe_hash=True will make the classes hashable, and it&aposs only unsafe if you mutate them after construction, which you won&apost. I guess you&aposll have to be careful when using a REPL or a different context where a typechecker might not hold sway, but I think that&aposs not too big of an issue.

If you&aposre still unconvinced, I&aposll leave you with two final thoughts: the memory in your computer is, ultimately, mutable. What makes immutable data structures in other languages immutable is just the amount of hoops you have to jump through to apply a mutation. This also demonstrates a basic technique statically-compiled languages use to be fast: move part of the work out of runtime into a separate, pre-runtime step. Which is exactly what we&aposve done here.

Happy New Year!

Categories: FLOSS Project Planets

Talk Python to Me: #443: Python Bytes Crossover 2023

Fri, 2023-12-29 03:00
Special crossover episode of Python Bytes to wrap up 2023. Topics include: <br/> <br/> <ol> <ul><strong>Michael #1</strong>: <a href="https://hatch.pypa.io/latest/blog/2023/12/11/hatch-v180/">Hatch v1.8</a></ul> <ul><strong>Brian #2:</strong> <a href="https://svcs.hynek.me/en/stable/">svcs : A Flexible Service Locator for Python</a></ul> <ul><strong>Michael #3:</strong> <a href="https://discuss.python.org/t/steering-council-election-results-2024-term/40851">Steering Council 2024 Term Election Results</a></ul> <ul><strong>Brian #4:</strong> <a href="https://typethepipe.com/post/python-protocols-when-to-use">Python protocols. When to use them in your projects to abstract and decoupling</a></ul> <ul>Extras</ul><ul>Joke: <strong>Joke:</strong> <a href="https://mastodon.social/@tveskov/111289358585305218">The dream is dead?</a></ul></ol><br/> <strong>--- Episode sponsors ---</strong><br/> <a href='https://talkpython.fm/posit'>Posit</a><br> <a href='https://talkpython.fm/training'>Talk Python Training</a>
Categories: FLOSS Project Planets

Go Deh: How not to check for a key in a dictionary.

Thu, 2023-12-28 14:03

 I skim the Linkedin Python group and sometimes comment.

A few days there was a poll asking for the way to check if a key is in a Python dictionary and as I write, more than half of the 900+ respondents have chosen dict.get rather than key in dict, which is the correct answer!

When pushed, it seems they are relying on dict.get returning None if the key is not in the dict, but fail to see tha if the key being tested is in the dict, but has a value of None then their test fails. 

Here's some code:

# %%mydict1 = {'a': 1, 'b': 2}print(mydict1.get('c'))  # -> None
# %%
def key_in_dict1(key_, dict_: dict) -> bool:    "FAULTY IMPLEMENTATION"    return dict_.get(key_) != None
# This worksmydict1 = {'a': 1, 'b': 2}print(key_in_dict1('a', mydict1))  # -> Trueprint(key_in_dict1('x', mydict1))  # -> False# %%
# But adding a key of x with value None gives# the wrong resultmydict2 = {'a': 1, 'b': 2, 'x': None}print(key_in_dict1('x', mydict2))  # -> False
# %%
# The correct way is to use 'in'def key_in_dict2(key_, dict_: dict) -> bool:    "Pythonic IMPLEMENTATION"    return key_ in dict_
# Tests:print(key_in_dict2('a', mydict1))  # -> Trueprint(key_in_dict2('x', mydict1))  # -> False
# %%
# And now for keys with None as a value:print(key_in_dict2('x', mydict2))  # -> True

Ugly, code-golf solution using dict.get()You can alter the default value returned from dict.get if a key is not found. This allows one to use two calls of dict.get with different defaults  to be used to give a correct solution:
def key_in_dict3(key_, dict_: dict) -> bool:    "CODE-GOLF IMPLEMENTATION USING GETs"    default1 = None    default2 = ""    return default1 != default2 \        and ( dict_.get(key_, default1) != default1             or dict_.get(key_, default2) != default2)
# Tests:print(key_in_dict3('a', mydict1))  # -> Trueprint(key_in_dict3('x', mydict1))  # -> Falseprint(key_in_dict3('x', mydict2))  # -> True


Of course, don't use  key_in_dict3, use version 2, i.e. key in dict.
END.
Categories: FLOSS Project Planets

TechBeamers Python: Top 30 Data Engineer Interview Questions with Answers

Thu, 2023-12-28 07:47

If you are planning a job in data engineering, then you should be well prepared for it. We have identified 30 data engineer interview questions that can help in your endeavor. During the interview, you can be asked questions from different related areas. So, we tried to cover these in this tutorial. 30+ Data Engineer […]

The post Top 30 Data Engineer Interview Questions with Answers appeared first on TechBeamers.

Categories: FLOSS Project Planets

Seth Michael Larson: Security Developer-in-Residence Weekly Report #23

Wed, 2023-12-27 19:00
Security Developer-in-Residence Weekly Report #23 AboutBlogNewsletterLinks Security Developer-in-Residence Weekly Report #23

Published 2023-12-28 by Seth Larson
Reading time: minutes

This critical role would not be possible without funding from the OpenSSF Alpha-Omega project. Massive thank-you to Alpha-Omega for investing in the security of the Python ecosystem!

It was a short week this week due to Christmas and New Years holidays in the United States.

This week I spent time learning about the current state of proposals for solving the "index trust" problem for the Python Package Index. There are a few different proposals that have happened in the past being PEP 458 and RSTUF. This learning was to better review a draft proposal for publish provenance (stay tuned!) that also contained some mentions of the index trust problem. Some outcomes we're looking for when solving the index trust problem:

  • Reduce absolute trust in the Python Package Index.
  • Allow for integrity recovery of artifacts hosted on the index in the case of compromise.
  • Allow installers to detect tampering of existing artifacts in a trust-on-first-use (TOFU) fashion even when dependencies aren't pinned using hashes.

Back to publish provenance, I learned more about how NPM is handling their own provenance and made some recommendations about future provenance attestations that might come after publish provenance (like build provenance with the builder workflow ID, git commit, and repo being verifiable!). After thinking about build provenance and how to keep the choice of which builder workflows are "trusted" in the hands of consumers instead of PyPI I created the following quick mockup:

@import url("https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.5.0/css/all.min.css");'PyPIOIDC ProviderBuilder Workflow (Reusable Workflow)Publishing Workflow (User-controlled)PyPIOIDC ProviderBuilder Workflow (Reusable Workflow)Publishing Workflow (User-controlled)#graph-div{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}#graph-div .error-icon{fill:#552222;}#graph-div .error-text{fill:#552222;stroke:#552222;}#graph-div .edge-thickness-normal{stroke-width:2px;}#graph-div .edge-thickness-thick{stroke-width:3.5px;}#graph-div .edge-pattern-solid{stroke-dasharray:0;}#graph-div .edge-pattern-dashed{stroke-dasharray:3;}#graph-div .edge-pattern-dotted{stroke-dasharray:2;}#graph-div .marker{fill:#333333;stroke:#333333;}#graph-div .marker.cross{stroke:#333333;}#graph-div svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#graph-div .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#graph-div text.actor>tspan{fill:black;stroke:none;}#graph-div .actor-line{stroke:grey;}#graph-div .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#graph-div .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#graph-div #arrowhead path{fill:#333;stroke:#333;}#graph-div .sequenceNumber{fill:white;}#graph-div #sequencenumber{fill:#333;}#graph-div #crosshead path{fill:#333;stroke:#333;}#graph-div .messageText{fill:#333;stroke:none;}#graph-div .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#graph-div .labelText,#graph-div .labelText>tspan{fill:black;stroke:none;}#graph-div .loopText,#graph-div .loopText>tspan{fill:black;stroke:none;}#graph-div .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#graph-div .note{stroke:#aaaa33;fill:#fff5ad;}#graph-div .noteText,#graph-div .noteText>tspan{fill:black;stroke:none;}#graph-div .activation0{fill:#f4f4f4;stroke:#666;}#graph-div .activation1{fill:#f4f4f4;stroke:#666;}#graph-div .activation2{fill:#f4f4f4;stroke:#666;}#graph-div .actorPopupMenu{position:absolute;}#graph-div .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#graph-div .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#graph-div .actor-man circle,#graph-div line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#graph-div :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;}loop[For each distribution ...]Call Builder workflow with git repo, commit SHA parametersProvide OIDC token for signing provenanceBuild Python distributionsSubmit name and digest of distribution and build provenanceVerify provenance is validStore build provenance document awaiting distributionHTTP 200 OKReturn distributions to Publishing WorkflowUpload distributions to PyPIAssociate build provenance with distributionsMake build provenance available programmatically and in the UI
This diagram doesn't represent a concrete proposal.

The only standards that would need agreeing on would be the upload API, the build provenance statement itself, and which parameters have special behavior/meaning for build provenance parameters (ie git repository, git commit SHA). This architecture would allow arbitrary build workflows provenance statements to be submitted to PyPI and verified.

Build provenance provides different guarantees to publish provenance, publish provenance only proves that a given artifact was at one point in the "hands" of the workflow but can't assert where the artifact actually came from without deeper inspection of the workflow. In a way, publish provenance is only making public and verifiable the configuration of a projects' Trusted Publisher at the time of release.

Build provenance from an isolated and trusted build workflow can assert that the artifact was built using a known set of steps and dependencies. It's also likely that there will be far fewer build workflows than projects, meaning that the list of trusted build workflows will be more surmountable to review compared to every publishing workflow of every project and dependency that would be required to review if relying on publish provenance for build integrity.

There's lots of cool stuff coming in this space next year, look forward to more from me and others.

Other items
  • Submitted a pull request for adding the new Software Bill-of-Materials Devguide URL to validation errors.
  • Triaged and responded to multiple security reports submitted to the Python Security Response Team at security@python.org.

That's all for this year! 👋 If you're interested in more you can read last week's report.

Thanks for reading! ♡ Did you find this article helpful and want more content like it? Get notified of new posts by subscribing to the RSS feed or the email newsletter.

This work is licensed under CC BY-SA 4.0

Categories: FLOSS Project Planets

Ned Batchelder: Coverage.py with sys.monitoring

Wed, 2023-12-27 18:03

New in Python 3.12 is sys.monitoring, a lighter-weight way to monitor the execution of Python programs. Coverage.py 7.4.0 now can optionally use sys.monitoring instead of sys.settrace, the facility that has underpinned coverage.py for nearly two decades. This is a big change, both in Python and in coverage.py. It would be great if you could try it out and provide some feedback.

Using sys.monitoring should reduce the overhead of coverage measurement, often lower than 5%, but of course your timings might be different. One of the things I would like to know is what your real-world speed improvements are like.

Because the support is still a bit experimental, you need to define an environment variable to use it: COVERAGE_CORE=sysmon. Eventually, sys.monitoring will be automatically used where possible, but for now you need to explicitly request it.

Some things won’t work with sys.monitoring: plugins and dynamic contexts aren’t yet supported, though eventually they will be. Execution will be faster for line coverage, but not yet for branch coverage. Let me know how it works for you.

This has been in the works since at least March. I hope I haven’t forgotten something silly in getting it out the door.

Categories: FLOSS Project Planets

Brett Cannon: My proof-of-concept record type

Wed, 2023-12-27 14:46

Back in June, I proposed a struct syntax for Python. I shared the post on Mastodon and got some feedback. Afterwards I thought about what I heard and talked it over with some folks. I&aposve now coded up a proof-of-concept to share to get some more feedback from people to gauge whether people in general like this idea.

And so I created the record-type project on PyPI to share a proof-of-concept of what I think a record type could look like if one was ever added to Python. I shared this on discuss.python.org and the feedback was generally positive, so now I&aposm seeking wider feedback via this blog post. To help show what the record-type does, here&aposs the opening example of the dataclasses documentation converted to use record-type:

from records import record @record def InventoryItem(name: str, price: float, *, quantity: int = 0): """Class for keeping track of an item in inventory."""

An example of using records.record

As listed in the README of the project&aposs repository, that decorator creates a class with:

  • __slots__ for performance
  • __match_args__ for pattern matching
  • __annotations__ for runtime type annotations
  • __eq__() for equality based on structure (via __slots__), not inheritance
  • __hash__() for hashing
  • __repr__() which is suitable for eval()
  • Immutability

The goal of this design is:

  • Create a simple data type that&aposs easy to explain to beginners
  • Creating the data type itself should be fast (i.e. no concerns over importing a module with a record type)
  • Type annotations are supported, but not required
  • Instances are immutable to make them (potentially) hashable
  • Support Python&aposs entire parameter definition syntax idiomatically for instance instantiation
  • Support structural typing as much as possible (e.g., equality based on object "shape" instead of inheritance)

Now, in all situations where you try to do something that simplifies classes leads to a comparison with dataclasses. Here&aposs the same example, but with dataclasses:

from dataclasses import dataclass, KW_ONLY @dataclass(frozen=True, slots=True) class InventoryItem: """Class for keeping track of an item in inventory.""" name: str price: float _: KW_ONLY quantity: int = 0

The same example using dataclasses.dataclass

Is that worse than the example using record-type? Is any of this compelling enough to turn what record-type proposes into actual syntax? I&aposve had some ask for method support, but I personally like the simplicity of leaning into a data-only approach (see my struct syntax post for more of an explanation). I personally like the structural equality, but I suspect some would be willing to give it up if performance could be improved for equality comparisons.

Anyway, based on the response I may write a PEP to see if there&aposs enough traction to add syntax for this (replies on Mastodon are probably the easiest way to express an opinion).

Categories: FLOSS Project Planets

Pages