FLOSS Project Planets

New article by Richard Stallman: https://www.gnu.org/philosophy/the-moral-and-the-legal.html

Over the past six months, we’ve met hundreds of developers that are using Python to build AI chat apps for their own knowledge domains, using the RAG (Retrieval Augmented Generation) approach to send chunks of knowledge to an LLM model along with the user question.

We’ve also heard from many developers that they’d like to learn how to build their own RAG chat apps, but they don’t know where to start. So we’re hosting a virtual hackathon to help you learn how to build your own RAG chat app with Python!

.

From January 29th to February 12th, we’ll host live streams showing you how to build on our most popular RAG chat sample repository, while also explaining the core concepts underlying all modern RAG chat apps. Live stream topics will include vector search, access control, GPT-4 with vision. We’re hoping to get developers from all over the world involved, so we’ll also have live streams in Spanish, Portuguese, and Chinese. There will be prizes for the best chat apps, and even a prize for our most helpful community member.

To learn more, visit the AI Chat App Hack page, and follow the steps there to register and meet the community. Hope to see you there!

More RAG resources for Python developers

If you’re interested in learning more about RAG chat apps but can’t join the hack, here are some resources to get you started:

• Tutorial: Get started with the Python enterprise chat sample using RAG
• GitHub Universe: Quickly build and deploy OpenAI apps on Azure, infused with your own data
• Azure AI resources for Python developers
• Using Llamaindex with Azure AI Search
• AI Discord community

The post Join us for AI Chat App Hack from Jan. 29 – Feb.12 appeared first on Python.

Do you use NumPy, Pandas, or scikit-learn and want to get faster results? Nvidia has created GPU-based replacements for each of these with the shared promise of extra speed.

For example, if you visit the front page of NVidia’s RAPIDS project, you’ll see benchmarks showing cuDF, a GPU-based Pandas replacement, is 15× to 80× faster than Pandas!

Unfortunately, while those speed-ups are impressive, they are also misleading. GPU-based libraries might be the answer to your performance problems… or they might be an an unnecessary and expensive distraction.

Read more...

Defending against the PyTorch supply chain attack PoC About • Blog • Newsletter • Links Defending against the PyTorch supply chain attack PoC

Published 2024-01-17 by Seth Larson
Reading time: minutes

This critical role would not be possible without funding from the OpenSSF Alpha-Omega project. Massive thank-you to Alpha-Omega for investing in the security of the Python ecosystem!

Last week there which a publication into a proof-of-concept supply chain attack against PyTorch using persistence in self-hosted GitHub runners, capturing tokens from triggerable jobs as a third-party contributor, and modifying workflows. This report was #1 on Hacker News for most of Sunday. In the comments of this publication there was a lot of discussion and folks questioning "how do you defend from this type of attack"?

Luckily for open source users, there are already techniques that can be used today to mitigate the downstream impact of a compromised dependency:

• Using a lock file with pinned hashes like pip with --require-hashes, poetry.lock, or Pipfile.lock.
• Reviewing diffs between currently pinned and new candidate releases. The diff must be of the installed artifacts, not using git tags or source repository information. Tools like diffoscope are useful for diffing wheel files which are actually zip files in disguise.
• For larger organizations the cost of manual review can be amortized by mirroring PyPI and only updating dependencies that have been manually reviewed.
• Binary or compiled dependencies can be compiled from source to ensure malicious code isn't hidden from human inspection.

These are tried-and-true methods to protect yourself and ensure dependencies aren't compromised regardless of what happens upstream. Obviously the suggestions above take time and effort to implement. Generally there's desire from me and others to make the above steps easier for consumers like exposing build provenance for easier reviewing of source code or by improving the overall safety of PyPI content using malware scanning and reporting.

Part of my plans for 2024 is to create guidance for Python open source consumers and maintainers for how to safely use packaging tools both from the perspective of supply chain integrity but also for vulnerabilities, builds, etc. So stay tuned for that!

CPython Software Bill-of-Materials update

Last week I published a draft for CPython's SBOM document specifically for the source tarballs in order to solicit feedback from consumers of SBOMs and developers of SBOM tooling. I received great feedback from Adolfo Garcia Veytia and Ritesh Noronha including the following points:

• Strip version information from the fileName attribute
• The top-level CPython component had no relationships to non-file components, should have DEPENDS_ON relationships to all its dependent packages.
• Fix the formatting of the "Tool: " name and version. Correct format is {name}-{version}.
• Use the fileName attribute on the CPython package instead of using a separate file component for the tarball containing CPython source code.
• Include an email address for all "Person" identities.
• Guidance on alternatives to the documentNamespace field.

After applying this feedback we now have an SBOM which meets NTIA's Minimum Elements of an SBOM and scores 9.6 out of 10 for the SBOM Quality Score.

Next I'm working on the infrastructure for actually generating and making the SBOM available for consumers:

• Created a PR for generating the draft SBOM. Next need to hook into the actual release process and opportunistically generate an SBOM.
• Applied changes according to feedback from SBOM reviewers.
• Created a PR for enabling hosting an SBOM artifact on https://python.org/downloads

Other items

• Reviewed PEP 740 proposal for arbitrary attestation mechanism for PyPI artifacts.
• Triaged multiple reports to the Python Security Response Team.

That's all for this week! 👋 If you're interested in more you can read last week's report.

Thanks for reading! ♡ Did you find this article helpful and want more content like it? Get notified of new posts by subscribing to the RSS feed or the email newsletter.

This work is licensed under CC BY-SA 4.0

We are excited to announce the creation of a European project December 1st 2023, which will run for the next 36 months. This Next Generation Internet pilot named "NGI TALER" is operated by a consortium of 11 partners from 8 European countries with the mandate to roll out an innovative electronic payment system for the greater benefit of European citizens, merchants, and banks. This payment system is different from current online payment methods, like credit cards or bank transfers, in that it offers privacy for the buyer: neither merchants nor banks can trace or link the payments. It is also a no-risk payment option for the merchant as there is no equivalent of fake or stolen credit cards, as payments are cleared and confirmed instantly. The payment system is socially, ecologically and fiscally responsible: it is not a new currency, there is no energy-consuming proof-of-work or proof-of-stake method and clearing is processed much faster than payments by credit cards. NGI TA [...]

Disclaimer: I am not one of KDE's masterminds or spokespersons. I am a mere bystander with few unimportant commits. I follow KDE's ecosystem and other developments in the free software world. In the following, I share some thoughts and my personal opinion.

Talks about new programming languagesAfter 30 years of C code, the Linux kernel opens itself to a second high-level language: Rust. Since fall of 2022 the kernel mainly gained infrastructure work. Some experiments show promising results like a Rust-based network driver or a scheduler.Recently, Git developers started to discuss how to allow Rust code in our beloved version control system. Far from having reached a consensus, its media coverage and heated discussions in forums show how interested the public is in this topic.Other projects try to replace established software by rewritten from scratch Rust ones: uutils coreutils, sudo-rs, librsvg, Rustls. Heck, Rewrite it it Rust (RiiR) has become a meme.
We already have a new programming language!KDE is close to its 6th Megarelease, with one major change being based on Qt 6. Qt 6 requires C++17 which -- as of today -- is perceived as modern C++ and is a leap compared to C++11. It is possible to write modern software with C++17. Still, additional tools like C++ Core Guidelines or Cppcheck are advised to keep the number of preventable bugs low.Most of the projects mentioned in the introduction are using C. This inflicts more pain to the developers and thus using Rust is more attractive. For sure, a fair portion of RiiR arguments do not apply to KDE's C++ code base.Problems with C++ remainC++ cannot adapt to modern ways like including a borrow checker or a less complicated syntax, as this would break compatibility. As much as C++ improved as a language, its compilers, and its ecosystem, it is not enough to be considered a good choice for new projects. NIST and NSA advice to move away from C++.Other problems like complicated tooling with variations on different platforms (build systems, compiler, linker, debugger, dependency management), mixed-in C-style code, difficult to parse C++ code, cannot be solved.I fear that in a not to distant future, C++ might be perceived as an outdated choice to learn and people might less likely consider to join KDE as contributors.
What can be done?In the past, GNOME adopted Vala as a new language to solve the short-comings of C. Vala seems to be dead. Going with Rust did not lead to a project-wide adoption.Some people are working on Qt bindings for Rust, e.g., CXX-Qt from KDAB. I am not sure if Qt itself is working on something similar. At least there is no go-to binding.
Beside the hot topic Rust, two big players invest in ways to have good interoperability with existing code bases and a modern language: Cpp2 / cppfront and Carbon.Cpp2 is a new language from Herb Sutter, who chairs the C++ working group. The idea is to have a transpiler cppfront producing modern C++ code. Cpp2 is not backward compatible to C++ and thus not limited in introducing new ways or removing existing parts. Cpp2 promises to integrate seamlessly in existing C++ code bases as it is compiled into C++ code.Carbon is a project by Google developers and follows a different approach. It aims to provide a new language that can use all C++ features in interfaces, even templates with all bells and whistles.Discuss our futureI do not want to whine about C++. I want to start a discussion on how KDE's future might look like. KDE was always driving innovations. We helped CMake to become one of the most important build systems for C++. KDE 4.0 introduced the semantic desktop. KHTML's code base was the nucleus for today's big browsers.Probably we should have this discussion as a BoF at Akademy 2024 or other places where KDE's masterminds and people with a feeling for future trends come together and form/formulate future directions. In the meantime, I start a discourse thread.Personally, I would like to see some push for Cpp2. More important, I want to see that we are actively shaping KDE's future.

Here are my top 10 articles from The DropTimes in 2023. From igniting my Drupal journey with essential resources to exploring the visionary world of Drupal, these articles have shaped my understanding of Drupal's evolution.

#612 – JANUARY 16, 2024
View in Browser »

Exploring Python in Excel

Are you interested in using your Python skills within Excel? Would you like to share a data science project or visualization as a single Office file? This week on the show, we speak with Principal Architect John Lam and Sr. Cloud Developer Advocate Sarah Kaiser from Microsoft about Python in Excel.
REAL PYTHON podcast

Python 3.13 Gets a JIT

This article does a deeper dive into the JIT recently added to the CPython 3.13 pre-release. This JIT is a bit different, it is called a copy-and-patch JIT, and the post explains what that means.
ANTHONY SHAW

NumPy 2 Is Coming: Preventing Breakage, Updating Your Code

NumPy 2 is coming, and it’s backwards incompatible. Learn how to keep your code from breaking, and how to upgrade.
ITAMAR TURNER-TRAURING

Build Invincible Apps With Temporal’s Python SDK

Get an introduction to Temporal’s Python SDK by walking through our easy, free tutorials. Learn how to build Temporal applications using Python, including building a data pipeline Workflow and a subscription Workflow. Get started here →
TEMPORAL sponsor

PSF Says: EU’s Cyber Resilience Act Has Wins for Open Source

PYTHON SOFTWARE FOUNDATION

Articles & Tutorials Learn From 2023’s Most Popular Python Tutorials and Courses

Revisit your favorite Real Python tutorials and video courses from 2023. Explore various topics, from Python basics to web development, machine learning, and effective coding environments. It’s been a busy year of learning, and there’s something for everyone to discover and build upon in 2024.
REAL PYTHON

Python’s Array: Working With Numeric Data Efficiently

In this tutorial, you’ll dive deep into working with numeric arrays in Python, an efficient tool for handling binary data. Along the way, you’ll explore low-level data types exposed by the array module, emulate custom types, and even pass a Python array to C for high-performance processing.
REAL PYTHON

Data Deduplication in Python with RecordLinkage

Duplicate detection is a critical process in data preprocessing, especially when dealing with large datasets. In this tutorial, you will explore data deduplication using Python’s RecordLinkage package, paired with Pandas for data manipulation.
PATRYK SZLAGOWSKI • Shared by Izabela Pawlik

The Curious Case of Pydantic and the 1970s Timestamps

When parsing Unix timestamps, Pydantic guesses whether to interpret them in seconds or milliseconds. While this is certainly convenient and works most of the time, it can drastically (and silently) distort timestamps from a few decades ago.
ARIE BOVENBERG • Shared by Arie Bovenberg

A Critical Supply Chain Attack on PyTorch

This post describes how coders found an exploit in the PyTorch supply chain, leaving the repo and its maintainers up for attack. Full details on what the vulnerability was and how to avoid the problem in your own repos is covered.
JOHN STAWINSKI

Comparing Coroutines, by Example, in Kotlin and Python

This series of 3 articles compares Python and Kotlin, with a focus on coroutines and generators. It compares, through examples, how coroutines are used in both languages, to read files and perform network requests.
MEDIUM.COM • Shared by Carmen Alvarez

Enhance Your Flask Web Project With a Database

Adding a database to your Flask project elevates your web app to the next level. In this tutorial, you’ll learn how to connect your Flask app to a database and how to receive and store posts from users.
REAL PYTHON

SQLALchemy vs Django ORM

If you are working with Django ORM most of the time and then switching to SQLAlchemy, you may face some unexpected behavior. This post describes the most important differences between them.
ALEXEY EVSEEV

Annotating *args and **kwargs in Python

“Typing *args and **kwargs has always been a pain since you couldn’t annotate them precisely before.” This article shows you what your options are when typing function signatures.
REDOWAN DELOWAR

Python Gotcha: Modifying a List While Iterating

Python makes it easy to modify a list while you are iterating through it’s elements. This will bite you. Read on to find out how and what can be done about it.
ANDREW WEGNER

A Deep Dive Into Python’s functools.wraps Decorator

Take a deep dive into Python’s functools.wraps decorator to learn how it maintains metadata in your code. A concise guide to effective decorator use.
JACOB PADILLA

max() is broken

The built-in function max in Python is broken and this article explains why, drawing parallels with other programming and mathematics concepts.
MATHSPP.COM • Shared by Rodrigo Girão Serrão

All PyCon 2023 (US and AU) Talks Sorted by the View Count

A full list of PyCon talks given in the US and Australia which are available on YouTube, and sorted by popularity.
SUBSTACK.COM

Projects & Code PikaPython: Python Interpreter in 4KB of RAM

GITHUB.COM/PIKASTECH

Fontimize: Optimize Fonts to the Glyphs on Your Site

GITHUB.COM/VINTAGEDAVE

instructor: Structured Outputs for LLMS

GITHUB.COM/JXNL

Pint: Units for Python

PYPI.ORG

Events Weekly Real Python Office Hours Q&A (Virtual)

January 17, 2024
REALPYTHON.COM

PyData Bristol Meetup

January 18, 2024
MEETUP.COM

PyLadies Dublin

January 18, 2024
PYLADIES.COM

Chattanooga Python User Group

January 19 to January 20, 2024
MEETUP.COM

IndyPy: Models & AI For Dummies (Hybrid)

January 23, 2024
MEETUP.COM • Shared by Laura Stephens

Happy Pythoning!
This was PyCoder’s Weekly Issue #612.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

At The DropTimes (TDT), we constantly evolve. The formative steps for us were challenging. We might be just out of infancy, but still in childhood. That does not give us any excuse not to be a better version of what we were yesterday. As we align with the build-in-open philosophy, let us record our recent decisions in response to community feedback.

Authored by: Nadiia Nykolaichuk.

Upon entering the new year, many of us have started working towards resolutions or aspiring achievements. In the digital space, Drupal websites, too, can aspire to be on the cutting edge of trends and provide the most compelling online journeys to users.

A classic childhood game is tic-tac-toe, also known as naughts and crosses. It’s simple and enjoyable, and coding a version of it with Python is an exciting project for a budding programmer. Now, adding some artificial intelligence (AI) using Python can make an old favorite even more thrilling.

In this comprehensive tutorial, you’ll construct a flexible game engine. This engine will include an unbeatable computer player that employs the minimax algorithm to play tic-tac-toe flawlessly. Throughout the tutorial, you’ll explore concepts such as immutable class design, generic plug-in architecture, and modern Python coding practices and patterns.

In this video course, you’ll learn how to:

• Develop a reusable Python library containing the tic-tac-toe game engine
• Create a Pythonic code style that accurately models the tic-tac-toe domain
• Implement various artificial players, including one using the powerful minimax algorithm
• Construct a text-based console front end for the game, enabling human players to participate
• Discover effective strategies for optimizing performance

Are you ready to embark on this step-by-step adventure of building an extensible game engine with an unbeatable AI player using the minimax algorithm?

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Pamela Fox is a Python Cloud Developer Advocate at Microsoft. 

Topics include:

• Girl Develop It
• Django Girls
• Girls Who Code
• Teaching a language vs teaching a tool
• What a dev advocate does
• Accessibility (A11y) testing
• Playwright
• axe-core
• Snapshot testing
• pytest plugin authoring
• Flask SQLAlchemy
• Relearning Go

Links from the show:

• Python Bytes 323 with Pamela: AI search wars have begun
• Python Test 199 with Pamela: Is Azure Right for a Side Project?
• gdi: Girl Develop It
• Django Girls
• Girls Who Code
• "Automated accessibility audits" - Pamela Fox (North Bay Python 2023)
• Playwright
• axe-core
• pytest-axe-playwright-snapshot, plugin from Pamela
• pytest-crayons plugin is from a PyCascades talk about building plugins
• pytest-check, yet another plugin
• FlaskSQLAlchemy
• Concurrency is not Parallelism by Rob Pike

The Complete pytest Course

• Level up your testing skills and save time during coding and maintenance.
• Check out courses.pythontest.com

★ Support this podcast on Patreon ★ <p>Pamela Fox is a Python Cloud Developer Advocate at Microsoft. </p><p><br>Topics include:</p><ul><li>Girl Develop It</li><li>Django Girls</li><li>Girls Who Code</li><li>Teaching a language vs teaching a tool</li><li>What a dev advocate does</li><li>Accessibility (A11y) testing</li><li>Playwright</li><li>axe-core</li><li>Snapshot testing</li><li>pytest plugin authoring</li><li>Flask SQLAlchemy</li><li>Relearning Go</li></ul><p>Links from the show:</p><ul><li><a href="https://pythonbytes.fm/episodes/show/323/ai-search-wars-have-begun">Python Bytes 323 with Pamela: AI search wars have begun</a></li><li><a href="https://podcast.pythontest.com/episodes/199-is-azure-right-for-a-side-project">Python Test 199 with Pamela: Is Azure Right for a Side Project?</a></li><li><a href="https://girldevelopit.com">gdi: Girl Develop It</a></li><li><a href="https://djangogirls.org/en/">Django Girls</a></li><li><a href="https://girlswhocode.com">Girls Who Code</a></li><li><a href="https://www.youtube.com/watch?v=J-4Qa6PSomM">"Automated accessibility audits" - Pamela Fox (North Bay Python 2023)</a></li><li><a href="https://playwright.dev">Playwright</a></li><li><a href="https://github.com/dequelabs/axe-core">axe-core</a></li><li><a href="https://github.com/pamelafox/pytest-axe-playwright-snapshot">pytest-axe-playwright-snapshot</a>, plugin from Pamela</li><li><a href="https://www.youtube.com/watch?v=kevcz8NRcQU">pytest-crayons plugin is from a PyCascades talk about building plugins</a></li><li><a href="https://github.com/okken/pytest-check">pytest-check</a>, yet another plugin</li><li><a href="https://flask-sqlalchemy.palletsprojects.com/en/3.1.x/">FlaskSQLAlchemy</a></li><li><a href="https://www.youtube.com/watch?v=oV9rvDllKEg">Concurrency is not Parallelism by Rob Pike</a></li></ul><p><br></p> <br><p><strong>The Complete pytest Course</strong></p><ul><li>Level up your testing skills and save time during coding and maintenance.</li><li>Check out <a href="https://courses.pythontest.com/p/complete-pytest-course">courses.pythontest.com</a></li></ul> <strong> <a href="https://www.patreon.com/PythonPeople" rel="payment" title="★ Support this podcast on Patreon ★">★ Support this podcast on Patreon ★</a> </strong>