Feeds

Season of KDE 2022

Planet KDE - Tue, 2022-01-25 19:00
Background

I am Ayush Singh, a second-year student of the Indian Institute of Technology, Dhanbad, India. My application has been accepted in the Season of KDE 2022. I will be working on writing a Rust wrapper for KConfig KDE Framework. This post describes my journey with KDE and why I submitted this Project for the Season of KDE.

My Introduction to Linux

I was introduced to the world of Linux back in 2016 when I was working on compiling android from source. Like most people, I started with Ubuntu. I didn’t explore Linux much at the time other than building android, and that was it.

However, things changed in 2018 when I got my PC. Since I already knew about Linux, I decided to dual-boot and explore a new operating system. Slowly, I stopped using the Windows partition and removed it altogether in early 2020.

My Introduction to KDE

I was first introduced to KDE when I switched to OpenSUSE Tumbleweed. I didn’t pay much attention to KDE at the time and quickly moved on. After this, I started using standalone window managers (like i3, AwesomeWM) for quite a while. The next time I actively used KDE was in the summer of 2020 when the lockdown began, and my school exams got postponed indefinitely. During that time, I discovered KDE Activities which completely changed my workflow. Since then, I have tried Gnome and standalone windows managers; however, none seem to offer the same level of coherence and flexibility as KDE. Also, did I already mention how much I love KDE Activities?

As for KDE Development, I am somewhat inexperienced. I have also opened Bugs in KDE and two merge requests, but they were pretty minor. Season of KDE will also allow me to get better acquainted with KDE Development.

Past Experience with Qt

I have used Qt Framework to write GUI applications in C++ for my School projects.

  1. In 2019, I made a Calculator.
  2. In 2020, I made a Library Management software.

Both of them used Qt Widgets since I didn’t know about QML at the time.

My Introduction to Rust

I had been interested in Systems Programming in the past. During the lockdown, I came across Rust Language. I quickly got interested in its promise of performance and safety and being more modern than C in general. After reading the Rust Book, I used Rust to create simple projects, like a web crawler and a magic bytes reader. I now understand that, like all programming languages, Rust has its strengths and weaknesses. However, I still love working with Rust.

Trying to use Rust in KDE

In June of 2021, I thought about writing a Web Browser for some reason. I did not want to use the Blink engine. I had previously heard about Servo and decided to use it. While doing that, I started searching for GUI toolkits in Rust since Servo is written in Rust. I wanted to use Qt with Rust, but I quickly found out that using Rust with Qt was more difficult than I would like. At around the same time, Gtk-rs was made official with Gtk4, which seemed to work great. I tried switching to Gnome for a while after that but ultimately failed.

Sadly, I never created that web browser. However, I did come out with a new conviction. I decided to make the missing tooling required to develop KDE/QML applications with Rust without writing any C++.

I started by looking into already existing crates for Qt development with Rust. I found qmetaobject crate, which was closely aligned with how I wanted Rust + Qt development to be. To create Kirigami applications using qmetaobject, I created bindings for the KI18n Localization framework (kconfig crate). It was used with almost all Kirigami applications and thus was a natural starting point. I also created a Rust crate for linking Kirigami Framework to the Rust application. While working on these bindings, I also contributed the qmetaobject upstream since some of the methods and enums I required were missing for qmetaobject. This helped me get familiar with Rust memory management, which I had previously taken for granted.

Applying for Season of KDE

I already knew about the Season of KDE and decided to apply for it. I submitted a proposal about writing “Rust wrapper of KConfig”. Since I was already working on the KI18n wrapper, I had the necessary experience to make this possible.

After some searching in the KDE mailing list, I came across Jos van den Oever, the author of Rust Qt Bindings Generator. He agreed to become my mentor for the Season of KDE. This project might have been delayed if he hadn’t decided to mentor me. He also helped me with this blog and some other KDE Community stuff.

Now, I will cover some information about my Season of KDE Project.

My Season of KDE Project Overview

I will create a wrapper crate for KConfig KDE Framework in Season of KDE. It will allow the usage of KConfig from Rust projects without writing C++. The crate will be dependent on qttypes, which wraps a lot of QML types and makes them available in Rust. While I will mainly be testing it with qmetaobject, I would like to avoid having any hard dependency on it. My goal is to make the bindings safe while having as little overhead as possible.

Finally, I would like to find a way to have something similar to KConfigXT for Rust. This is a bit of a lofty goal, and I’m not entirely sure how to achieve it. However, I think KConfigXT is one of the best features of KConfig, and I don’t believe the wrapper would be complete without it.

The kconfig bindings crate is available here. Feel free to test it out and report Bugs.

Goals

Creating bindings for more languages allows bringing more people to the KDE Community. Currently, Qt only has good support for C++ and Python. Rust promises to provide a safe and performant language with modern conveniences like a good package manager, async support, etc. As such, I think it can be pretty helpful to promote the use of Rust in KDE.

C++ is a great language, so I don’t believe in rewriting everything in Rust. However, we should use it in places where it is better than C++. For example, Rust is excellent for parsers and thus can be used in KDevelop and Kate.

Currently, the barrier for using Rust in KDE is very high, which discourages most people from giving it a shot. This means that most Rust programmers never give KDE much of a chance, and in turn, there has been very little progress with Rust in KDE.

I want to create bindings for enough components that simple Kirigami applications can be written entirely in Rust. Since QML UI is already mostly decoupled from the C++ backend in most applications, writing the backend in Rust instead of C++ is not as difficult as trying to use QtWidgets from Rust. This should attract new application developers to consider Rust a viable option.

Future

Making bindings between KDE and Rust shows surpringing contrasts between the two languages. In the next post, I’ll explore how QtFlags can be used in Rust.

Helpful Links for Rust/Qt Development
  1. qmetaobject
  2. Rust Qt Binding Generattor
  3. ki18n crate
  4. kconfig crate
  5. rust-qt-template
  6. kirigami crate
  7. kde frameworks crate
Categories: FLOSS Project Planets

How not to execve()

Planet KDE - Tue, 2022-01-25 18:00

There is a local privilege escalation in Polkit (formerly PolicyKit, and used in plenty of places where privilege escalation is needed). It was found by Qualys, and carefully documented on the oss-sec mailing list. It has hit the mainstream media fairly hard, too – probably because it follows closely on unrelated log4j and faker.js issues in Open-Source-land. I’m not a security specialist by a long shot (not by at least 3 light-seconds, even), but let’s take a brief look at execve() in FreeBSD.

The description below applies to FreeBSD’s execve() as it is today. This morning. Right now when I write it, because there is a review to fix the root cause of the issue which will surely roll out quickly. In the meantime I landed updates to the polkit package last night, although navigating-the-security-notifications parts have not finished yet.

In UNIX talk, when one process wants to start another, there is a “fork-exec dance”. The one process makes a copy of itself, proceeding as two separate processes, and then exec is called, which replaces a process by a different one from another executable. The most common program (process) that starts other processes is the shell. When you do this in a terminal:

$ ls

Aside from all the other things going on, the shell is going to fork itself and then one of the two will replace itself by the executable for ls. The ls executable then runs, does its thing, and exits. Afterwards, there is one shell process left.

The current API for exec – for replacing the running process by different one from another executable – is execve(). Here is a snippet from the manpage:

int execve(const char *path, char *const argv[], char *const envp[]); The execve() system call transforms the calling process into a new process. The new process is constructed from an ordinary file, whose name is pointed to by path, called the new process file. The argument argv is a pointer to a null-terminated array of character pointers to null-terminated character strings. These strings construct the argument list to be made available to the new process. At least one argument must be present in the array; by custom, the first element should be the name of the executed program (for example, the last component of path).

Here is a complete program that uses execve(). The intention is that this program just replaces itself with ls. Running ls with no arguments usually lists the files in the current directory. With arguments, there’s command-line option processing and then the named files or directories are listed.

#include <unistd.h> int main(int argc, char **argv) { char * const bogus_argv[2] = {NULL, NULL}; char * const bogus_envp[2] = {"PS1=$", NULL}; int r = execve("/bin/ls", bogus_argv, bogus_envp); return 0; }

Let’s take a look at some of the lines of this program.

char * const bogus_argv[2] = {NULL, NULL};

Here we have an array of pointers, ready to act as the argv for the program we’re going to run. Note that both pointers are set to NULL, so we’re violating the constraint mentioned in the manpage: here there are zero arguments in the null-terminated array. But there’s two NULLs, so we can at least pretend that we’re compliant: if there were a non-null pointer which would customarily point to the name of the program, the array would look like {"ls", NULL}. So we have a suitable null-terminator in place.

char * const bogus_envp[2] = {"PS1=$", NULL};

This is a very minimal environment. PS1 is the name of the variable – it’s the shell prompt. The value $ is what you traditionally have in the POSIX shell.

int r = execve("/bin/ls", bogus_argv, bogus_envp);

The current process (this program whose source I show above) is replaced by the executable which is in the file /bin/ls – that’s the ls program we know and love. We pass in the bogus argv, which violates the constraints described for execve(), and the very simple environment.

Personally I expected ls to run and list the current directory. After all, even without the name-of-the-program as the first argument, the argv array is null-terminated. Nope.

$ cc t.c && ./a.out : PS1=$: No such file or directory

Somehow we’re reaching the environment pointers, which are interpreted as arguments to ls. Replace the first NULL in the assignment to bogus_argv by anything else, including "", and the program behaves as I expected. Not with a bunch of NULLs, though. Making the bogus_argv longer, e.g. with 16 NULLs, doesn’t help: all of them are ignored and the environment leaks into the arguments of ls.

Takeaway

OpenBSD did it right in 2015 (quoting a Twitter message from Bryan Steele, @canadianbryan):

date: 2015/02/07 08:47:49; author: tedu; state: Exp; lines: +7 -1; forbid execve() with argc == 0. prompted by a millert email. ok deraadt miod

FreeBSD is going to do it right (in review), and maybe even Linux is going to do it right (via Ariadne Conill, @ariadneconill on Twitter, this patch).

Categories: FLOSS Project Planets

FSF News: Waag founder Marleen Stikker to keynote LibrePlanet 2022

GNU Planet! - Tue, 2022-01-25 16:35
BOSTON, Massachusetts, USA -- Tuesday, January 25, 2022 -- The Free Software Foundation (FSF) today announced Marleen Stikker as its opening keynote speaker for LibrePlanet 2022. The annual technology and social justice conference will be held virtually on March 19 and 20, 2022, with the theme "Living Liberation."
Categories: FLOSS Project Planets

Dirk Eddelbuettel: RcppArmadillo 0.10.8.1.0 on CRAN: Upstream Updates

Planet Debian - Tue, 2022-01-25 15:33

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 950 other packages on CRAN, downloaded over 22.9 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint/vignette) by Conrad and myself has been cited 451 times according to Google Scholar.

This release brings another upstream update 10.8.0, and first bug fix release 10.8.1. As updates by Conrad can come a little quicker than the desired monthly cadence CRAN aims for, we skipped the 10.8.0 release for CRAN only but of course generally provide them via the Rcpp drat repo as well as via general updates to the repo, and full reverse dependency testing (for which results are always logged here).

The full set of changes (since the last CRAN release 0.10.7.5.0) follows.

Changes in RcppArmadillo version 0.10.8.1.0 (2022-01-23)
  • Upgraded to Armadillo release 10.8.1 (Realm Raider)

    • fix interaction between OpenBLAS and LAPACK

    • emit warning if find() is incorrectly used to locate NaN elements

Changes in RcppArmadillo version 0.10.8.0.0 (2022-01-02)
  • Upgraded to Armadillo release 10.8 (Realm Raider)

    • faster handling of symmetric matrices by pinv() and rank()

    • faster handling of diagonal matrices by inv_sympd(), pinv(), rank()

    • expanded norm() to handle integer vectors and matrices

    • added datum::tau to replace 2π

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Categories: FLOSS Project Planets

PyCoder’s Weekly: Issue #509 (Jan. 25, 2022)

Planet Python - Tue, 2022-01-25 14:30

#509 – JANUARY 25, 2022
View in Browser »

PEP 646: Variadic Generics Was Accepted

This PEP introduces TypeVarTuple, enabling parameterisation with an arbitrary number of types. For example, it allows the type of array-like structures in numerical computing libraries such as NumPy and TensorFlow to be parameterised with the array shape, enabling static type checkers to catch shape-related bugs. The PEP was recently accepted by the Steering Council.
PYTHON.ORG

Build a Dice-Rolling Application With Python

In this step-by-step project, you’ll build a dice-rolling simulator app with a minimal text-based user interface using Python. The app will simulate the rolling of up to six dice. Each individual die will have six sides.
REAL PYTHON

How to Quickly Label Data for Machine Learning

With Toloka, you can control the accuracy and speed of data labeling to develop high performing ML models. Our platform supports annotation for image classification, semantic segmentation, object detection, named entity recognition, sentiment analysis, speech recognition, text classification →
TOLOKA AI sponsor

PEP 679: Allow Parentheses in assert Statements

“It is a common user mistake when using the form of the assert statement that includes the error message to surround it with parentheses. Unfortunately, this mistake passes undetected as the assert will always pass, because it is interpreted as an assert statement where the expression is a two-tuple, which always has truth-y value.”
PYTHON.ORG

Strict Python Function Parameters

Learn about keyword-only and positional-only parameters and see how using strict function signatures can help you write more resilient code.
SETH MICHAEL LARSON

Guiding Design Principles for Scientific Python

Tips on how to design and organize scientific Python code.
NSLS-II.GITHUB.IO

Python Jobs Senior Full-Stack Web Developer (Anywhere)

MonetizeMore

Python Engineer (Web Scraping) (Asia, TX, America)

NewsCatcher

PyCoder's Weekly Curator and Podcast Co-Host (Anywhere)

Real Python

Senior Software Engineer (Anywhere)

Brew

More Python Jobs >>>

Articles & Tutorials Designing for Users and Building a Social Network With Django

Are you looking for a project to practice your Django skills? Designing the fundamental interactions of a social network is an instructive way to explore models and relationships while learning advanced Django skills. This week on the show, previous guest Martin Breuss talks about his new four-part tutorial series, “Build a Social Network With Django.”
REAL PYTHON podcast

Solving Wordle With Python

“Wordle seems to be trending and it got me thinking of what would be the best algorithm to solve it. I decided to do a naive implementation in Deepnote with Python that only uses NLTK’s list of all english words and base letter frequencies in english texts. This way, I was able to solve the January 18 challenge, but I used up all available attempts.”
SIMON SOTAK

Scrape Your Web Data From Any Target

With Oxylabs Scraper APIs, extract public data from the most complex targets. Our Scraper APIs handle JavaScript-heavy websites and support large data requests using over 102 million global proxy infrastructure. Receive data in JSON or CSV format and pay per successful request only. Start free trial →
OXYLABS sponsor

Starting With Python IDLE

In this course, you’ll learn how to use the development environment included with your Python installation. Python IDLE is a small program that packs a big punch! You’ll learn how to use Python IDLE to interact with Python directly, work with Python files, and improve your development workflow.
REAL PYTHON course

Modulo String Formatting in Python

You can use the % modulo operator for string formatting in Python. It’s a commonly used technique in older Python versions, especially in Python 2. Therefore, you might see it when digging into existing code bases, and it can be helpful to understand how it works.
REAL PYTHON

Understand Django: Go Fast With Django

How do you make your Django app fast? You measure what is slow, scale your system when necessary, and use a combination of fast database queries and strategic caching. This artcile explores those topics and more to help you get a performant Django app.
MATT LAYMAN • Shared by Matt Layman

Static Typing for Python Decorators

Accurately static typing decorators in Python is tricky. The wrapper function obfuscates type information required to statically determine the types of the wrapped function parameters. Here’s how you can circumnavigate that.
REDOWAN DELOWAR • Shared by Redowan Delowar

Python Type Hints: How to Type a Descriptor

“The descriptor protocol allow us to completely customize attribute access. Python’s documentation describes the protocol with types involved described with words. Let’s look at how we can write those as type hints.”
ADAM JOHNSON

The Power of Python Descriptors

An introduction to the Python descriptor protocol, with some example use cases. The descriptor protocol allows you to implement custom logic when a variable is accessed, or assigned a new value.
PICCOLO-ORM.COM

The Developer / Product Team Notification Template Problem

Notifications are hard. They require infrastructure that is reliable, scalable and observable as well as an end user experience that is helpful and respectful. That’s why we built Courier.
COURIER sponsor

Dealing With YAML With Arbitrary Tags in Python

Using the PyYAML library to safely read and write YAML with any tags, in a way that’s as straightforward as interacting with built-in types.
ANDGRAVITY.COM

An Introduction to Mypy’s @overload

“Sometimes the type of the returned value in a function depend on the type of the arguments in ways that can’t be captured with a Union.”
PATRICK NSUKAMI • Shared by Patrick Nsukami

How Vectorization Speeds Up Your Python Code

Vectorization allows you to speed up processing of homogeneous data in Python. Learn what it means, when it applies, and how to do it.
ITAMAR TURNER-TRAURING

A Trick to Have Arbitrary Infix Operators In Python

Discussing an interesting trick which adds support for infix operators in Python, e.g. 5 |add| 6
TOMER FILIBA

DIY Self Driving: A Holiday Side Project With Python

TRISTAN RICE

Projects & Code pyccolo: Declarative Instrumentation for Python

GITHUB.COM/SMACKE

Qtile: Hackable Tiling Window Manager Written in Python

QTILE.ORG

sonora: A gRPC-Web Implementation for Python

GITHUB.COM/PUBLIC

EasyRPC: Sharing Python Functions Between Processes and Applications

GITHUB.COM/CODEMATION • Shared by Joshua Jamison

Events Weekly Real Python Office Hours Q&A (Virtual)

January 25, 2022
REALPYTHON.COM

PyCamp Leipzig 2022

January 29 to January 31, 2022
BARCAMPS.EU

PyDelhi User Group Meetup

January 29, 2022
MEETUP.COM

PythOnRio Meetup

January 29, 2022
PYTHON.ORG.BR

Introduction to the Python Programming Language (In Persian)

February 1, 2022
INSTAGRAM.COM

PyCon Iran Conference (Virtual)

February 1 to February 2, 2022
PYCON.ORG • Shared by PyCon Iran Team

DjangoCon Europe 2022

September 21 to 25, 2022 in Porto, Portugal
DJANGO SOFTWARE FOUNDATION

Happy Pythoning!
This was PyCoder’s Weekly Issue #509.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

Categories: FLOSS Project Planets

Qt Quick 3D: interactive 2D content

Planet KDE - Tue, 2022-01-25 11:43

Qt Quick 3D has some new features in 6.2. One of them is that you can map interactive Qt Quick scenes onto 3D objects.

During a hackathon last year, we developed the Kappa Tau Station demo: a model of a space station in which you can use the WASD keys to walk around (as in many games), but also containing some 2D UI elements on some surfaces. For example you can:

Categories: FLOSS Project Planets

Python Morsels: How to remove spaces in Python

Planet Python - Tue, 2022-01-25 11:00

Need to remove spaces from your a string in Python? Let's talk about how to remove all spaces, remove duplicate spaces, remove spaces from the ends of our strings, remove trailing newlines, and remove spaces from the beginning and end of each line.

Remove spaces from a string

Need to remove all spaces from a string in Python?

If it's just space characters you could use the string replace method to replace all spaces by an empty string.

If we call the replace method on this greeting string:

>>> greeting = " Hello world! " >>> no_spaces = greeting.replace(" ", "")

The resulting no_spaces string will have all space characters removed:

>>> no_spaces 'Helloworld!' Remove all whitespace from a string

If you're trying to remove all sorts of whitespace characters (space, tab, newline, etc.) you could use the string split and join methods:

If we call split on this version string, Python will split on all consecutive whitespace characters:

>>> version = "\tpy 310\n" >>> version.split() ['py', '310']

The string join method can join an iterable of strings by a delimiter (see turning a list into a string). If we join with a delimiter of empty string (""), we'll effectively remove all spaces:

>>> no_spaces = "".join(version.split()) >>> no_spaces 'py310'

If you're comfortable with regular expressions, you can also use a regular expression to replace all consecutive whitespace by an empty string:

>>> import re >>> no_spaces = re.sub(r"\s+", r"", version) >>> no_spaces 'py310' Remove duplicate whitespace

What if you just need to get rid of extra spaces (collapsing consecutive spaces)?

We could use the string split and join methods, as before, but join on a space character instead of an empty string:

>>> version = "\tpy 310\n" >>> normalized_spaces = " ".join(version.split()) >>> normalized_spaces 'py 310'

Note that this normalizes all whitespace characters (so newlines and tabs will be converted as well) and it removes spaces from the ends of our string.

Strip whitespace from the beginning and end of a string

What if you only need to remove whitespace from the beginning and end of your string?

You can use the string strip method:

>>> version = "\tpy 310\n" >>> stripped_version = version.strip() >>> stripped_version 'py 310'

By default the strip method removes all whitespace characters (not just spaces).

The strip method also accepts an optional argument if you prefer to strip just a specific character. It also has two cousin methods: lstrip (for splitting from the left-hand side) and rstrip (for splitting from the right-hand side).

If you just need to remove an optional trailing newline character from the end of your string, you can use strip (passing in a \n character):

>>> version = "\tpy 310\n" >>> no_trailing_newline = version.rstrip("\n") >>> no_trailing_newline '\tpy 310' Strip whitespace from the beginning and end of each line

What if you need to strip whitespace from the beginning and end of each line in your string?

You could split your lines with splitlines, use a comprehension to call the strip method on each line, and then use the join method to join your lines back together with newline characters:

>>> string = " Line 1\nLine 2 \n Line 3 \n" >>> stripped = "\n".join([ ... line.strip() ... for line in string.splitlines() ... ]) ... >>> stripped 'Line 1\nLine 2\nLine 3'

Although this is complex enough that I'd usually start to reach for regular expressions at this point:

>>> import re >>> stripped = re.sub(r"^\s+|\s+$", r"", string, flags=re.MULTILINE) >>> stripped 'Line 1\nLine 2\nLine 3'
Categories: FLOSS Project Planets

TEN7: Adding Bundle Subclasses to Drupal Core 9.3.0 (Part 2)

Planet Drupal - Tue, 2022-01-25 10:00

In Part 1 of this blog series, we explored the concepts of entities, bundles, entity classes, and the new feature of bundle subclasses now available in Drupal Core 9.3.0. #2570593: Allow entities to be subclassed using "bundle classes" is one of the biggest changes to Drupal Core in many releases, although it’s only directly available to developers. In this post, we’ll look at how to start making use of this functionality to improve your life.

Categories: FLOSS Project Planets

The Open Sourcerer: The Software Upgrade Threadmill and Life’s crazy chain of dependencies — an epic tale about Firefox, GTG, Python, and Linux distros

Planet Python - Tue, 2022-01-25 09:30

This blog post talks about the general types of hurdles I’ve been encountering on the desktop side of the “upgrade threadmill” when running Linux. It is part two of a three-part blog post série. If you’re into servers, you may be interested in part one, the Linux server landscape’s post-2020 metamorphosis.

Modern software development happens at a breakneck pace, and while staying on ancient versions (hello, Debian Stable / Ubuntu LTS / Android users) is not really a safe and realistic option anymore (try reporting bugs without getting laughed out of the room by upstream maintainers), it is becoming a challenge for users to keep up. When it works, it works… but when something breaks down in the upgrade threadmill, the chain of dependencies to get back on track can become absolutely ludicrous and throw your digital life in turmoil. Just like needing to replace that one light bulb…

Case in point: I’m finally publishing this article in 2022, while I initially meant to blog about this way back in 2017… but more stuff kept breaking all the time, resetting my productivity and accidentally adding more potential content for this blog post. More value for you, dear reader!

As someone who has been running Linux for 19 years (as of 2022), I think I know my way around most hurdles you can possibly encounter. Undoubtedly, running Linux-based operating systems on desktop/laptop computers has overall gotten incredibly easier compared to 2003, but also, as one gradually becomes highly dependent on specific tools and committed to well-oiled workflows, the upgrade threadmill can become a real high-stakes pursuit.

I’ll skim over minor anecdotes like the fact that I once had to “dnf versionlock alsa” for a while because ALSA was misbehaving with my Sound Blaster™ Audigy™ (don’t laugh! That’s how you get analog surround sound when your motherboard doesn’t do 5.1) while testing on a live USB version of a new Fedora release but, as it turns out, it worked without problems after installation… or the fact that I just realized, in 2022, that nautilus-sendto doesn’t exist anymore and since most apps haven’t yet migrated to the Flatpak-compatible replacement “portal”, that means I can’t right-click a file in Nautilus to send it via email now… and I’ll instead dive into the rather more interesting reason why I ran an old insecure web browser—and operating system—for nearly two years.

A tale of foxes

In 2017, Mozilla unleashed Firefox Quantum, quite possibly the most awesome and groundbreaking release they did in that decade. Except there was just one little problem: it also completely changed the extensions system (for good technical reasons).

I happened to be critically dependent, for my day-to-day productivity, on the now incompatible TabGroups extension (which was, itself, a reimplementation of Mozilla’s previously deprecatedPanorama” feature from Firefox 4, previously known as “Tab Candy“), whose author had burned out and given up completely on porting it to the new WebExtensions API.

As a power user, I was utterly foxed.

There was an awfully long period of time where Firefox had no replacement and no migration path for users like me.

Because the modern web (a.k.a. the web obesity crisis) was unbearable, I needed to stay on an Electrolysis-capable-but-pre-Quantum version of Firefox, and that meant specifically Firefox 57, and no other. Not even Firefox ESR. I had to deep-freeze my Firefox version to prevent any upgrade. Thankfully, operating systems like Fedora allow you to do that easily, with DNF’s “versionlock” feature.

This is what “dnf versionlock firefox” looks like:

Hey, don’t give me that look! I had no other choice at that time, and I was too busy in my day-to-day mercenary CMO job to be messing with my personal productivity tooling. And I kept waiting, checking and hoping that someday, someone would write a replacement WebExtension for it.

I therefore spent a painfully long period of time stuck on the same old (and eventually EoL‘ed) version of Fedora Workstation, because upgrading Fedora eventually became impossible due to dependency issues arising from the versionlocked Firefox. While Fedora 27 was released in 2017, I had to remain on that frozen-in-time, unmaintained, insecure version until the end of 2019.

This is what waiting in suspense, on an EoL’ed operating system, looks like:

This moment was excruciating.

Yes, I can hear all of you from the InfoSec crowd gasping for air, in shock, after having stared at the screen for a solid 30 seconds wondering how I could have subjected myself to such risk in this day and age. What can I say, I have NERVes of StEELE.

Thankfully, one day in 2019, Simple Tab Groups was recommended to me and, as I tested it extensively to see how solid it actually was, I determined it to be the sole viable, Nekohayo-Proof™, Enterprise-Grade™ replacement for Panorama/TabGroups. It provided a migration path and it was pretty much bullet-proof (especially compared to things like Conex, which I found to be unreliable for my needs). I’ve been happily using it since 2019 and it works better than anything else ever did. With a couple of tweaks in the settings, it has much better performance than its predecessors—especially when you disable thumbnails, use the tabs “discarding” (unloading) feature, and after my suggested search activation technique was implemented—so that’s neat.

A tale of pure hard drive

Eventually, Firefox being a solved problem at last, and Fedora 27 reaching end-of-life status, I really had to migrate, not just for security reasons, but also because my computer’s hard drive partitions were badly space-bound, as I had really outgrown the layout I had made all the way back in… 2010.

Repartitioning my workstation, and upgrading some SSDs in the process, was also blocking my ability to reassign the older/smaller SSDs to my server, which in turn was blocking my ability to migrate from Centos 7 to Centos 8, which was hurting my ability to upgrade PHP sustainably, which was blocking my ability to upgrade Matomo.

If your head hurts while you’re reading this, at this point, it’s not just because of the hard drives. But wait, I’m not done yet!

A tale of serpents

Fedora, as a fast-moving operating system, generally favors leanness and innovation over backwards compatibility (“Friends, Features, First“). This means that if you want to get rid of Python 2 for example, you hunt down and exile any application that still hasn’t ported to Python 3 (they were not alone by the way; OpenSUSE and Sabayon Linux, among others, did the same thing roughly a year later). This works well in an ideal world where application developers have the time and resources to quickly port to new technologies, but the Python 2 to 3 migration has been a notoriously long process in terms of ecosystem. And sometimes, developers have to shave two dozen massive yaks in lockstep in order to achieve that. I know, because I have lived through, as a co-maintainer, the multi-years effort that was porting Pitivi to Python 3, GTK 3, GObject Introspection and GStreamer 1.0 simultaneously.

As a result, when Fedora decided to cut the cord for Python 2 apps in version 31, I had a number off desktop applications simply cease to exist on Fedora. Among them were DisplayCAL (which I had just discovered as the only way to actually make my Colormunki spectrophotometer work for display calibration without requiring a PhD in quantum physics, as colord unfortunately never worked reliably for me), and… Getting Things GNOME.

I could live without DisplayCAL (“we had barely just met!”), but I could not live without GTG, so I had to go begging for a good samaritan to emergency-package it as a Flatpak. Thankfully, Bilal Elmoussaoui answered the call to package the old version. Whew, I had some breathing room and could finally upgrade from the EoL’ed Fedora 27 to latest stable version at the time, Fedora 30, while I figured out a way to put the GTG project back on track.

  • I eventually did put the GTG project back onto the rails (with Diego’s admirable bugfixing frenzy, the much-awaited 0.4 version was released in 2020 from development hell). Two years down the road, GTG is now slated to re-enter Fedora (hopefully) 🤞
  • DisplayCAL, on the other hand, is still a Python 2 application so… as Larry “Solo Wing Pixy” Foulke said in the introduction scene of The Belkan War, “It’s going to take a while.”

As you can see, even for an experienced Linux user used to troubleshooting cutting-edge technology, the upgrade threadmill can be a problem. I mean, just look at how absolutely ridiculous the “chain of dependencies” has been for me here just to solve the “critical path”:

  • Objective: “run the latest Fedora version!”
    • Blocked by Firefox version
      • Blocked by needing to wait for a viable replacement for Panorama / Tab Groups
    • Blocked by GTG
      • Needs a temporary Flatpak package to live through the winter
      • Needs a new Python 3 + GObject Introspection + GTK3-based release
        • Needs a new community of coders to reform around the project
          • Needs product management, including proper open-source processes and an opiniated direction
            • Needs updating/rewriting the majority of contributors’ documentation
            • Needs talking to previous maintainers and the public to ascertain what led to previous failure
          • Needs remarketing & open-source tech evangelism, including outreach & community management
            • Needs to evaluate the feasibility and available resources
              • Need to ascertain the technical status of the previously stalled development efforts (by persistently tracking down and pressing previous maintainers for answers via personal email exchanges) and see if any of the onlookers of this ticket would be interested in embarking on that project
              • Needs to survey the market to gauge interest, with a marketing piece that also provides historical context and a pitch on GTG’s unique value proposition, preceded by:
            • Needs a clear, actionable, lean, rapidly achievable “MVP” roadmap
              • Needs heavy bug triaging
                • Needs building & testing code that had been abandoned for 7 years
                • Needs claiming administrative ownership of the multiple bug trackers
            • Needs to build a social media presence from scratch
            • Needs to rewrite the project’s front page to market it to users (who may become potential contributors)
          • The things in bold above are done as a public service, but they also are what companies would hire me for, if they have enough Japanese Yen. I didn’t spend years training as a mercenary in Restricted Airspace B7R for nothin’!

That’s not even counting the repartitioning of all my drives (which was another yak shave that had its own chain of dependencies), nor the years-long epic investigation to solve weird hardware issues some years prior to all this. Goodness gracious, it’s a miracle we get anything done around here.

You may think the story ends here. After all, with those issues resolved, I finally was able to thaw everything and upgrade Fedora… and it was a huge sigh of relief indeed; late 2019 to late 2020 was pretty great in terms of being able to run a normal up-to-date Fedora Workstation experience. But nothing is eternal (except Doom)…

Keep your popcorn nearby. Next, I will write about the Curious Case of Fedora and Unison, which warrants its own blog post. Keep your eyes open for a new article on Monday, January 31st!

Categories: FLOSS Project Planets

Real Python: Looping With Python enumerate()

Planet Python - Tue, 2022-01-25 09:00

In Python, a for loop is usually written as a loop over an iterable object. This means that you don’t need a counting variable to access items in the iterable. Sometimes, though, you do want to have a variable that changes on each loop iteration. Rather than creating and incrementing a variable yourself, you can use Python’s enumerate() to get a counter and the value from the iterable at the same time!

In this course, you’ll see how to:

  • Use enumerate() to get a counter in a loop
  • Apply enumerate() to display item counts
  • Implement your own equivalent function to enumerate()
  • Unpack values returned by enumerate()

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Categories: FLOSS Project Planets

Agiledrop.com Blog: 2021 in review

Planet Drupal - Tue, 2022-01-25 08:49

As we kick off another year, we’d like to look back at Agiledrop’s new developments and greatest accomplishments from 2021.

READ MORE
Categories: FLOSS Project Planets

Mike Driscoll: PySimpleGUI - An Intro to Laying Out Elements

Planet Python - Tue, 2022-01-25 08:30

PySimpleGUI is a Python GUI that wraps other Python GUI toolkits (Tkinter, PySide, wxPython). By abstracting away the complexities of the other GUIs into a common API, you can quickly write code that can be rendered using any of those other toolkits just by changing the import at the top of your program.

If you were to use wxPython and you wanted to layout your widgets, you would use a wx.Sizer. In PySide or PyQt, you would use Layout object. Tkinter uses geometry managers. The general concept is the same. When you use these tools, the GUI toolkit uses relative positioning of the widgets to arrange them onscreen.

In PySimpleGUI, the same thing is done using nested lists. You will look at a couple of different examples in this tutorial that will give you a general idea of how this all works.

Creating a Horizontal Layout

Creating a series of Elements that are horizontally oriented (left-to-right) is done with a small list of lists in PySimpleGUI.

Open up a Python editor and add the following code:

import PySimpleGUI as sg # Horizontal layout layout = [[sg.Button(f"OK {num}") for num in range(1, 6)]] # Create the window window = sg.Window("Demo", layout) # Create an event loop while True: event, values = window.read() # End program if user closes window or # presses the OK button if event == "OK" or event == sg.WIN_CLOSED: break window.close()

Here you use a Python list comprehension to create a nested list that contains five Button objects in it. When you run this code, you will see the buttons aligned horizontally, with one next to the other going from left-to-right.

Now you are ready to see how you can rewrite the code to create a vertical layout!

Creating a Vertical Layout

To create a vertically oriented layout with PySimpleGUI, you need to make the layout contain a series of nested lists that each contain a one or more Elements in them.

Create a new Python file and add this code to it:

import PySimpleGUI as sg layout = [[sg.Button("OK")], [sg.Button("OK 2")], [sg.Button("OK 3")], [sg.Button("OK 4")], [sg.Button("OK 5")]] # Create the window window = sg.Window("Demo", layout) # Create an event loop while True: event, values = window.read() # End program if user closes window or # presses the OK button if event == "OK" or event == sg.WIN_CLOSED: break window.close()

When you run this code, the Button Elements will be stacked from top to bottom vertically, instead of horizontally.

Using Columns for Complex Layouts

If you need to add two or more columns of Elements next to each other in PySimpleGUI, you can use a sg.Column Element. It is a type of container Element specifically made for creating stacked sets of Elements!

Create another Python file and add the following code:

import PySimpleGUI as sg import os.path # First the window layout in 2 columns file_list_column = [ [sg.Text("Image Folder"), sg.In(size=(25, 1), enable_events=True, key="-FOLDER-"), sg.FolderBrowse(),], [sg.Listbox(values=[], enable_events=True, size=(40, 20), key="-FILE LIST-")], ] # For now will only show the name of the file that was chosen image_viewer_column = [ [sg.Text("Choose an image from list on left:")], [sg.Text(size=(40, 1), key="-TEXT-")], [sg.Image(key="-IMAGE-")], ] # ----- Full layout ----- layout = [ [sg.Column(file_list_column), sg.VSeperator(), sg.Column(image_viewer_column),] ] window = sg.Window("Column Demo", layout) # Run the Event Loop while True: event, values = window.read() if event == "Exit" or event == sg.WIN_CLOSED: break # Folder name was filled in, make a list of files in the folder if event == "-FOLDER-": folder = values["-FOLDER-"] try: # Get list of files in folder file_list = os.listdir(folder) except: file_list = [] fnames = [ f for f in file_list if os.path.isfile(os.path.join(folder, f)) and f.lower().endswith((".png", ".gif")) ] window["-FILE LIST-"].update(fnames) elif event == "-FILE LIST-": # A file was chosen from the listbox try: filename = os.path.join(values["-FOLDER-"], values["-FILE LIST-"][0]) window["-TEXT-"].update(filename) window["-IMAGE-"].update(filename=filename) except: pass window.close()

In this example, you create two layouts! The first layout is called file_list_column and contains four Elements in it. The second layout is called image_viewer_column and contains three Elements. Then you put these layouts inside of Columns that are themselves inside a layout. As you can see, PySimpleGUI layouts are lists inside of lists all the way down!

Wrapping Up

PySimpleGUI removes some of the complexity of creating layouts by using Python lists. The PyQt / PySide and wxPython GUI toolkits use an object-oriented approach which includes a significant amount of boilerplate and flags to do much the same thing. They each have their tradeoffs, but PySimpleGUI's learning curve is that much flatter because of their design choice.

Related Reading

The post PySimpleGUI - An Intro to Laying Out Elements appeared first on Mouse Vs Python.

Categories: FLOSS Project Planets

Python for Beginners: String Indexing in Python

Planet Python - Tue, 2022-01-25 08:30

Strings are used for processing text data in Python. While processing strings, we often need to access a certain part of the string. In this article, we will see how we can extract parts of a string using indexing in Python. 

What is String Indexing ?

If we have an ordered sequence or container object such as a string, a list, or a tuple, we can access the elements of the objects using their relative position in the sequence. The relative position of the elements in the ordered sequence is termed as index. With Indexing, we can access any element from an ordered sequence using the indices. 

In python, string indexing is zero based. It means that we start the counting from 0 and the first character of the string is assigned the index 0, the second character is assigned the index 1, the third character is assigned the index 2 and so on. 

We can understand this using the following example.

Suppose that we have a string “PythonForBeginners”

Here, the index of the letter “P” is 0. The index of the letter “y” is 1. The index of letter ”t” is 2, The index of letter “h” is 3 and so on. The index of the last letter “s” is 17.

In python, we can use positive as well as negative numbers for string indexing. Let us discuss them one by one.

String Indexing using Positive Numbers

As we have seen above, Strings are indexed using positive numbers from 0 to string length -1. We can access a character at any position between 0 to (length of the string) -1 using positive indices as follows.

myString = "PythonForbeginners" index = 0 character = myString[index] print("Character at index {} in the string '{}' is {}.".format(index, myString, character)) index = 1 character = myString[index] print("Character at index {} in the string '{}' is {}.".format(index, myString, character)) index = 2 character = myString[index] print("Character at index {} in the string '{}' is {}.".format(index, myString, character)) index = 3 character = myString[index] print("Character at index {} in the string '{}' is {}.".format(index, myString, character)) index = 17 character = myString[index] print("Character at index {} in the string '{}' is {}.".format(index, myString, character))

Output:

Character at index 0 in the string 'PythonForbeginners' is P. Character at index 1 in the string 'PythonForbeginners' is y. Character at index 2 in the string 'PythonForbeginners' is t. Character at index 3 in the string 'PythonForbeginners' is h. Character at index 17 in the string 'PythonForbeginners' is s.

Always remember that an index greater than or equal to the string length will cause an IndexError exception as follows.

myString = "PythonForbeginners" index = 20 character = myString[index] print("Character at index {} in the string '{}' is {}.".format(index, myString, character))

Output:

Traceback (most recent call last): File "/home/aditya1117/PycharmProjects/pythonProject/string12.py", line 3, in <module> character = myString[index] IndexError: string index out of range

You can either avoid the IndexError exception by checking the value of index before accessing any character in the string. Alternatively, you can use python try except block to handle the exceptions if they occur.

Here, I will suggest you to use the try except blocks. Checking the index every time we access a character may be redundant and costly if we are accessing using indices less than the length of the string. While using try except blocks, the program will not check the value of index each time we access a character from the string. If IndexError occurs, it will be handled by the code in the except block. 

Indexing using Negative Numbers

We can also use negative indices for accessing characters from a string. In python, the last character of the string is assigned an index -1. The second last character is assigned an index -2. Similarly, the first character of the string is assigned an index -(length of the string).

We can understand this using the following example.

Suppose that we have a string “PythonForBeginners”

Here, the index of the letter “s” is -1. The index of the letter “r” is -2. The index of letter ”n” is -3 , The index of letter “n” is -4 and so on. The index of the first letter “P” is -18.

You can verify this using the following program.

myString = "PythonForbeginners" index = -1 character = myString[index] print("Character at index {} in the string '{}' is {}.".format(index, myString, character)) index = -2 character = myString[index] print("Character at index {} in the string '{}' is {}.".format(index, myString, character)) index = -3 character = myString[index] print("Character at index {} in the string '{}' is {}.".format(index, myString, character)) index = -4 character = myString[index] print("Character at index {} in the string '{}' is {}.".format(index, myString, character)) index = -18 character = myString[index] print("Character at index {} in the string '{}' is {}.".format(index, myString, character))

Output:

Character at index -1 in the string 'PythonForbeginners' is s. Character at index -2 in the string 'PythonForbeginners' is r. Character at index -3 in the string 'PythonForbeginners' is e. Character at index -4 in the string 'PythonForbeginners' is n. Character at index -18 in the string 'PythonForbeginners' is P.

While using negative numbers as indices, Make sure that you do not pass an index less than  -(length of the string). Otherwise, your program will run into an IndexError as follows.

myString = "PythonForbeginners" index = -20 character = myString[index] print("Character at index {} in the string '{}' is {}.".format(index, myString, character))

Output:

Traceback (most recent call last): File "/home/aditya1117/PycharmProjects/pythonProject/string12.py", line 3, in <module> character = myString[index] IndexError: string index out of range Conclusion

In this article, we have studied string indexing in python. We have seen how we can use negative as well as positive numbers to access characters from a string. To study more about strings in python, you can read this article on string concatenation.

The post String Indexing in Python appeared first on PythonForBeginners.com.

Categories: FLOSS Project Planets

Latte Dock v0.10.8 | Bug Fix Release

Planet KDE - Tue, 2022-01-25 07:19

 


Let's welcome Latte Dock v0.10.8 the 8th Official Bug Fix Release of v0.10.x branch!   Go get it from, download.kde.org*

Fixes

  • multi-screen: fix docks/panels screen repositioning when the user specifies different screen for dock or panel
  • fix borders identification for autopositioning vertical docks/panels
  • fix vertical docks/panels autopositioning which is relevant to top and bottom panels
  • position kwin edges helper window properly on startup after offscreen positioning
  • x11: center applets config window on screen
  • do not shrink vertical docks/panels on startup after switching from offscreen to onscreen
  • make dock and panel work properly when they undo their removal


  Donations

You can ping me at https://www.reddit.com/user/psifidotos in order to give you my paypal account.
or you can split your donation between my active projects in kde store. -----   * archive has been signed with gpg key: 325E 97C3 2E60 1F5D 4EAD CF3A 5599 9050 A2D9 110E

 

Categories: FLOSS Project Planets

Kubuntu 21.04 (Hirsute Hippo) Reaches End of Life

Planet KDE - Tue, 2022-01-25 06:50

Kubuntu Hirsute Hippo was announced on April 22, 2021 with 9 months support.

As of January 20, 2022, 21.04 reached ‘end of life’.

No more package updates will be accepted to 21.04, and it will be archived in the coming weeks.

You can read the official end of life announcement for Ubuntu as a whole.

Kubuntu 20.04 Focal Fossa and 21.10 Impish Indri continue to be supported.

Users of 21.04 can follow the Kubuntu 21.04 to 21.10 Upgrade instructions.

Should for some reason your upgrade be delayed, and you find that the 21.04 repositories have been archived, instructions to perform a EOL Upgrade can be found on the Ubuntu wiki.

Thank you for using Kubuntu 21.04 Hirsute Hippo.

Categories: FLOSS Project Planets

Python⇒Speed: The fastest way to read a CSV in Pandas

Planet Python - Mon, 2022-01-24 19:00

You have a large CSV, you’re going to be reading it in to Pandas—but every time you load it, you have to wait for the CSV to load. And that slows down your development feedback loop, and might meaningfully slows down your production processing.

But it’s faster to read the data in faster. Let’s see how.

In this article we’ll cover:

  1. Pandas’ default CSV reading.
  2. The faster, more parallel CSV reader introduced in v1.4.
  3. A different approach that can make things even faster.
Read more...
Categories: FLOSS Project Planets

Talking Drupal: Talking Drupal #331 - Migrating Paragraphs for The National Zoo

Planet Drupal - Mon, 2022-01-24 14:00

Today we are talking about Migrating Paragraphs for the National Zoo with Mohammed El-Khatib.

TalkingDrupal.com/331

Topics
  • Nic - Family flew home
  • Abby - Little free library – Hades game
  • Mohammed - Migrating D9 to Tailwind CSS and Alpine – Travel plans fell through
  • John - Listening to TD with kids
  • National Zoo
    • Favorite animal
  • How the National Zoo uses Drupal
  • Why the zoo needed to migrate paragraphs
  • Mapping migration strategy
  • Tool
    • Migrate Plus
    • Migrate Tools
    • Migrate Upgrade
  • Nested Paragraphs
  • Translation
  • Any strategies to migrate
  • Resources for help
  • Tips and Tricks
  • What is next for National Zoo
  • Anything to add?
Resources Guests

Mo El-Khatib - mmelkhatib

Hosts

Nic Laflin - www.nLighteneddevelopment.com @nicxvan John Picozzi - www.epam.com @johnpicozzi Abby Bowman - www.linkedin.com/in/arbowman @abowmanr

MOTW
  • Draggable views DraggableViews makes rows of a view “draggable” which means that they can be rearranged by Drag’n’Drop.
Categories: FLOSS Project Planets

TEN7: Adding Bundle Subclasses to Drupal Core 9.3.0 (Part 1)

Planet Drupal - Mon, 2022-01-24 11:47
The technical details of why and how TEN7 contributed bundle subclasses to Drupal Core 9.3.0.
Categories: FLOSS Project Planets

PyCharm: Together, We Supported Python!

Planet Python - Mon, 2022-01-24 11:03

Last November, PyCharm joined forces with the Python Software Foundation (PSF) for their end-of-the-year fundraiser. From November 9 to December 1, all proceeds from every new PyCharm Professional Edition license purchased with the discount code ‘SUPPORTPYTHON21’ went to the PSF’s general fund.

The Python Software Foundation is the main organization behind the Python programming language. As a non-profit organization, the PSF depends on sponsorships and donations to support its work. You can always donate yourself through their website.

JetBrains and the PSF would like to thank all of you who took part in this campaign. Together, we raised $25,000 in less than a month! Contributions like those from PyCharm users help the PSF maintain a healthy balance and continue to support the Python community and its various outreach and diversity programs.

We hope that you enjoy using Python in 2022 and that PyCharm will be a powerful and reliable partner for your journey!

The PyCharm Team

Categories: FLOSS Project Planets

ItsMyCode: Adding new column to existing DataFrame in Pandas

Planet Python - Mon, 2022-01-24 10:49

In this article, we will look at different ways to adding new column to existing DataFrame in Pandas. 

Let us create a simple DataFrame that we will use as a reference throughout this article to demonstrate adding new columns into Pandas DataFrame.

# import pandas library import pandas as pd # create pandas DataFrame df = pd.DataFrame({'team': ['India', 'South Africa', 'New Zealand', 'England'], 'points': [10, 8, 3, 5], 'runrate': [0.5, 1.4, 2, -0.6], 'wins': [5, 4, 2, 2]}) # print the DataFrame print(df)

Output

team points runrate wins 0 India 10 0.5 5 1 South Africa 8 1.4 4 2 New Zealand 3 2.0 2 3 England 5 -0.6 2

Now that we have created a DataFrame let’s assume that we need to add a new column called “lost”, which holds the count of total matches each team has lost.

Method 1: Declare and assign a new list as a column

The simplest way is to create a new list and assign the list to the new DataFrame column. Let us see how we can achieve this with an example.

# import pandas library import pandas as pd # create pandas DataFrame df = pd.DataFrame({'team': ['India', 'South Africa', 'New Zealand', 'England'], 'points': [10, 8, 3, 5], 'runrate': [0.5, 1.4, 2, -0.6], 'wins': [5, 4, 2, 2]}) # print the DataFrame print(df) # declare a new list and add the values into the list match_lost = [2, 1, 3, 4] # assign the list to the new DataFrame Column df["lost"] = match_lost # Print the new DataFrame print(df)

Output

team points runrate wins lost 0 India 10 0.5 5 2 1 South Africa 8 1.4 4 1 2 New Zealand 3 2.0 2 3 3 England 5 -0.6 2 4 Method 2: Using the DataFrame.insert() method

The disadvantage of the above approach is that we cannot add the column at the specified position, and by default, the column is inserted towards the end, making it the last column.

We can overcome the issue using the pandas.DataFrame.insert() method. This method is useful when you need to insert a new column in a specific position or index.

In the below example, let us insert the new column “lost” before the “wins” column. We can achieve this by inserting a new column at index 2.

# import pandas library import pandas as pd # create pandas DataFrame df = pd.DataFrame({'team': ['India', 'South Africa', 'New Zealand', 'England'], 'points': [10, 8, 3, 5], 'runrate': [0.5, 1.4, 2, -0.6], 'wins': [5, 4, 2, 2]}) # print the DataFrame print(df) # insert the new column at the specific position df.insert(3, "lost", [2, 1, 3, 4], True) # Print the new DataFrame print(df)

Output

team points runrate lost wins 0 India 10 0.5 2 5 1 South Africa 8 1.4 1 4 2 New Zealand 3 2.0 3 2 3 England 5 -0.6 4 2 Method 3: Using the DataFrame.assign() method

The pandas.DataFrame.assign() method is used if we need to create multiple new columns in a DataFrame.

This method returns a new object with all original columns in addition to new ones. All the existing columns that are re-assigned will be overwritten.

In the below example, we are adding multiple columns to Pandas DataFrame.

# import pandas library import pandas as pd # create pandas DataFrame df = pd.DataFrame({'team': ['India', 'South Africa', 'New Zealand', 'England'], 'points': [10, 8, 3, 5], 'runrate': [0.5, 1.4, 2, -0.6], 'wins': [5, 4, 2, 2]}) # print the DataFrame print(df) # append multiple columns to Pandas DataFrame df2 = df.assign(lost=[2, 1, 3, 4], matches_remaining=[2, 3, 1, 1]) # Print the new DataFrame print(df2)

Output

team points runrate wins lost matches_remaining 0 India 10 0.5 5 2 2 1 South Africa 8 1.4 4 1 3 2 New Zealand 3 2.0 2 3 1 3 England 5 -0.6 2 4 1 Method 4: Using the pandas.concat() method

We can also leverage the pandas.concat() method to concatenate a new column to a DataFrame by passing axis=1 as an argument. This method returns a new DataFrame after concatenating the columns.

# import pandas library import pandas as pd # create pandas DataFrame df = pd.DataFrame({'team': ['India', 'South Africa', 'New Zealand', 'England'], 'points': [10, 8, 3, 5], 'runrate': [0.5, 1.4, 2, -0.6], 'wins': [5, 4, 2, 2]}) # print the DataFrame print(df) # create a new DataFrame df2 = pd.DataFrame([[1, 2], [2, 1], [3, 4], [0, 3]], columns=['matches_left', 'lost']) # concat and Print the new DataFrame print(pd.concat([df, df2], axis=1))

Output

team points runrate wins matches_left lost 0 India 10 0.5 5 1 2 1 South Africa 8 1.4 4 2 1 2 New Zealand 3 2.0 2 3 4 3 England 5 -0.6 2 0 3 Method 5: Using the Dictionary

Another trick is to create a dictionary to add a new column in Pandas DataFrame. We can use the existing columns as Key to the dictionary and assign values respectively to the new column.

# import pandas library import pandas as pd # create pandas DataFrame df = pd.DataFrame({'team': ['India', 'South Africa', 'New Zealand', 'England'], 'points': [10, 8, 3, 5], 'runrate': [0.5, 1.4, 2, -0.6], 'wins': [5, 4, 2, 2]}) # print the DataFrame print(df) # Create a new dictionary with keys as existing column # and the values of new column match_lost = {2: 'India', 1: 'South Africa', 3: 'New Zealand', 0: 'England'} # assign the dictionary to the DataFrame Column df['lost'] = match_lost # print Dataframe print(df)

Output

team points runrate wins lost 0 India 10 0.5 5 2 1 South Africa 8 1.4 4 1 2 New Zealand 3 2.0 2 3 3 England 5 -0.6 2 0 Conclusion

In this article, we saw the 5 approaches creating and assigning a list, insert(), assign(), concat() and dictionary to insert new columns into Pandas DataFrame or overwrite the existing ones. Depending on the need and the requirement, you can choose one of the methods specified which are more suitable.

Categories: FLOSS Project Planets

Pages