Feeds

Russ Allbery: Review: The Story of the Treasure Seekers

Planet Debian - Sun, 2022-01-30 23:00

Review: The Story of the Treasure Seekers, by E. Nesbit

Publisher: Amazon Copyright: 1899 Printing: May 2012 ASIN: B0082ZBXSI Format: Kindle Pages: 136

The Story of the Treasure Seekers was originally published in 1899 and is no longer covered by copyright. I read the free Amazon Kindle version because it was convenient. My guess is that Amazon is republishing the Project Gutenberg version, but they only credit "a community of volunteers."

There are six Bastable children: Dora, Oswald, Dicky, the twins Alice and Noel, and Horace Octavius (H.O.), the youngest. Their mother is dead and the family's finances have suffered in the wake of her death (or, as the first-person narrator puts it, "the fortunes of the ancient House of Bastable were really fallen"), which means that their father works long hours and is very absorbed with his business. That leaves the six kids largely to fend for themselves, since they can't afford school. Clearly the solution is to find treasure.

This is a fix-up novel constructed from short stories that were originally published in various periodicals, reordered and occasionally rewritten for the collected publication. To be honest, calling it a fix-up novel is generous; there are some references to previous events, but the first fourteen chapters can mostly stand alone. The last two chapters are closely related and provide an ending. More on that in a moment.

What grabs the reader's attention from the first paragraph is the writing style:

This is the story of the different ways we looked for treasure, and I think when you have read it you will see that we were not lazy about the looking.

There are some things I must tell before I begin to tell about the treasure-seeking, because I have read books myself, and I know how beastly it is when a story begins, "Alas!" said Hildegarde with a deep sigh, "we must look our last on this ancestral home" — and then some one else says something — and you don't know for pages and pages where the home is, or who Hildegarde is, or anything about it.

The first-person narrator of The Story of the Treasure Seekers is one of the six kids.

It is one of us that tells this story — but I shall not tell you which: only at the very end perhaps I will.

The narrator then goes on to elaborately praise one of the kids, occasionally accidentally uses "I" instead of their name, and then remembers and tries to hide who is telling the story again. It's beautifully done and had me snickering throughout the book. It's not much of a mystery (you will figure out who is telling the story very quickly), but Nesbit captures the writing style of a kid astonishingly well without making the story poorly written. Descriptions of events have a headlong style that captures a child's sense of adventure and heedless immortality mixed with quiet observations that remind the reader that kids don't miss as much as people think they do.

I think the most skillful part of this book is the way Nesbit captures a kid's disregard of literary convention. The narrator in a book written by an adult tends to fit into a standard choice of story-telling style and follow it consistently. Even first-person narrators who break some of those rules feel like intentionally constructed characters. The Story of the Treasure Seekers is instead half "kid telling a story" and half "kid trying to emulate the way stories are told in books" and tends to veer wildly between the two when the narrator gets excited, as if they're vaguely aware of the conventions they're supposed to be following but are murky on the specifics. It feels exactly like the sort of book a smart and well-read kid would write (with extensive help from an editor).

The other thing that Nesbit handles exceptionally well is the dynamic between the six kids. This is a collection of fairly short stories, so there isn't a lot of room for characterization. The kids are mostly sketched out with one or two memorable quirks. But Nesbit puts a lot of effort into the dynamics that arise between the children in a tight-knit family, properly making the group of kids as a whole and in various combinations a sort of character in their own right. Never for a moment does either the reader or the kids forget that they have siblings. Most adventures involve some process of sorting out who is going to come along and who is going to do other things, and there's a constant but unobtrusive background rhythm of bickering, making up, supporting each other, being frustrated by each other, and getting exasperated at each other's quirks. It's one of the better-written sibling dynamics that I've read.

I somehow managed to miss Nesbit entirely as a kid, probably because she didn't write long series and child me was strongly biased towards books that were part of long series. (One book was at most a pleasant few hours; there needed to be a whole series attached to get any reasonable amount of reading out of the world.) This was nonetheless a fun bit of nostalgia because it was so much like the books I did read: kids finding adventures and making things up, getting into various trouble but getting out of it by being honest and kind, and only occasional and spotty adult supervision. Reading as an adult, I can see the touches of melancholy of loss that Nesbit embeds into this quest for riches, but part of the appeal of the stories is that the kids determinedly refuse to talk about it except as a problem to be solved.

Nesbit was a rather famous progressive, but this is still a book of its time, which means there's one instance of the n-word and the kids have grown up playing the very racist version of cowboys and indians. The narrator also does a lot of stereotyping of boys and girls, although Nesbit undermines that a bit by making Alice a tomboy. I found all of this easier to ignore because the story is narrated by one of the kids who doesn't know any better, but your mileage may vary.

I am always entertained by how anyone worth writing about in a British children's novel of this era has servants. You know the Bastables have fallen upon hard times because they only have one servant. The kids don't have much respect for Eliza, which I found a bit off-putting, and I wondered what this world looks like from her perspective. She clearly did a lot of the work of raising these motherless kids, but the kids view her as the hired help or an obstacle to be avoided, and there's not a lot of gratitude present.

As the stories unfold, it becomes more and more clear that there's a quiet conspiracy of surrounding adults to watch out for these kids, which the kids never notice. This says good things about society, but it does undermine the adventures a little, and by the end of the book the sameness of the stories was wearing a bit thin. The high point of the book is probably chapter eight, in which the kids make their own newspaper, the entirety of which is reproduced in the book and is a note-perfect recreation of what an enterprising group of kids would come up with.

In the last two stories, Nesbit tacks on an ending that was probably obligatory, but which I thought undermined some of the emotional subtext of the rest of the book. I'm not sure how else one could have put an ending on this book, but the ending she chose emphasized the degree to which the adventures really were just play, and the kids are rewarded in these stories for their ethics and their circumstances rather than for anything they concretely do. It's a bit unsatisfying.

This is mostly a nostalgia read, but I'm glad I read it. If this book was not part of your childhood, it's worth reading if only for how well Nesbit captures a child's narrative voice.

Rating: 7 out of 10

Categories: FLOSS Project Planets

Podcast.__init__: Building A Detailed View Of Your Software Delivery Process With The Eiffel Protocol

Planet Python - Sun, 2022-01-30 21:04
Summary

The process of getting software delivered to an environment where users can interact with it requires many steps along the way. In some cases the journey can require a large number of interdependent workflows that need to be orchestrated across technical and organizational boundaries, making it difficult to know what the current status is. Faced with such a complex delivery workflow the engineers at Ericsson created a message based protocol and accompanying tooling to let the various actors in the process provide information about the events that happened across the different stages. In this episode Daniel Ståhl and Magnus Bäck explain how the Eiffel protocol allows you to build a tooling agnostic visibility layer for your software delivery process, letting you answer all of your questions about what is happening between writing a line of code and your users executing it.

Announcements
  • Hello and welcome to Podcast.__init__, the podcast about Python’s role in data and science.
  • When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With the launch of their managed Kubernetes platform it’s easy to get started with the next generation of deployment and scaling, powered by the battle tested Linode platform, including simple pricing, node balancers, 40Gbit networking, dedicated CPU and GPU instances, and worldwide data centers. Go to pythonpodcast.com/linode and get a $100 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show!
  • Your host as usual is Tobias Macey and today I’m interviewing Daniel Ståhl and Magnus Bäck about Eiffel, an open protocol for platform agnostic communication for CI/CD systems
Interview
  • Introductions
  • How did you get introduced to Python?
  • Can you describe what Eiffel is and the story behind it?
    • What are the goals of the Eiffel protocol and ecosystem?
    • What is the role of Python in the Eiffel ecosystem?
  • What are some of the types of questions that someone might ask about their CI/CD workflow?
    • How does Eiffel help to answer those questions?
  • Who are the personas that you would expect to interact with an Eiffel system?
  • Can you describe the core architectural elements required to integrate Eiffel into the software lifecycle?
    • How have the design and goals of the Eiffel protocol/architecture changed or evolved since you first began working on it?
  • What are some example workflows that an engineering/product team might build with Eiffel?
  • What are some of the challenges that teams encounter when integrating Eiffel into their delivery process?
  • What are the most interesting, innovative, or unexpected ways that you have seen Eiffel used?
  • What are the most interesting, unexpected, or challenging lessons that you have learned while working on Eiffel?
  • When is Eiffel the wrong choice?
  • What do you have planned for the future of Eiffel?
Keep In Touch Picks Closing Announcements
  • Thank you for listening! Don’t forget to check out our other show, the Data Engineering Podcast for the latest on modern data management.
  • Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
  • If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@podcastinit.com) with your story.
  • To help other people find the show please leave a review on iTunes and tell your friends and co-workers
Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Categories: FLOSS Project Planets

d-ptr pitfalls

Planet KDE - Sun, 2022-01-30 18:00

It’s been a week of obscure-bug-hunting. A bug report (PR in FreeBSD words) pointed out that KOrganizer was crashing. Since it works on Linux, it falls to the KDE-FreeBSD team to figure it out. A quick session with lldb shows it’s a nullptr-dereference. Since we had exciting nullptr differences before, my first idea is to look for smart pointers. And they’re there, and can lead the unaware into a bog, a fen, a morass of undefined behavior (but it happens to work on Linux).

Symptoms and Implementations

A “smart” pointer manages a chunk of memory it points to. Examples in Qt include QScopedPointer and QSharedPointer. C++ standard examples are std::shared_ptr and std::unique_ptr.

Smart pointers need a particular implementation: the Qt source code implements the Q-flavored ones (in Qt source code, and there’s only one Qt source code), but the standard ones are implemented by multiple standard libraries. There’s GNU libstdc++ and LLVM libcxx, for instance. There are differences in the implementations.

One important difference lies in the implementation of the destructor of a std::unique_ptr. The LLVM implementation replaces an internal pointer by a nullptr and then calls the destructor of the held object, while GNU calls the destructor of the held object and leaves the internal pointer alone (the std::unique_ptr is being destroyed anyway, so why bother updating the pointer-value).

This becomes visible in some situations where a not-completely-destroyed smart pointer is used: with the GNU implementation it may still hold a valid pointer, with LLVM it holds nullptr. Some will crash, some will not – it doesn’t really matter because to get into this situation you need to be in Undefined Behavior territory anyway and you should be glad that your computer doesn’t catch fire, fall over, and then sink into the swamp.

The visible symptom in a backtrace is an unexpectedly nullptr “smart” pointer.

D-ptring

A d-ptr is a technique for maintaining binary-compatibility when a class’ implementation details change. It’s explained fairly well on the Qt wiki.

When the size of (instances of) a class changes, all of its consumers need to be recompiled. Consider this very simple class that describes the state of a Game. It can print the state of the game, too.

struct Game { int score = 0; void print() const { std::cout << "Score=" << score << '\n'; } };

If we now realise we need to keep track of which turn it is in the game, we add a data member – but that changes the size of the instances of the class to accomodate the new data member. Now all the code that relies on class Game needs to be recompiled.

struct Game { int score = 0; int turn = 0; void print() const { std::cout << "Score=" << score << '\n'; } };

To avoid having to recompile consumers, the d-ptr technique moves the implementation details – those pesky data members – to a private class that consumers don’t know about. The Game class now holds a single smart pointer to that private class. Pointers are always the same size, so the Game class won’t change size any more no matter what we do with the implementation.

In Qt-related code, it is common to have a pointer from the internal, private class instance, to the owning instance. It’s just common practice, and this is called the q-ptr (a “q” is just a “d” pointing in a different direction, get it?).

So we introduce a private inner class, and give the Game class a smart pointer to an instance of that. While we’re at it, give the game a method to win some points. This is the last time we need to recompile our consumers.

This example code puts everything in one declaration of class Game and fits in one file. Typically you would hide the implementation detail in a separate translation unit or a .cpp file. So “last time” is a bit of a lie: in a typical realistic implementation this would be the last time.

class Game { struct Private { Game* const q; int score = 0; int turn = 0; Private(Game* owner) : q(owner) {} }; std::unique_ptr<Private> const d = std::make_unique<Private>(this); public: void print() const { std::cout << "Score=" << d->score << '\n'; } void win(int n) { d->score += n; } }; Diagram with d- and q-pointers between classes. The d-pointer is smart, unlike q.

Now we can add data members all we like in the Private class, and it won’t affect the size of Game objects and everything is hunky-dory – except that the stage has been set for the bog monster.

Object Lifetime

In C++, objects have a lifetime. Before an object is alive, it cannot be used. As it lies dying, special rules apply, and once it is dead, it cannot be used. Accessing an object outside of its lifetime (there isn’t an object then!) is Undefined Behavior, and I hear the bog monster has quite shocking teeth.

Here’s a pretty innocuous idea: when the game ends (e.g. it is destroyed), we should print the score as well. There’s already a function implementing this for us, and in the interest of not adding functions to the Game class, let’s use the destructor of the internal Private class to do the work: we’ll just add one line,

~Private() { q->print(); }

See, when the Game destructor is called, the Private destructor is called, and we just print the score. (Narrator: this was not the case)

A cute little bog monster

Depending on the environment your bog monster lives in, it may be a cute one or a monstrous one. Keep in mind that q is a raw pointer, and it points to the owning Game instance, which is “in the process of being destroyed” (not true, more on that later).

We call the Game::print() method, which dereferences the std::unique_ptr to get at the score variable held by the Private instance. The GNU implementation still has a pointer to memory, score is read, and the print() function seems to work.

The LLVM implementation has already set the pointer to null, so the dereference falls over with a SEGV. It doesn’t really matter, since it’s undefined behavior to call print() at all here.

Lifetime Details A fire-breathing bog monster

Thinking about the destruction of Game and all of its data members (including Private, which is held through that std::unique_ptr) as “one action” is conceptually convenient until things go wrong. Then the details of object lifetime matter:

  • The destructor ~Game is called (the destructor starts). At this point the lifetime of the object is over. Dereferencing pointers to this object is undefined behavior from now on. That includes the q pointer held by the Private object. Basically, q is a dangling raw pointer.
  • During the destructor call it is still permitted to call member functions from the destructor (with some caveats), so calling print() here would be ok.
  • Destructors for members are called after the destructor of Game has run – so Game is really really dead by now. Only now does the destructor for the member d run, so that’s how we get to the destructor of Private. There is no Game object any more, so using q is undefined behavior (still).

We can apply the same analysis to the destruction of Private even without looking at Game. Suppose a Private object is owned by some specific std::unique_ptr d, and we call d.reset() to destroy the held object.

  • d.reset() might destroy the held object first, and then change the internal pointer to null, or might do it the other way around, it doesn’t really matter.
  • The destructor ~Private is called. At this point the lifetime of the object is over. Dereferencing pointers to this particular object is undefined behavior from now on. That includes the internal pointer of d.
  • During the destructor call it is still permitted to call member functions (and methods of other objects that are still alive and free functions and everything). So q->print() is legal to call, but ..
  • Dereferencing d in the implementation of print() is either going to dereference an invalid pointer to the Private object whose lifetime has ended, or dereference an invalid null pointer. Either way is Undefined Behavior.
Takeaway

Looking for a pithy rhyming takeaway I can’t get more than

If you Q_Q it, don’t do it.

Which is overly dismissive of Q_Q. Somewhat less pithy:

When d-pointering, the destructor of the Private class must not use the q pointer.

Following that advice avoids both of the scenarios leading to UB sketched in the previous section. It’s easy to overlook, and seems innocuous, until it crawls out of the bog and bites someone (in Kldap, this has been patched).

Categories: FLOSS Project Planets

FSF Blogs: January GNU Spotlight with Mike Gerwitz: Twenty-two new releases

GNU Planet! - Sun, 2022-01-30 16:40
22 new GNU releases in the last month (as of January 27, 2022):
Categories: FLOSS Project Planets

January GNU Spotlight with Mike Gerwitz: Twenty-two new releases

FSF Blogs - Sun, 2022-01-30 16:40
22 new GNU releases in the last month (as of January 27, 2022):
Categories: FLOSS Project Planets

Russell Coker: Links Jan 2022

Planet Debian - Sat, 2022-01-29 21:56

Washington Post has an interesting article on how gender neutral language is developing in different countries [1].

pimaker has an interesting blog post about how they wrote a RISCV CPU emulator to boot a Linux kernel in a pixel shader in the VR Chat platform [2].

ZD has an interesting article about the new Solo Bumblebee platform for writing EBPF programs to run inside the Linux kernel [3]. EBPF is an interesting platform and it’s good to have new tools to help people develop for it.

Big Think has an interesting article about augmented reality suggesting that it could be worse than social media for driving disputes [4]. Some people would want tags of racial status on all people they see.

Vice has an insightful article making the case for 8 hours of work per week [5].

The Guardian has an insightful article about how our attention is being stolen by modern technology [6].

Interesting article about the Ilobleed rootkit that targets the HP ILO server management system [7], it’s apparently designed for persistent attacks as it bypasses the firmware upgrade process and updates only the version number so anyone who thinks that a firmware update will fix it is horribly mislead.

Nick Bostrom wrote an insightful and disturbing article for Aeon titles None of Our Technologies has Managed to Destroy Humanity Yet [8] about the existential risk of new technologies and how they might be mitigated, much of which involves authoritarian governments unlike any we have seen before. As a counterpoint the novel A Deepness in the Sky claims that universal surveillance would be as damaging to the future of a society as planet-buster bombs.

Euronews has an informative article about eco-fascism [9]. The idea that the solution to ecological problems is to have less people in the world and particularly less non-white people is a gateway from green politics to fascism.

The developer of the colors and faker npm libraries (for NodeJS) recently uploaded corrupted versions of those libraries as a protest against companies that use them without paying him [10]. This is the wrong way to approach the issue, but it does demonstrate a serious problem with systems like NPM that allow automatic updates. It gives a better result to just use software packaged by a distribution which has QA checks applied including on security updates.

VentureBeat has an interesting article on the way that AI is taking over jobs [11]. Lots of white collar jobs can be replaced by machine learning systems, we need to plan for the ways this will change the economy.

The Atlantic has an interesting article about tapeworms that infect some ants [12]. The tapeworms make the ants live longer and produce pheromones to make the other ants serve them. This increases the chance that the ants will be eaten by birds which is the next stage in the tapeworm lifecycle.

Related posts:

  1. Links January 2021 Krebs on Security has an informative article about web notifications...
  2. Links September 2020 MD5 cracker, find plain text that matches MD5 hash [1]....
  3. Links April 2021 Dr Justin Lehmiller’s blog post comparing his official (academic style)...
Categories: FLOSS Project Planets

ItsMyCode: How to Add a Title to Seaborn Plots

Planet Python - Sat, 2022-01-29 20:41

There are four different methods to add a title to seaborn plots. Let us explore each of these methods in detail with examples.

Method 1: Using set() method

The set() method is used to add different elements to the plot, and we can use it to add the title by using the title parameter in the case of a single seaborn plot. 

# import pandas library import pandas as pd from pyparsing import line import seaborn as sns import matplotlib.pyplot as plt # create pandas DataFrame df = pd.DataFrame({'team': ['India', 'South Africa', 'New Zealand', 'England'], 'points': [10, 8, 3, 5], 'runrate': [0.5, 1.4, 2, -0.6], 'wins': [5, 4, 2, 2]}) # plot the data frame line_plt = sns.lineplot(data = df) line_plt.set(title = "ICC World Cup Standings") plt.show()

Output

How to Add a Title to Seaborn Plots 5 Method 2: Using set_title() method

The seaborn plot returns an instance of Matplotlib axes, and for such objects, we can use the set_title() method, which accepts one parameter title as an argument.

# import pandas library import pandas as pd from pyparsing import line import seaborn as sns import matplotlib.pyplot as plt # create pandas DataFrame df = pd.DataFrame({'team': ['India', 'South Africa', 'New Zealand', 'England'], 'points': [10, 8, 3, 5], 'runrate': [0.5, 1.4, 2, -0.6], 'wins': [5, 4, 2, 2]}) # plot the data frame line_plt = sns.lineplot(data = df) line_plt.set_title("ICC World Cup Standings") plt.show()

Output

Add a Title to Seaborn Plots Method 3: Using title() method 

We can also leverage matplotlib.pyplot.title() method, which offers a lot of customization such as location, color, font size, etc.

# import pandas library import pandas as pd import seaborn as sns import matplotlib.pyplot as plt # create pandas DataFrame df = pd.DataFrame({'team': ['India', 'South Africa', 'New Zealand', 'England'], 'points': [10, 8, 3, 5], 'runrate': [0.5, 1.4, 2, -0.6], 'wins': [5, 4, 2, 2]}) # plot the data frame line_plt = sns.lineplot(data = df) # set title using matplotlib title plt.title("ICC World Cup Standings") plt.show()

Output

Add a Title to Seaborn Plots Method 4: Using suptitle() method

We can also use suptitle() method to add an overall title to the seaborn plot. This is mainly useful if you have many subplots, but you still need to use one single title to represent the seaborn charts.

import pandas as pd import seaborn as sns import matplotlib.pyplot as plt # create pandas DataFrame df = pd.DataFrame({'wins': [12, 11, 10, 3, 11, 20, 2, 30, 12,7], 'lost': [6, 4, 5, 3, 10, 7, 2, 12, 0, 6], 'team': ['A', 'A', 'A', 'A', 'A', 'B', 'B', 'B', 'B', 'B']}) # plot the data frame rel = sns.relplot(data=df, x='wins', y='lost', col='team') # add overall title rel.fig.suptitle('ICC World Cup Standings') plt.show()

Output

How to Add a Title to Seaborn Plots 6
Categories: FLOSS Project Planets

ItsMyCode: How to Fix: module &#8216;pandas&#8217; has no attribute &#8216;dataframe&#8217;

Planet Python - Sat, 2022-01-29 20:04

We get AttributeError: module ‘pandas’ has no attribute ‘dataframe’ when the Pandas module is unable to resolve and initialize the DataFrame class. The AttributeError usually occurs if the class is not defined in the module or if it has a different name.

In this article, let us take a look at what AttributeError: module ‘pandas’ has no attribute ‘dataframe’ error means and how to resolve this with examples.

AttributeError: module ‘pandas’ has no attribute ‘dataframe’ Solution

There are four main reasons behind this error. 

  1. Ignoring the case of while creating DataFrame
  2. Declaring the module name as a variable name
  3. Naming file as pd.py or pandas.py
  4. Pandas module is not installed
Reason 1 – Ignoring the case of while creating DataFrame

The DataFrame class name is case-sensitive and, it is represented in camel-case, if you are using pd.dataframe() all in lower case then you will get module ‘pandas’ has no attribute ‘dataframe’ as shown below.

# import pandas library import pandas as pd import seaborn as sns import matplotlib.pyplot as plt # create pandas DataFrame df = pd.dataframe({'team': ['India', 'South Africa', 'New Zealand', 'England'], 'points': [10, 8, 3, 5], 'runrate': [0.5, 1.4, 2, -0.6], 'wins': [5, 4, 2, 2]}) print(df)

Output

raceback (most recent call last): File "c:\Personal\IJS\Code\main.py", line 7, in <module> df = pd.dataframe({'team': ['India', 'South Africa', 'New Zealand', 'England'], raise AttributeError(f"module 'pandas' has no attribute '{name}'") AttributeError: module 'pandas' has no attribute 'dataframe'. Did you mean: 'DataFrame'?

We can fix the issue by changing the dataframe to DataFrame(lowercase to camel-case) to create the Pandas DataFrame. When we run the program now, we don’t get any errors and, the code runs successfully.

# import pandas library import pandas as pd import seaborn as sns import matplotlib.pyplot as plt # create pandas DataFrame df = pd.DataFrame({'team': ['India', 'South Africa', 'New Zealand', 'England'], 'points': [10, 8, 3, 5], 'runrate': [0.5, 1.4, 2, -0.6], 'wins': [5, 4, 2, 2]}) print(df)

Output

team points runrate wins 0 India 10 0.5 5 1 South Africa 8 1.4 4 2 New Zealand 3 2.0 2 3 England 5 -0.6 2 Reason 2 – Declaring the module name as a variable name

We get this error if you have named any variable as ‘pd‘ or ‘pandas‘ and try to create the DataFrame. 

# import pandas library import pandas as pd import seaborn as sns import matplotlib.pyplot as plt # variable named as pd pd = [2, 3] # create pandas DataFrame df = pd.DataFrame({'team': ['India', 'South Africa', 'New Zealand', 'England'], 'points': [10, 8, 3, 5], 'runrate': [0.5, 1.4, 2, -0.6], 'wins': [5, 4, 2, 2]}) print(df)

Output

Traceback (most recent call last): File "c:\Personal\IJS\Code\main.py", line 9, in <module> df = pd.DataFrame({'team': ['India', 'South Africa', 'New Zealand', 'England'], AttributeError: 'str' object has no attribute 'DataFrame'

In our above code, we have a local variable named ‘pd‘ and Python will not be able to resolve the DataFrame as it is a local variable and not the Pandas module.

We can resolve this error by renaming the variable to something else.

# import pandas library import pandas as pd import seaborn as sns import matplotlib.pyplot as plt # variable named as pd data = [2, 3] # create pandas DataFrame df = pd.DataFrame({'team': ['India', 'South Africa', 'New Zealand', 'England'], 'points': [10, 8, 3, 5], 'runrate': [0.5, 1.4, 2, -0.6], 'wins': [5, 4, 2, 2]}) print(df)

Output

team points runrate wins 0 India 10 0.5 5 1 South Africa 8 1.4 4 2 New Zealand 3 2.0 2 3 England 5 -0.6 2 Reason 3 – Naming file as pd.py or pandas.py

If you have named the script as pd.py or pandas.py then you will get module ‘pandas’ has no attribute ‘dataframe’ error. 

This mainly happens because the file name will shadow the Pandas module and, it can mess up the module imports.

We can fix this issue by renaming the script to some other name such as “my_script.py”.

Reason 4- Pandas package is not installed

The last reason could be if Pandas library is not installed in the proper Python path. You can go and verify the Lib/site-packages path and see if the Pandas library exists. 

If it’s not present then you can install pandas by running the below command.

pip install pandas

Installation instructions for Anaconda is found here

Categories: FLOSS Project Planets

Armin Ronacher: Uninitialized Memory: Unsafe Rust is Too Hard

Planet Python - Sat, 2022-01-29 19:00

Rust is in many ways not just a modern systems language, but also quite a pragmatic one. It promises safety and provides an entire framework that makes creating safe abstractions possible with minimal to zero runtime overhead. A well known pragmatic solution in the language is an explicit way to opt out of safety by using unsafe. In unsafe blocks anything goes.

If you have read this article before you might be surprised that it looks quite different now. This article in itself was a victim of the author being confused by the rules surrounding unsafe. It has since been changed with an alternative example that better explains the pitfalls. A thank you goes to eddyb who pointed out my mistakes on reddit.

I made the case on Twitter a few days ago that writing unsafe Rust is harder than C or C++, so I figured it might be good to explain what I mean by that.

From C to Rust

So let's start with something simple: we have some struct that we want to initialize with some values. The interesting value here will be the name. It's a pointer to an allocated string. Other than that where it's allocated doesn't matter to us so we keep the struct itself on the stack. The idea is that after the initialization that thing can be passed around safely and printed.

#include <stdio.h> #include <stdlib.h> #include <stdbool.h> struct role { char *name; bool disabled; int flag; }; int main() { struct role r; r.name = strdup("basic"); r.flag = 1; r.disabled = false; printf("%s (%d, %s)\n", r.name, r.flag, r.disabled ? "true" : "false"); free(r.name); }

Now let's write this in Rust. Let's not read the docs too much, let's just do a 1:1 translation to more or less the same but by using unsafe. One note here before you read the code: we're purposefully trying to create an object that looks familiar to Rust programmers and can be seen as public API. So we use a String here instead of a C string so there are some changes to the C code.

use std::mem; struct Role { name: String, disabled: bool, flag: u32, } fn main() { let role = unsafe { let mut role: Role = mem::zeroed(); role.name = "basic".to_string(); role.flag = 1; role.disabled = false; role }; println!("{} ({}, {})", role.name, role.flag, role.disabled); }

So immediately one will ask why unsafe is needed here and the answer is that of course you don't need it here. However this code is also using a suboptimal function: std::mem::zeroed. If you run this on a recent Rust compiler you will get this result:

thread 'main' panicked at 'attempted to zero-initialize type `Role`, which is invalid', src/main.rs:11:30

On older Rust compilers this code will run but it was never really correct. So how do we solve this? The compiler already tells us that we need to use something else:

warning: the type `Role` does not permit zero-initialization --> src/main.rs:11:30 | 11 | let mut role: Role = mem::zeroed(); | ^^^^^^^^^^^^^ | | | this code causes undefined behavior when executed | help: use `MaybeUninit<T>` instead, and only call | `assume_init` after initialization is done |

So why does this type not support zero initialization? What do we have to change? Can zeroed not be used at all? Some of you might think that the answer is #[repr(C)] on the struct to force a C layout but that won't solve the problem. We in fact need to reach for MaybeUninit as the compiler indicates. So let's try that first and then afterwards we figure out why we need it:

use std::mem::MaybeUninit; struct Role { name: String, disabled: bool, flag: u32, } fn main() { let role = unsafe { let mut uninit = MaybeUninit::<Role>::zeroed(); let role = uninit.as_mut_ptr(); (*role).name = "basic".to_string(); (*role).flag = 1; (*role).disabled = false; uninit.assume_init() }; println!("{} ({}, {})", role.name, role.flag, role.disabled); }

By swapping out zeroed for MaybeUninit::zeroed everything changes. We can no longer manipulate our struct directly, we now need to manipulate a raw pointer. Because that raw pointer does not implement deref and because Rust has no -> operator we now need to dereference the pointer permanently to assign the fields with that awkward syntax.

So first of all: does this work now? The answer is yes. But is it correct? The answer is not. But let's see what changed? The answer lies in the fact that any construct like a mutable reference (&mut) or value on the stack in itself (even in unsafe) that would be valid outside of unsafe code still needs to be in a valid state at all times. zeroed returns a zeroed struct and there is no guarantee that this is a valid representation of either the struct or the fields within it. In our case it happens that our String is valid with everything zeroed out but this is not guaranteed and undefined behavior.

One important note is that a mutable reference must also never point to an invalid object, so doing let role = &mut *uninit.as_mut_ptr() if that object is not fully initialized is also wrong.

So let's change from zeroed to uninit. If we run it again we're crashing. So why are we crashing? The answer is that by assigning a string to name we also drop the old string that was there before. We just happened to not encounter this before because Drop happened to be able to deal with a zeroed out string, but we were deep in undefined behavior there. Now how do we solve that? We need to somehow directly write to the pointer there.

So let's just accept that MaybeUninit is necessary and we need to deal with raw references here. It's somewhat cumbersome but it doesn't look too bad. So now we have two new problems: we know that &mut X is not allowed, but *mut X is. How do we get a *mut X without using &mut X first? Ironically until Rust 1.51 it was impossible to construct such a thing without breaking the rules. Today you can use the addr_of_mut! macro. So we can do this:

let name_ptr = std::ptr::addr_of_mut!((*role).name);

Great, so now we have this pointer. How do we write into it? We can use the write method instead:

addr_of_mut!((*role).name).write("basic".to_string());

Are we okay now? Remember how we used a regular struct? If we read the documentation we learn that there are no guarantees of such a struct at all. It turns out that despite what the documentation currently says we can rely on fields being aligned. If however we were dealing with #[repr(packed)] we would have to use write_unaligned instead which is legal if Rust were to pick for a member of the struct to be unaligned. So this could be the final version:

use std::mem::MaybeUninit; use std::ptr::addr_of_mut; struct Role { name: String, disabled: bool, flag: u32, } fn main() { let role = unsafe { let mut uninit = MaybeUninit::<Role>::uninit(); let role = uninit.as_mut_ptr(); addr_of_mut!((*role).name).write("basic".to_string()); (*role).flag = 1; (*role).disabled = false; uninit.assume_init() }; println!("{} ({}, {})", role.name, role.flag, role.disabled); } When to use addr_of_mut!

There are two cases to consider: uninitialized memory and unaligned references. You're not allowed to (even temporarily) create an unaligned reference to something and you're not allowed to create a reference to uninitialized memory. So when are these references created?

If you write (*role).flag = 1; this is fine by Rust rules if the type does not Drop. If it does, then we have more a problem: Drop::drop gets called and it gets called on uninitialized memory. So in that case we need to go via addr_of_mut!. This is why we can directly assign to flag, but we need to go via addr_of_mut! for the name as it is a String.

MaybeUninit

A meta issue is that the understanding of safety changed with time. At one point mem::uninitialized was considered a sound API. At a later point MaybeUninit was added to address the detected short comings. However MaybeUninit in practical terms not ideal because of partially initialized types. While MaybeUninit<T> and T are memory compatible thanks to #[repr(transparent)] this does not work well with nested use.

It's not uncommon that you need to have a MaybeUninit on a field of a struct, but at a later point you want this abstraction not to be there. Actually working with MaybeUninit in practice can be a very challenging experience which this blog post does not sufficiently capture.

Is my Unsafe Correct?

It's 2022 and I will admit that I no longer feel confident writing unsafe Rust code. The rules were probably always complex but I know from reading a lot of unsafe Rust code over many years that most unsafe code just did not care about those rules and just disregarded them. There is a reason that addr_of_mut! did not get added to the language until 1.53. Even today the docs both say there are no guarantees on the alignment on native rust struct reprs.

Over the last few years it seem to have happened that the Rust developers has made writing unsafe Rust harder in practice and the rules are so complex now that it's very hard to understand for a casual programmer and the documentation surrounding it can be easily misinterpreted. An earlier version of this article for instance assumed that some uses of addr_of_mut! were necessary that really were not. And that article got quite a few shares overlooking this before someone pointed that mistake out!

These rules have made one of Rust's best features less and less approachable and also harder to understand. The requirement for the existence MaybeUninit instead of “just” having the old mem::uninitialized API is obvious but shows how complex the rules of the language are.

I don't think this is good. In fact, I believe this is not at all a great trend that fewer and fewer people seem to understand unsafe rust. C interop is a bit part of what made Rust great, and that we're creating such massive barriers should be seen as undesirable. More importantly: the compiler is not helpful in pointing out when I'm doing something wrong.

Making unsafe more ergonomic is a hard problem for sure but it might be worth addressing. Because one thing is clear: people won't be stopping writing unsafe code any time soon.

Categories: FLOSS Project Planets

Kay Hayen: Next Nuitka Live Stream

Planet Python - Sat, 2022-01-29 18:00

Today, Sunday 30.01.2022, there will be the third live stream of me coding on Nuitka, and talking and chatting with visitors in the Discord channel created specifically for this. I will go from 9-12 CEST and from 18 CEST until probably at least 20 CEST, but it seems I tend to go overtime.

Last time

So the last two streams are on my Youtube, these are around 4h videos, done on the same day.

In the second stream I was working on Onefile compression for all Python versions, by making the location of another Python that for Scons we sort of did already, reusable. So now, you can run Nuitka with Python2.6 and compile your legacy code, while with say 3.6 also installed on the same system, you do the zstandard compression. That will be used for other things in the future as well. This is going to be part of the next release and currently on develop.

In the first stream, I also did a little bit of performance plans, but mostly only showing people what I have there in stock, not actually went there and I started work on a upx plugin, such that DLLs for standalone are compressed with that. That also required a bit of plugin interface changes and research. This one works mostly, but will need more live. Also I think I looked at reducing what is included with the follow standard library option, getting very minimal distributions out of it.

This time

All around, this last stream, was a huge success. I was a bit under the weather last weekend, but we go on now.

Not sure yet, what to do. I might be debugging issues I have with 2.6 and a recent optimization, that prevents factory branch from becoming the next pre-release, I might be looking at macOS still. And I might be looking at the caching of bytecode demoted modules, that is kind of ready to be used, but is not currently, which is a pity. And of course, the Python PGO may get a closer look, such that e.g. it works for standalone mode.

How to Join

There is a dedicated page on the web site which has the details. Spoiler, it’s free and I have no plans for anything that involves a subscription of any kind. Of course, talking of subscription, do also checkout the Nuitka commercial offering. That is a subscription with adds that protect your IP even more than regular Nuitka.

Join me

Come and join me there. Instructions here. You know you want to do it. I know I want you to do to it!

Yours, Kay
Categories: FLOSS Project Planets

Brett Cannon: Unravelling ellipsis

Planet Python - Sat, 2022-01-29 17:53

... is the Ellipsis singleton.

Shortest syntactic sugar blog post I have written. &#x1F601;

Categories: FLOSS Project Planets

Abiola Ajadi: Debci- An introduction for beginners!

Planet Debian - Sat, 2022-01-29 17:25

Hello again!

Been a minute! for this blog i will continue from my previous article where i explained Debci you can read more about it here.

In my previous article I mentioned Debci stands for Debian Continous Integration and it exist to make sure packages work currently after an update by testing all of the packages that have tests written in them to make sure it works and nothing is broken. For my internship, I am working on improving the user experience through the UI of the debci site making it easier to use.

Debci consist of the following major terms:
  • Suite
  • Architecture
  • Packages
  • Trigger
How it works together

There are three releases in the active maintenance which are Unstable, Testing, and stable(these are known as the suite). What do they mean? Unstable: This is where active development occurs and packages are initially uploaded. This distribution is always called sid. Testing: After packages undergone some degree of testing in unstable, they are installed into the testing directory. This directory contains packages that have not yet been accepted into the stable release but are on their way there. Stable: The stable distribution includes Debian’s most recent officially released distribution. The current stable release which is Debian 11 is codenamed Bullseye. Also we have the oldstable which is the previous stable release. The Debian 10 is now old stable which was codenamed Buster.

Architectures: These are known as the CPUs achitecture and there are various ports like amd64, arm64, i386 et.c.

An scenerio for example is if a user wants to test a package such as acorn in Testing on arm64 along with a package X from Unstable this would be a pin-package (Pin packages are packages that need to be obtained from a different suite than the main suite that selected.), which means the package the user wants to test with the initial Package selected.Finally, trigger can be described as the name of the test job which is optional. This test is done to check if those packages in unstable can be migrated to Testing.

This is a break down of Debci and I hope you enjoyed learning about what my internship entails. Till next time!

references: Debian releases. Ports

Categories: FLOSS Project Planets

New Year, New Job

Planet KDE - Sat, 2022-01-29 17:00

Next week I’ll be starting a new job after 4 years since my last one. I’m very happy because I’m finally going to work with C++ and Qt; it was not easy to find this kind of job on the Italian market… I will also move from Rome to Florence which should be a nice quality of life upgrade.

My new employer is Develer. Some of you might know it because they organize the annual QtDay. They also sponsor other conferences such as the italian Pycon and they are in general friendly to open source.

In the past few months I had very little time for KDE stuff, hopefully this will change once I settle down in the next few weeks.

Categories: FLOSS Project Planets

ItsMyCode: How to Create a Pie Chart in Seaborn

Planet Python - Sat, 2022-01-29 15:53

We do not have any built-in function to create Pie chart in seaborn, but with the help of Matplotlib, we can create a pie chart and leverage seaborn for color pallets.


A pie chart (or a circle chart) is a circular statistical graphic, which is divided into slices to illustrate numerical proportion. In this article, let us take a look at creating a pie chart using Matplotlib and Seaborn with examples.

Create a simple Pie chart in Seaborn

Let’s say we need to plot a pie chart for sales delivery of cars sold in different locations.

The Pie chart below demonstrates the proportion of numerical sales data for different locations. We represent the sales data in the form of slices to understand it easily.

In the below example, we have used Matplotlib to draw a pie chart, and we are using a pastel color pallet of seaborn. 

The seaborn color_palette() method accepts the color pallet in the string format, and it has a wide variety of color pallets such as dark, bright, pastel, Paired etc.

import matplotlib.pyplot as plt import seaborn as sns # car sales data total_sales = [3000, 2245, 1235, 5330, 4200] location = ['Bangalore', 'Delhi', 'Chennai', 'Mumbai', 'Kolkatta'] # Seaborn color palette to plot pie chart colors = sns.color_palette('pastel') # create pie chart using matplotlib plt.pie(total_sales, labels=location, colors=colors) plt.show()

Output

Simple Pie chart in Seaborn Create an advanced Pie chart in Seaborn

Now that we know how to create a Pie chart using Matplotlib and seaborn, let us explore the advanced features to customize the pie chart.

  • We have used autopct property to set the percentage of sales inside each slice, making it more effective.
  • We have the highest car sales in Mumbai. We can represent it differently by making it a standout slice using explode property.
  • We can rotate the labels in the direction of the slices using the rotatelabel property.
  •  We have changed the color pallet to bright to make the pie chart look more attractive.
import matplotlib.pyplot as plt import seaborn as sns # car sales data total_sales = [3000, 2245, 1235, 5330, 4200] explode = [0, 0, 0, 0.2, 0] location = ['Bangalore', 'Delhi', 'Chennai', 'Mumbai', 'Kolkatta'] # Seaborn color palette to plot pie chart colors = sns.color_palette('bright') # create pie chart using matplotlib plt.pie(total_sales, labels=location, colors=colors, autopct='%.0f%%', explode=explode, shadow=True, rotatelabels='true') plt.show()

Output

Advanced Pie chart in Seaborn
Categories: FLOSS Project Planets

Kate & KTextEditor Bug Fixing

Planet KDE - Sat, 2022-01-29 15:40
Bugs, bugs, bugs…

The Kate & KTextEditor projects have at the moment 189 open bugs.

That is a lot given only a few people work on this projects regularly.

Therefore it would be really appreciated if people could help out with fixing.

If you have spare time and you are interested in help to improve our projects, head over to our long list of bugs.

Some might even be already fixed and just need to be tried out with a recent version. Other might lack ways to reproduce, providing such ways or even just asking the reporter and setting the bug to the NEEDSINFO/WAITINGFORINFO state will help us to keep some better overview.

Contributing to Kate & KTextEditor is easier then ever nowadays.

Head over to our build documentation, with kdesrc-build it is very easy to setup some development environment on any recent Linux/BSD distribution.

We have already a really large number of merge requests that can be used as hints how to actually submit you patches and how the process works. Normally, if you are responsive to our feedback, I would say the patch contribution works really well since we are on our GitLab instance.

Wishes?

Beside actual bugs, we have 128 wish list items lingering around.

If you need some ideas what could be implemented, you can take a look at that list, too.

We have some more high-level collection of ideas to work on our GitLab instance.

Naturally, you might be more interested in contributing patches for your own ideas, that is definitively welcome!

These lists are just collection of potential things you could work on if you have no own ideas but want to help out with feature development for people that can’t implement them on their own.

Rambling of a maintainer about bug reporting

Now, after reaching out for a bit help with the bugs and wishes we already have around, some feedback for the actual bug reporters.

I naturally appreciate that you report bugs and wishes.

Only with such feedback we are able to know what is broken and what is wanted.

This is very important.

Sometimes even I myself just report a bug for some crash or other misbehavior that I can’t directly fix or reproduce, just to have it marked for future review.

And it is totally valid to report wishes you have for our stuff.

But, there is the other side, too, our team.

We work on this in our spare time, no Kate or KTextEditor main contributor was ever paid for that. I guess more or less all current work is done purely as hobby and out of fun or responsibility for the code base.

Implementing new features is fun, normally you add stuff you wants to use yourself, therefore the work on that is gratifying.

On the other side, fixing bugs is often tedious work. Even more, if the bug doesn’t hurt you at all.

Just to pick a random example (we had a input method bug like this, but without any of the negative feedback mentioned at the end, that’s fiction for that bug):

Some input method breaks. I don’t use any complex input method, therefore I am not hurt by that at all. As long as I can type äüöß… I am happy. But yes, naturally, such a breakage is very bad for a large part of the user base.

I personally have a hard time to fix such stuff myself, as I have close to zero experience with it and have not even implemented the current support for that. In such cases if I find the time I try to reach out for people that are more fluent with that and eventually such things then get worked out.

If nobody is found that actually cares for such things or no such person has time, such bugs will stay. No amount of pinging, shouting or similar inside the bug will help. I feel responsible as maintainer, but if I have not the needed skill or time or both nothing will happen. I can not coerce somebody to fix it even if I know somebody that has the needed skills. Nor do I want to.

Over the last years, we had such negative events in not that few bugs. Sometimes it seems like people think we are actually their paid support. No, we are not. If you report some issue and we ignore it, ping us friendly, that can’t hurt. For sure at least I did already just ignore bugs because I didn’t read them well enough or just skipped them completely. Some gentle ping is ok, but some “this is very important and must be directly fixed or else I will never ever use this again” won’t help you.

The same if you want some behavior change or feature and we don’t want to implement this. Let’s say somebody wishes in a bug “I want the option that the text is painted in rainbow colors with some configurable rainbow color scheme”. And then I and others in the team decide this is a feature out of scope for us and we won’t do it. You should respect this. Even more if you actually don’t want to contribute such a behavior change or feature yourself.

Executive summary about bug reporting

Bugs & wishes are very welcome, you help us to improve our projects by reporting them.

But keep in mind, we have not sold you some product with a support contract that will earn us our living.

We provide you with hopefully valuable tools in our spare time and we would appreciate at least a bit respect for that in the communication with us.

Comments?

A matching thread for this can be found here on r/KDE.

Categories: FLOSS Project Planets

Bitflags in Rust

Planet KDE - Sat, 2022-01-29 15:24
Background

While working on Rust bindings for KConfig as a part of Season of KDE 2022, I came across a few problems while trying to represent QFlags in Rust:

  1. Most QFlags are defined as C++ enums in which multiple members can have the same value. This is not possible in Rust enum.
  2. It is possible to enable multiple flags using BitwiseOr. Rust enums cannot do bitwise operations.

This post will guide you through the various implementations I came up with and their tradeoffs.

The C++ enum

The enum I was trying to implement was KConfig::OpenFlags. The enum is given below:

enum OpenFlag { IncludeGlobals = 0x01, ///< Blend kdeglobals into the config object. CascadeConfig = 0x02, ///< Cascade to system-wide config files. SimpleConfig = 0x00, ///< Just a single config file. NoCascade = IncludeGlobals, ///< Include user's globals, but omit system settings. NoGlobals = CascadeConfig, ///< Cascade to system settings, but omit user's globals. FullConfig = IncludeGlobals | CascadeConfig, ///< Fully-fledged config, including globals and cascading to system settings }; Implementation 1: Using Rust modules

This method uses a combination of Rust modules and consants. The sample implementation is as follow:

pub mod OpenFlags { type E = u32; const INCLUDE_GLOBALS: Self::E = 0x01; const CASCADE_CONFIG: Self::E = 0x02; const SIMPLE_CONFIG: Self::E = 0x00; const NO_CASCASE: Self::E = Self::INCLUDE_GLOBALS; const NO_GLOBALS: Self::E = Self::CASCADE_CONFIG; const FULL_CONFIG: Self::E = Self::INCLUDE_GLOBALS | Self::CASCADE_CONFIG; } fn something(flag: OpenFlags::E) {} Advantages
  1. Const is replaced at compile time, so no performance cost.

  2. All values can be documented in the same way using Rust comments.

  3. Multiple flags can be activated.

Drawbacks
  1. Not an enum. Just a collection of constants.
Implementation 2: Using const in Impl

This method defines the problematic members as const in impl. The sample implementation is as follows:

#[repr(C)] pub enum OpenFlags { IncludeGlobals = 0x01, CascadeConfig = 0x02, SimpleConfig = 0x00, FullConfig = 0x01 | 0x02, } #[allow(non_upper_case_globals)] impl OpenFlags { const NoCascade: Self = Self::IncludeGlobals; const NoGlobals: Self = Self::CascadeConfig; } fn something(flag: OpenFlags) {} Advantages
  1. Enum, for the most part.
Drawbacks
  1. Inconsistent documentation. The constants don’t show up as enum variants.
  2. Multiple flags cannot be activated
Implementation 3: Converting standard Rust enums when passing to C++

This method uses standard rust enums. The sample implementation is as follows:

pub enum OpenFlags { IncludeGlobals, CascadeConfig, SimpleConfig, NoCascade, NoGlobals, FullConfig } impl OpenFlags { type E = u32; const INCLUDE_GLOBALS: Self::E = 0x01; const CASCADE_CONFIG: Self::E = 0x02; const SIMPLE_CONFIG: Self::E = 0x00; pub fn to_cpp(&self) -> Self::E { match self { Self::IncludeGlobals => Self::INCLUDE_GLOBALS, Self::CascadeConfig => Self::CASCADE_CONFIG, Self::SimpleConfig => Self::SIMPLE_CONFIG, Self::NoCascade => Self::INCLUDE_GLOBALS, Self::NoGlobals => Self::CASCADE_CONFIG, Self::FullConfig => Self::INCLUDE_GLOBALS | Self::CASCADE_CONFIG, } } } fn something(flag: OpenFlags) { let flag = flag.to_cpp(); ... } Advantages
  1. Completely Enum.

  2. Documentation works as expected.

Drawbacks
  1. Function call every time passing from Rust to C++. I don’t think this will have much performance penalty, but still worth mentioning.

  2. Cannot set multiple flags at once. Eg OpenFlag::IncludeGlobal | OpenFlag::CascadeConfig not possible

Implementation 4: use bitflags crate

This is the implementation that I finally settled on. The implementation is as follows:

use bitflags::bitflags bitflags! { /// Determines how the system-wide and user's global settings will affect the reading of the configuration. /// This is a bitfag. Thus it is possible to pass options like `OpenFlags::INCLUDE_GLOBALS | /// OpenFlags::CASCADE_CONFIG` #[repr(C)] pub struct OpenFlags: u32 { /// Blend kdeglobals into the config object. const INCLUDE_GLOBALS = 0x01; /// Cascade to system-wide config files. const CASCADE_CONFIG = 0x02; /// Just a single config file. const SIMPLE_CONFIG = 0x00; /// Include user's globals, but omit system settings. const NO_CASCADE = Self::INCLUDE_GLOBALS.bits; /// Cascade to system settings, but omit user's globals. const NO_GLOBALS = Self::CASCADE_CONFIG.bits; /// Fully-fledged config, including globals and cascading to system settings. const FULL_CONFIG = Self::INCLUDE_GLOBALS.bits | Self::CASCADE_CONFIG.bits; } } fn something(flag: OpenFlags) {} Advantages
  1. Multiple flags can be used together.

  2. Documentation is consistent.

Drawbacks
  1. Not enum. Shows up as struct in docs.
Documentation Screenshot Conclusion

I think I will be using bitflags for representing all QFlags in kconfig for the foreseeable future.

Categories: FLOSS Project Planets

Andrea Grandi: Using pyenv and pyenv-virtualenv to install Python and create a virtual environment on MacOS

Planet Python - Sat, 2022-01-29 10:00

How to use pyenv and pyenv-virtualenv to install a specific version of Python and create a virtual environment with that version on MacOS

Categories: FLOSS Project Planets

Sylvestre Ledru: An update on rust/coreutils

Planet Debian - Sat, 2022-01-29 06:50

TLDR: we are making progress on the Rust implementation of the GNU coreutils.

Well, it is an understatement to say my previous blog post interested many people. Many articles, blog posts and some podcasts talked about it! As we pushed coreutils 0.0.12 a few days ago and getting closer to the 10 000 stars on github, it is now time to give an update!

This has brought a lot of new contributors to this project. Instead of 30 to 60 patches per month, we jumped to 400 to 472 patches every month. Similarly, we saw an increase in the number of contributors (20 to 50 per month from 3 to 8). Two new maintainers (Michael Debertol & Terts Diepraam) stepped in and have been doing a much better job than myself as reviewers now! As a silly metric, according to github, we had 5 561 clones of the repository over the last 2 weeks!

The new contributors focused on:

  • Performances. Now, some binaries are significantly faster than GNU (ex: head, cut, etc)
  • Adding missing binaries or options (see below)
  • Improve the testsuite: we grew the overall code coverage from 55% to 75% (in general, we consider that a 80% code coverage on a project is excellent).
  • Refactoring the code to simplify the maintenance. Examples:

    • Using the same code for permissions for chgrp and chown
    • Managing error the same way in the various binaries - (Kudos to Jeffrey Finkelstein for the huge work)
    • Improving the GNU compatibility (thanks to Jan Verbeek, Jan Scheer, kimono-koans and many others)
    • Move to clap 3. Upgrade by Terts which unblocks us on various problems.
  • ...

Closing the gap with GNU

As far as I know, we are only missing stty (change and print terminal line settings) as a program.

Thanks to some heroes, basenc, pr, chcon and runcon have been implemented. For example, for the two last programs, Koutheir Attouchi wrote new crates to manage SELinux properly. This crate has been used for some other utilities like cp, ls or id.

Leveraging the GNU testsuite to test this implementation

Because the GNU testsuite is excellent, we now have a proper CI using it to run the tests. It is pretty long on the Github action CI (almost two hours to run it) but it is an amazing improvement to the way we work. It was a joint work from a bunch of folks (James Robson, Roy Ivy III, etc). To achieve this, we also made it easier to run the GNU testsuite locally with the Rust implementation but also to ignore some tests or adjust some error messages (see build-gnu.sh and run-gnu-test.sh).

Following a suggestion of Brian G, a colleague at Mozilla (he did the same for some Firefox major change), we are now collecting the history of fail/pass/error into a separate repository and generating a daily graph showing the evolution of regression.  At this date, we have, with GNU/Coreutils 9.0:

Total 611 tests Pass 214 Skip 84 Fail 298 Error 15

We are now automatically identifying new passing tests and regressions in the CI.

For example:

Warning: Congrats! The gnu test tests/chmod/c-option is now passing!
<br />Warning: Congrats! The gnu test tests/chmod/silent is now passing!
<br />Warning: Congrats! The gnu test tests/chmod/umask-x is now passing!
<br />Error: GNU test failed: tests/du/long-from-unreadable. tests/du/long-from-unreadable is passing on 'master'. Maybe you have to rebase?
[...]
<br />Warning: Changes from master: PASS +4 / FAIL +0 / ERROR -4 / SKIP +0

This is also beneficial to GNU as, by implementing some options, Michael Debertol noticed some incorrect behaviors (with sort and cat) or an uninitialized variable (with chmod).

Documentations

Every day, we are generating the user documentation and of the internal coreutils.

User documentation: https://uutils.github.io/coreutils-docs/user/ Example: ls or cp

The internal documentation can be seen on: https://uutils.github.io/coreutils-docs/dev/uucore/
For example, the backup style is documented here: https://uutils.github.io/coreutils-docs/dev/uucore/backup_control/index.html

More?

Besides my work on Debian/Ubuntu, I have also noticed that more and more operating systems are starting to look at this:

In parallel, https://github.com/uutils/findutils/, a rust dropped-in replacement for find, is getting more attention lately! Here, the graph showing the evolution of the program using the BFS testsuite (much better than GNU's).

What is next?

  1. stty needs to be implemented
  2. Improve the GNU compatibility on key programs and reduce the gap
  3. Investigate how to reduce the size of the binaries
  4. Allow Debian and Ubuntu to switch by default without tricky manipulation

How to help?

I have been maintaining a list of good first bugs for new comers in the repo!

Don't hesitate to contribute, it is much easier than it seems and a terrific way to learn Rust!

 

Categories: FLOSS Project Planets

December/January in KDE Itinerary

Planet KDE - Sat, 2022-01-29 05:45

Since the last summary two month ago KDE Itinerary has had its 21.12 release and is already receiving new features for the next one, such as manually added train and bus trips or new additional information displayed in the timeline.

New Features Manually added train trips

Getting train or bus trips into KDE Itinerary so far was only possible by importing the corresponding ticket, manual editing wasn’t available. This makes sense as in order to function properly we need trip data that actually matches the schedule and contains coordinates or identifiers for efficiently querying for realtime updates.

However that’s a problem for users with some form of flat rate ticket, they might not have a corresponding ticket for each trip. Therefore it will become possible to add train or bus trips from a online journey search, basically as if we’d have an embedded KTrip for this.

To make this work conveniently we needed a location picker, which involves a name-based online search for stations. KPublicTransport provides that since the very beginning, but is does have a problem: which online backend do we query? Without further instructions that will be all 70+ of them, which is very inefficient and will produce poor results.

KTrip addresses this by asking the user to explicitly select an online backend. That works, but given the amount of backends we have this isn’t really ideal. For Itinerary we took a slightly different approach. For the location search you just have to select the corresponding country, and with that information KPublicTransport is now able to automatically pick as few as possible backends to perform the location search.

Train station picker in KDE Itinerary.

Since Itinerary knows the location you will be in at a given time, that is already selected for you correctly in most cases, and a shared location history further speeds up the location search.

Unfortunately there is still one blocking issue, this needs the so far unreleased kirigami-addons module for selecting the departure time. We therefore cannot enable this feature unconditionally yet. If your package/platform has kirigami-addons and you can’t wait to try this, it is accessible in the timeline context menu with development mode enabled.

Public holidays in the timeline

KDE Itinerary can now also show more information in timeline day header elements. Most prominently this is used to show public holidays, at the location you are in on the given day.

KDE Itinerary timeline day header showing a public holiday.

This should help to avoid surprises like the public holiday in the middle of the week during Akademy 2018 :)

Currency conversion rates

Similarly, the timeline location information cards now also contain currency information when entering a country with a different currency than is used in your home country. When enabled this also includes the current conversion rate (off by default, as this requires online access).

Currency information in the itinerary timeline. Infrastructure Work

On Android there is a standardized mechanism to bring up the user’s favorite map application for a given location, using geo: URIs. Those work on Linux too, but so far we couldn’t rely on them, requiring hardcoded alternatives which result in duplicate code and sub-optimal user experiences.

This has two reasons:

  • No graphical user interface for selecting the preferred application to handle geo: URIs, in case there are multiple applications able to handle them.
  • No guarantee that there would be anything able to handle geo: URIs at all.

Both problems have now been addressed. With Plasma 5.24 there the “Default Applications” system settings module will allow to pick the default map application.

Plasma System Settings default geo: URL handler configuration.

And with KDE Frameworks 5.91 there is now always at least a fallback handler for geo: URIs that translates them to corresponding OpenStreetMap URLs and opens those in a web browser. This is not limited to OSM, it only takes a .desktop file to add support for other web-based maps (see MR for examples).

While this will help Itinerary it’s in no way limited to that, similar code is found in KAddressBook or Kalendar as well for example.

Fixes & Improvements Travel document extractor
  • Only consider rectangular strokes for the PDF vector barcode detection. This fixes other vector graphics in e.g. Dutch vaccination certificate PDF documents interfering with vector barcode detection.
  • Make the VIA Rail extractor work both with a full PDF or just the barcode.
  • Add new extractor scripts for Amtrak and Skymark booking emails.
  • Fixes for the SNCF and Vitolus extractor scripts.
Public transport data
  • Make platform comparison consider platform section information. This fixes erroneous platform change warnings when schedule and live data differ in the amount of platform section details only.
  • Fix Navitia location queries when querying for multiple different location types.
  • Support EFA floor level difference attributes. This results in more reliable indoor navigation instructions.
  • Support the alternative Hafas platform format.
Indoor maps
  • Fix station map platforms being selected wrongly without live data.
  • Improved translations of OSM map element details.
  • Improved opening hours parsing for expressions containing common errors or other deviations from the specification, or use localized wording.
Itinerary app
  • Fix issues with wrong trip groupings due to the same city detection getting confused in densely populated areas.
  • Hide the booking section for train and bus reservations when empty. That can happen now that we can create such trips manually.
  • Fix health certificate ordering when facing invalid dates, and remember the last selected one.
  • Improve the display of vaccination times.
Contribute

Feedback and travel document samples are very much welcome, but with travel remaining difficult there are plenty of other things that can be done from home as well. The KDE Itinerary workboard or the more specialized indoor map workboard show what’s on the todo list, and are a good place for collecting new ideas. For questions and suggestions, please feel free to join us on the KDE PIM mailing list or in the #kontact channel on Matrix.

Categories: FLOSS Project Planets

ItsMyCode: Python PermissionError: [Errno 13] Permission denied

Planet Python - Sat, 2022-01-29 04:28

If we provide a folder path instead of a file path while reading file or if Python does not have the required permission to perform file operations(open, read, write), you will encounter PermissionError: [Errno 13] Permission denied error

In this article, we will look at what PermissionError: [Errno 13] Permission denied error means and how to resolve this error with examples.

What is PermissionError: [Errno 13] Permission denied error?

We get this error mainly while performing file operations such as read, write, rename files etc. 

There are three main reasons behind the permission denied error. 

  1. Insufficient privileges on the file or for Python
  2. Passing a folder instead of file
  3. File is already open by other process
How to Fix PermissionError: [Errno 13] Permission denied error?

Let us try to reproduce the “errno 13 permission denied” with the above scenarios and see how to fix them with examples.

Case 1: Insufficient privileges on the file or for Python

Let’s say you have a local CSV file, and it has sensitive information which needs to be protected. You can modify the file permission and ensure that it will be readable only by you.

Now let’s create a Python program to read the file and print its content. 

# Program to read the entire file (absolute path) using read() function file = open("python.txt", "r") content = file.read() print(content) file.close()

Output

Traceback (most recent call last): File "C:/Projects/Tryouts/python.txt", line 2, in <module> file = open("python.txt", "r") PermissionError: [Errno 13] Permission denied: 'python.txt'

When we run the code, we have got  PermissionError: [Errno 13] Permission denied error because the root user creates the file. We are not executing the script in an elevated mode(admin/root).

In windows, we can fix this error by opening the command prompt in administrator mode and executing the Python script to fix the error. The same fix even applies if you are getting “permissionerror winerror 5 access is denied” error

In the case of Linux the issue we can use the sudo command to run the script as a root user.

Alternatively, you can also check the file permission by running the following command.

ls -la # output -rw-rw-rw- 1 root srinivas 46 Jan 29 03:42 python.txt

In the above example, the root user owns the file, and we don’t run Python as a root user, so Python cannot read the file.

We can fix the issue by changing the permission either to a particular user or everyone. Let’s make the file readable and executable by everyone by executing the following command.

chmod 755 python.txt

We can also give permission to specific users instead of making it readable to everyone. We can do this by running the following command.

chown srinivas:admin python.txt

When we run our code back after setting the right permissions, you will get the following output.

Dear User, Welcome to Python Tutorial Have a great learning !!! Cheers Case 2: Providing the file path

In the below example, we have given a folder path instead of a valid file path, and the Python interpreter will raise errno 13 permission denied error.

# Program to read the entire file (absolute path) using read() function file = open("C:\\Projects\\Python\\Docs", "r") content = file.read() print(content) file.close()

Output

Traceback (most recent call last): File "c:\Personal\IJS\Code\program.py", line 2, in <module> file = open("C:\\Projects\\Python\\Docs", "r") PermissionError: [Errno 13] Permission denied: 'C:\\Projects\\Python\\Docs'

We can fix the error by providing the valid file path, and in case we accept the file path dynamically, we can change our code to ensure if the given file path is a valid file and then process it.

# Program to read the entire file (absolute path) using read() function file = open("C:\\Projects\\Python\\Docs\python.txt", "r") content = file.read() print(content) file.close()

Output

Dear User, Welcome to Python Tutorial Have a great learning !!! Cheers Case 3: Ensure file is Closed

While performing file operations in Python, we forget to close the file, and it remains in open mode.

Next time, when we access the file, we will get permission denied error as it’s already in use by the other process, and we did not close the file.

We can fix this error by ensuring by closing a file after performing an i/o operation on the file. You can read the following articles to find out how to read files in Python and how to write files in Python.

Conclusion

In Python, If we provide a folder path instead of a file path while reading a file or if the Python does not have the required permission to perform file operations(open, read, write), you will encounter PermissionError: [Errno 13] Permission denied error.

We can solve this error by Providing the right permissions to the file using chown or chmod commands and also ensuring Python is running in the elevated mode permission.

Categories: FLOSS Project Planets

Pages