Python GUIs: Using Layouts to Position Widgets in PySide6 — Use layouts to effortlessly position widgets within the window (updated for PySide6)

Planet Python - Tue, 2022-01-11 04:00

So far we've successfully created a window, and we've added a widget to it. However we normally want to add more than one widget to a window, and have some control over where it ends up. To do this in Qt we use layouts. There are 4 basic layouts available in Qt, which are listed in the following table.

Layout Behaviour QHBoxLayout Linear horizontal layout QVBoxLayout Linear vertical layout QGridLayout In indexable grid XxY QStackedLayout Stacked (z) in front of one another

You can also design and lay out your interface graphically using the Qt designer. Here we're using code, so you can understand the underlying system.

As you can see, there are three positional layouts available in Qt. The VBoxLayout, QHBoxLayout and QGridLayout. In addition there is also QStackedLayout which allows you to place widgets one on top of the other within the same space, yet showing only one layout at a time.

Before we start we need a simple application outline. Save the following code in a file named app.py -- we'll modify this application to experiment with different layouts.

python import sys from PySide6.QtWidgets import QApplication, QMainWindow, QWidget from PySide6.QtGui import QPalette, QColor class MainWindow(QMainWindow): def __init__(self): super(MainWindow, self).__init__() self.setWindowTitle("My App") app = QApplication(sys.argv) window = MainWindow() window.show() app.exec_()

To make it easier to visualize the layouts, we'll first create a simple custom widget that displays a solid color of our choosing. This will help to distinguish widgets that we add to the layout. Add the following code to your file as a new class at the top level --

python class Color(QWidget): def __init__(self, color): super(Color, self).__init__() self.setAutoFillBackground(True) palette = self.palette() palette.setColor(QPalette.Window, QColor(color)) self.setPalette(palette)

In this code we subclass QWidget to create our own custom widget Color. We accept a single parameter when creating the widget — color (a str). We first set .setAutoFillBackground to True to tell the widget to automatically fill its background with the window cooler. Next we get the current palette (which is the global desktop palette by default) and change the current QPalette.Window color to a new QColor described by the value color we passed in. Finally we apply this palette back to the widget. The end result is a widget that is filled with a solid color, that we specified when we created it.

If you find the above confusing, don't worry too much. We'll cover custom widgets in more detail later. For now it's sufficient that you understand that calling you can create a solid-filled red widget by doing the following:

python Color('red')

First let's test our new Color widget by using it to fill the entire window in a single color. Once it’s complete we can add it to the QMainWindow using .setCentralWidget and we get a solid red window.

python class MainWindow(QMainWindow): def __init__(self): super(MainWindow, self).__init__() self.setWindowTitle("My App") widget = Color('red') self.setCentralWidget(widget)

Run it! The window will appear, filled completely with the color red. Notice how the widget expands to fill all the available space.

Next we'll look at each of the available Qt layouts in turn. Note that to add our layouts to the window we will need a dummy QWidget to hold the layout.

QVBoxLayout vertically arranged widgets

With QVBoxLayout you arrange widgets one above the other linearly. Adding a widget adds it to the bottom of the column.

A QVBoxLayout, filled from top to bottom.

Let’s add our widget to a layout. Note that in order to add a layout to the QMainWindow we need to apply it to a dummy QWidget. This allows us to then use .setCentralWidget to apply the widget (and the layout) to the window. Our colored widgets will arrange themselves in the layout, contained within the QWidget in the window. First we just add the red widget as before.

python class MainWindow(QMainWindow): def __init__(self): super(MainWindow, self).__init__() self.setWindowTitle("My App") layout = QVBoxLayout() layout.addWidget(Color('red')) widget = QWidget() widget.setLayout(layout) self.setCentralWidget(widget)

Run it! Notice the border now visible around the red widget. This is the layout spacing — we'll see how to adjust that later.

If you add a few more colored widgets to the layout you’ll notice that they line themselves up vertical in the order they are added.

python class MainWindow(QMainWindow): def __init__(self): super(MainWindow, self).__init__() self.setWindowTitle("My App") layout = QVBoxLayout() layout.addWidget(Color('red')) layout.addWidget(Color('green')) layout.addWidget(Color('blue')) widget = QWidget() widget.setLayout(layout) self.setCentralWidget(widget) QHBoxLayout horizontally arranged widgets

QHBoxLayout is the same, except moving horizontally. Adding a widget adds it to the right hand side.

A QHBoxLayout, filled from left to right.

To use it we can simply change the QVBoxLayout to a QHBoxLayout. The boxes now flow left to right.

python class MainWindow(QMainWindow): def __init__(self): super(MainWindow, self).__init__() self.setWindowTitle("My App") layout = QHBoxLayout() layout.addWidget(Color('red')) layout.addWidget(Color('green')) layout.addWidget(Color('blue')) widget = QWidget() widget.setLayout(layout) self.setCentralWidget(widget) Nesting layouts

For more complex layouts you can nest layouts inside one another using .addLayout on a layout. Below we add a QVBoxLayout into the main QHBoxLayout. If we add some widgets to the QVBoxLayout, they’ll be arranged vertically in the first slot of the parent layout.

python class MainWindow(QMainWindow): def __init__(self): super(MainWindow, self).__init__() self.setWindowTitle("My App") layout1 = QHBoxLayout() layout2 = QVBoxLayout() layout3 = QVBoxLayout() layout2.addWidget(Color('red')) layout2.addWidget(Color('yellow')) layout2.addWidget(Color('purple')) layout1.addLayout( layout2 ) layout1.addWidget(Color('green')) layout3.addWidget(Color('red')) layout3.addWidget(Color('purple')) layout1.addLayout( layout3 ) widget = QWidget() widget.setLayout(layout1) self.setCentralWidget(widget)

Run it! The widgets should arrange themselves in 3 columns horizontally, with the first column also containing 3 widgets stacked vertically. Experiment!

You can set the spacing around the layout using .setContentMargins or set the spacing between elements using .setSpacing.

python layout1.setContentsMargins(0,0,0,0) layout1.setSpacing(20)

The following code shows the combination of nested widgets and layout margins and spacing. Experiment with the numbers til you get a feel for them.

python class MainWindow(QMainWindow): def __init__(self): super(MainWindow, self).__init__() self.setWindowTitle("My App") layout1 = QHBoxLayout() layout2 = QVBoxLayout() layout3 = QVBoxLayout() layout1.setContentsMargins(0,0,0,0) layout1.setSpacing(20) layout2.addWidget(Color('red')) layout2.addWidget(Color('yellow')) layout2.addWidget(Color('purple')) layout1.addLayout( layout2 ) layout1.addWidget(Color('green')) layout3.addWidget(Color('red')) layout3.addWidget(Color('purple')) layout1.addLayout( layout3 ) widget = QWidget() widget.setLayout(layout1) self.setCentralWidget(widget) QGridLayout widgets arranged in a grid

As useful as they are, if you try and using QVBoxLayout and QHBoxLayout for laying out multiple elements, e.g. for a form, you’ll find it very difficult to ensure differently sized widgets line up. The solution to this is QGridLayout.

A QGridLayout showing the grid positions for each location.

QGridLayout allows you to position items specifically in a grid. You specify row and column positions for each widget. You can skip elements, and they will be left empty.

Usefully, for QGridLayout you don't need to fill all the positions in the grid.

A QGridLayout with unfilled slots.

python class MainWindow(QMainWindow): def __init__(self): super(MainWindow, self).__init__() self.setWindowTitle("My App") layout = QGridLayout() layout.addWidget(Color('red'), 0, 0) layout.addWidget(Color('green'), 1, 0) layout.addWidget(Color('blue'), 1, 1) layout.addWidget(Color('purple'), 2, 1) widget = QWidget() widget.setLayout(layout) self.setCentralWidget(widget) QStackedLayout multiple widgets in the same space

The final layout we’ll cover is the QStackedLayout. As described, this layout allows you to position elements directly in front of one another. You can then select which widget you want to show. You could use this for drawing layers in a graphics application, or for imitating a tab-like interface. Note there is also QStackedWidget which is a container widget that works in exactly the same way. This is useful if you want to add a stack directly to a QMainWindow with .setCentralWidget.

QStackedLayout — in use only the uppermost widget is visible, which is by default the first widget added to the layout.

QStackedLayout, with the 2nd (1) widget selected and brought to the front.

python from PySide6.QtWidgets import QStackedLayout # add this import class MainWindow(QMainWindow): def __init__(self): super().__init__() self.setWindowTitle("My App") layout = QStackedLayout() layout.addWidget(Color("red")) layout.addWidget(Color("green")) layout.addWidget(Color("blue")) layout.addWidget(Color("yellow")) layout.setCurrentIndex(3) widget = QWidget() widget.setLayout(layout) self.setCentralWidget(widget)

QStackedWidget is exactly how tabbed views in applications work. Only one view ('tab') is visible at any one time. You can control which widget to show at any time by using .setCurrentIndex() or .setCurrentWidget() to set the item by either the index (in order the widgets were added) or by the widget itself.

Below is a short demo using QStackedLayout in combination with QButton to to provide a tab-like interface to an application:

python import sys from PySide6.QtCore import Qt from PySide6.QtWidgets import ( QApplication, QHBoxLayout, QLabel, QMainWindow, QPushButton, QStackedLayout, QVBoxLayout, QWidget, ) from layout_colorwidget import Color class MainWindow(QMainWindow): def __init__(self): super().__init__() self.setWindowTitle("My App") pagelayout = QVBoxLayout() button_layout = QHBoxLayout() self.stacklayout = QStackedLayout() pagelayout.addLayout(button_layout) pagelayout.addLayout(self.stacklayout) btn = QPushButton("red") btn.pressed.connect(self.activate_tab_1) button_layout.addWidget(btn) self.stacklayout.addWidget(Color("red")) btn = QPushButton("green") btn.pressed.connect(self.activate_tab_2) button_layout.addWidget(btn) self.stacklayout.addWidget(Color("green")) btn = QPushButton("yellow") btn.pressed.connect(self.activate_tab_3) button_layout.addWidget(btn) self.stacklayout.addWidget(Color("yellow")) widget = QWidget() widget.setLayout(pagelayout) self.setCentralWidget(widget) def activate_tab_1(self): self.stacklayout.setCurrentIndex(0) def activate_tab_2(self): self.stacklayout.setCurrentIndex(1) def activate_tab_3(self): self.stacklayout.setCurrentIndex(2) app = QApplication(sys.argv) window = MainWindow() window.show() app.exec_()

A custom tab-like interface implemented using QStackedLayout.

Helpfully. Qt actually provide a built-in TabWidget that provides this kind of layout out of the box - albeit in widget form. Below the tab demo is recreated using QTabWidget:

python import sys from PySide6.QtCore import Qt from PySide6.QtWidgets import ( QApplication, QLabel, QMainWindow, QPushButton, QTabWidget, QWidget, ) from layout_colorwidget import Color class MainWindow(QMainWindow): def __init__(self): super().__init__() self.setWindowTitle("My App") tabs = QTabWidget() tabs.setTabPosition(QTabWidget.West) tabs.setMovable(True) for n, color in enumerate(["red", "green", "blue", "yellow"]): tabs.addTab(Color(color), color) self.setCentralWidget(tabs) app = QApplication(sys.argv) window = MainWindow() window.show() app.exec_()

A tabbed interface using the QTabWidget.

As you can see, it's a little more straightforward — and a bit more attractive! You can set the position of the tabs using the cardinal directions, toggle whether tabs are moveable with .setMoveable. You'll notice that the macOS tab bar looks quite different to the others -- by default on macOS tabs take on a pill or bubble style. On macOS this is typically used for tabbed configuration panels. For documents, you can turn on document mode to give slimline tabs similar to what you see on other platforms. This option has no effect on other platforms.

python tabs = QTabWidget() tabs.setDocumentMode(True)

QTabWidget in document mode on macOS.

We'll encounter more of these advanced widgets later.

For an in-depth guide to building Python GUIs with PySide6 see my book, Create GUI Applications with Python & Qt6.

Categories: FLOSS Project Planets

Russ Allbery: Review: Hench

Planet Debian - Mon, 2022-01-10 21:56

Review: Hench, by Natalie Zina Walschots

Publisher: William Morrow Copyright: September 2020 ISBN: 0-06-297859-4 Format: Kindle Pages: 403

Anna Tromedlov is a hench, which means she does boring things for terrible people for money. Supervillains need a lot of labor to keep their bases and criminal organizations running, and they get that labor the same way everyone else does: through temporary agencies. Anna does spreadsheets, preferably from home on her couch.

On-site work was terrifying and she tried to avoid it, but the lure of a long-term contract was too strong. The Electric Eel, despite being a creepy sleazeball, seemed to be a manageable problem. He needed some support at a press conference, which turns out to be code for being a diversity token in front of the camera, but all she should have to do is stand there.

That's how Anna ended up holding the mind control device to the head of the mayor's kid when the superheroes attack, followed shortly by being thrown across the room by Supercollider.

Left with a complex fracture of her leg that will take months to heal, a layoff notice and a fruit basket from Electric Eel's company, and a vaguely menacing hospital conversation with the police (including Supercollider in a transparent disguise) in which it's made clear to her that she is mistaken about Supercollider's hand-print on her thigh, Anna starts wondering just how much damage superheroes have done. The answer, when analyzed using the framework for natural disasters, is astonishingly high. Anna's resulting obsession with adding up the numbers leads to her starting a blog, the Injury Report, with a growing cult following. That, in turn, leads to a new job and a sponsor: the mysterious supervillain Leviathan.

To review this book properly, I need to talk about Watchmen.

One of the things that makes superheroes interesting culturally is the straightforwardness of their foundational appeal. The archetypal superhero story is an id story: an almost pure power fantasy aimed at teenage boys. Like other pulp mass media, they reflect the prevailing cultural myths of the era in which they're told. World War II superheroes are mostly all-American boy scouts who punch Nazis. 1960s superheroes are a more complex mix of outsider misfits with a moral code and sarcastic but earnestly ethical do-gooders. The superhero genre is vast, with numerous reinterpretations, deconstructions, and alternate perspectives, but its ur-story is a good versus evil struggle of individual action, in which exceptional people use their powers for good to defeat nefarious villains.

Watchmen was not the first internal critique of the genre, but it was the one that everyone read in the 1980s and 1990s. It takes direct aim at that moral binary. The superheroes in Watchmen are not paragons of virtue (some of them are truly horrible people), and they have just as much messy entanglement with the world as the rest of us. It was superheroes re-imagined for the post-Vietnam, post-Watergate era, for the end of the Cold War when we were realizing how many lies about morality we had been told. But it was still put superheroes and their struggles with morality at the center of the story.

Hench is a superhero story for the modern neoliberal world of reality TV and power inequality in the way that Watchmen was a superhero story for the Iran-Contra era and the end of the Cold War.

Whether our heroes have feet of clay is no longer a question. Today, a better question is whether the official heroes, the ones that are celebrated as triumphs of individual achievement, are anything but clay. Hench doesn't bother asking whether superheroes have fallen short of their ideal; that answer is obvious. What Hench asks instead is a question familiar to those living in a world full of televangelists, climate denialism, manipulative advertising, and Facebook: are superheroes anything more than a self-perpetuating scam? Has the good superheroes supposedly do ever outweighed the collateral damage? Do they care in the slightest about the people they're supposedly protecting? Or is the whole system of superheroes and supervillains a performance for an audience, one that chews up bystanders and spits them out mangled while delivering simplistic and unquestioned official morality?

This sounds like a deeply cynical premise, but Hench is not a cynical book. It is cynical about superheroes, which is not the same thing. The brilliance of Walschots's approach is that Anna has a foot in both worlds. She works for a supervillain and, over the course of the book, gains access to real power within the world of superheroic battles. But she's also an ordinary person with ordinary problems: not enough money, rocky friendships, deep anger at the injustices of the world and the way people like her are discarded, and now a disability and PTSD. Walschots perfectly balances the tension between those worlds and maintains that tension straight to the end of the book. From the supervillain world, Anna draws support, resources, and a mission, but all of the hope, true morality, and heart of this book comes from the ordinary side.

If you had the infrastructure of a supervillain at your disposal, what would you do with it?

Anna's answer is to treat superheroes as a destructive force like climate change, and to do whatever she can to drive them out of the business and thus reduce their impact on the world. The tool she uses for that is psychological warfare: make them so miserable that they'll snap and do something too catastrophic to be covered up. And the raw material for that psychological warfare is data.

That's the foot in the supervillain world. In descriptions of this book, her skills with data are often called her superpower. That's not exactly wrong, but the reason why she gains power and respect is only partly because of her data skills. Anna lives by the morality of the ordinary people world: you look out for your friends, you treat your co-workers with respect as long as they're not assholes, and you try to make life a bit better for the people around you. When Leviathan gives her the opportunity to put together a team, she finds people with skills she admires, funnels work to people who are good at it, and worries about the team dynamics. She treats the other ordinary employees of a supervillain as people, with lives and personalities and emotions and worth. She wins their respect.

Then she uses their combined skills to destroy superhero lives.

I was fascinated by the moral complexity in this book. Anna and her team do villainous things by the morality of the superheroic world (and, honestly, by the morality of most readers), including some things that result in people's deaths. By the end of the book, one could argue that Anna has been driven by revenge into becoming an unusual sort of supervillain. And yet, she treats the people around her so much better than either the heroes or the villains do. Anna is fiercely moral in all the ordinary person ways, and that leads directly to her becoming a villain in the superhero frame. Hench doesn't resolve that conflict; it just leaves it on the page for the reader to ponder.

The best part about this book is that it's absurdly grabby, unpredictable, and full of narrative momentum. Walschots's pacing kept me up past midnight a couple of times and derailed other weekend plans so that I could keep reading. I had no idea where the plot was going even at the 80% mark. The ending is ambiguous and a bit uncomfortable, just like the morality throughout the book, but I liked it the more I thought about it.

One caveat, unfortunately: Hench has some very graphic descriptions of violence and medical procedures, and there's an extended torture sequence with some incredibly gruesome body horror that I thought went on far too long and was unnecessary to the plot. If you're a bit squeamish like I am, there are some places where you'll want to skim, including one sequence that's annoyingly intermixed with important story developments.

Otherwise, though, this is a truly excellent book. It has a memorable protagonist with a great first-person voice, an epic character arc of empowerment and revenge, a timely take on the superhero genre that uses it for sharp critique of neoliberal governance and reality TV morality, a fascinatingly ambiguous and unsettled moral stance, a gripping and unpredictable plot, and some thoroughly enjoyable competence porn. I had put off reading it because I was worried that it would be too cynical or dark, but apart from the unnecessary torture scene, it's not at all. Highly recommended.

Rating: 9 out of 10

Categories: FLOSS Project Planets

Codementor: One-Hot Encoding in Data Science

Planet Python - Mon, 2022-01-10 20:06
What is One-Hot Encoding in Data Science? and How to implement it in Python using Pandas or Scikit-Learn.
Categories: FLOSS Project Planets

Event Organizers: A new chapter for Drupal Event Organizers. Thank You All!

Planet Drupal - Mon, 2022-01-10 19:17

Over the past three years, it has been a privilege to work alongside some of the most fun and dedicated groups of people that I have ever been around. To meet every month with other Drupal event planners from around the world has been the personal and professional support I desperately needed during our world's most challenging times, and for that, I can’t say thank you enough. 

During our short tenure we accomplished some important foundational objectives, but what stands out for me personally are the three initiatives that we decided on as a strategic focus.

These initiatives are: 

We would like to thank everyone who helped contribute over the past few years. These individuals' terms have ended and we could not be more grateful for their commitments.

2019 - 2021 Term
  • Baddý Sonja Breidert - DrupalCamp Iceland, Germany, Europe, Splash Awards - Europe
  • Kaleem Clarkson - DrupalCamp Atlanta, USA
  • Suzanne Dergacheva - DrupalNorth - Montreal, QC CANADA
  • Andrii Podanenko - DrupalCamp Kiev - Ukraine, Europe
A New Chapter for Drupal Event Organizers.

In October of 2021, the Event Organizers Working group solicited nominations for board members to take this important working group to the next level in supporting event organizers around the world.

Please welcome the next event organizers working group board of directors:

  • Leslie Glynn - Design 4 Drupal Boston, NEDCamp
  • Surabhi Gokte - Drupal India Association
  • Lenny Moskalyk - DrupalCamp Kyiv
  • John Picozzi - New England Drupal Camp (NEDCamp)
  • Matthew Saunders - DrupalCamp Colorado
  • Avi Schwab - MidCamp, Midwest Open Source Alliance
  • Sean Walsh - DrupalCamp NJ

Are you a Drupal event organizer or are looking to plan a Drupal event?

Did you know you can join other event organizers for our monthly open meeting?

Let’s see you at the next meetings:

  • Tuesday, Jan 11 @ 9:00 am ET
  • Tuesday, Feb 8 @ 9:00 am ET
View Meeting Information

ps - Always promoting :). You can catch me trying to help push the Drupal Event Platform Initiative. - Kaleem

Categories: FLOSS Project Planets

The importance of window to desktop file mapping

Planet KDE - Mon, 2022-01-10 19:00

In my last post I talked about what application developers can do to fix their applications not showing up correctly in Plasma’s task manager. I motivated this by the fact that we need this on Wayland to display the app’s icon. However icons are not the only reason why correctly mapping windows to desktop files is important. It brings substantial benefits even on X11:

  • Application titles: In addition to showing the window title Plasma’s task manager also shows the application name (the Name key in the desktop file). If the desktop file can’t be determined it falls back to the executable name, which isn’t particularly nice.
  • Jumplist actions: Desktop files allow apps to specify additional application actions. For example Firefox allows you to open a new window or a new private browsing window by right-clicking on the entry in the task manager.
  • Media player controls: Plasma’s task manager allows to control application’s media playback. You can e.g. pause a music player by right-clicking on its task manager entry. For this to work the window has to be matched to the MPRIS player instance, which happens based on the desktop file name.
  • Recent files: Plasma’s task manager shows you recently used files for an application in the context menu for an application and allows to open that file in the app. This also relies on the desktop file mapping.

Now that we established why it is important to map a window to a desktop file, how is it done?

On Wayland the xdg-shell protocol, which is responsible for application windows, has builtin support for passing a desktop file name in form of set_app_id.

On X11, it’s more complicated.

For Qt applications the plasma-integration Qt Platform Theme sets a KDE-specific window property that contains the desktop file name. The task manager reads this property and handles it accordingly.

GTK apps have a very similar window property, named _GTK_APPLICATION_ID. However, until now Plasma did not use this information at all! Beginning with Plasma 5.25 the task manager will take _GTK_APPLICATION_ID into account, which fixes matching Gedit and other apps.

When neither _KDE_NET_WM_DESKTOP_FILE nor _GTK_APPLICATION_ID are set the task manager will use a wild guessing game heuristic to try and match the window to a desktop file. It takes into account things like the X11 window class, executable name, and Name and Exec from the desktop file. Sometimes this works well, sometimes it doesn’t.

These are just some examples why this mapping is so important. In a future post I will talk about an improvement to Plasma’s task manager in Plasma 5.24 that also requires this mapping to work.

Categories: FLOSS Project Planets

Talking Drupal: Talking Drupal #329 - The Penguin Corps

Planet Drupal - Mon, 2022-01-10 14:00

Today we are talking about The Penguin Corps with Stu Keroff and Students from the Penguin Corps.


  • Stephen - AZ trip
  • Nic - Computer build
  • Stu - Back to school
  • Favorite things
  • Rania Grade 7
    • Walking up and down stairs 10 times to get to sleep
  • Michael - Grade 7
    • Sports, Basketball or Swim
  • Cam - Grade 7
    • Working on cars, 1986 Ford Mustang
  • Geoffrey - Grade 6
    • Soccer
  • Nithya - Grade 6
    • Reading
  • Penguin Corps
  • How it got started
  • Getting support
  • Why Linux
  • Computers in the classroom
  • Importance
  • Digital Divide
  • Hardware
  • Donations
  • Beyond the classroom
  • Corporate support
Resources Guests

Stu Keroff - @studoeslinux Rania Michael Cam Geoffrey Nithya


Nic Laflin - www.nLighteneddevelopment.com @nicxvan John Picozzi - www.epam.com @johnpicozzi Stephen Cross - @stephencross

Categories: FLOSS Project Planets

Inspired Python: Solving Wordle Puzzles with Basic Python

Planet Python - Mon, 2022-01-10 10:38
Solving Wordle Puzzles with Basic Python

Have you heard of Wordle? It’s a deceptively simple word puzzle. You’re asked to guess the word of the day, which is a five-letter word in English. If you guess wrong, you’re given a few hints: a letter in the word is green if your guess for that letter in that position is right; a yellow letter if that letter is present in the word, but not that position; and gray, if the letter is not in the word at all.

Deceptively simple, and yet quite challenging! Here’s how you can write a Wordle Solver with Python sets, list comprehensions, a bit of good luck!

Categories: FLOSS Project Planets

Ask A KDE Dev Anything - TEST!

Planet KDE - Mon, 2022-01-10 09:55
Yo, come at me and ask me stuff! I'll use this steram to check if things work or not.
Categories: FLOSS Project Planets

Real Python: Build and Handle POST Requests in Django – Part 3

Planet Python - Mon, 2022-01-10 09:00

In this four-part tutorial series, you’re building a social network with Django that you can showcase in your portfolio. This project is strengthening and demonstrating your understanding of relationships between Django models and showing you how to use forms so that users can interact with your app and with each other. You’re also making your site look good by using the CSS framework Bulma.

In the previous part of this tutorial series, you built templates and views for displaying a list of all profiles, as well as individual profile pages. You also learned how to apply style rules defined by the CSS framework Bulma to make your pages look good.

In the third part of this tutorial series, you’ll learn how to:

  • Create the front-end interface to let users follow and unfollow profiles
  • Submit and handle a POST request in Django using buttons
  • Set up the model for your text-based content
  • Build styled templates to display content on the front end
  • Use intricate model relationships in template code

At the end of this part, your social network will have all the pages you’d expect, and you’ll be able to create dweets on the back end and display them on the front end. Your users will be able to discover and follow other users and read the contents of the profiles that they follow. They’ll also be able to click a button that sends an HTTP POST request handled by Django to unfollow a profile if they want to stop reading a certain user’s content.

You can download the code that you’ll need to start the third part of this project by clicking the link below and going to the source_code_start/ folder:

Get Source Code: Click here to get the source code you’ll use to build and handle POST requests with Django.


In this four-part tutorial series, you’re building a small social network that allows users to post short text-based messages. The users of your app can also follow other user profiles to see the posts of those users or unfollow them to stop seeing their text-based posts:

You’re also learning how to use the CSS framework Bulma to give your app a user-friendly appearance and make it a portfolio project you can be proud to show off!

In the third part of this tutorial series, you’ll continue working with Bulma-styled templates, and you’ll create the model for your text-based message content. You’ll also set up and handle HTTP POST request submissions on your Django app’s front end, which will allow users to follow or unfollow each other by clicking a button:

At the end of this tutorial, each user will be able to go to a new dashboard page to access a personal feed of dweets that’s based on which profiles they follow. They’ll also be able to follow and unfollow other users. That’s a giant leap ahead for your Django social network!

Project Overview

In this section, you’ll get an overview of what topics you’ll cover in this third part of the tutorial series. You’ll also get a chance to revisit the full project implementation steps, in case you need to skip back to a previous step from earlier in the series or if you want to see what’s still up ahead.

At this point, you should have finished working through parts one and two of this tutorial series. If you did, then you’re now ready to continue with the next steps of this tutorial series. These steps focus on the code logic for following and unfollowing profiles, as well as how to set up dweets:

    Step 7 Follow and Unfollow Other Profiles Step 8 Create the Back-End Logic For Dweets Step 9 Display Dweets on the Front End

After completing all the steps in the third part of the tutorial series, you can continue with part four.

To get a high-level idea of how you’re working through all four parts of this tutorial series on building your Django social network, you can expand the collapsible section below:

Full Project Implementation StepsShow/Hide

You’re implementing the project in a number of steps spread out over multiple separate tutorials in this series. There’s a lot to cover, and you’re going into detail along the way:

✅ Part 1: Models and Relationships

  • Step 1: Set Up the Base Project
  • Step 2: Extend the Django User Model
  • Step 3: Implement a Post-Save Hook

✅ Part 2: Templates and Front-End Styling

  • Step 4: Create a Base Template With Bulma
  • Step 5: List All User Profiles
  • Step 6: Access Individual Profile Pages

📍 Part 3: Follows and Dweets

  • Step 7: Follow and Unfollow Other Profiles
  • Step 8: Create the Back-End Logic For Dweets
  • Step 9: Display Dweets on the Front End

⏭ Part 4: Forms and Submissions

  • Step 10: Submit Dweets Through a Django Form
  • Step 11: Prevent Double Submissions and Handle Errors
  • Step 12: Improve the Front-End User Experience

Each of these steps will provide links to any necessary resources. By approaching the steps one at a time, you’ll have the opportunity to pause and come back at a later point in case you want to take a break.

With the high-level structure of this tutorial series in mind, you’ve got a good idea of where you’re at and which implementation steps you’ll handle in the last part.

Before getting started with the next step, take a quick look at the prerequisites to skim any links to other resources that might be helpful along the way.


To successfully work through this part of your project, you need to have completed the first part on models and relationships and the second part on templates and styling, and you should confirm that your project is working as described there. It would be best if you’re also comfortable with the following concepts:

Read the full article at https://realpython.com/django-social-post-3/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Categories: FLOSS Project Planets

David Amos: 5 Ways To Use Python On An iPad

Planet Python - Mon, 2022-01-10 07:00

When Apple released the M1 iPad Pros in March of 2021, I traded in my MacBook Pro to get the latest tablet. I’d already been using a mac Mini as my daily workhorse, so I wasn’t concerned about my day-to-day coding workflow. But I was very curious to know what coding on the iPad looks like, and if a full-featured professional coding set-up was even possible.

While a native Python IDE experience is still unavailable for iPadOS — and might never be — it turns out that it’s actually pretty easy to code in Python on the iPad, especialy if you’re willing to work in Jupyter Notebooks. You don’t even need an iPad Pro!

Here are five ways you can code in Python on any iPad right now.

1. a-Shell

My goto app for using Python on the iPad is Nicolas Holzschuch’s fantastic a-Shell app. a-Shell gives you a Unix-style terminal on your iPad or iPhone and it’s completely free. It’s also quite powerful.

Click here to open a-Shell in the Apple Store \(\rightarrow\)

Once you install and open a-Shell, type help to get a quick overview of the app:

a-Shell comes with vim and ed for editing files, and it includes Python 3.9 out-of-the box. Here’s what editing a Python file in vim looks like:

Very nice!

💡Tip: If you use Apple’s Magic Keyboard for iPad, then you don’t have a physical Esc key. This makes working in vim painful until you figure out that Cmd + . works like Esc. If you&aposd like, you can change Caps Lock to work as Esc in a-Shell’s settings.

You can also map the globe key on the Magic Keyboard to function as Esc in the Settings app.

a-Shell plays nicely with iPadOS 15’s multi-tasking features. You can open new windows, put a-Shell side-by-side with another app, and — my favorite — use a-Shell in slideover mode.

I do a lot of reading on my iPad. When I come across something that I want to check in the Python REPL, it’s incredibly helpful to be able to swipe in from the right-hand-side of my iPad, quickly check something in the REPL, and then dismiss the app by swiping right:

You can install Python packages using pip in a-Shell as long as those package are pure Python. This is admittedly a serious limitation for a lot of folks, but it does allow you to install some pretty awesome packages — including Will McGugan’s awesome rich library:

Besides being a great way to use Python on your iPad, a-Shell has a lot of other useful features. You can navigate your iPad’s file system, transfer files using curl, generate SSH keys, SSH into remote servers, and more. You can even write programs in C and C++ and run them on your iPad 🤯

One of a-Shell’s major downsides is the lack of support for Python virtual environments. This means a-Shell is great for testing things out, or for doing some basic, pure-Python programming, but it’s not very well suited to professional development.

2. Carnets

Carnets is a free, standalone Jupyter notebook app available on iPad and iPhone. You get a full Python and Jupyter implementation — no need to connect to an external Jupyter server — as well as a handful of useful Python packages, including NumPy, pandas, and Matplotlib.

Click here to view Carnets on the App Store \(\rightarrow\)

You can create, view, and edit notebook files, including ones that you created elsewhere or were sent to you by a colleague. The thing that I like most about it is that it “just works.” Download the app and in a few minutes you’re running Jupyter notebooks right on your iPad.

Carnet’s interface looks just like Jupyter in a browser. But what you see is what you get. There aren’t any bells and whistles here.

If you need to install a package that doesn’t come with Carnets, you can use %pip install inside of a notebook cell to install the package:

To see all of the Python packages you get with Carnets, run %pip list. There are quite a few — although many you see in the following list were installed by me or as dependencies of packages I installed:

There are two versions of Carnets available in the App Store:

1. Carnets – Jupyter
2. Carnets – Jupyter (with scipy)

Carnets – Jupyter (with scipy) includes a few additional packages for doing machine learning right on your iPad: scipy, seaborn, sklearn, and coremltools. If you can afford the extra space, I highly recommend downloading Carnets – Jupyter (with scipy) instead of the base Carnets app.

Like a-Shell, the Carnets app doesn’t let you create isolated Python environments. You’re stuck using the global environment and whatever package versions come pre-built with the app.

3. Juno

Juno is another Jupyter notebook app for the iPad that bills itself as a Jupyter IDE. Like Carnets, you get Python bundled with some extra packages. Unlike Carnets, Juno costs $15 and comes with some nice bonus features.

Click here view Juno on the App Store \(\rightarrow\)

Juno really stands out from Carnets with its sleek and modern iPad interface:

You can run cells by pressing Shift + Enter inside of the cell or by tapping the blue Run Cell button at the bottom right-hand corner of the screen.

The lightning bolt button gives you quick access to some common tasks. You can change the cell type from code to Markdown, move cells up and down, and cut, copy, paste, and delete cells:

While Carnets can open Jupyter notebooks from anywhere on your iPad, the interface isn’t very iPad-friendly. Contrast this to Juno’s file picker, which really feels at home on iPad and iOS devices:

Another area where Juno shines is the IDE-like tab completion and tooltips that come built-in:

Like Carnets, Juno comes with a suite of built-in packages. But you can’t run %pip list in a cell to see them all like you can in Carnets:

Package management in Juno is actually a bit of a disappointment, especially for an app that costs $15.

I couldn’t find a complete list of packages that come pre-installed with Juno, but here are a few imports that worked out of the box:

To install a package, click the name of the notebook at the top-center of the screen and select Install Python Package. You’ll see the following dialog box:

Like Carnets and a-Shell, you can only install pure Python packages. But unlike Carnets and a-Shell, any dependencies of a package you install won’t be installed automatically. You’ll need to install them manually one-by-one.

One nice thing about Juno’s package manger is that you get a chance to see some metadata for a package before you install it, including dependencies — which you can install at the touch of a button:

As I mentioned before, you can’t use %pip list in Juno to view the packages you have installed into Juno’s environment. In fact, there is no way to view your installed packages from within Juno.

Instead, you must open the Files app and navigate to the site_packages/ folder in Juno’s on-device storage:

If you want to remove one of your installed packages, you need to do that manually from within site_packages/. I find this to be a major shortcoming. I really appreciate the quick access to pip using the %pip magic command supported by Carnets.

Despite the clunky package manager, Juno does look much nicer than Carnets and the tab-completion and tooltips do boost productivity. If those features matter to you and you’re willing to fork over the $15, then Juno is a nice option.

4. Juno Connect

Juno Connect is a Jupyter notebook client app that can be used to access Jupyter on a remote server. So, technicaly, Python isn’t running on your iPad, but Juno Connect provides a beautiful interface for working with remote notebook servers. You can purchase it from the App Store for $10.

Click here to view Juno Connect on the App Store \(\rightarrow\)

When you first launch Juno Connect, you’ll a see a screen with some notebooks ready to try out, as well as several options to connect to remote notebook servers:

Juno Connect support services like Cocalc and Binder right out of the box. You can also connect to Jupyter servers hosted elsewhere, such as on a Digital Ocean droplet.

When you connect to a notebook server, you’ll see a screen like the one below, where you can create a new notebook or select one to open:

Once you open or create a new notebook, you’ll see an interface that looks exactly like the typical Juno interface. And since the notebook is running on an external server, you get access to all of the typical Jupyter features, including the %pip magic that doesn’t work with the local-only version of Juno:

One nice feature of Juno Connect is the ability to export a remote notebook to a local file on your iPad:

You can even export the notebook as a new notebook, allowing you to save a local copy to work with offline in either Juno or the Carnets app.

5. Codeanywhere Cloud IDE

The last option on my list isn’t an iPad app and it doesn’t run Python locally on your iPad, but it’s absolutely essential if you need access to a full-blown development environment. That solution is the Codeanywhere Cloud IDE.

Plans start at $6 per month and you can get a 40% discount if you prepay for two years. This unlocks a VS Code style IDE that runs entirely in your browser.

Click here to get a 10% discount off any Codeanywhere subscription \(\rightarrow\)

Once you choose a plan and create an account, you’ll be taken to your dashboard where you can create new containers for working on projects:

When you click on New Container, you’ll have the opportunity use a blank Ubuntu Linux container or to select from some pre-defined containers with various languages pre-installed:

Once you create a container, it takes a few minutes for it to spin-up and become available to you in your dashboard:

The resources you get per container depend on the plan you selected. My plan gives me 15GB of storage and 4GB of memory.

Click the green Open IDE button to launch the IDE in a new browser tab:

If you’re familiar with VS Code, you’ll feel right at home in Codeanywhere’s IDE. It’s incredibly full-featured, including things like linter and debugging support:

You can even install extensions!

Codeanywhere is pricy compared to the other options mentioned in this list. It also requires an internet connection. But, I can’t live without it. It’s the best way I’ve found to do some hardcore coding on my iPad.

Additional Ways To Code In Python On Your iPad

The five tools I&aposve mentioned are what I currently use day-in and day-out to run Python on my iPad, but there are a number of other options that deserve a mention.


Whenever I bring up coding in Python on the iPad, I get a flurry of comments praising the Pythonista3 app. There was a time when Pythonista3 was a great option, but that’s no longer the case in my opinion.

First of all, Pythonista3 appears to be completely abandoned. The last version was released over a year ago before iPadOS 14 was available (the current version of iPadOS at the time of writing is 15). And, second, Pythonista3 only supports Python 3.6, which reached it’s end of life in December 2021.


The PyTo app is a serious contender in the Python-for-iPad space. You get Python 3.10 running locally with a nice IDE-style experience, support for editing multiple files, several bonus packages including NumPy, SciPy, Scikit-learn, and statsmodels. The full app experience costs $10.

I&aposm currently giving PyTo a run for it&aposs money to see how it fits into my daily use. If it lives up to it&aposs promise, you might find it higher up on the list in a future update to this article!

Google Colab

Google’s Colab is a browser-based notebook based on Jupyter notebooks. Your code runs on a private virtual machine that comes with most of the Python scientific stack pre-installed. You even get free access to GPUs, which makes Colab a great option for machine learning projects. If you need guaranteed uptime and more power, check out Colab Pro.


If you don&apost need Python running locally but want a solid IDE experience, check out vscode.dev. You can quickly clone an external code repository and start coding with Python right from your browser. The Python language doesn&apost have first-class support yet — stick with "webby" languages like JavaScript and CSS for the best experience — but it could be a good, free alternative in a pinch.

GitHub Codespaces

I&aposve never used GitHub&aposs Codespaces project, but if you have a GitHub organization on the Team or Enterprise plan, this might be a good option. It looks very similar to Codeanywhere.


The pyodide project brings Python and over seventy-five package in Python&aposs scientific stack to any browser by compiling them all to WebAssembly. You can try pyodide in a REPL and bookmark it for quick access in the future.

iPad + RaspberryPi

Another option is to connect your iPad to a Raspberry Pi via USB, which allows you to access the Raspberry Pi as an ethernet device. This is really cool, but personaly I don’t want to have to carry any more devices around with me than I need to. It does make for a fun little project, though, and gets you access to a full local development server.

Final Thoughts

I’d love to see a native version of an IDE like VS Code on the iPad, but I don’t think it’s ever going to happen. However, between a-Shell, Carnets, and Codeanywhere, I’ve been able to do everything that I’ve needed to from my iPad. Of course, your mileage may vary.

So, in 2022, coding in Python on the iPad is not only feasible, it’s actually quite fun!

Get a curated list of five curiosity-inducing articles and resources each week from around the Python and Julia communities by subscribing to my Curious About Code newsletter.

And if you&aposre interested in taking your coding skills to the next level, I offer private one-on-one coaching for Python programming and technical writing.

Categories: FLOSS Project Planets

Palantir: The Palantir Fellowship Program - 2022

Planet Drupal - Mon, 2022-01-10 07:00

Our DrupalEasy fellowships offer full tuition, stipends, and paid internships for candidates underrepresented in the tech industry

We are excited to announce the second round of Palantir.net's Fellowship program, a wonderful opportunity for us to provide qualifying candidates with the chance to attend DrupalEasy's Drupal Career Online (DCO). The first two fellowships will be awarded for the Spring 2022 session, and the next two will be awarded for the Fall 2022 session.

Fellowships At A Glance
  • Award: Full scholarship to cover the cost of Drupal Career Online and need-based support stipends. Paid internship/junior developer position with Palantir.net, dependent upon level of expertise and capabilities upon graduation
  • Eligibility: Available to candidates with diminished industry representation and/or who have been discriminated against and marginalized
  • Requirements: All candidates must meet the background requirements, apply to, and be accepted to Drupal Career Online. Recipients must commit to a paid internship and/or junior staff position with Palantir.net
  • Additional Perks: To celebrate your graduation, you'll receive additional stipends, professional mentoring, and an opportunity to work here with us

Think this is the perfect fit for you? Check out the application process!

You can also read more about the key dates and deadlines, and how DCO provides its participants with a clear pathway towards a successful Drupal career.

Our fellowships were created to provide the foundational tools and vital resources necessary for each of our recipients to become successful professionals who will bring valuable and unique perspectives to the Drupal community. 

As a company that believes in fostering, cultivating, and preserving diversity, equity, and inclusion makes us - and the world - a better place, this fellowship program is one part of our ongoing efforts to proactively identify and remove systemic barriers to equality. 

Here at Palantir.net, we're more than just a group of web professionals - we are curious, independent thinkers with exceptional passion, initiative, and talent. And if you're still curious about us, we invite you to learn more about our CultureTeam, and Work.

Or, better yet, reach out to me (Rachel Waddick) directly at marketing@palantir.net. As an extroverted communications person, I can guarantee that I'd love to hear from you! 

Photo by Arlington Research at Unsplash

Community Culture Drupal People
Categories: FLOSS Project Planets

ItsMyCode: How to Fix in Python ValueError: Trailing data?

Planet Python - Mon, 2022-01-10 05:18

ItsMyCode |

In Python ValueError: Trailing data occurs when you try to load the JSON data or file into pandas DataFrame, and the data is written in lines separated with newline characters such as ‘\n’.

Typically, we import data from the JSON files, and there are higher chances that JSON data contains newline characters.

Let’s take a simple example to reproduce this error. We have a JSON file of employees, and the address property in the JSON has ‘\n’ 

JSON File # import pandas library import pandas as pd # create pandas DataFrame df = pd.read_json('employee.json') # print names of employee print(df)


ValueError: Trailing data Note: If the JSON data is malformed or the file path is invalid you will get an error ValueError: Expected object or value Solution Python ValueError: Trailing data

The simplest way to fix this error is to pass the lines=True argument in the read_json() method while importing the JSON file.

The lines=True parameter ensures to read the JSON file as an object per line.

Now when we import the JSON file into a pandas DataFrame, it will load and print the data without any issue. 

# import pandas library import pandas as pd # create pandas DataFrame df = pd.read_json('employee.json',lines=True) # print names of employee print(df)


ID name age address 0 123 Jack 25 #3, 5th Main \nIndia 1 124 Chandler 25 #5, 2nd Main \nUS 2 123 Jack 25 #3/2, 6th Main \nCanada

Another way is to remove the \n character from the address column. We can simply replace the \n character with an empty '' character, as shown below.

# import pandas library import pandas as pd # create pandas DataFrame df = pd.read_json('employee.json',lines=True) df['address'] = df['address'].str.replace('\n', ' ') # print names of employee print(df)


ID name age address 0 123 Jack 25 #3, 5th Main India 1 124 Chandler 25 #5, 2nd Main US 2 123 Jack 25 #3/2, 6th Main Canada

The post How to Fix in Python ValueError: Trailing data? appeared first on ItsMyCode.

Categories: FLOSS Project Planets

Web Omelette: Dedicated entity bundle classes in Drupal 9.3

Planet Drupal - Mon, 2022-01-10 05:09

In this article we are going to see how to create and define specific classes for your entity bundles starting with Drupal 9.3.

Categories: FLOSS Project Planets

Mike Driscoll: PyDev of the Week: Lais Carvalho

Planet Python - Mon, 2022-01-10 01:05

This week we welcome Lais Carvalho (@lais_bsc) as our PyDev of the Week! Lais is active in multiple Python user groups in Ireland and has helped organize multiple Python conferences. She is on a recent episode of Python Bytes too, so be sure to check that out!

If you'd like to see what projects Lais is working on, head on over to GitHub.

Let's spend a few moments getting to know Lais better!

Can you tell us a little about yourself (hobbies, education, etc.)

My name is Lais Carvalho and I work as Developer and Community Advocate for Quansight, a consultancy company that offers support for most of the Open Source libraries available, specially on the Python sphere. I have just graduated on IT (Information Technology), after studying engineering for two years. I also volunteer for Python Ireland and have helped organise a few Python conferences. My favourite ones are EuroPython (the biggest Python conference in Europe) and PyJamas, an online event where everyone is invited to present wearing pajamas.

Why did you start using Python?

I started using Python in my first year of engineering university. I was struggling to learn WolframAlpha to check if my handmade calculations were correct, but everything used to be so hard and take so long. That was when chatting to one of our teachers, he mentioned I might prefer to use Python for that. I thought “well, it doesn't cost me much to try” and have become a fan since then, which I believe was in 2014.

What other programming languages do you know and which is your favorite?

I am not a programmer, which means that my main line of work is with people (not code) but I can build small things using Java and Python. My R skills are not something to be proud of but I can make beautiful graphs and pretend I understand data analysis well enough. I would like to try and learn GoLang or Julia next.

My favourite do-it-all language is Python due to its versatility. But languages are tools, one is welcome to choose whichever (s)he knows best in order to get the job done. To expand on the metaphor, it is elementary that a plier will not be the most effective to hammer nails on a wall, but it might do a fine job on acceptable time if the amount of nails is small or the time is limited and hammers are not available.

What projects are you working on now?

I am currently on medical license due to a cancer diagnosis received two weeks before my thesis delivery deadline. I am now finished with the treatment (in theory) and waiting to hear results back from the hospital. Keeping fingers crossed.

Before that, I was working on updating the documentation of one of Quansight’s Open Source libraries named QHub.

Which Python libraries are your favorite (core or 3rd party)?

There are so many wonderful ones, which makes this a very challenging question but in summary, I quite like requests, making HTTP requests human-friendly since 2011. In addition, the more scientific ones like NumPy and SciPy, and the classy Manim for mathematical animations. Aw yes, and Flask for web apps! Not to mention the standard stuff, Python is all wonderful.

How did you become so active with Python user groups?

I started volunteering for PyCon Ireland in my second year in college (2019), because I believed networking was the most effective way to get into the industry. At the event, I met so many wonderful people and everyone had a project or initiative in need of help, so I volunteered. I remember being extremely excited about partaking in such an inclusive and fun community, so much so that when COVID restrictions arrived I became almost fully dedicated to the user groups. It was a good way to cope with lockdown and to meet wonderful people that ended up becoming close-personal friends of mine.

What advice do you have for people who would like to start their own local Python user groups?

Look around and see if there are people interested in joining you. Look for inspiration in existing groups and persevere. This is a huge community we are part of and there will always be someone willing to help in some way. Be vocal, ask for help, and don’t give up. Finally, if you are thinking about starting your own Python User Group in Ireland, come and speak to us at Python Ireland, we would be happy to help in any way we can.

Is there anything else you’d like to say?

I have just recorded an episode of Python Bytes that you can find on their website or on Spotify. Wanna ask me something? DM me on Twitter at @lais_bsc.

Python Ireland:

  • A Speakers Coaching Session is happening later this month and if you want to improve your public speaking skills, it is a great opportunity to do so.
  • Also, our monthly meetups happen on the second Wednesday of each month, online for now (due to COVID regulations) but working on coming back in person. The recorded sessions are added to the YouTube channel.
  • For more information, follow us on Twitter. @pythonireland

PyLadies Dublin:

  • All the PyLadies workgroups are incredible and they also need volunteers. If you’re looking for a safe place to join the Python Community, I would seriously recommend PyLadies! And the Dublin cohort is led by the wonderful Vicky.

EuroPython will start looking for volunteers for the in person event as well soon

Pyjamas happens in December, want to be notified about it? Follow us on twitter @PyjamasConf and subscribe to our YouTube channel.

Thanks for doing the interview, Lais!

The post PyDev of the Week: Lais Carvalho appeared first on Mouse Vs Python.

Categories: FLOSS Project Planets

Zato Blog: API development workflow with Zato

Planet Python - Sun, 2022-01-09 23:40

Zato is an integration platform and backend application server which means that, during most of their projects, developers using Zato are interested in a few specific matters.

The platform concentrates on answering these key, everyday questions that Python backend developers routinely ask:

  • How do I connect to external systems or databases to store or extract data?
  • How do I map messages from one format to another?
  • How do I automate my work so that it is repeatable across environments?
  • How can I focus on my job instead of thinking of trivial low-level details?
  • How do I debug my code?
  • How can I reuse the skills and knowledge that I obtained elsewhere and how can I better myself?
  • How do I manage the complexity of having to integrate at least several systems and dozens or hundreds of APIs?
The typical workflow Dashboard
  • Dashboard is a web-based administration console that lets you define all the various connection pools, security types, scheduler jobs or other integration assets that you later make use of in your Python-based API services

  • When you install Zato, it is comfortable to use its Dashboard to define resources like REST, MongoDB or SOAP connections pools because it does not require any programming, you just fill out a form and that resource is created

  • All of your code is in Python - that makes it easy to manipulate any kind of data that you receive or may need to access

  • In your code, you only deal with the business logic - if your service is invoked, it means that all the security layers, serializers and other low-level parts have already completed their job and you are given a nice Python-based business object that represents your input. Based on this input information, you create output that your business logic dictates, still working purely on a high-level of Python objects, without any low-level details.

  • Static typing is encouraged through the use of dataclasses for your data models

  • Your code is hot-deployed from your IDE, no matter where your servers are, e.g. you work on Mac but the server is a remote Linux instance

  • You can debug your services from your IDE as well and, again, the server can be a remote one

# -*- coding: utf-8 -*- # stdlib from dataclasses import dataclass # Zato from zato.server.service import Model, Service @dataclass(init=False) class GetClientRequest(Model): client_id: int @dataclass(init=False) class GetClientResponse(Model): name: str email: str segment: str class GetClient(Service): class SimpleIO: input = GetClientRequest output = GetClientResponse def handle(self): # Enable type checking and type completion request = self.request.input # type: GetClientRequest # Log details of our request self.logger.info('Processing client `%s`', request.client_id) # Produce our response - in a full service this information # will be obtained from one or more databases or systems. response = GetClientResponse() response.name = 'Jane Doe' response.email = 'hello@example.com' response.segment = 'RNZ' # Return the response to our caller self.response.payload = response

Such a service can be assigned to one or more REST channels and we can invoke it now:

$ curl localhost:17010/api/v1/client -d '{"client_id":123}' {"name":"Jane Doe","email":"hello@example.com","segment":"RNZ"} $ Automation
  • Using Dashboard is very handy, and, once you are satisfied with what you created using it, the resources can be exported from command line using a built-in tool called enmasse - the result is a regular YAML file that describes every resource that you created using Dashboard

  • In the next step, an emasse file can be imported in a new environment, also from command line. For instance, you can keep adding new connection definitions to the export file each day and they can be imported in a separate testing environment by CI/CD regression testing processes.

  • Thanks to the enmasse export and import cycle, you do not need to manually recreate any resources that you initially created in your development environment. Moreover, because enmasse files are simple YAML files, you can script their usage in any way required when they are being import, e.g. by filling out credentials or other details that each environment may differ by.

  • Similarly, with your Python code, you can commit your code to git and configure each environment to hot-deploy it directly from a git checkout. Alternatively, Python code can be deployed statically, which requires a server restart. The latter approach is useful when you, for instance, prepare an AWS AMI with your solution that you create new runtime instance from, because it lets you embed your entire enmasse definition and code directly in your AMIs.

  • Definitions of your services can be exported to OpenAPI

  • Note that, by default, your services can be multi-protocol - it means that you may have a service that only uses WebSockets, or perhaps is a background one that has no REST endpoint at all, and yet, you still will be able to export it to OpenAPI

  • Taken together, it means that you can use any OpenAPI-compatible tool, such as Postman, for accessing and testing your API services, no matter if they use REST or not

  • You can also generate an entire static HTML site that will include OpenAPI as well as static pages describing your APIs - this is useful, for instance when you need to share with your tech partners not only an OpenAPI definition but a pretty-looking API documentation site as well

Let’s generate an OpenAPI definition for the GetClient service above:

$ zato openapi /path/to/server/server1 \ --include "client*" \ --file /tmp/openapi.yaml

Now, we can import the OpenAPI definition to Postman and invoke our services from it:

Python once more
  • Python is at the core of Zato and the platform allows you to leverage your already existing skills and knowledge of third-party libraries

  • For instance, the code below is a stand-alone program that connects to Redis to insert a key and read it back:

# -*- coding: utf-8 -*- # Redis import redis # Connect to the database conn = redis.Redis(host='localhost', port=6379, db=0) # Set a key .. conn.set('key', 'value') # .. read it back .. result = conn.get('key') # .. and log what we received. print('Result', result)
  • Now, the same as a Zato-based service:
# -*- coding: utf-8 -*- # Zato from zato.server.service import Service class MyService(Service): def handle(self): # Set a key .. self.kvdb.conn.set('key', 'value') # .. read it back .. result = self.kvdb.conn.get('key') # .. and log what we received. self.logger.info('Result %s', result)
  • In both cases, it is exactly the same amount of code yet a couple of differences are notable.

    • First, because the configuration was created earlier using Dashboard or enmasse, your code does not need to be concerned with where Redis is, it just makes use of it.
    • Second, already at this point, your Zato code is a scalable API service that can be hot-deployed to clusters for other systems to invoke it.
    • Yet, the actual business logic, the usage of Redis, is the same, which means that you can reuse what you already know and build upon it while developing with Zato.
  • Redis was but one example and the same principle applies to everything else. Be it SQL, MongoDB, ElasticSearch or any other connection type, you can take your existing code and simply embed it in Zato services.

  • This also means that it is easy to find programming examples specific to a particular system because, in the end, when you connect to such an external system through Zato, you issue requests using a library which already is widely popular among Python programmers, like redis-py in this example above.

  • Similarly, as a data scientist you may already have some Pandas or Spark code developed earlier - the same principle applies, you can embed it in your services or you can import it from your libraries, just like with regular Python code.

Next steps
  • Start the tutorial to learn how to integrate systems. After completing it, you will have a multi-protocol service representing a sample scenario often seen in banking systems with several applications cooperating to provide a single and consistent API to its callers.

  • Visit the support page if you need assistance.

  • Para aprender más sobre las integraciones de Zato y API en español, haga clic aquí

  • Pour en savoir plus sur les intégrations API avec Zato en français, cliquez ici

Categories: FLOSS Project Planets

ItsMyCode: How to Fix: KeyError in Pandas?

Planet Python - Sun, 2022-01-09 23:10

ItsMyCode |

The KeyError in Padas occurs when you try to access the columns in pandas DataFrame, which does not exist, or you misspell them. 

Typically, we import data from the excel name, which imports the column names, and there are high chances that you misspell the column names or include an unwanted space before or after the column name.

The column names are case-sensitive, and if you make a mistake, then Python will raise an exception KeyError: ‘column_name

Let us take a simple example to demonstrate KeyError in Pandas. In this example, we create a pandas DataFrame of employee’s data, and let’s say we need to print all the employee names.

# import pandas library import pandas import numpy as np # create pandas DataFrame df = pandas.DataFrame(np.array([["Jack", 22, "US"], ["Chandler", 55, "Canada"], ["Ross", 48, "India"]]), columns=['name', 'age', 'country']) # print names of employee print(df["Name"])


raise KeyError(key) from err KeyError: 'Name'

When we run the program, Python raises KeyError, since we have misspelled the “name” column as “Name”.

Solution KeyError in Pandas

We can fix the issue by correcting the spelling of the key. If we are not sure what the column names are, we can print all the columns into the list as shown below.

# import pandas library import pandas import numpy as np # create pandas DataFrame df = pandas.DataFrame(np.array([["Jack", 22, "US"], ["Chandler", 55, "Canada"], ["Ross", 48, "India"]]), columns=['name', 'age', 'country']) # print names of employee print(df["name"])


0 Jack 1 Chandler 2 Ross Name: name, dtype: object

We can now see a column called “name,” and we can fix our code by providing the correct spelling as a key to the pandas DataFrame, as shown below.

We can also avoid the KeyErrors raised by the compilers when an invalid key is passed. The DataFrame has a get method where we can give a column name and retrieve all the column values.

Syntax : DataFrame.get( 'column_name' , default = default_value_if_column_is_not_present)

If there are any misspelled or invalid columns, the default value will be printed instead of raising a KeyError. Let’s look at an example to demonstrate how this works.

# import pandas library import pandas import numpy as np # create pandas DataFrame df = pandas.DataFrame(np.array([["Jack", 22, "US"], ["Chandler", 55, "Canada"], ["Ross", 48, "India"]]), columns=['name', 'age', 'country']) # print names of employee print(df.get("Name", default="Name is not present"))


Name is not present

And if we provide the correct column name to the DataFrame.get() method, it will list all the column values present in that.

# import pandas library import pandas import numpy as np # create pandas DataFrame df = pandas.DataFrame(np.array([["Jack", 22, "US"], ["Chandler", 55, "Canada"], ["Ross", 48, "India"]]), columns=['name', 'age', 'country']) # print names of employee print(df.get("name", default="Name is not present"))


0 Jack 1 Chandler 2 Ross Name: name, dtype: object

The post How to Fix: KeyError in Pandas? appeared first on ItsMyCode.

Categories: FLOSS Project Planets

Matt Layman: Most Abstract Function First Is Better

Planet Python - Sun, 2022-01-09 19:00
Python is one of many languages that will let you define your module’s functions in whatever order you want. If your language of choice allows functions to be order independent, what order should you choose? Let’s illustrate what I’m talking about so that we’re all on the same page. First, here’s a Python example that puts the helper function first. def child(): print("hello world") def parent(): child() parent() And here’s a version that does the opposite.
Categories: FLOSS Project Planets

Armin Ronacher: Dependency Risk and Funding

Planet Python - Sun, 2022-01-09 19:00

I have a love/hate relationship with dependencies. I wrote about this extensively on this blog. Once about the challenges with scaling trust in dependencies and earlier about the problem with micro dependencies. Somehow very unsurprisingly nothing has actually improved in that regard in the last 5 years. In fact, I think the problem has become significantly worse. Where a few years back the main fear here was high profile developers being targeted, the dependency discussion is now overlapped and conflated with discussions about funding and sustainability.

I'm sure everybody remembers the XKCD on dependencies:

Comic by XKCD, #2347: Dependency. CC BY-NC 2.5

What I like about this comic is that you can insert a whole bunch of projects in your head into that comic. I like to imagine that the mentioned project is Curl. It's maintained largely by a single person — Daniel Stenberg — for more than 20 years. Curl is a good example of an actual crucial dependency. It's everywhere. I have seen it on game consoles, in cars, on MP3 players, smart speakers, bluray players, embedded devices, command line utilities, backend servers, … It's not only an incredible useful software, it's also solving a hard problem. It's also not a small dependency either. Curl is a whole package of useful functionality. If curl ceases to exist it would be clearly bad for society.

However. How can curl disappear? Curl is not just one of the most important dependencies, it's also one of the most resilient dependencies. When you or me install curl, we rarely install it from the official website. Curl is more likely to come from a mirror, vendored into a library we're using, there are a lot of forks in proprietary code bases etc. Curl is an unkillable dependency. Not only can the website go down, also the original developer could probably go away and someone would pick up the work, it's that useful.

Let's contrast this for a second with the situation on npm. One of the most dependent on libraries is in fact colors. The library is effectively emitting ANSI codes for colorization. A useful feature for sure, but not world shattering. I would go out on a lib and say that this type of functionality very often is implemented directly instead of dependent on. For instance when I wrote click I purposefully decided to implement ANSI coloring right in my own library without depending on something. My hunch is that it wouldn't take long to rip out and replace that library.

A few days ago the developer behind that library decided to release a new version of the library that no longer does what it advertised on the tin. Since it was a minor update quite a few people ended up with that version. They didn't however even know that they were depending on “that one package”, they probably pulled it in because something else in their dependency chain needed it.

If you went to the GitHub repo of that developer you found two things: some conspirational content in the readme of the repo, but also a justification for why their library no longer did what it was supposed to do: the developer was dissatisfied with “fortune 500” using their code for free and asked for a six figure contract or for people to fork it.

What I wish people would actually start discussing when it comes to these things is that npm (and other package managers) have developed into incredible levers. Someone who has a package with a lot of dependents one can easily knock out that piece of all modern digital infrastructure. Daniel Stenberg of curl doesn't wield that power (and probably also doesn't want to).

The risk a dependency poses is high with small, more commonly used dependencies, by a single unvetted developer, installed through a package manager like npm, cargo, pypi or similar. Yet when something goes wrong there, everybody immediately notices and people quickly call for funding. Yet those are not the dependencies that actually support our economy. Many of those dependencies became that foundational, not because they are solving a hard problem, but because we collectively started embracing laziness over everything else. When we then focus our funding discussions around these types of dependencies, we're implicitly also putting the focus away from the actually important packages.

I appreciate what GitHub does with sponsors and I think it's an awesome idea. I also appreciate that GitHub puts a finger at funding Open Source being an issue, but unfortunately there is a dark side to this: it points the finger to where it's easy. GitHub like npm point the finger to what computers can easily explain. My code flew to mars. That's awesome. But that Badge of honor I now carry on my GitHub profile I got because they crawled the Python dependency list. Together with my badge the folks that created lxml got a badge. However Daniel Veillard who maintains the underling libxml2 library received no such badge. In fact many people probably forget that libxml2 even exists or that they might be using it, because it's hidden behind a much more fancy high level facade that hides it. Unlike an npm package, you don't download libxml2 from somewhere when you install lxml. libxml2 like curl doesn't have the lever or visibility. Yet the amount of work and dedication that went into the library is significant. And he's just one of thousands of developers who have created incredible libraries we all still use.

Clearly we need to solve funding of Open Source projects and I love that GitHub sponsors is a thing. But I think we need to find a better way to assess impact of libraries than just how many people depend on this on npm or other package managers. Because that's by far not the whole picture.

Categories: FLOSS Project Planets

Dirk Eddelbuettel: Rblpapi 0.3.13: Some Fixes and Documentation

Planet Debian - Sun, 2022-01-09 18:07

A new version, now at 0.3.13, of the Rblpapi package just arrived at CRAN. Rblpapi provides a direct interface between R and the Bloomberg Terminal via the C++ API provided by Bloomberg (but note that a valid Bloomberg license and installation is required).

This is the thirteenth release since the package first appeared on CRAN in 2016. It comprises the PRs from three different contributors (with special thanks once again to Michael Kerber), and extends test and documentation, and extends two function interfaces to control explicitly whether returned lists of length one should be simplified.

The list of changes follow below.

Changes in Rblpapi version 0.3.13 (2022-01-09)
  • Add a test for bds (Michael Kerber in #352)

  • Add simplify argument (and option) to bdh and bds (Dirk in #354 closing #353, #351)

  • Improve documentation for bsearch (John in #357 closing #356)

Courtesy of my CRANberries, there is also a diffstat report for the this release. As always, more detailed information is on the Rblpapi page. Questions, comments etc should go to the issue tickets system at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Categories: FLOSS Project Planets

Year in Review: personal (and GPG)

Planet KDE - Sun, 2022-01-09 18:00

It’s the start of a new year, which means some retrospective – Year in Review posts upcoming – but also a new GPG signing key. This is the key I use to sign tags of software I release, such as Calamares, ARPA2CM, and others. It’s also the key I might use to send signed email.

GPG Bits

My current GPG key for general use is this one:

sec rsa4096/0x7FEA3DA6169C77D6 2016-06-11 [SC] [expires: 2023-02-11] Key fingerprint = 00AC D15E 25A7 9FEE 028B 0EE5 7FEA 3DA6 169C 77D6 uid [ultimate] Adriaan de Groot <groot@kde.org> uid [ultimate] Adriaan de Groot <adridg@freebsd.org> ssb rsa3072/0x717ED90DAF7DC23B 2021-09-12 [E] [expires: 2022-09-12] ssb rsa3072/0x328D742D8807A435 2022-01-10 [S] [expires: 2023-01-10]

You can get it from reputable keyservers, or from my about page. The previous signing key 0xCFDDC96F12B1915C expired on january 2, 2022. The new signing key is 0x328D742D8807A435 and is good until next year.

Personal Bits

My two kids are in university. They have two remaining grandparents. Those were the biggest changes in my immediate family this year. I moved my mom’s house so it is closer to us, but that ran into medical complications as well, although all is now well.

With conferences shut down – there was a brief upswing of commercial software conferences after the summer, but all the Open Source ones I know of remain stubbornly and safely virtual – my usual travel has been reduced to nil. I am starting to miss the occasional Open Source get-together, though. A walk in the woods with Scarlett and Leinir, for instance, is one of my hoped-for yearly activities. There was just one KDE-related trip to Germany.

Vacation-wise we spent the summer outdoors on bicycles, with a minor COVID-related hiccup, and over Christmas I went to Rome to visit my brother. That was the first time in big crowds (ugh, and airplanes) in a long long time, and I’m not particularly convinced it was justified, sustainable, or responsible in the face of the current situation. Italy, though, seems more disciplined than the Netherlands when it comes to following COVID-related health regulations. With regular self-testing (especially a few days after travel) things were ok (but what if they hadn’t been?)

Planning the next 50 weeks

Srsly, I don’t think there’s a whole lot that is plannable right now. There’s some job changes coming up in my household, projects that are being finished up (so at some point I’ll be looking for new ones), that kind of thing, but there’s no way that “I’ll be in Edmonton for the month of April because I want to see snow again” or some similar hare-brained scheme is going to work out.

So I plan on plugging away at Calamares, at FreeBSD, at making a healthy balanced dinner for my family every evening, at playing badminton when it’s possible, and just getting on with things.

Categories: FLOSS Project Planets