Russell Coker: Basics of Linux Kernel Debugging

Planet Debian - Wed, 2021-04-14 10:27

Firstly a disclaimer, I’m not an expert on this and I’m not trying to instruct anyone who is aiming to become an expert. The aim of this blog post is to help someone who has a single kernel issue they want to debug as part of doing something that’s mostly not kernel coding. I welcome comments about the second step to kernel debugging for the benefit of people who need more than this (which might include me next week). Also suggestions for people who can’t use a kvm/qemu debugger would be good.

Below is a command to run qemu with GDB. It should be run from the Linux kernel source directory. You can add other qemu options for a blog device and virtual networking if necessary, but the bug I encountered gave an oops from the initrd so I didn’t need to go further. The “nokaslr” is to avoid address space randomisation which deliberately makes debugging tasks harder (from a certain perspective debugging a kernel and compromising a kernel are fairly similar). Loading the bzImage is fine, gdb can map that to the different file it looks at later on.

qemu-system-x86_64 -kernel arch/x86/boot/bzImage -initrd ../initrd-$KERN_VER -curses -m 2000 -append "root=/dev/vda ro nokaslr" -gdb tcp::1200

The command to run GDB is “gdb vmlinux“, when at the GDB prompt you can run the command “target remote localhost:1200” to connect to the GDB server port 1200. Note that there is nothing special about port 1200, it was given in an example I saw and is as good as any other port. It is important that you run GDB against the “vmlinux” file in the main directory not any of the several stripped and packaged files, GDB can’t handle a bzImage file but that’s OK, it ends up much the same in RAM.

When the “target remote” command is processed the kernel will be suspended by the debugger, if you are looking for a bug early in the boot you may need to be quick about this. Using “qemu-system-x86_64” instead of “kvm” slows things down and can help in that regard. The bug I was hunting happened 1.6 seconds after kernel load with KVM and 7.8 seconds after kernel load with qemu. I am not aware of all the implications of the kvm vs qemu decision on debugging. If your bug is a race condition then trying both would be a good strategy.

After the “target remote” command you can debug the kernel just like any other program.

If you put a breakpoint on print_modules() that will catch the operation of printing an Oops which can be handy.

Related posts:

  1. Finding Corrupt Files that cause a Kernel Error There is a BTRFS bug in kernel 3.13 which is...
  2. Kernel Security vs Uptime For best system security you want to apply kernel security...
  3. Debugging as a Demonstration Sport I was watching So You Think You Can Dance [1]...
Categories: FLOSS Project Planets

Acro Media: Advantages of Open Source Commerce | Acro Media

Planet Drupal - Wed, 2021-04-14 10:00

Companies are breaking free of restrictive proprietary platforms in favour of custom open source solutions. Find out why in this comprehensive article.

Advantages of Open Source Commerce

Ownership of data & technology If you use an open source commerce platform, you own the code.

You need to look at your website the same way you would view a brick-and-mortar storefront. Paying a monthly licensing fee for a hosted platform is like having a leased property -- you’re only adding to your overhead costs and you have no control over your future. Hosted solutions don’t allow you to own the code, which business owners often don’t think of as a problem until something bad happens. If a hosted solution goes under, they take you down with them. You lose your online presence and have to rebuild from the beginning. That’s a long and expensive process.

If you use an open source commerce platform you own the code. If you work with an agency and choose to move to an in-house development team or a different agency, you can do so at no cost or risk to your business.

Integration with virtually any business system The code is completely exposed for you to use.

If you judge ecommerce solutions solely on their feature set, hosted solutions like Magento, Shopify, and Volusion will always look good up front. But your ecommerce platform is more than just window dressing. Open source frameworks can have an impressive feature set, but the biggest advantage is the expansive list of back-end systems they can integrate with.

Proprietary platforms can offer standard integrations with customer relationship management (CRM) systems and fulfillment solutions, but if you’re a big retailer, you may find you need a higher degree of integration for your sales process.

Open source platforms are exactly that. Open. The code is completely exposed for your use. Combine this with the modular architecture and you have a platform with the ability to integrate with virtually any business system that allows access, from CRMs and shipping vendors to payment gateways and accounting software. Your ecommerce site can become an automated business rather than just a storefront.

A custom user experience A custom user experience gives more power to the marketer.

When it comes to user experience, hosted platforms give you a best-practice, industry-standard, cookie-cutter execution of a shopping cart. It’s a templated design that is sold as a finished product, so you know you’ll have a catalogue, a simple check-out, account pages, etc. Outside of choosing a theme, there is very little room for customization. Open source allows for all the same functionality plus a powerful theme system that allows you to add unique and advanced features very easily.

A custom user experience gives more power to the marketer, allowing them to create custom conversion paths for different user types and integrate those paths within the content experience. You can generate personalized content based on customer data and/or provide different content to users based on their geographic location.

Open source commerce is also ideal for omnichannel selling. The consumer path is seamless across all sales channels, devices, websites and retail tools throughout the entire customer experience. You can set up content, layout and functionality to respond to the device being used, such as smartphones and tablets.

The omnichannel experience & a single source of data Open source platforms use a single data source which makes it optimal for creating omnichannel strategies.

Today’s ecommerce landscape is rapidly evolving. It’s no longer just about selling products online. Companies are expected to create immersive shopping experiences for users. The advances in mobile technology have given consumers constant and instant access to information. They expect their favourite brands to be able to deliver an integrated shopping experience across all channels and devices complete with personalized content, consistent product information, and simple conversion paths. This is not an easy task. For retailers that sell through both online and in-store channels, the challenge is even greater.

Open source platforms use a single data source which makes it optimal for creating omnichannel strategies. Rather than having to force together multiple platforms that pull data from various systems, open source allows for one centralized data centre that can be accessed by any of the systems you need.

What does this mean exactly?

Customer data, product details, promotions & sales information, inventory numbers and more can all be easily defined and streamlined across multiple channels. For example:

  • Your customers can start a purchase online and then pick up where they left off in your store. 
  • Customer data can be accessed easily for automated marketing; loyalty programs, birthday “gifts”, personalized recommendations.
  • If your products are sold on your ecommerce store as well as third party marketplaces, your product info is always consistent without having to apply multiple updates on various backends.
  • Easily define and promote location-based pricing and offers.
  • Real-time inventory numbers can be shown online to ensure product availability and minimize the risk of back-orders.
  • Tax & shipping rules can be defined per city, state, country to ensure all customers are shown the correct cost of items at checkout.
A flexible platform that aligns with your needs Exceed the boundaries of a traditional sales platform.

Any ecommerce platform today needs the ability to adapt. If your platform is locked down, you risk losing to your competitors. Hosted ecommerce solutions are just shopping carts with conventional catalogue management and the ability to sell physical and/or digital products.

Open source commerce releases you from these industry-standard restraints. Organize your products using virtually any attribute. Display your products in any style, from lists, grids, or tables to a customized look that can make you stand out from your in-the-box competition. Integrate features that go beyond commerce, such as custom applications, web services, and customer portals. Exceed the boundaries of a traditional sales platform.

Don’t be tied to someone else’s development path. By leveraging an open source platform, you allow yourself to be the frontrunner in your market.

No licensing fees, revenue sharing or mandatory support contracts. Open source commerce is free to use.

Anyone with the appropriate development skills can pick up an open source framework and begin working with it immediately at no charge. If you require development help you will need to pay a contractor or agency and depending on your needs these upfront costs can seem like a lofty investment. However, after the upfront development, there are no mandatory ongoing costs associated with open source.

If you are utilizing a SAAS or proprietary platform start-up costs are minimal but the majority of them have various ongoing costs.

  • Monthly contracts — SAAS platforms will charge you a monthly fee to use their platform, in addition to this fee you may have to pay for additional functionality, integrations, and/or support.
  • Licensing fees — The big enterprise platforms (Demandware, Hybris, Magento) charge a yearly license fee to use their software platforms. These fees can range from $50,000 - $700,000 per year.
  • Revenue sharing — SAAS and proprietary platforms will often require a revenue share contract to supplement their low monthly fee. A typical revenue share agreement is a 2% transaction fee. Depending on your yearly gross revenue this can be a major blow.
1000’s of supporters and continued development

Open source platforms are pushed forward by thousands of developers and supporters worldwide; agencies, contractors, & enthusiasts all have a shared goal of bettering their software and creating an accessible and stable platform. Proprietary systems simply can’t compete with a workforce this large or this focused. Open source evolves at the pace of the web. By leveraging this type of platform, you can be a front-runner in your market. Often before a retailer even knows it needs a specific new integration or piece of functionality, someone is already building it.

Drupal Commerce & Acro Media

Drupal Commerce is the powerful ecommerce software that extends from the open source Drupal platform. Drupal Commerce was built onto the content management system using the same architecture, allowing for a true marriage of content and commerce. It is a truly unrestricted platform that provides both structure and flexibility.

Acro Media is the leading Drupal Commerce agency in North America. We work exclusively with Drupal and Drupal Commerce, and currently, develop and support one of the biggest Drupal Commerce websites in the world. Our Drupal services include:

  • Drupal Commerce
  • Drupal consultation and architecture
  • Drupal visualizations and modelling
  • Drupal integrations to replace or work with existing platforms
  • Drupal website migrations (rescues) from other web platforms
  • Custom Drupal modules
Are you ready to escape?

Break free from the proprietary platforms and legacy software you’re handcuffed to and create the commerce experience you want. Open source commerce gives the power to the business owner to create a commerce experience that meets the ever-changing conditions of your marketplace as well as the complexities of your inner company workings.

Next steps

Want to learn more about open source, Drupal Commerce, or Acro Media? Book some time with one of our business developers for an open conversation to answer any questions and provide additional insight. Our team members are here to help provide you with the best possible solution, no sales tricks. We just want to help, if we can.

Categories: FLOSS Project Planets

Real Python: Build a Python Directory Tree Generator for the Command Line

Planet Python - Wed, 2021-04-14 10:00

Creating applications with a user-friendly command-line interface (CLI) is a useful skill for a Python developer. With this skill, you can create tools to automate and speed up tasks in your working environment. In this tutorial, you’ll build a Python directory tree generator tool for your command line.

The application will take a directory path as an argument at the command line and display a directory tree diagram on your screen. It’ll also provide other options to tweak the output.

In this tutorial, you’ll learn how to:

  • Create a CLI application with Python’s argparse
  • Recursively traverse a directory structure using pathlib
  • Generate, format, and display a directory tree diagram
  • Save the directory tree diagram to an output file

You can download the code and other resources required to build this directory tree generator project by clicking the link below:

Get Sample Code: Click here to get the sample code you’ll use to build a directory tree generator with Python in this tutorial.

Demo: A Directory Tree Generator Tool in Python

In this tutorial, you’ll build a command-line tool to list the contents of a directory or folder in a treelike diagram. There are already several mature solutions out there that perform this task. You’ll find tools like the tree command, which is available on most operating systems, plus other tools, like treelib, dirtriex, and so on. However, figuring out your own solution to this problem would be a good learning exercise.

This tutorial refers to the kind of tool described above as a directory tree generator. The tool you’ll build here will allow you to generate and display a treelike diagram listing the internal structure of a given directory in your file system. You’ll also find this diagram referred to as a directory tree diagram throughout the tutorial.

Your directory tree generator will have a user-friendly CLI. It’ll also provide some interesting features, such as displaying a tree diagram with the contents of a directory on your terminal window and saving the diagram to an external file.

Here’s how the application will look and work once you get to the end of this tutorial:

Your directory tree generator will provide a fully functional but minimal CLI with a couple of options that allow you to generate and display a tree diagram listing all the files and directories in a given root directory.

Project Overview

The project you’ll build in this tutorial consists of a command-line application that takes a directory path as an argument, walks through its internal structure, and generates a treelike diagram listing the contents of the directory at hand. In this section, you’ll take a first look at the problem and a possible solution. You’ll also decide how to lay out the project.

Laying Out the Project

To build your directory tree generator, you’ll create a few modules and a package. Then you’ll give the project a coherent Python application layout. At the end of this tutorial, your project’s root directory will have the following directory structure:

./rptree_project/ │ ├── rptree/ │ ├── rptree.py │ ├── __init__.py │ └── cli.py │ ├── README.md └── tree.py

The rptree_project/ directory is the project’s root directory. There, you’ll place the following files:

  • README.md provides the project description and instructions on installing and running the application. Adding a descriptive and detailed README file to your projects is considered a best practice in programming, especially if you’re planning to release the project as an open source solution.

  • tree.py provides an entry-point script for you to run the application.

Then you have the rptree/ directory that holds a Python package with three modules:

  1. rptree.py provides the application’s main functionalities.
  2. __init__.py enables rptree/ as a Python package.
  3. cli.py provides the command-line interface for the application.

Your directory tree generator tool will run on the command line. It’ll take arguments, process them, and display a directory tree diagram on the terminal window. It can also save the output diagram to a file in markdown format.

Outlining the Solution

Traversing a directory in your file system and generating a user-friendly tree diagram that reflects its contents might not look like a difficult task at first glance. However, when you start thinking about it, you realize that it hides a lot of complexity.

Read the full article at https://realpython.com/directory-tree-generator-python/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Categories: FLOSS Project Planets

Civic UK: DWP GOV.UK Cookie consent using CIVIC Cookie Control

Planet Drupal - Wed, 2021-04-14 08:00
CIVIC has implemented the DWP Consent to cookies pattern for Drupal 8/9 and Wordpress websites using our own cookie compliance solution, Cookie Control.
Categories: FLOSS Project Planets

Qt Creator 4.15 RC1 released

Planet KDE - Wed, 2021-04-14 05:35

We are happy to announce the release of Qt Creator 4.15 RC1 !

Categories: FLOSS Project Planets

Droptica: How to Manage Multimedia in Drupal? Media Module

Planet Drupal - Wed, 2021-04-14 05:33

Working with multimedia is one of the areas that large websites have to deal with. When multiple editors upload a large number of files, keeping your photos and videos in order can become difficult and time-consuming. Drupal has several proven recipes for managing the media library, which I'll present in this article.

Drupal modules for managing multimedia

Drupal offers great flexibility when working with files on a website. In the simplest terms, you can use fields like "File upload". A more complicated option is to use one of the modules to work with multimedia. Let's take a look at three interesting modules worth knowing about.

IMCE module

IMCE is a file viewer available to administrators and editors that also allows you to add, copy, and delete files and directories. It’s fast and convenient, it offers a preview of photos (including creating thumbnails), allows you to sort them and limit the disk space for individual users.

There was a time when the IMCE module was an absolute must-have for any Drupal website. To this day, over 340 thousand websites use it, but its popularity is steadily declining. The reason for this is that with more editors it's difficult to keep your file directories in order. Searching for a specific item is problematic, because the files have only a name and type. It's also impossible to filter them according to selected criteria. Despite the existence of the IMCE version for Drupal 8, new websites usually use other solutions.


SCALD project

SCALD is a module designed for Drupal 7, with a very rich base of add-ons (over 50 additional modules). It allows you to add photos, videos, and other types of files as entities called atoms. Each atom can have any field list based on which the media list is filtered. There is no module version for Drupal 8, but there is a migration path for atoms to the Media module entity.

I mention the SCALD module due to its significant contribution to building the concept of media management. With the premières of Drupal 6 and 7, the era of entities and fields was slowly approaching. The first ideas began to emerge to present files as entities that could be described in any way. This is how the SCALD project was created – quite niche, although used over the years on large and popular sites with millions of multimedia, such as Radio France, ARTE TV and Le Figaro.

Media module

This is by far the most important of the modules presented here. It follows an idea similar to SCALD, because it turns files into entities. It’s currently used by over 180 thousand Drupal 7-based websites, as well as all the websites based on Drupal 8 and 9. That's because since version 8.4 it’s in the core.

Media is a mature module, prepared for years by a team of experienced programmers. I can confidently say that it's one of the best multimedia management solutions available on the Internet. What makes the Media module so special? I'd point out its four main advantages:

  1. Effective embedding of a file library in the Drupal's entity/field/view model. This solution gives virtually endless possibilities for web developers. The files can have different view modes and form view modes, they can be used in many independent contexts. A photo sent via the Media module can be used multiple times, both as a thumbnail in the content of the article and, for example, as a background of a large banner.
  2. Leaving the directory/file model. When uploading a photo or video, the editor no longer decides about the order in the directory structure. It’s done automatically. Ordering is done using the fields defined in the media. You can, for example, add a category to the files, and then use it to search through the list of multimedia.
  3. Independence from media sources. On a well-designed website, videos will work the same way whether they are uploaded directly or via YouTube or Vimeo. Photos can also be loaded from multiple sources (such as Pixabay or Google Drive). Such an approach makes the work of editors easier and allows quickly adding subsequent sources as needed.
  4. Media isn't only multimedia anymore. You can put documents, Facebook posts or excerpts from Google Maps in the media library. Your only limit is your imagination.

When it comes to migration, remember that the Media module comes in three forms:

In each of these cases, a migration path to the core version is available, so you can easily transfer the multimedia library to a newer Drupal.

Why isn't the Media module used on every website?

This begs the question – since the Media module is so great and also available in the core, why isn't it enabled by default and used in every Drupal project? This is because many Media elements need to be set up by a programmer first. Therefore, this module is a tailor-made solution that is strongly adapted to the needs of a specific website.

It's particularly problematic to correctly display videos from external sources (due to the limited capabilities of external players) and to design appropriate display modes for the media. Getting through these issues can take a long time. Due to this fact, at Droptica we use a ready-made solution included in Droopler, the Drupal distribution.

Media module in Droopler

Since version 2.0, Droopler has built-in support for the Media module. All photos, graphics, icons and videos on the website are stored in the file library.


It's fine if you have a website based on Droopler 1.x. When upgrading to Droopler 2.x all "File upload" fields will be automatically converted to the new format.

The files put in the library can be used in many places on the website - as a paragraph background, an icon, a photo in a gallery or a cover for a blog post. In each of these contexts, the multimedia will be trimmed and adjusted accordingly. Here are examples of using the same photo in a completely different capacity:

As a tile


As a form background


As a blog cover


Once you start using Droopler, you'll be amazed at the ease with which you can use YouTube and Vimeo videos. For example, try putting your video in a paragraph with a banner. In the paragraph edit options add the media in the "Background" field:


Then go to the "Video" tab, enter the video address, click ADD and choose a title for the newly added item. Finally, put the video in a paragraph.


The effect of this action will be a moving background that stretches across the entire screen and is played in a loop. The YouTube controls will be hidden, and the video will have a transparent black overlay, improving the readability of the text.


You can add a video to a gallery in the same way, and e.g. next to the text, in the "Sidebar Image" paragraph. There are practically no restrictions here, all the elements fit together well.


The multimedia library included in the latest Drupal works great with even with a large number of subpages and blog posts. You will quickly notice that reusing the existing photos and videos saves time and effort. Even though the Media module requires a lot of coding for every new project, you can use "prepackages" such as Droopler to start working on the content right away.

Categories: FLOSS Project Planets

Python Morsels: Python doesn't have constants

Planet Python - Wed, 2021-04-14 02:30

Python doesn't have constants.

The CONSTANT convention

We have two variables here that are named with a fully uppercase naming convention:

>>> NUMBERS = [2, 1, 3, 4, 7, 11, 18] >>> COORDINATES = (1, 2, 3)

That naming convention is pretty common for constant variables in other programming languages. Sometimes you'll it in Python as well (PEP 8 actually notes this convention).

Variables point to objects

If we point another variable, nums, to first variable, NUMBERS:

>>> nums = NUMBERS

And then we call the append method on this new nums variable:

>>> nums.append(29)

This will mutate the object that nums points to:

>>> nums [2, 1, 3, 4, 7, 11, 18, 29]

We've changed the list that nums is pointing to. But NUMBERS and nums point to the same object, which means we've changed NUMBERS too!

>>> NUMBERS [2, 1, 3, 4, 7, 11, 18, 29]

That's in fact, what assignment does.

Variables point to objects in Python and we've pointed two variables to the same object. So when we change nums, NUMBERS seems to change as well because the object which NUMBERS points to have changed (remember there's 2 types of change in Python).

You can see this mutation by stepping through this interactive Python Tutor visualization:

Immutable objects cannot be changed

This COORDINATES variable points to a tuple:

>>> COORDINATES (1, 2, 3)

Unlike lists, tuples are immutable objects, meaning tuples cannot be changed.

So, if we point the variable p to COORDINATES:


And then we try to mutate the object that p points to, we'll get an error.

For example we can't assign to an index in that tuple:

>>> p[0] = 10 Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'tuple' object does not support item assignment

And we can't append to the tuple:

>>> p.append(29) Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'tuple' object has no attribute 'append'

As hard as we might try, there is no way to mutate the tuple that p points to. We can't change tuples because tuples are immutable objects.

Assignment changes what a variable points to

Immutability is about changing objects; it's about mutation, and mutating the objects. But we have two types of change in Python.

Objects are changed through mutations. Variables are changed through assignment statements. Variables point to objects, and you can change which object a variable points to by assigning to that variable.

So, can we take COORRDINATES and assign it to a different tuple?

>>> COORDINATES = (4, 5, 6) >>> COORDINATES (4, 5, 6)

We can! There's nothing stopping us from doing this.

So COORDINATES isn't actually a constant variable. And in fact, there's no way to make it a constant variable. Python doesn't have constants.


In Python, we have this concept of immutability. For each object we can ask, is the object mutable (meaning you can change it), or is it immutable (meaning it can't be changed)?

Variables point to objects. There's no way to lock a variable down and stop it from being reassigned. Constant variables in other languages cannot be reassigned. We don't have any equivalent of that in Python. In Python, there's no way to stop a variable from being reassigned.

As long as you're in the same scope as a variable, you can always reassign it. Likewise, as long as an object is mutable, you can always mutate it. However, you can make an immutable object, but you cannot make a constant variable.

Categories: FLOSS Project Planets

My current Plasma Wayland from git

Planet KDE - Wed, 2021-04-14 01:28
However much distress the current scenario and some personal stuff might cause, I’ve really been having fun with my laptop. Running everything on bleeding edge is exciting: kernel 5.12.rc6, openSUSE Tumbleweed with Plasma built daily from master (so openSUSE Krypton), using only the Wayland session, switching entirely to pipewire and so on. I figured I … Continue reading "My current Plasma Wayland from git"
Categories: FLOSS Project Planets

Codementor: Basic Tutorial: Using Docker and Python

Planet Python - Wed, 2021-04-14 00:12
How I write Dockerfiles for python applications and why.
Categories: FLOSS Project Planets

Spinning Code: SC DUG April 2021 – Getting Started with Electron

Planet Drupal - Tue, 2021-04-13 23:44

This month I gave a talk at South Carolina Drupal User Group on Getting Started with Electron. Electron allows you to use your web developer skills to create desktop applications. I based this talk on some of my recent side projects and the Electron Project Starter I posted the end of last year.

If you would like to join us please check out our up coming events on MeetUp for meeting times, locations, and remote connection information.

We frequently use these presentations to practice new presentations, try out heavily revised versions, and test out new ideas with a friendly audience. So if some of the content of these videos seems a bit rough please understand we are all learning all the time and we are open to constructive feedback. If you want to see a polished version checkout our group members’ talks at camps and cons.

If you are interested in giving a practice talk, leave me a comment here, contact me through Drupal.org, or find me on Drupal Slack. We’re excited to hear new voices and ideas. We want to support the community, and that means you.

The post SC DUG April 2021 – Getting Started with Electron appeared first on Spinning Code.

Categories: FLOSS Project Planets

hussainweb.me: My thoughts on Automatic Updates in Drupal

Planet Drupal - Tue, 2021-04-13 23:20
It’s spring and I decided to come out to a park to work and write today’s post. I sat on a bench and logged in to my WordPress site to start writing the post when I noticed that one of the plugins had updates available. I didn’t have to think about this and straightaway hit the update button. Less than 30 seconds later, the plugin was updated, the red bubble had disappeared, and I had my idea of today’s post. That is why I want to talk about automatic updates on Drupal today.
Categories: FLOSS Project Planets

Sumana Harihareswara - Cogito, Ergo Sumana: Python Packaging Tools: Security Work And An Open Position

Planet Python - Tue, 2021-04-13 20:59

Two exciting bits of news regarding massively improving how we package, distribute, and install Python software!

First: a new grant. New York University (specifically Professor Justin Cappos) and I have successfully asked the US National Science Foundation for a grant to improve Python packaging security. The NSF is awarding NYU $800,000 over two years, from mid-2021 to mid-2023, to further improve the pip dependency resolver and to integrate The Update Framework further into the packaging toolchain. I shared more details in this announcement on an official Python packaging forum.

I'll be part of this work, paid to work on this part-time, doing some outreach, coordination, project management, and similar. Thanks to the NSF, Justin, the Secure Systems Lab at NYU, and all the people who work on Python packaging tools!

Second: the Python Software Foundation is hiring a full-time project manager and community manager for Python's packaging toolchain. Thanks to Bloomberg for the funding! Please check out the job description and spread the news. Please apply by May 18th, 2021.

The job is remote and you can apply from anywhere in the world. As the description says: "Total compensation will range from $100k-$125k USD based on qualifications and experience." And you'd report to Ee W. Durbin III, a colleague I strongly recommend and love working with.

I'm thoroughly grateful that we've now gotten to the point where the PSF can hire for a full-time person for this role. As a volunteer and as a contractor, I've performed -- in many cases initiated -- the activities that this person will do, and I've seen the critical need. We deeply need a full-time coordinator for holistically assessing and improving the user and developer experience of Python packaging, because -- as Russell Keith-Magee said in his PyCon US 2019 keynote -- the status quo poses "an existential threat" to the future of the language. And so one of the desired qualifications for the role is: "Belief that Python packaging problems are of critical importance for the Python language... but that those problems are solvable."

We've gotten better and better at attracting corporate and grant funding -- and yes, I'll take some credit for that, with my past work researching and writing grant proposals, leading funded projects, and volunteering with the Packaging Working Group and cofounding the Project Funding Working Group. So, now, what should we focus on? We need to prioritize improvements for strategic value (e.g., should we first concentrate on overhauling the Warehouse API, or making a generic wheel-builder service, or tightening metadata compliance, or ....?). What can we learn from other package management toolchains, especially those that emerged after PyPI and pip (e.g., yarn, npm, cargo), and what should we copy? In my opinion, you do not need to already have an opinion on these questions to apply for this role -- you just have to be interested in talking with a bunch of stakeholders, poking through past discussions, and collaboratively developing some answers.

I won't be applying for this PSF role -- I'm going to be, instead, excited to collaborate with that person and help them learn all the stuff I know, so that in the long run, we'll have more people, with that set of skills and domain knowledge, working on Python packaging. I'll concentrate on the Python supply chain security piece specifically (via the NSF-funded work at NYU), plus finishing my book and maybe creating and leading associated trainings, and taking what I've learned to other languages and ecosystems through client work.

So: please spread the word and apply!

Categories: FLOSS Project Planets

Dirk Eddelbuettel: RcppArmadillo on CRAN: New Upstream ‘Plus’

Planet Debian - Tue, 2021-04-13 20:22

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 852 other packages on CRAN.

This new release brings us the just release Armadillo 10.4.0. Upstream moves at a speed that is a little faster than the cadence CRAN likes. We release RcppArmadillo on March 9; and upstream 10.3.0 came out shortly thereafter. We aim to accomodate CRAN with (roughly) monthly (or less frequent) releases) so by the time we were ready 10.4.0 had just come out.

As it turns, the full testing had a benefit. Among the (currently) 852 CRAN packages using RcppArmadillo, two were failing tests. This is due to a subtle, but important point. Early on we realized that it would be beneficial if the standard R control over random-number creation and seeding affected Armadillo too, which Conrad accomodated kindly with an optional RNG interface—which RcppArmadillo supplies. With recent changes he made, the R side saw normally-distributed draws (via the Armadillo interface) changed, which lead to the two changes. All hail unit tests. So I mentioned this to Conrad, and with the usual Chicago-Brisbane time difference late my evening a fix was in my inbox. The CRAN upload was then halted as I had missed that due to other changes he had made random draws from a Gamma would now call std::rand() which CRAN flags. Another email to Brisbane, another late (one-line) fix back and all was good. We still encountered one package with an error but flagged this as internal to that package’s setup, so Uwe let RcppArmadillo onto CRAN, I contacted that package’s maintainer—who was very receptive and a change should be forthcoming. So with all that we have on CRAN giving us Armadillo 10.4.0.

The full set of changes follows. As Armadillo 10.3.0 was not uploaded to CRAN, its changes are included too.

Changes in RcppArmadillo version (2021-04-12)
  • Upgraded to Armadillo release 10.4.0 (Pressure Cooker)

    • faster handling of triangular matrices by log_det()

    • added log_det_sympd() for log determinant of symmetric positive matrices

    • added ARMA_WARN_LEVEL configuration option, to control the degree of emitted warning messages

    • reduced the default degree of warning messages, so that failed decompositions, failed saving/loading, etc, no longer emit warnings

  • Apply one upstream corrections for arma::randn draws when using alternative (here R) generator, and arma::randg.

Changes in RcppArmadillo version (2021-03-10)
  • Upgraded to Armadillo release 10.3 (Sunrise Chaos)

    • faster handling of symmetric positive definite matrices by pinv()

    • expanded .save() / .load() for dense matrices to handle coord_ascii format

    • for out of bounds access, element accessors now throw the more nuanced std::out_of_range exception, instead of only std::logic_error

    • improved quality of random numbers

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Categories: FLOSS Project Planets

Awesome Python Applications: Baserow

Planet Python - Tue, 2021-04-13 18:17

Baserow: Web-based no-code persistence platform, like a database meets a spreadsheet, with a REST API.


Categories: FLOSS Project Planets

Sumana Harihareswara - Cogito, Ergo Sumana: Trying to Notice What's Missing

Planet Python - Tue, 2021-04-13 16:16
I'm ploughing through some open source project email threads and thinking:

In 2010, people got together in Berlin for a Wikimedia developers' meeting .... and then a bunch of them hung around a lot longer than they'd expected, because a volcano erupted and so their flights got cancelled. As I understand it, you can trace certain architectural decisions and improvements to the discussions and pair programming from that chunk of unexpected extra in-person time.

It's conference season, at least in the northern hemisphere, and we're going into our second year of virtualized or missing technology conferences. The maintainers, users, and stakeholders of the open source software you depend on have gone more than a year without getting to quietly gossip with each other over a snack or while walking to a sponsored party. It's been more than a year since one guy has been able to make that other guy laugh and remember "ah, he's not so bad really". It's been more than a year since people could easily scribble boxes and arrows together on the back of a conference schedule or poke at the demo on someone's laptop.

We come together every once in a while to refill on trust and camaraderie and a shared understanding of what we're trying to do and who we're trying to do it for; I assume that, for some folks, those wells have now run dry.

In a tree's rings you can see the years of drought. Where, in our code and our conversations, will we see the record of this separation? Do you already see it?

Categories: FLOSS Project Planets

PyCoder’s Weekly: Issue #468 (April 13, 2021)

Planet Python - Tue, 2021-04-13 15:30

#468 – APRIL 13, 2021
View in Browser »

Python 3 Types in the Wild: A Tale of Two Type Systems

This academic paper from researchers at the Rensselaer Polytechnic Institute and IBM TJ Watson Research Center examines the MyPy and PyType tools and explores how Python developers use type annotations. The researchers collected over 70,000 Python GitHub repositories and found that only 2,678 had Python 3-style type annotations, most of which fail to type-check with either of the two tools. The paper’s third section is quite accessible and has a lot of interesting analysis.

How to Make an Awesome Python Package in 2021

The headache often associated with Python packaging is starting to fade away. Don’t believe me? Check out this step-by-step guide to creating and setting up a package repository. You’ll learn how to create a test package on TestPyPI, create a pyproject.toml file with flit, set-up linters and tests, GitHub Actions workflows, and more.

Datadog APM Provides Deeper, More Meaningful Trace Data to Quickly Resolve Application Problems

Datadog’s distributed tracing and APM generates flame graphs from real requests, enabling teams to visualize app performance in real-time. Engineering teams can seamlessly pivot to related logs and metrics without switching tools for fast troubleshooting and MTTR. Try Datadog APM free →
DATADOG sponsor

Start Contributing to Python: Your First Steps

In this quick introduction, you’ll see how you can take your first steps toward contributing to Python. You’ll discover various ways you can contribute and get to know some of the resources that will help you along the way.

Start Managing Multiple Python Versions With pyenv

Learn how to install multiple Python versions and switch between them with ease, including project-specific virtual environments, with pyenv.

PyCharm 2021.1 Is Out!

This release includes faster indexing, enhanced WSL 2 support, and an exciting new collaboration tool called Code With Me.
JETBRAINS.COM • Shared by Bartosz Zaczyński

PDFx V1.4.1 Is Now Available

PDFx is a tool to extract text, links, references, and metadata from PDF files and URLs. This release doesn’t include many new features but is rather a full update of the package repository to current Python standards, including testing and coverage, linting and static checks, GitHub workflows, and more.

Wing Python IDE 7.2.9 Released

This release includes remote development for 64-bit Raspberry Pi, improved auto-closing of quotes, optimized change tracking, and more.

PyPy v7.3.4 Released

This release includes two interpreters supporting the syntaxes for Python 2.7 and 3.7.

Discussions Why Does Pandas None | True Return False When Python None or True Returns True?

| represents the “or” operation, but when used in a boolean index in Pandas, it doesn’t behave the same way as Python’s or as you might expect — if you go off of name alone. The | operator is a bitwise operator, which only works on integer values. In fact, one could say that Python doesn’t really have a true logical “or” operator, since technically or is a short-circuit operator.

Python Jobs Intermediate Python Developer (Boulder, CO, USA)


Software Engineer (New York, NY, USA)


Data Engineer (Seattle, WA, USA)


Software Development Engineer (Indianapolis, IN, USA)

TOC Logistics International, Inc.

More Python Jobs >>>

Articles & Tutorials Using PyTorch + NumPy? You’re Making a Mistake

There’s a subtle bug that’s easy to introduce when using these packages, and it’s likely that many projects suffer from the bug. The issue has to do with how data is loaded, pre-processed, and augmented in PyTorch. If your training pipeline is bottlenecked by data pre-processing, you might decide to load the data in parallel. The canonical way of achieving this results in identical augmentations and can lead to performance degradation, but there’s a way to fix the problem.

How Python Dictionaries Work

Dictionaries are an important part of Python — not just because Python programmers use them a lot, but also because they are used internally by the Python interpreter to run Python code. In this in-depth article, you’ll learn about hash tables and hash functions as well as how Python dictionaries work behind the scenes.

[Free Virtual] How We Built Real-Time Full-Text Website Search with RediSearch

Join us at RedisConf where we feature this topic and more than 60 breakout sessions, 25% of which will be presented by Redis Enterprise customers. Learn about real-world use cases across three tracks. Register today →
REDIS LABS sponsor

Computer Vision and Embroidery

Andrew Healey’s wife wanted to find out what thread colors were used in some of the embroidery hoop images posted to the r/embroidery subreddit, so he embarked on a weekend project to solve the problem using the OpenCV computer vision library. Learn how he did it in this short, fun read, and then check out the source code over on his GitHub repository.

Getting Started With Refactoring Your Python Code

Do you think it’s time to refactor your Python code? What should you think about before starting this task? Listen Brendan Maginnis and Nick Thapen from discuss Sourcery in this episode of the Real Python Podcast. Sourcery is an automated refactoring tool that integrates into your IDE and suggests improvements to your code.

The k-Nearest Neighbors (kNN) Algorithm in Python

k-Nearest Neighbors (kNN) is a non-linear supervised machine learning algorithm suitable for both classification and regression problems. In this tutorial, you’ll learn all about the kNN algorithm in Python, including how to implement kNN from scratch, kNN hyperparameter tuning, and improving kNN performance using bagging.

Projects & Code superset: A Data Visualization and Data Exploration Platform


Python-Raytracer: A Basic Ray Tracer That Exploits NumPy Arrays and Functions to Work Fast


layout-parser: A Python Library for Document Layout Understanding


jurigged: Hot Reloading for Python


github-actions-updater: Like GitHub’s Dependabot but for GitHub Actions

GITHUB.COM/SAADMK11 • Shared by Maksudul Haque

Events Real Python Office Hours (Virtual)

April 14, 2021

GeoPython 2021

April 22 – 24, 2021

PyCon Israel 2021 (Virtual)

May 2 – 3, 2021

PyCon 2021 (Virtual)

May 12 – 18, 2021

DjangoCon Europe 2021 (Virtual)

June 2 – 6, 2021

PyCon India 2021

September 17 – 20, 2021

Happy Pythoning!
This was PyCoder’s Weekly Issue #468.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

Categories: FLOSS Project Planets

Django Weblog: Announcement of 4.x Technical Board Election Registration

Planet Python - Tue, 2021-04-13 15:16

The release last week of Django 3.2 represents the final feature release of a major release series of Django. Per DEP-10 this release triggers the election for the Technical board for the Django 4.x release cycle.

All DSF members are automatically registered electors.

If you are not a DSF member and would like to apply to vote you need to register here.

The process of electing a new Technical Board will be:

  • Registration of Electors is open for one week and ending Tuesday April 20th, 2021 AoE
  • Registration of Candidates will then be open for one week ending Tuesday April 27th, 2021 AoE
  • On Tuesday May 4th all registered and approved electors will receive an email to the email address they are registered with along with a unique code to be used when voting
  • Voting will be open for one week from May 4th
  • The results of the election will be announced when voting is finished.

If you have any questions about the elevation please contact foundation@djangoproject.com.

Chaim Kirby,

Secretary, Django Software Foundation

Categories: FLOSS Project Planets

OpenSense Labs: Drupal 9 Upgrade: FAQs

Planet Drupal - Tue, 2021-04-13 12:36
Drupal 9 Upgrade: FAQs Gurpreet Kaur Tue, 04/13/2021 - 22:06

Drupal has been in the content management game for over two decades. That is a long period of time and in that time span the CMS has accomplished a lot. It’s community of over a million Drupalists is just one part of it. With 20 years gone, Drupal is still running strong and it’s latest update is proof of that. 

Yes, in the June of 2020, amidst the pandemic, Drupal 9 was launched and it was my only sunshine at that time and I am sure many of my Drupal friends would agree with me. Drupal 9 can be deemed as one of the best offerings of Drupal, not there were many that were anything but great.

Job done easy: Drupal 9 upgrade

The latest version meant we had to take it on, why be comfortable with the past when the present can offer more comfort, right? However, the mammoth task that was the Drupal 8 upgrade, you must remember that, made many of us apprehensive and rightfully so. And that is where we were surprised, pleasantly might I add, by Drupal. 

“Easiest upgrade in a decade.”

This is how the Drupal 9 upgrade is being described and it is because you won’t have to replatform your site, you’d only be updating. It's like taking a connecting flight from London to Paris rather than leaving the airport and taking a train from the station. Connecting flights are easy.

Timeline of Drupal 9 future releases | Source: Drupal.org

Considering the fact that Drupal 9 is not a migration, the upgrade is a breeze. With the addition of new features like Olivero and Claro, being built and stabilised respectively, the new versions of Drupal 9 will become all the more appealing and a release can be expected half yearly much like Drupal 8. So, can we afford to sit back and not take the step towards upgrading our Drupal sites. The fact the Drupal 7 and Drupal 8’s end-of-life has been decided has given us all the more reason to make the upgrade. 

To help you in finding the answer to the popular question of “How to upgrade Drupal 9?”, here I am with an all inclusive checklist for this massive step you are about to take. So, let’s begin.

Is the upgrade different for Drupal 7 and Drupal 8?

This is one of the most commonly asked questions for Drupal 9 upgrade. There are still a lot of Drupal sites that are still operating on the 7th version, so it is understandable to want to know whether ‘the easiest upgrade of the decade' is applicable to them or is it different. 

Drupal 9 upgrade is different for Drupal 7 and 8 sites. This is mainly because Drupal 8 was a whole other ballgame than Drupal 7, while Drupal 9 is just a new and improved version of Drupal 8 with no deprecated APIs. Therefore, the upgrading process was bound to be different.

Let’s see how. 

Drupal 8 to Drupal 9 

Support for Drupal 8 will end on 2nd November, 2021, it will receive patch releases, but there won’t be any major updates in its feature simply because we have Drupal 9. Therefore, upgrading to the newer version is the better option. 

To upgrade from Drupal 8 to Drupal 9, 

  • You would have to see that all your projects are Drupal 9 compatible; 
  • You would have to see that your codebase is free of deprecated APIs; 
  • And then all you would have to do is run update.php. 

It sounds simple enough, right? That is because it is. Of course, you would have to perform some more steps than these three and there would be a number of tools that you would have to use for them. However, it is these three steps that are essentially the crux of upgrading from Drupal 8 onto 9.

Drupal 7 to Drupal 9

Drupal 7’s support is said to end on 28th November, 2022. Although your D7 site has quite some time ahead of it, you still have to start thinking about the upgrade as the process is not as easy as the D8 to D9 upgrade. 

You can take two routes here; 

  • You can first update to Drupal 8, avail functionality from all the modules that are yet to come in Drupal 9 and then take the easy route of upgrading to Drupal 9; 
  • Or you can directly upgrade to Drupal 9 and skip the 8th version. This would be ideal because your upgrade would have a much longer lifespan.

For the upgrade from Drupal 7, you would need three modules; 

  • The Migration Module Suite; 
  • Upgrade Status; 
  • And Drupal Module Upgrader. 

These three do everything from data and content migrations and checking the availability of modules to scanning them and updating them for the upgrade. 

So yes, the Drupal 9 upgrade from Drupal 7 and 8 is different, simply because they are very different versions of the CMS. 

Now that we have this notion clarified, let’s move on to all the nitty-gritty details of the upgrade.  

What tools would you need for the Drupal 9 upgrade?

Can you cook without the right ingredients? Can you make the perfect lasagna without the melt-in-the-mouth pasta? I know you know the answer to these two questions and that’s a big fat no. 

Therefore, the incredible journey towards the Drupal 9 experience has to start by talking about the tool you would need for it, the ingredients that’ll make the upgrade all the more appetising. I’ll stop with the cooking analogy now and come to the point. There are two mandatory tools that you will be needing.

Upgrade Status 

Upgrade Status is a module that ll but prepares your site for the upcoming upgrade you are planning to execute. It is a pretty versatile module that does all the grunt work required before making the leap to D9. 

  • Your site has to be the latest version of Drupal, that is Drupal 8.8 or 8.9, so the first thing this module does is inspects whether you are on that version, if not, it’ll tell you to update.
  • Your system needs to meet Drupal 9’s set requirements for the upgrade to go on smoothly, so the next thing it does is check if you meet them.
  • Your contributed projects also need to be updated and be compatible with Drupal 9. If they are Drupal 8 compatible, there is a high chance they’ll be for Drupal 9. So, Upgrade Status works with Update Status to ensure every project is in sync.
  • Your site needs to be compatible with Drupal 9 in other ways as well, so that is also checked and verified through phpstan.
  • Your system’s integration drush is also taken care of here.
  • Your system might have deprecated APIs, Upgrade Status finds, although there are any fixes provided; for that you'll need another tool.

Upgrade Status’ enchantment doesn’t end here, there is more to this module yet. 

  • I’ve told you that it prepares your Drupal 8 site, but it also gives you the leeway for downloading it with and without the Composer. 
  • It is also available in Drupal 7 and this version would help you in preparation for the Drupal 9 as well as Drupal 8 upgrade, if that is what you want. 
  • Now, the most alluring part, once you’ll upgrade to Drupal 9, it’ll start preparing you for the Drupal 10 upgrade! Yes, you read it right, Drupal 10 launch is in the works and you can be ready for that when it happens with this module.
Deprecation Check

Remember I mentioned deprecated APIs above, well, there are more than APIs that can be deprecated and that needs your attention before you start creating the upgrade environment for your site. That is why, Deprecation Checking and Correction Tools become a necessity.  

All in all, there are five tools for your help. 

  • The first one is an IDE or code editor, which comprehends deprecation to kickstart the process. 
  • Then there is the Drupal-check, this one performs a PHP run and let’s you know whether deprecated code is used and where.
  • Third is the Upgrade Status I talked about above, which essentially scans your entire system and finds deprecated code for you. 
  • The Drupal.org Testing System also supports deprecation inspection.
  • Finally, the Upgrade Rector Contributed Project provides a solution for all of the checking done above and that too in an automated manner.
What are the environment requirements for Drupal 9?

When you are roasting the perfect chicken, the temperature in the oven has to be just right, the timing has to be just right and the flavours have to be just right to make it come out all juicy and succulent. I know I have started with the cooking analogy again, but this is the best comparison I could think of. 

Like that perfect roast chicken, Drupal 9 also needs just the right environment from you and your system to make it work. And this involves; 

Web Servers

Drupal 9 can be run using two different web servers and you need to have the latest versions of these two.

  • For nginx, you have to have the version 0.7.x or above; 
  • And for Apache, it has to be at least 2.4.7.

PHP is the language Drupal is built on, of course you must know that. What you may not know is that you need PHP 7.3 for operating Drupal 9. The latest version 9.1.0 supports PHP 8.

Database and Hosting Structures 

Drupal 9 has specific requirements for its backend database as well as its hosts. These are; 

  • Version 5.7.8+ for MuSQL or Percona; 
  • Version 10.3.7+ for MariaDB;
  • Version 3.26+ for SQLite; 
  • Version 10 for PostgreSQL; 

And if you are planning to use Drush to build your interfaces, go for version 10, since only that is compatible with Drupal 9.

Have you prioritised the update of the core codebase?

Drupal 8 was the last major release before Drupal 9. Like any Drupal version, there are a number of minor releases being launched and you must keep up with them. If you have done so, your core codebase would be up-to-date and you’d be all set for the upgrade. 

One of the Drupal 9 upgrade requirements is that your site be updated to Drupal 8.8 or 8.9 for you to be prepared for the easiest upgrade of the decade. 

What happens if you are on a version older than 8.8?

What’ll happen is the upgrade won’t go through. Now you might ask why? I’ll give you an analogy, can you skip a few grades and land from 6th grade directly to high school? Even if you could, would you say that getting the hang of the classes you’ll have be quite arduous? It most definitely will be. So, like you can’t skip grades and you can’t skip the minor releases of Drupal 8 before going on to Drupal. Because Drupal 8.9 and Drupal 9 are somewhat similar the upgrade is a breeze, however, Drupal 8.4 and 9 do not share that kind of similarity and you might need to upgrade fast. 

From upgrades in PHP to core modules, from themes to contributed modules and from path aliases and database, almost every aspect requires updating to be compatible with Drupal 9. And only Drupal 8.8 and 8.9 have that level of API compatibility. 

How to upgrade if you have a Drupal version that precedes 8.7? 

For a version older than Drupal 8.7, you would have to perform some small code changes and database updates that are not very complex. An update to the latest version of the core would be required, and this would make the database and contributed modules ready for the 9 upgrade.

You can use Upgrade Status to ensure that all of what you just did is compatible with Drupal 9 or not. 

Then you’ll have to check for deprecated code, like we discussed two sections above. Thereafter, you become ready to update your core codebase to Drupal.

How to upgrade if you have a Drupal version that succeeds 8.8?

If you are already on Drupal 8.8, there isn’t much you have to do in order to prepare yourself. The only task to perform is running the Upgrade Status and checking compatibility. 

There is one more thing that you will have to do. And that is what your existing site is based upon because that will decide how the actual upgrade will flow. 

Your site could be based on Composer or it could be based on Tarball. Make sure you keep that in mind before starting the upgrade, as both work on different upgrade mechanisms. While the former has its own list of steps to follow, the latter is upgraded using Drush.

What about updating the custom code?

After upgrading the core codebase, you would have to pay close attention to your custom code. There is a high chance that your project will have custom modules and themes, these mean a custom code would also there and it is your responsibility to update it. 

You wouldn’t need new tools and techniques to help you here. 

  • Upgrade Status will scan all the projects for custom code deprecations; 
  • Upgrade Rector will fix any minor deprecations by itself. 

Updating the custom code is not that big a task, so it is often neglected and overlooked. However, it is important enough to hamper the entire upgrade. So, don’t skip it.

Can the contributed projects be neglected?

If you have taken care of the core and custom projects, you can’t think about leaving the contributed project without any attention. So, updating them is the next agenda on the Drupal 9 upgrade checklist. 

The reason for updating the contributed projects is the same as the core and custom projects, to check the compatibility of the projects with Drupal 9. And you must know by now, since I am repeating it for what feels like the 100th time, Upgrade Status will help you in the compatibility check.

You must know that even though a module is Drupal 9 compatible, there is the off chance that it can be a version of a major release that’ll bring API changes along with it. You have to be mindful of these modules because they can be detrimental to your site’s health. API changes that you are not prepared for can’t be efficacious by any means.

But what if a module is not Drupal 9 compatible?

There are, of course, not many of them, however there are some and you must know how to handle them. You can do any of these two things in this scenario.

  • Creating, you can create a custom code to update it;  
  • Or waiting, you can wait to see if it gets updated by the time you are ready for the final upgrade;  

What about the contributed modules with patches, do they need attention?

There are situations where you may want to update a contributed module, but an update is not in the picture, rather you have patches available and sometimes, not even that. What would you do? Pay attention to the patches, that is what.

Usually if a patch is available for a module, Upgrade Status would inform you about that and you can implement it. 

Then there is the scenario, when a contributed module has neither an update and nor a patch. For such an instance, the combination of Upgrade Status and Upgrade Rector will have you sorted. You can also check Drupal’s Contributed Modules Guidelines to help you further. 

Have you examined your content and field types thoroughly?

Auditing for site building is a must. Adding fields and customisations in Drupal is quite easy, anyone can do it. This is both good and bad. Good because anyone can do it leading to less dependence on developers, bad because anyone can do it meaning there might be redundant field types and outdated content. And in this step, you check for just that. 

  • Check and eliminate fields that were created for a once-in-a-lifetime event; don’t let them sit and waste away.
  • Remember those content types you created to add content later, well, you haven’t done that yet, so they have to go as well.
  • Update the help text; this might seem trivial to you, but being someone who relied on that piece of text to perform all my initial duties instead of pinging my manager every minute, trust me, it is important. If it is outdated, it is useless and why would you want something useless on your upgraded Drupal 9 site?

Customisations are only great until they don’t start bloating your site causing regression. You do not want that, so don’t just keep all your focus on the code, modules and themes, give some time to the content as well.

How do you become sure that your upgrade is successful?

You will execute the upgrade to make your site better than it already is, that should be the aim, right? However, despite you doing everything to the T, there is a chance that you may go in regression. Your upgrade may be doing everything that it is supposed to, but still your site’s performance can be slower than it was. 

Noone wants that scenario to become their reality, so how do you avoid that? How do you ensure that the upgrade is successful? There is certainly a way to ensure that and that’s through testing.

When you perform tests, you get to know what is truly happening with your site. Where you went wrong and what you did right won’t be hidden under a veil. For instance, performing a test to check the administrative and editorial working of the site is ideal. On paper, everything may appear sound, but in reality your editors could be struggling to make even the smallest of edits. And it is only through testing that you’ll be able to find that out. 

You can start at a smaller level with a hello world test and gradually scale up to examine the major aspects of the upgrade. 

Another thing that’ll help you in making the upgrade a success is going slow. Yes, upgrading to Drupal 9 is deemed as the easiest update of the decade, but it is still a huge task to take on. Just the sheer number of modules that wild require updating can become overwhelming, so go slow, update a few modules at one time. This is the chances of errors would be slim and the chances of success would be quite high. 

The Bottom Line 

Drupalists from across the globe had been awaiting the release of Drupal 9 for a long time, now that it is here, there is nothing holding us back from taking the upgrade leap. If I am to be honest, I’d say that I was fearful of the upgrade process. However, when Dries Buytaert say that one of the fundamental issues to address in the Drupal 9 release was the ease of installation, you cannot do anything but believe him. That’s what we should do. 

blog banner blog image Drupal 9 Drupal 9 Upgrade Blog Type Articles Is it a good read ? On
Categories: FLOSS Project Planets

Python for Beginners: Shortcut to comment out multiple lines in Python

Planet Python - Tue, 2021-04-13 10:05

We often need to comment out block of codes in python while testing or debugging the code. When a block is turned into a python comment, it doesn’t contribute in output of the program and helps to determine which function or block is generating error in the program. In this article, we will look at some shortcut to comment out multiple lines of code at once in different python IDEs. Lets see examples for each IDE one by one.

Shortcut to comment out multiple lines in Spyder

In spyder python IDE, we can comment a single line of code by selecting the line and then using the key combination ctrl+1 . This will turn the selected single line to a comment as shown below. The function given in the example adds a number and its square to a python dictionary as as key-value pair.

print("This line will be commented out.") def add_square_to_dict(x,mydict): a=x*x mydict[str(x)]=a return mydict

After pressing ctrl+1:

#print("This line will be commented out.") def add_square_to_dict(x,mydict): a=x*x mydict[str(x)]=a return mydict

The shortcut to comment out multiple lines of code in spyder IDE is to first select all the lines which need to be commented out and then the key combination ctrl+4 is pressed. This turns the entire selected lines into a python comment as shown below.

class MyNumber(): """This is the docstring of this class. It describes what this class does and all its attributes.""" def __init__(self, value): self.value=value def increment(self): """This is the docstring for this method. It describes what the method does, what are its calling conventions and what are its side effects""" self.value=self.value+1 return self.value print (MyNumber.increment.__doc__)

After pressing ctrl+4:

# ============================================================================= # # class MyNumber(): # """This is the docstring of this class. # # It describes what this class does and all its attributes.""" # def __init__(self, value): # self.value=value # def increment(self): # """This is the docstring for this method. # # It describes what the method does, what are its calling conventions and # what are its side effects""" # self.value=self.value+1 # return self.value # print (MyNumber.increment.__doc__) # =============================================================================

We can also use ctrl+1 to uncomment the lines after selecting them when they are commented out or comment out the lines after selecting them to comment them out. In some versions of Spyder ctrl+5 can be used to uncomment the lines of code.

Shortcut to comment out multiple lines in IDLE

To comment out a block of code in IDLE, we have to first select the line and then press the key combination ctrl+D. This will comment out the selected lines of code as shown below.

class MyNumber(): """This is the docstring of this class. It describes what this class does and all its attributes.""" def __init__(self, value): self.value=value def increment(self): """This is the docstring for this method. It describes what the method does, what are its calling conventions and what are its side effects""" self.value=self.value+1 return self.value print (MyNumber.increment.__doc__)

After pressing ctrl+D:

## class MyNumber(): ## """This is the docstring of this class. ## ## It describes what this class does and all its attributes.""" ## def __init__(self, value): ## self.value=value ## def increment(self): ## """This is the docstring for this method. ## ## It describes what the method does, what are its calling conventions and ## what are its side effects""" ## self.value=self.value+1 ## return self.value ## print (MyNumber.increment.__doc__)

To uncomment the lines of code, we just have to select the lines and then press ctrl+shift+d. This will uncomment the selected lines.

Shortcut to comment out multiple lines in Jupyter Notebook

We can use ctrl+/ to comment out the selected lines of python code in Jupyter Notebook. This turns selected lines of code into comment as shown below.

class MyNumber(): """This is the docstring of this class. It describes what this class does and all its attributes.""" def __init__(self, value): self.value=value def increment(self): """This is the docstring for this method. It describes what the method does, what are its calling conventions and what are its side effects""" self.value=self.value+1 return self.value print (MyNumber.increment.__doc__)

After pressing ctrl+/:

# class MyNumber(): # """This is the docstring of this class. # It describes what this class does and all its attributes.""" # def __init__(self, value): # self.value=value # def increment(self): # """This is the docstring for this method. # It describes what the method does, what are its calling conventions and # what are its side effects""" # self.value=self.value+1 # return self.value # print (MyNumber.increment.__doc__)

To uncomment the selected lines, we just have to again press ctrl+/.

Comment out multiple lines in Pycharm

If we have to comment out multiple lines of code in Pycharm, we can select the lines to be commented out and then press ctrl+shift+/ . To uncomment the lines, we just have to select the lines and then again press ctrl+shift+/.


In this article, we have seen shortcuts to comment out multiple lines at once in python different IDEs like spyder, IDLE, Jupyter Notebook and PyCharm. Stay tuned for more informative articles.

The post Shortcut to comment out multiple lines in Python appeared first on PythonForBeginners.com.

Categories: FLOSS Project Planets

Real Python: Start Managing Multiple Python Versions With pyenv

Planet Python - Tue, 2021-04-13 10:00

Have you ever wanted to contribute to a project that supports multiple versions of Python but aren’t sure how you would easily test all the versions? Are you ever curious about the latest and greatest versions of Python? Maybe you’d like to try out these new features, but you don’t want to worry about messing up your development environment. Luckily, managing multiple versions of Python doesn’t have to be confusing if you use pyenv.

This course will provide you with a great overview of how to maximize your time spent working on projects and minimize the time spent in headaches trying to use the right version of Python.

In this course, you’ll learn how to:

  • Install multiple versions of Python
  • Install the latest development version of Python
  • Switch between the installed versions
  • Use virtual environments with pyenv
  • Activate different Python versions and virtual environments automatically

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Categories: FLOSS Project Planets