Feeds

subtext: in my opinion, and for companies (and their users) that want a good balance between protecting their core business with Open Source ideals.

Following up to my thoughts on the case for funding Open Source, there is a second topic I want to discuss in more detail: Open Source and commercialization. As our founder likes to say: Open Source is not a business model. And indeed it really isn't. However, this does not mean that Open Source and Open Source licenses aren't a critical consideration for a technology company and a fascinating interconnection between the business model and license texts.

As some of you might know I'm a strong proponent of the concept now branded as “Fair Source” which we support at Sentry. Fair Source is defined by a family of springing licenses that give you the right to read and modify code, while also providing an exclusivity period for the original creator to protect their core business. After a designated time frame, the code transitions into Open Source via a process called DOSP: Delayed Open Source Publication. This is not an entirely new idea, and I have been writing about it a few times before [1] [2].

A recurring conversation I have in this context is the AGPL (Affero General Public License) as an alternative vehicle for balancing business goals and Open Source ideals. This topic also has resurfaced recently because of Elasticsearch'es Open Source, Again post where they announced that they will license Elasticsearch under the AGPL.

In my view, while AGPL is a true Open Source license, it is an inferior choice compared to the FSL (the Functional Source License, a Fair Source license) for many projects. Let me explain my reasoning.

The Single Vendor Model

When you take a project like Sentry, which started as an Open Source project and later turned into a VC funded company, its model revolves around a commercial entity being in charge. That model is often referred to as “single vendor.” This is also the case with companies like Clickhouse Inc. or Elastic and their respective projects.

Sentry today is no longer Open Source, it's Fair Source (FSL licensed). Elastic on the other hand is indeed unquestionable Open Source (AGPL among others). What both projects have in common is that they value brand (including trademarks), that they have strong opinions on how that project should be run, and they use a CLA to give themselves the right to re-licenses it under other terms.

In a "single vendor" setup, the company behind the project holds significant power (for ~150 years give or take).

The Illusion of Equality

When you look at the AGPL as a license it's easy to imagine that everybody is equal. Every contributor to a project agrees with the underlying license assumptions of the AGPL and acts accordingly. However, in practice, things are more complicated — especially when it comes to commercial usage. Many legal departments are wary of the AGPL and the broader GPL family of licenses. Some challenges are also inherent to the licenses such as not being able to publish *GPL code to the app store.

You can see this also with Elasticsearch. The code is not just AGPL licensed, you can also retrieve it under the ELv2 and SSPL licensing terms. Something that Elastic can do due to the CLAs in place.

Compare this to Linux, which is licensed under GPLv2 with a syscall exception. This very specific license was chosen by Linus Torvalds to ensure the project's continued success while keeping it truly open. In Linux' case, no single entity has more rights than anyone else. There is not even a realistic option to relicense to a newer version of the GPL.

The FSL explicitly recognizes the reality that the single vendor holds significant power but balances it by ensuring that this power diminishes over time. This idea can also be found in copyright law, where a creator's work eventually enters the public domain. A key difference with software though is that it continuously evolves, making it hard to pinpoint when it might eventually become public domain as thousands of people contribute to it.

The FSL is much more aggressive in that aspect. If we run Sentry into the ground and the business fails, within two years, anyone can pick up the pieces and revive it like a Phoenix from the ashes. This isn't just hypothetical. Bryan Cantrill recently mentioned the desire of Oxide forking CockroachDB once its BUSL change date kicks in. While that day hasn't come yet, it's a real possibility.

Dying Companies

Let's face it: companies fail. I have no intentions for Sentry to be one of them, but you never know. Companies also don't just die just once, they can do so repeatedly. Xapian is an example I like to quote here. It started out as a GPL v2+ licensed search project called Muscat which was built at Cambridge. After several commercial acquisitions and transitions, the project eventually became closed source (which was possible because the creators held the copyright). Some of the original creators together with the community forked the last GPLv2 version into a project that eventually became known as Xapian.

What's the catch? The catch is that the only people who could license it more liberally than GPLv2 are long gone from the project. Xapian refers to its current license “a historical accident”. The license choice causes some challenges specifically to how Xapian is embedded. There are three remaining entities that would need to agree to the relicensing. From my understanding none of those entities commercially use Xapian's original code today but also have no interest in actually supporting a potential relicensing.

Unlike trademark law which has a concept of abandonment, the copyright situation is stricter. It would take two lifetimes for Xapian to enter the public domain and at that point it will be probably be mostly for archival purposes.

Equal Grounds Now or Later

If Xapian's original code would have been FSL licensed, it would have been Apache 2.0 (or MIT with the alternative model) many times over. You don't need to hope that the original license holder still cares, by the time you get hold of the source code, you already have an irrevocable promise that it will eventually turn into Apache 2.0 (or MIT with the alternative license choice) which is about as non-strings attached as it can get.

So in some ways a comparison is “AGPL now and forever” vs “FSL now, Apache 2.0/MIT in two years”.

That's not to say that AGPL (or SSPL) don't have their merits. Xapian as much as it might suffer from their accidental license choice also is a successful Open Source project that helped a lot of developers out there. Maybe the license did in fact work out well for them, and because everybody is in the same boat it also has created a community of equals.

I do believe however it's important to recognize that “single-vendor AGPL with a CLA” is absolutely not the same as “community driven AGPL project without the CLA”.

The title claims that FSL balances Open Source better than AGPL, and it's fair to question how a license that isn't Open Source can achieve that. The key lies in understanding that Fair Source is built on the concept of delayed Open Source. Yes, there's a waiting period, but it’s a relatively short one: just two years. Count to two and the code transitions to full, unshackled openness. And that transition to Open Source is a promise that can't be taken from you.

[1]Originally about the BUSL license which introduced the idea (Open Source, SaaS and Monetization) [2]Later about our own DOSP based license, the FSL (FSL: A License For the Bazaar, Not the Cathedral).

Linux snapshots? Back in the day (mid 2000’s) ReiserFS was my go to Linux filesystem, it was fast & reliable. But then after its creator...

Since dolphin-plugins 24.05, you can git clone from dolphin with dolphin-plugins git plugin.

Once the plugins are installed and Git is enabled in the context menu settings, you have this context menu action available:

And this shows this git clone dialog (with my french locale):

You can paste a git repository url and it will fetch its branches. If you happen to have a url in your clipboard or a git clone command line, it will directly extract it as the repository url.

This was spearheaded by Nikolai Krasheninnikov, thanks. I participated a bit as well and reviewed it.

There is still opportunity to improve the git implementation, like having a better commit dialog. That would be a nice and simple new contributor opportunity :)

The Skrooge Team announces the release 2.33.0 version of its popular Personal Finances Manager based on KDE Frameworks. Changelog

• Correction bug 485366: Differnce in different Report-Selections (2)
• Correction bug 484156: "Monthly Report" Last month grahic failure
• Correction bug 489784: Importing a QIF the account type is changed
• Correction bug 492287: Skrooge 2.32.0 freezes while opening existing .skg files, but import is fast
• Correction bug 493062: Another Problem with QIF and Character "/"
• Correction bug: Fix mimetype of .sta file
• Correction bug: Remove dependency on QCA. So, old password protected files are no more supported.
• Correction bug: Fix translation issue in "Incomes vs Expenditures" dashboard widget

Welcome to the second post in our "This Week in KDE Apps" series! If you missed it we just announced this new series last week and our goal is to cover as much as possible of what's happening in the KDE world and complete Nate's This Week in Plasma.

This week we had a new Ruqola, KDE's Rocket.chat client, release and a new GCompris release. There is also news regarding NeoChat, KDE's Matrix chat client; Itinerary, the travel assistant that lets you plan all your trips; the Dolphin file browser; Marble, KDE's map application application; the Okular document view and more.

Let's get started!

Dolphin

Dolphin now ensures that the Trash always correctly shows all trashed files of all connected storage devices, even if they get dynamically connected or disconnected. (Akseli Lahtinen, 24.12.0. Link)

Made the lists of recent files and locations update more reliably. (Méven Car, 24.12.0. Link)

Filelight

Resolved a bug that caused the graphs to sometimes be mis-rendered until hovered with the pointer. (Harald Sitter, 24.12.0. Link)

Itinerary

Itinerary can now handle geo:// URLs by opening the "Plan Trip" page with a pre-selected arrival location. This is supported both on Android and Linux. (David Redondo, 24.12.0. Link)

Itinerary now defaults to showing the new two-level trips/timeline view. (Volker Krause, 24.12.0. Link)

Trip groups can now be restored from backups. (Volker Krause, 24.12.0. Link)

KDE PIM

Fixed a crash when auto-discovery of or connection to Exchange Web Resources has failed (Daniel Vrátil, 24.08.2. Link)

Okular

Implemented support for more types of items in comboboxes of PDF forms. (Pratham Gandhi, 24.12.0. Link)

Improved the speed and correctness of printing for the common case of not needing to rasterize the document and not needing to print annotations. (Oliver Sander, Albert Astals Cid, and Nicolas Fella sponsored by TU Dresden, 24.12.0. Link)

Improved the UX of digitally signing a document (Nicolas Fella sponsored by TU Dresden, 24.12.0. Link)

Spectacle

Improved visual quality for screenshots taken at non-fractional scale factors. (Noah Davis, 24.08.2. Link)

Marble

Marble was ported to Qt6. (Gilles Caulier & Carl Schwan, 24.12.0. Link)

The Kirigami UI — which was last updated in 2017 — was significantly rewritten and modernized. (Carl Schwan, 24.12.0. Link).

NeoChat

The NeoChat team meet at the Matrix Conference in Berlin which ended up being productive! Learn more at Carl's mastodon post.

Fix two semi-common crashes reported to Sentry. (Tobias Fella & James Graham, 24.08.2. Link 1, Link 2, Link 3)

An F-Droid build is again available in KDE's F-Droid repository. (Tobias Fella & Volker Krause)

Fixed various visual papercuts for NeoChat on Android and Plasma Mobile. (Carl Schwan, 24.08.2 and 24.12.0. Link, Link, Link, Link, ...)

KDE Connect

The Ping and Find Remote Device plugins were ported to Kotlin. (TPJ Schikhof, Link 1, Link 2)

LabPlot

Add possibility to apply functions on curves directly on the plot. This makes it is possible to, for example, calculate the differences between curves, scale or shift curves, etc. (Martin Marmsoler. Link)

Kate

Improved the visuals of Kate's inline code formatting tooltips. The content of these tooltips can now also be displayed in a special context tool view which can be enabled in the Behavior settings (Karthik Nishanth, 24.12.0. Link)

GCompris

GCompris 4.2 is out with some bug fixes and graphical improvements for multiple activities. More information is available in the release announcement.

Ruqola

Ruqola 2.3.0 — the KDE Rocket.chat client — is out with an administrator mode, a new welcome page, and better support for the custom markdown syntax of Rocket.chat. More information is available in the release announcement.

Other

Eamonn Rea made many Kirigami application remember their size across launches:

• AudioTube
• Kontrast
• KRecorder
• KWeather

...And Everything Else

This blog only covers the tip of the iceberg! If you’re hungry for more, check out Nate's blog about Plasma and KDE's Planet, where you can find more news from other KDE contributors.

Get Involved

The KDE organization has become important in the world, and your time and contributions have helped achieve that status. As we grow, it’s going to be equally important that your support become sustainable.

We need you for this to happen. You can help KDE by becoming an active community member and getting involved. Each contributor makes a huge difference in KDE; you are not a number or a cog in a machine! You don’t have to be a programmer, either. There are many things you can do: you can help hunt and confirm bugs, even maybe solve them; contribute designs for wallpapers, web pages, icons and app interfaces; translate messages and menu items into your own language; promote KDE in your local community; and a ton more things.

You can also help us by donating. Any monetary contribution, however small, will help us cover operational costs, salaries, travel expenses for contributors and in general help KDE continue bringing Free Software to the world.

PyCon Taiwan 2024 Keynote About • Blog • Newsletter • Links PyCon Taiwan 2024 Keynote

Published 2024-09-22 by Seth Larson
Reading time: minutes

Here are my slides and overview of my PyCon Taiwan 2024 Keynote titled "Bytes, Pipes, and People". The video will be published to YouTube, subscribe to the PyCon Taiwan YouTube channel to be notified when available.

Software security has historically been treated as extra or "nice-to-have", not a core feature that users expect. This means we have accumulated plenty of tech debt. Now there are growing incentives and requirements for producing secure software to meet user expectations.

Luckily for us, many of the tools, data, and systems already exist to help us build a culture of security for Python. These tools help relay messages between software creators and users so we can collaborate on this shared goal.

By actively participating you are starting the positive feedback loop of software security, making users safer faster!

Below is a list of items that actions can implement to build a culture of security for Python:

Maintainers

• Adopt Trusted Publishers if you use GitHub Actions, GitLab CI/CD, Google Cloud Build, or ActiveState to publish Python packages.
• Use lock files for the build and publish workflow, such as pip-tools, Poetry, or PDM.
• Adopt a lightweight security policy. Do not stress about CVEs: fix, release, publish a CVE.
• Contribute new insecure code detections to Bandit.

Users

• Update dependencies that have vulnerabilities. Prioritize projects that are connected to the internet.
• Update software on a semi-regular basis to avoid out-of-date and end-of-life software. Staying up-to-date helps you being able to upgrade to fixed versions in the future.
• Run tests with PYTHONWARNINGS with DeprecationWarning and PendingDeprecationWarning set to errors to avoid missing deprecated features.
• Create a secure open source usage policy, using verified data to evaluate open source projects. Do not install new projects without checking your policy first.
• If you need a Software Bill-of-Materials document there are tools available to generate one. Those tools will improve over time from new Python package SBOM standards.
• Add a vulnerability scanner like pip-audit, Grype, or Trivy.

Tools and Links

• What is Software Bill-of-Materials ("SBOM")?
• Trusted Publishers
• PyPI blog
• Bandit
• Warnings in Python
• pip-audit
• Scientific Python SPEC-8: "Securing the Release Process"
• Supply chain security threats (SLSA)
• Grype
• Trivy
• Ecosystem.ms
• Libraries.io
• Deps.dev
• Trusty
• pip-tools
• Poetry
• PDM
• uv
• Sigstore Python
• Yanking Python packages

References

• HTTP Archive
• Sonatype 2023 Annual State of Software Supply Chain Report
• Kushal Das for Python Language Summit photos
• StackOverflow

Thanks for reading! ♡ Did you find this article helpful and want more content like it? Get notified of new posts by subscribing to the RSS feed or the email newsletter.

This work is licensed under CC BY-SA 4.0

The beta of Kubuntu Oracular Oriole (to become 24.10 in October) has now been released, and is available for download.

This milestone features images for Kubuntu and other Ubuntu flavours.

Pre-releases of Kubuntu Mantic Minotaur are not recommended for:

• Anyone needing a stable system
• Regular users who are not aware of pre-release issues
• Anyone in a production environment with data or workflows that need to be reliable

They are, however, recommended for:

• Regular users who want to help us test by finding, reporting, and/or fixing bugs
• Kubuntu, KDE, and Qt developers
• Other Ubuntu flavour developers

The Beta includes some software updates that are ready for broader testing. However, it is an early set of images, so you should expect some bugs.

We STRONGLY advise testers to read the Kubuntu 24.10 Beta release notes before installing, and in particular the section on ‘Known issues‘.

You can also find more information about the entire 24.10 release (base, kernel, graphics etc) in the main Ubuntu Beta release notes and announcement.



To enable Flatpaks in KDE’s Discover in Kubuntu 24.10, run this command:

sudo apt install flatpak plasma-discover-backend-flatpak


To enable the largest Flatpak repository, run this command:

flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo


Log out and log back in (or restart) to re-initialize the XDG_DATA_DIRS variable, otherwise, newly installed Flatpak apps will not run or appear in the startup menu.

We can prompt our users for input with Python's built-in input function.

Table of contents

1. Prompting for user input
2. Customizing the prompt text
3. Prompt users with Python's built-in input function

Prompting for user input

Python has a built-in input function that we can use to prompt a user of our program to enter some text:

>>> color = input("Favorite color:") Favorite color:▯

Our Python REPL is now hanging and waiting for us (the user) to input text. Let's type purple:

>>> color = input("Favorite color:") Favorite color:purple▯

After the user has typed the text they'd like to enter, they can hit the Enter key to lock-in the value.

>>> color = input("Favorite color:") Favorite color:purple >>>

Now the color variable contains the string purple:

>>> color 'purple' Customizing the prompt text

Note that the input function …

Read the full article: https://www.pythonmorsels.com/prompting-for-input/

After years on the waiting list, May First was just given a /24 block of IP addresses. Excellent.

Now we want to start using them for, among other things, sending email.

I haven’t added a new IP address to our mail relays in a while and things seems to change regularly in the world of email so I’m curious: what’s the best 2024 way to warm up IP addresses, particularly using postfix?

Sendergrid has a nice page on the topic. It establishes the number of messages to send per day. But I’m not entirely sure how to fit messages per day into our setup.

We use round robin DNS to direct email to one of several dozen email relay servers using postfix. And unfortunately our DNS software (knot) doesn’t have a way to add weights to ensure some IPs show up more often than others (much less limit the specific number of messages a given relay should get).

Postfix has some nice knobs for rate limiting, particularly: default_destination_recipient_limit and default_destination_rate_delay

If default_destination_recipient_limit is over 1, then default_destination_rate_delay is equal to the minimum delay between sending email to the same domain.

So, I’m staring our IP addresses out at 30m - which prevents any single domain from receiving more than 2 messages per hour. Sadly, there are a lot of domain names that deliver to the same Google addresses, so I am not sure I can accurately control how many messages Google sees coming from any given IP. But it’s a start.

A bigger problem is that messages that exceed the limit hang out in the active queue until they can be sent without violating the rate limit. Since I can’t fully control the number of messages a given queue receives (due to my inability to control the DNS round robin weights), a lot of messages are going to be severely delayed, especially ones with an @gmail.com domain name.

I know I can temporarily set relayhost to a different queue and flush deferred messages, however, as far as I can tell, it doesn’t work with active messages.

To help mitigate the problem I’m only using our bulk mail queue to warm up IPs, but really, this is not ideal.

Suggestions welcome!

With DrupalCon Barcelona 2024 only two weeks later, the focus this week is on tying up loose ends. That already started last week, but of the milestone 0.1.0 priorities that product lead Lauri identified, 15 are still left after last week: we need to fix 1.5/day to get to zero.

Major loose ends

I titled last week’s update drag and drop party because there were so many improvements on that front. But one critical piece was still missing: it was painful to drag components into locations that are difficult to visualize: top and bottom of slots, even more so for adjacent slots. Bálint “balintbrews” Kléri brought one additional improvement:

Your browser does not support playing videos. You can download it instead.

Drag a component and drop it into truly any location.
Issue #3471169, video by Bálint.

Beautiful isn’t it?! I couldn’t say it better than Jesse “jessebaker” Baker in his review: This is such a marked improvement and honestly it’s already better than I dared hope it would be, especially this early. Excellent work.

Ben “bnjmnm” Mullins finished the highest-impact loose end: he ensured all single-value field types work correctly.

Quick recap: the back end and front end were developed independently, with the back end focusing on reusing Drupal concepts (i.e. field types + field widgets), the front end on the ideal UX, and then Ben having bridged the gap between the two by lifting a subset of the JSX Theme Engine. Ben had gotten the foundations in place in week 10 and then brought basic Redux integration for live updates of the preview in week 11. We knew there were lots of loose ends, but we had no choice but to move on to higher priorities.

This week, Ben finally had the chance to introduce sorely needed end-to-end (E2E) test coverage, which ensures that a whole range of field widgets work as expected not only in the original field widget sense (focus on server side), but also the live update sense (focus on client side): the Redux integration must be aware of when the values the user is typing is valid. More work remains to be done there, but the introduction of the prop-types.cy.js E2E Cypress test marks a significant milestone! 1 Next steps will be tracked in the corresponding meta/plan issue.

Lots of usability bugs squashed

Jesse coordinated the squashing of many usability bugs — not really broken things, but things that should happen to not have a clunky UX:

• Omkar “omkar-pd” Deshpande and Jesse: #3470933: Close the insert panel when clicking outside the panel — why force the users to do extra clicking when we can close something when user actions indicate a UI piece is not currently necessary?
• Amine “boulaffasae” Boulaffass and Bálint: #3472634: Component List doesn’t scroll — when installing additional Single Directory Components (SDCs), the list can grow beyond the viewport height and this fix allows accessing those :D
• Gaurav “gauravvvv”, Shyam “shyam_bhatt” Bhatt and Bálint: #3472488: The preview container height will overflow outside the canvas — a fixed canvas size prevented building tall pages!
• Utkarsh “utkarsh_33” and fazilitehreem expanded upon the right-click support they added last week: #3472053: Show context menu on right clicking components in the Layers panel — because what is possible on the canvas/in the preview, should also be possible in the “layers” panel

Missed a prior week? See all posts tagged Experience Builder.

Goal: make it possible to follow high-level progress by reading ~5 minutes/week. I hope this empowers more people to contribute when their unique skills can best be put to use!

For more detail, join the #experience-builder Slack channel. Check out the pinned items at the top!

Back-end improvements

• Feliksas “f.mazeikis” Mazeikis landed the most important server-side change of the week: surfacing the reason for an SDC not being available in XB. This is obviously critical information for SDC developers. More needs to happen in this space, but this in particular is a huge productivity boost for the people working on the Starshot Demo Design System, because that too will be featured in the Driesnote at DrupalCon Barcelona :)

Initial UI that shows the reason why an SDC is not available in XB.”
Issue #3469684, image by me.

• Ted “tedbow” Bowman and Dave “longwave” Long made the most critical response a CacheableJsonResponse instead of a JsonResponse, which means it can now be served by Drupal’s Dynamic Page Cache while guaranteeing up-to-date information. Similar to the “component list scroll” bug fix above, this was only perceptible with many SDCs installed (and Xdebug enabled), but it still is an important leap forward. 2
• Deepak “deepakkm” Mishra, Ted and I added support for propless SDCs — a silly oversight :D
• Finally, I clarified the “shape matching bits” by improving the docs, code organization (it dates back to the earliest XB days!), making its location and experimental nature more explicit in the CODEOWNERS file, and introducing a Shape matching issue queue component. 3

Unfortunately, during the week, Lauri prioritized several more issues, so we ended this week with … 12 — only 3 fewer than we started with :/ That means we’ll need to land >2/day in the next week to get to zero. Fortunately, most of those twelve are already pretty far along.

Who of you have ever been able to chill towards a deadline, and how did you do that? :D

Week 18 was September 9–15, 2024.

1. It is not comprehensive yet, but then again, nor does Drupal core have field types that can populate all of the possible prop shapes that Single Directory Components’ JSON schema are able to express. Next steps here are growing the test coverage to reach full confidence for the prop shapes that we do have field types + widgets for. ↩︎

2. And as Ben pointed out: this is very cacheable by design, so it’s not a case of premature optimization. ↩︎

3. I think it’d be good to have each area in the CODEOWNERS file correspond to an issue queue component. Thoughts? Let me know! ↩︎

• Drupal
• Acquia
• Experience Builder

• Drag a component and drop it into truly any location.
• Initial UI that shows the reason why an SDC is not available in XB.

We talk with Ryan Price from ICF about a recent project he worked on involving updating a legacy Drupal 7 install with modern Drupal 10 techniques. 

URLs mentioned

• https://www.iesabroad.org/ 

• https://www.studyabroadfoundation.org/

• Pacific Northwest Drupal Summit - October 11-13, 2024.

DrupalEasy News

• Professional module development - 15 weeks, 90 hours, live, online course.  

• Drupal Career Online - 12 weeks, 77 hours, live online, beginner-focused course. 

Audio transcript

We're using the machine-driven Amazon Transcribe service to provide an audio transcript of this episode.

Subscribe

Subscribe to our podcast on iTunes, iHeart, Amazon, YouTube, or Spotify.

If you'd like to leave us a voicemail, call 321-396-2340. Please keep in mind that we might play your voicemail during one of our future podcasts. Feel free to call in with suggestions, rants, questions, or corrections. If you'd rather just send us an email, please use our contact page.

Credits

Podcast edited by Amelia Anello.

The post Using Pandas to Read JSON from URL appeared first on Erik Marsja.

When working with data in Python, using Pandas to read JSON from URL is an excellent tool that lets you directly load JSON data from a web source into a Pandas dataframe. This tutorial will teach you the steps to accomplish this task, building upon our previous discussions on reading JSON with Python more generally.

Table of Contents

• How to use Pandas to Read JSON from URL
• Parameters of the read_json Method
• Summary
• Resources

How to use Pandas to Read JSON from URL

First, let us look at a simple example of using Pandas to read JSON from a URL.

import pandas as pd # URL containing JSON data url = "http://api.open-notify.org/astros.json" # Read JSON data from URL into a DataFrame df = pd.read_json(url) # Display the dataframe print(df)

In the code chunk above, we start by importing the Pandas library. The URL variable contains the web address where the JSON data is hosted. The pd.read_json(url) function is then used to read the JSON data from the URL and load it into a Pandas DataFrame, which is a two-dimensional labeled data structure with columns of potentially different types. Finally, print(df) displays the DataFrame, allowing us to see the imported data in tabular format.

Now that we have seen a basic example, let us learn more about the parameters of the pd.read_json() method to understand how we can customize the reading process.

Parameters of the read_json Method

The pd.read_json() method has several parameters that allow you to fine-tune how the JSON data is read and converted into a dataframe. Here is an overview of the most important parameters:

• path_or_buf: The string containing the URL or the path to the JSON file. This is the source of the JSON data that will be read.
• orient: Defines the expected JSON string format. Default is ‘columns’. This parameter specifies the orientation of the JSON data. Other options include ‘split’, ‘records’, ‘index’, and ‘values’.
• typ: Specifies the type of object to be returned. Default is ‘frame’. This parameter can be set to ‘series’ if you want to return a Series instead of a DataFrame.
• dtype: Determines whether to infer types of objects. Default is ‘None’. This parameter can be used to specify the data type for each column.
• convert_axes: Whether to convert the axes to another type. Default is ‘True’. This parameter allows you to convert the axes to a specified data type.
• convert_dates: List of columns to convert to dates. Default is ‘True’. This parameter can be used to specify which columns should be parsed as dates.
• keep_default_dates: Whether to include default date parsers. Default is ‘True’. This parameter determines whether to use the default date parsers provided by Pandas.
• precise_float: Whether to use a high precision floating point converter. Default is ‘False’. This parameter can be set to ‘True’ if you need high precision for float values.
• date_unit: Unit for encoding datetime. Default is ‘None’. This parameter can be used to specify the time unit for encoding datetime objects.
• encoding: Specifies the encoding to be used. Default is ‘utf-8’. This parameter determines the encoding for reading the JSON data.
• lines: Whether to read the JSON file as a JSON object per line. Default is ‘False’. This parameter can be set to ‘True’ if the JSON data is in a line-delimited format.

With these parameters allows you to better control how JSON data is read and processed, enabling you to tailor the DataFrame to your needs.

Summary

To summarize, we have learned how to use Pandas to read JSON data from a URL. We explored a practical example and detailed the parameters of the pd.read_json() method, enhancing our ability to customize the data reading process. Handling nested JSON data can be more challenging, but that will be covered in a future post.
I would appreciate it if you could share this post and leave your comments below. Your feedback is invaluable!

Resources

Here are some other reading data-related tutorials:

• How to Convert JSON to Excel in Python with Pandas
• How to use Pandas read_html to Scrape Data from HTML Tables

The post Using Pandas to Read JSON from URL appeared first on Erik Marsja.

This post is a review for Computing Reviews for 50 years of queries , a article published in Communications of the ACM

The relational model is probably the one innovation that brought computers to the mainstream for business users. This article by Donald Chamberlin, creator of one of the first query languages (that evolved into the ubiquitous SQL), presents its history as a commemoration of the 50th anniversary of his publication of said query language.

The article begins by giving background on information processing before the advent of today’s database management systems: with systems storing and processing information based on sequential-only magnetic tapes in the 1950s, adopting a record-based, fixed-format filing system was far from natural. The late 1960s and early 1970s saw many fundamental advances, among which one of the best known is E. F. Codd’s relational model. The first five pages (out of 12) present the evolution of the data management community up to the 1974 SIGFIDET conference. This conference was so important in the eyes of the author that, in his words, it is the event that “starts the clock” on 50 years of relational databases.

The second part of the article tells about the growth of the structured English query language (SEQUEL)– eventually renamed SQL–including the importance of its standardization and its presence in commercial products as the dominant database language since the late 1970s. Chamberlin presents short histories of the various implementations, many of which remain dominant names today, that is, Oracle, Informix, and DB2. Entering the 1990s, open-source communities introduced MySQL, PostgreSQL, and SQLite.

The final part of the article presents controversies and criticisms related to SQL and the relational database model as a whole. Chamberlin presents the main points of controversy throughout the years: 1) the SQL language lacks orthogonality; 2) SQL tables, unlike formal relations, might contain null values; and 3) SQL tables, unlike formal relations, may contain duplicate rows. He explains the issues and tradeoffs that guided the language design as it unfolded. Finally, a section presents several points that explain how SQL and the relational model have remained, for 50 years, a “winning concept,” as well as some thoughts regarding the NoSQL movement that gained traction in the 2010s.

This article is written with clear language and structure, making it easy and pleasant to read. It does not drive a technical point, but instead is a recap on half a century of developments in one of the fields most important to the commercial development of computing, written by one of the greatest authorities on the topic.

The core Plasma team has entered full-on bug-fixing mode until Plasma 6.2.1, and what a week of bug-fixes it was! We’re nailing regressions reported in the beta release as they appear, as well as older bugs we didn’t manage to get to yet. I’m incredibly impressed with the team and their tremendous work this week!

In addition, a few features and UI changes postponed from Plasma 6.2 have started to land in 6.3.

Notable New Features

Plasma’s Widget Explorer sidebar now gives you the opportunity to remove all instances of a widget, which can help when you have a stuck widget hidden somewhere (Niccolò Venerandi, Plasma 6.3.0. Link):

Notable UI Improvements

With multiple screens showing wallpaper slideshows, the transition times for each screen’s slideshow are now synced so they happen at the same moment (Sebastian Meyer, Plasma 6.3.0. Link)

Loosened the criteria for mouse button re-binding, so for example you can now swap the “Back” and “Middle Click” buttons (Paul Dann, Plasma 6.3.0. Link)

The Emoji Selector window now remembers its window position (on X11) and size (me: Nate Graham, Plasma 6.3.0. Link)

For notifications that you can pause, the “Pause” button now becomes a “Resume” button after you press it, instead of being checkable (me: Nate Graham, Plasma 6.3.0. Link)

In Plasma’s Networks widget, networks you’re not connected to no longer have Configure buttons, because clicking on them didn’t actually do anything useful anyway (me: Nate Graham, Plasma 6.3.0. Link)

The checkboxes on System Settings’ Bluetooth page have been replaced with switches, because they’re for instant-apply settings (Christoph Wolk, Plasma 6.3.0. Link)

Notable Bug Fixes

Fixed a bug that could cause System Settings to crash when leaving its Wallpaper page (Méven Car, Plasma 6.2.0. Link)

Fixed a bug that could cause Plasma to crash under certain circumstances when applying certain global themes (Nicolas Fella, Plasma 6.2.0. Link)

Fixed a case where plasma-browser-integration-host (the process that communicates with web browsers that have Plasma Browser Integration installed) could crash when windows were closed (Méven Car, Plasma 6.2.0. Link)

Closing a Plasma widget while a tooltip for one of its UI elements is open no longer causes the tooltip to remain visible and awkwardly jump to the panel itself; now the tooltip also disappears as you would expect. This also fixes a related crash (Niccolò Venerandi, Plasma 6.2.0. Link 1 and link 2)

Fixed a bug that could cause the logout process to get stuck on a black screen on X11 (David Edmundson, Plasma 6.2.0. Link)

Moving a window to another virtual desktop using a method that does not switch to that desktop no longer leaves no window focused on the current desktop (David Edmundson, Plasma 6.2.0. Link)

Fixed a bug that caused real-fake-session-restore to not work properly on Wayland (David Edmundson, Plasma 6.2.0. Link)

The footer on Discover’s updates page no longer sometimes gets visually broken (Harald Sitter, Plasma 6.2.0. Link)

When configuring Plasma’s Digital Clock widget to show seconds and using 24-hour time, the time display no longer gets cut off on narrow vertical panels (Akseli Lahtinen, Plasma 6.2.0. Link)

Fixed an issue that could cause Plasma’s edit mode to misbehave in strange ways when there are widgets on the desktop and auto-hide panels (Marco Martin, Plasma 6.2.0. Link)

Plasma’s “Minimize All Windows” widget no longer sometimes fails to restore them on Wayland; now this always works (Christoph Wolk, Plasma 6.2.0. Link)

System Settings once again remembers its window size, position (on X11), and maximization state as expected (Akseli Lahtinen, Plasma 6.2.0. Link)

On X11, you can now open the color scheme editor on System Settings’ Colors page more than once (Albert Astals Cid, Plasma 6.2.0. Link)

Fixed the “two clicks to rename” feature on the Plasma desktop when using systemwide double-click mode so that it works properly (Christoph Wolk, Plasma 6.2.0. Link)

Fixed a bug that caused it to be impossible to change power profiles in the Plasma widget under certain circumstances. There may be other such bugs too BTW; this only fixed one of them (Jakob Petsovits, Plasma 6.2.0. Link)

KWin’s global “Move Mouse to Focus” shortcut now does something more sensible on Wayland by moving the pointer to the focused window instead of the top-left corner of the screen (Xaver Hugl, Plasma 6.2.0. Link)

The Audio Volume widget can once again display its complex view (rather than a dumb giant icon) when placed in an extremely thick panel (Christoph Wolk, Plasma 6.2.0. Link)

Made it possible to translate four labels on System Settings’ Touchpad page which were previously untranslatable and hence always shown in English (Victor Ryzhykh, Plasma 6.2.0. Link)

Plasma’s feature to sync your keyboard’s LED color with the system’s accent color (where supported) can now be permanently disabled if you don’t use it or like it (Natalie Clarius, Plasma 6.2.0. Link)

When using an auto-hide Plasma panel, it can no longer be accidentally opened invisibly while in KWin’s Overview effect, which would also cause it to briefly get stuck open after leaving Overview (Xaver Hugl, Plasma 6.2.0. Link)

Plasma’s Power and Battery widget no longer sometimes mis-labels batteries on systems with removable batteries or multiple batteries (Oliver Beard, Plasma 6.2.0. Link)

Fixed a bug that caused some Plasma widgets’ pop-ups to not have their top corners rounded as intended (Niccolò Venerandi, Plasma 6.2.0. Link)

Throughout QtQuick-based KDE software, pressing a keyboard’s “Show Menu” key now opens the menu under the focused item if there is one, rather than always opening it under the cursor (Evgeny Chesnokov, Frameworks 6.7. Link)

Throughout KDE software, the “Upload to Imgur” sharing plugin now only lets you try to upload file types that Imgur actually supports (Nicolas Fella, Frameworks 6.7. Link)

The informational tooltips seen all over the place in System Settings that you can access by clicking on a little button with the info symbol on it no longer flickers between visible and invisible if it opens right under the cursor (Ismael Asensio, Frameworks 6.7. Link)

Global shortcuts for activating Kickoff no longer break if you restart Plasma with plasmashell --replace (David Edmundson, Frameworks 6.7. Link). Also, friendly reminder that if you’re using Plasma’s systemd startup integration — which you probably are since it’s on by default for distros that ship systemd — the correct way to restart Plasma is systemctl restart --user plasma-plasmashell.service.

Other bug information of note:

• 1 Very high priority Plasma bug (same as last week). Current list of bugs
• 36 15-minute Plasma bugs (up from 33 last week). Current list of bugs
• 181 (!) KDE bugs of all kinds fixed over the last week. Full list of bugs

Notable in Performance & Technical

In Discover, moved the processing of app ratings to another thread so it can’t block the UI thread and make the app feel slow and laggy when the network is slow (Aleix Pol Gonzalez, Plasma 6.2.0. Link)

Launching an app that lacks a .desktop file (e.g. an app packaged as an AppImage) no longer causes a brief screen freeze whose length is proportional to the size of the app’s executable (Vlad Zahorodnii, Plasma 6.2.0. Link)

Optimized how KDE software in general finds mountpoints, which can result in a significant speed-up for various apps’ file-based use cases — up to 80% for one of them! (Kai Uwe Broulik, Frameworks 6.7. Link)

How You Can Help

Please test the Plasma 6.2 beta release! We’ve focused a lot on stability for this release and want to make sure we haven’t missed anything big before the final release in about a month. Your bug reports do not go into a black hole; we triage every one! So enthusiastic testing and bug reporting is encouraged. I’ve noticed fewer bug reports for this beta than previous ones, and I don’t believe for an instant that it’s because the release is already perfect! Go out there and file those bug reports!

Otherwise, visit https://community.kde.org/Get_Involved to discover additional ways to be part of a project that really matters. Each contributor makes a huge difference in KDE; you are not a number or a cog in a machine! You don’t have to already be a programmer, either. I wasn’t when I got started. Try it, you’ll like it! We don’t bite! Or consider donating instead! That helps too.

This blog post provides the heads-up about planned tablet input changes that are brewing for Plasma 6.3. KWin provides support for the tablet input protocol, but things are different on the client side. Some apps support it, some do not. If an application supports the tablet input protocol, great, it will receive tablet input events as is. On the other hand, if the application does not support the tablet input protocol, then KWin will fake tablet input as pointer input. In Plasma 6.3, KWin will stop doing that and I think that we should briefly talk what led us to such a decision and what impact it will have.

Originally, when the tablet input protocol support had landed in KWin, there were still pretty few applications and toolkits that supported it. Emulating tablet input was a fairly reasonable decision, otherwise you would have likely not been able to use tablet in the Plasma Wayland session at all. As time went by, more and more clients gained native support for the tablet protocol. Unfortunately, in meanwhile, we had also started noticing various issues with tablet emulation.

So, what’s the reasonable thing to do about it? Fix the bugs of course. And we did. But there is still a set of issues that cannot be addressed without bringing more complexity in already too complex code that we are struggling to keep afloat. Enough is enough.

Q: What’s new in 6.3?

A: Starting since 6.3, tablet input emulation will be deprecated and disabled by default. Note that you can enable it back by setting the KWIN_WAYLAND_EMULATE_TABLET=1 environment variable.

Q: When will tablet input emulation be dropped?

A: There is no concrete milestone at the moment.

Q: What does it mean to you? (as a user)

A: Hopefully, nothing. The major toolkits such as GTK, Qt, and SDL already provide support for the tablet protocol, so does Xwayland. So, you should be able to use tablet without any issues in X11 applications or Wayland native applications that use the aforementioned toolkits. Chromium/Electron still does not provide native support for tablet input on Wayland, but it’s also worth noting that most of those applications run through Xwayland by default unless the user sets some command-line arguments.

If your favorite application does not work with tablets, please tell it to the developers of that application so they know that there’s demand for such an operation mode.

Q: What should I do? (as a toolkit developer)

A: Please add support for tablets! If your toolkit already supports the tablet input protocol, wonderful, no work to do. \o/

Q: Is KWin alone by stopping emulating tablet input?

A: No, it is not. Mutter (the Wayland compositor in GNOME Shell) doesn’t emulate tablet input either.

Closing words

Deprecating tablet emulation is disappointing but the options that we have are not great either. It’s either bring in more complexity in order to fix the existing issues (plus even more code to ensure that the pointer focus is managed correctly when using both pointer and tablet) into an already too complex codebase or just do nothing special about applications that don’t opt in into tablet input. Hopefully, the remaining applications and toolkits that still miss tablet support add it in the near future. If you have more thoughts about it, please reach out to us at our matrix room.

Another one of the ASN list. This turned out longer than I expected (which is good). If you want to briefly understand what is an ASN, my Personal ASNs From India post carries an introduction to it.

Now, here’re the Educational and Research Institutions with their own ASN in India, which I could find:

• AS2697 Education and Research Network
• AS9885 NKN Internet Gateway
• AS23770 Tata Institute of Fundamental Research (used as National Centre for Biological Sciences network)
• AS38021 Network of Indian Institute of Foreign Trade
• AS38620 National Knowledge Network
• AS38872 Indian School of Business
• AS45340 B.M.S College of Engineering
• AS55296 National Institute of Public Finance and Policy
• AS55479 Indian Institute of Technology, Kanpur
• AS55566 Inter University Centre for Astronomy and Astrophysics
• AS55824 NKN Core Network
• AS56056 AMITY-IN
• AS55847 NKN Edge Network
• AS58703 Amrita Vishwa Vidyapeetham
• AS58758 Tata Institute of Fundamental Research (used as Homi Bhabha Centre for Science Education (HBCSE) network)
• AS59163 GLA University
• AS59193 Indian Institute of Technology, Hyderabad
• AS131226 Indian Institute of Technology, Roorkee
• AS131473 SRM University
• AS132423 Indian Institute of Technology, Bombay
• AS132524 Tata Institute of Fundamental Research (used as main campus network)
• AS132749 Indraprastha Institute of Information Technology, Delhi
• AS132780 Indian Institute of Technology, Delhi
• AS132984 Uka Tarsadia University
• AS132785 Shiv Nadar Institution of Eminence Deemed to be University
• AS132995 South Asian University
• AS133002 Indian Institute of Tropical Meteorology
• AS133233 S.N. Bose National Centre for Basic Sciences
• AS133273 Tata Institute of Social Sciences
• AS133308 Indira Gandhi Centre For Atomic Research
• AS133313 Saha Institute of Nuclear Physics
• AS133552 B.M.S. College of Engineering
• AS133723 Institute for Plasma Research
• AS134003 Centre For Cellular And Molecular Platforms
• AS134023 Aligarh Muslim University
• AS134322 Tata Institute of Fundamental Research (used as International Centre for Theoretical Sciences (ICTS) network)
• AS134901 Indian Institute of Science Education And Research
• AS134934 Institute For Stem Cell Biology And Regenerative Medicine
• AS135730 Datta Meghe Institute Of Medical Sciences
• AS135734 Birla Institute of Technology And Science
• AS135835 Sardar Vallabhbhai Patel National Police Academy
• AS136005 Raman Research Institute
• AS136304 Institute of Physics, Bhubaneswar
• AS136470 B.M.S. College of Engineering
• AS136702 Physical Research Laboratory
• AS137136 Indian Agricultural Statistics Research Institute
• AS137282 Kalinga Institute of Industrial Technology
• AS137617 Indian Institute of Management, Ahmedabad
• AS137956 Indian Institute of Technology, Ropar
• AS138155 Jawaharlal Nehru University
• AS138231 Indian Institute of Information Technology, Allahabad
• AS140033 Indian Institute of Technology, Bhilai
• AS140118 Indian Institute of Technology Banaras Hindu University
• AS140192 Indian Institute of Information Technology and Management, Kerala
• AS140200 Panjab University
• AS141270 Indian Institute Of Technology, Indore
• AS141340 Indian Institute Of Technology, Madras
• AS141477 Indira Gandhi National Open University
• AS141478 Director National Institute Of Technology, Calicut
• AS141288 National Institute of Science Education And Research Bhubaneswar
• AS141507 National Institute of Mental Health And Neurosciences
• AS142493 Sri Ramachandra Institute Of Higher Education And Research
• AS147239 Lal Bahadur Shastri National Academy of Administration (LBSNAA)
• AS147258 Dayalbagh Educational Institute
• AS149607 National Forensic Sciences University
• AS151086 Amrita Vishwa Vidyapeetham
• AS152533 National Institute of Technology, Karnataka

Special Mentions

• AS132926 Allen Career Institute
• AS141841 Indian Institute of Hardware Technology Limited

Some observations:

• Only 14 (out of 67) networks have/announce IPv6 prefixes.
• Two networks have IX presence; AS132785 Shiv Nadar at Extreme IX Delhi and AS56056 Amity at DE-CIX Delhi.
• TIFR as an organization has 4 ASNs (AS23770, AS58758, AS132524 and AS134322), BMS College of Engineering has 3 (AS45340, AS133552 and AS136470) and Amrita Vishwa Vidyapeetham has 2 (AS58703 and AS151086).
• AS2697, AS9885, AS38620, AS55824 and AS55847 ie Education and Research Network (ERNET) and National Knowledge Network (NKN) form the Indian NREN.
• 43 network have NKN as their upstream, while BSNL comes second as upstream for 21.

Let me know if I’m missing someone.

LagoonCon Barcelona 2024 offers a practical look at optimizing Drupal site management through the open-source Lagoon platform. Held on September 23, ahead of DrupalCon, the event features sessions on enterprise scaling, ROI improvement, and product engineering insights, making it a valuable opportunity for developers and tech leaders.

Do you have too many git branches on the go at once? Here is the command to list them in order of last modification:

git for-each-ref --sort=-committerdate refs/heads

Barcelona Update: Starshot Events Recipe Track

With DrupalCon Barcelona fast approaching I thought it was time to share some more updates on the progress of the events recipe for Drupal CMS a.k.a. the Starshot initiative.

mandclu Sep 20, 2024 - 9:27am Tags

• Drupal
• Dates and Times

• Read more about Barcelona Update: Starshot Events Recipe Track
• Add new comment

What are strategies for being a productive developer with ADHD? How can you help your team members with ADHD to succeed and complete projects? This week on the show, we speak with Chris Ferdinandi about his website and podcast "ADHD For the Win!"

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]