Feeds

In the last weeks qmpbackup has seen a bit more improvements.

• Adds support for CEPH/RBD backed devices.
• Allows to use unique bitmaps for having multiple, separate backup chains.
• Adds support for jsonified filename configurations like often used on proxmox systems.
• Adds support for saving attached pflash/nvram devices (storing UEFI related settings)
• qmprestore can now merge the backup chain into a new image file and the new snapshotrebase command can rebase the images and after committing, creates an internal qcow snapshot, so one can easily switch between different vm states in the backup.

Ive been running it lately to backup Virtual machines on proxmox systems, where the proxmox backup server is not an option.

BOSTON (October 22, 2024) -- The Free Software Foundation (FSF) has announced today that it is working on a statement of criteria for free machine learning applications, which will require the software, as well as the raw training data and associated scripts, to grant users the four freedoms.

#652 – OCTOBER 22, 2024
View in Browser »

Structural Pattern Matching in Python

In this tutorial, you’ll learn how to harness the power of structural pattern matching in Python. You’ll explore the new syntax, delve into various pattern types, and find appropriate applications for pattern matching, all while identifying common pitfalls.
REAL PYTHON

Combinatoric Iterators From itertools

The itertools module offers four combinatoric iterators that generate different combined outputs from one or more iterable. This post covers all of them: product, permutations, combinations, and combinations_with_replacement.
JUHA-MATTI SANTALA

Scrape Web Data at Scale with a 98.7% Success Rate

Extract all the data you need from any website without getting blocked with ZenRows’ Scraper API – a complete toolkit with premium proxies, anti-CAPTCHA, cloud-based scalable browsers, and more. Start your free trial now →
ZENROWS sponsor

CPython Internals: Your Guide to the Python 3 Interpreter

Unlock the inner workings of the Python language, compile the Python interpreter from source code, and participate in the development of CPython. Guido van Rossum, the creator of Python, says: “I can recommend CPython Internals to anyone who wants to get going with hacking on CPython” →
ANTHONY SHAW sponsor

SSH Scripting With Fabric and Python

Reading and writing files is a basic task that most software applications need to do, but what if you need to do that on remote machines? This tutorial introduces you to Fabric and how to connect over SSH in Python.
MIKE DRISCOLL

Python 3.14.0 Alpha 1 Released

CPYTHON DEV BLOG

Quiz: Structural Pattern Matching

REAL PYTHON

Quiz: Iterators and Iterables in Python

REAL PYTHON

Quiz: Python import: Advanced Techniques and Tips

REAL PYTHON

Articles & Tutorials Mind Your Image Metadata

Most devices record a variety of metadata when generating images. While some of that information may be innocuous, you could end up exposing the GPS coordinates to your home if you aren’t careful. In this article, Stefanie provides a brief introduction to image metadata, and then shows you how to remove it with exif-stripper.
STEFANIEMOLIN.COM • Shared by Stefanie Molin

Open Source in Python and JavaScript 2024

Python vs. JavaScript: Which open-source community is leading the way? This analysis of 36,000 GitHub repositories explores the evolution of Python and JavaScript ecosystems, highlighting key trends and popular topics. Discover how open-source communities of Python and JavaScript have shaped the tech landscape.
PYCHALLENGER.COM • Shared by Erik Nogueira Kückelheim

Accelerate Edge Devices with High-Performance AI Power

Experience the power of Edge AI—delivering lightning-fast, real-time processing where it matters. Optimize your applications to push performance and accuracy beyond limits with Intel’s OpenVINO toolkit.
INTEL CORPORATION sponsor

Using Type Hints for Multiple Return Types in Python

In this video course, you’ll learn how to define multiple return types using type hints in Python. This course covers working with single or multiple pieces of data, defining type aliases, and performing type checking using a third-party static type checker tool.
REAL PYTHON course

Narwhals: Expanding DataFrame Compatibility

How does a Python tool support all types of DataFrames and their various features? Could a lightweight library be used to add compatibility for newer formats like Polars or PyArrow? This week on the show, we speak with Marco Gorelli about his project, Narwhals.
REAL PYTHON podcast

Syntactic Sugar: Why Python Is Sweet and Pythonic

In this tutorial, you’ll learn what syntactic sugar is and how Python uses it to help you create more readable, descriptive, clean, and Pythonic code. You’ll also learn how to replace a given piece of syntactic sugar with another syntax construct.
REAL PYTHON

Entering Text in the Terminal Is Complicated

Julia asked some folks on Mastodon what they found confusing about working in a terminal. It turns out that entering text in the terminal is complicated. This post talks about why that is and how to understand it better.
JULIA EVANS

4 Lessons From Small Teams That Ship Fast

Software engineering provides a lot of leverage and small teams can do a large amount of work. This post talks about several common examples in the industry where a small group created a big product.
LEONARDO CREED

Perks of Being a Python Core Developer

Mariatta has been a Python Core Developer since 2017. If you want to know just what that means, this post talks about all the things she gets to do.
MARIATTA

Custom Dictionary Types in Pydantic

Pydantic lets you create custom types. This post talks about how to create a custom dictionary type using root models and Enums.
BRYAN ANTHONIO

How to Use Lambda Functions in Python

This article looks at some examples and best practices when using Lambda functions in Python.
FEDERICO TROTTA • Shared by AppSignal

Projects & Code ryp: R Inside Python

GITHUB.COM/WAINBERG

pyglove: Symbolic OO for Python

GITHUB.COM/GOOGLE

pipreqs: Generate requirements.txt Based on Imports

GITHUB.COM/BNDR

Bowler: Safe Code Refactoring for Modern Python

GITHUB.COM/FACEBOOKINCUBATOR

nanodjango: Full Django in a Single File

GITHUB.COM/RADIAC

Events Weekly Real Python Office Hours Q&A (Virtual)

October 23, 2024
REALPYTHON.COM

October Oslo Python Meetup

October 24, 2024
MEETUP.COM

PyCon APAC 2024

October 25 to October 27, 2024
PYCON.ID

PyCon Korea 2024

October 25 to October 28, 2024
PYCON.KR

PythonHo Conference 2024

October 26 to October 28, 2024
PYTHONHO.COM

PythOnRio Meetup

October 26, 2024
PYTHON.ORG.BR

Django Girls Aba

October 27, 2024
DJANGOGIRLS.ORG

PyCon FR 2024

October 31 to November 3, 2024
PYCON.FR

PyCon Zimbabwe

October 31 to November 3, 2024
PYCON.ORG

Happy Pythoning!
This was PyCoder’s Weekly Issue #652.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

The Python Global Interpreter Lock or GIL, in simple words, is a mutex (or a lock) that allows only one thread to hold the control of the Python interpreter.

This means that only one thread can be in a state of execution at any point in time. The impact of the GIL isn’t visible to developers who execute single-threaded programs, but it can be a performance bottleneck in CPU-bound and multi-threaded code.

Since the GIL allows only one thread to execute at a time even in a multi-threaded architecture with more than one CPU core, the GIL has gained a reputation as an “infamous” feature of Python.

In this video course you’ll learn how the GIL affects the performance of your Python programs, and how you can mitigate the impact it might have on your code.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

In this quiz, you’ll test your understanding of how to define your own Python function.

You’ll revisit theoretical knowledge about passing values to functions, when to divide your program into separate user-defined functions, and all the tools you’ll need to define complex and powerful functions in Python.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

With PHP attributes, your Drupal development process can be simplified, more performant, and easier to read. Learn the difference between PHP Attributes and Annotations and discover why attributes are the future of efficient coding.

Takeaway: With Drupal 7's end-of-life date set for January 5, 2025, public sector institutions face critical decisions about their web infrastructure. While migration to Drupal 10 offers clear advantages, organizations unable to migrate immediately need robust security measures to protect their sites.

PythonAnywhere has been around for over 10 years, and as our platform continues to grow with thousands of users, we’re committed to keeping it in top shape. Part of this involves upgrading some of the older parts of our infrastructure, with a special focus on our file storage servers—some of the oldest systems we have.

As we continue to evolve and adapt the Qt Framework to the needs of our users and upcoming regulation changes, we are excited to announce some significant changes to our Long-Term Support (LTS) policy from Qt 6.8 onwards. The changes are designed to provide a more robust and predictable support strategy, ensuring your projects remain secure and stable over their entire lifecycle. 

Today's exercice is just about turning a very nice example of the python soundevice module into something that works for me© to help me tune my bass.

Long story short, I suck at tuning my instrument and just lost my tuner...

This will require the python module soundevice and matplotlib.

So in order to tune my guitar I indeed need a spectrosonogram that displays the frequencies captured in real time by an audio device with an output readable enough I can actually know if I am nearing a legit frequency called a Note.

The frequencies for the notes are pretty arbitrary and I chose to only show the frequency for E, A , D, G, B since I have a 5 strings bass.
I chose the frequency between 100 and 2000 knowing that anyway any frequency below will trigger harmonics and above will trigger reasonance in the right frequency frame.

Plotting a spectrogram is done by tweaking the eponym matplotlib grapher with values chosen to fit my need and show me a laser thin beam around the right frequency. #!/usr/bin/env python3 """Show a text-mode spectrogram using live microphone data.""" import argparse import math import shutil import matplotlib.pyplot as plt from multiprocessing import Process, Queue import matplotlib.animation as animation import numpy as np import sounddevice as sd usage_line = ' press enter to quit,' def int_or_str(text): """Helper function for argument parsing.""" try: return int(text) except ValueError: return text try: columns, _ = shutil.get_terminal_size() except AttributeError: columns = 80 parser = argparse.ArgumentParser(add_help=False) parser.add_argument( '-l', '--list-devices', action='store_true', help='show list of audio devices and exit') args, remaining = parser.parse_known_args() if args.list_devices: print(sd.query_devices()) parser.exit(0) parser = argparse.ArgumentParser( description=__doc__ + '\n\nSupported keys:' + usage_line, formatter_class=argparse.RawDescriptionHelpFormatter, parents=[parser]) parser.add_argument( '-b', '--block-duration', type=float, metavar='DURATION', default=50, help='block size (default %(default)s milliseconds)') parser.add_argument( '-d', '--device', type=int_or_str, help='input device (numeric ID or substring)') parser.add_argument( '-g', '--gain', type=float, default=10, help='initial gain factor (default %(default)s)') parser.add_argument( '-r', '--range', type=float, nargs=2, metavar=('LOW', 'HIGH'), default=[50, 4000], help='frequency range (default %(default)s Hz)') args = parser.parse_args(remaining) low, high = args.range if high <= low: parser.error('HIGH must be greater than LOW') q = Queue() try: samplerate = sd.query_devices(args.device, 'input')['default_samplerate'] def plot(q): global samplerate fig, ( ax,axs) = plt.subplots(nrows=2) plt.ioff() def animate(i,q): data = q.get() ax.clear() axs.clear() axs.plot(data) ax.set_yticks([ 41.20, 82.41, 164.8, 329.6, 659.3, # E 55.00, 110.0, 220.0, 440.0, 880.0, # A 73.42, 146.8, 293.7, 587.3, # D 49.00, 98.00, 196.0, 392.0, 784.0, #G 61.74, 123.5, 246.9, 493.9, 987.8 ])#B ax.specgram(data[:,-1],mode="magnitude", Fs=samplerate*2, scale="linear",NFFT=9002) ax.set_ylim(150,1000) ani = animation.FuncAnimation(fig, animate,fargs=(q,), interval=500) plt.show() plotrt = Process(target=plot, args=(q,)) plotrt.start() def callback(indata, frames, time, status): if any(indata): q.put(indata) else: print('no input') with sd.InputStream(device=args.device, channels=1, callback=callback, blocksize=int(samplerate * args.block_duration /50 ), samplerate=samplerate) as sound: while True: response = input() if response in ('', 'q', 'Q'): break for ch in response: if ch == '+': args.gain *= 2 elif ch == '-': args.gain /= 2 else: print('\x1b[31;40m', usage_line.center(args.columns, '#'), '\x1b[0m', sep='') break except KeyboardInterrupt: parser.exit('Interrupted by user') except Exception as e: parser.exit(type(e).__name__ + ': ' + str(e))

Drupal CMS will come pre-installed with a set of modules and themes, using recipes, effectively replacing the "Standard" install profile. These recipes will provide the functionality that is considered must-have in modern CMSes, as well as what is deemed essential for our target persona and improve the overall user experience. 

We have been calling this the base recipe, which adds functionality on its own (e.g. installing the necessary core and contrib modules) and also selects other recipes to be applied by default. A while back we ran a survey to ask the community what features they felt were essential for the out-of-the-box offering and this has informed the inclusions. 

Along with the survey, we have done market research and benchmarking to see what our competitors include. But putting together a single proposal for the base recipe has proven challenging, because some features that we want are not yet available, or have some potential to conflict with Experience Builder or other upcoming initiatives. In some cases, contrib modules exist to provide a particular feature, but if it is not a high priority for our target user, we have left it out in order to focus our attention on what is. 

So this plan is for the initial release of Drupal CMS, scheduled for 15 January 2025. New features will of course be added to future releases, and we plan to launch new work tracks with this in mind soon.

Current state of the base recipe

If you are not up for parsing the recipe.yml file linked above, here is a summary of what it currently does:

What

Why

Installs a bunch of core modules and applies some core recipes

We are no longer using install profiles, so we have to add the foundational stuff somehow

Adds a redirect on access denied to the login form, and then to the original destination (via ECA)

So users can easily reach their intended destination even if their session has expired

Adds support for logging in with email in addition to username (via Login Email or Username)

So users don't have to remember a separate username. There is also an issue for supporting this in core, and when that lands we will no longer require a contrib module.
#111317: Allow users to login using either their username OR their e-mail address

Adds Gin as the admin theme

Because Gin provides a more modern UI, and as a contrib theme, is able to innovate faster than Drupal core admin themes

Adds Navigation (with a left-side menu) instead of the traditional admin toolbar

So the admin UI feels more modern and aligned with other similar systems. Navigation is an experimental module in core and has a roadmap outlining the path to stable.
#3421969: [PLAN] New Navigation and Top Bar to replace Toolbar Roadmap: Path to Stable

Adds a quick search for the admin menu (via Coffee)

So users can easily search for the admin page they are looking for.

Adds Trash module 

So users can recover deleted content

Adds Linkit support to CKEditor

So users can easily link to site content via search. Note there is an issue for adding a basic version of this in core, and we would prefer to use that. If it lands before 11.1, we will replace Linkit in the initial release.
#3317769: Drastically improve the linking experience in CKEditor 5

Adds a site dashboard (via Dashboard)

So users see a dashboard with relevant content when they first install, and when they log in (replacing /user as the default login page)

Adds focal point cropping to the image media type (via Focal point)

So users can select a focal point for their images to help them display nicely across aspect ratios

Adds Project Browser, Automatic updates, and Upgrade status

So users can add modules and keep their sites up to date from the UI, with no developer tools required

Adds some media management helper tools (Media entity download and Media file delete)

So the default media management experience is more intuitive. This will be extended and updated as part of the Media management track work.

Adds a Basic page content type

So every site has at least one content type available by default. See the full content strategy for more information.

Adds content cloning (via Quick node clone)

So users can duplicate content to easily create similar pages. This feature is a must-have, but the implementation is still up for discussion in #3474608: Evaluate cloning modules and #3477303: Create recipe to clone entities with ECA

Adds foundational SEO functionality: Pathauto, Redirect

Most sites require this functionality and the initial setup can be done generically

Coming soon

Some things that it does not yet include, but most likely will be in the initial release:

What

Why

Better default site search 

Drupal core search is very limited and not what site owners would expect from a modern platform. Drupal CMS will provide a more robust search experience using Search API. This is being done in the Advanced search work track, with the recipe in progress in #3468271: Add recipe for search backend

Autosave on forms (via Autosave Form)

So users don't lose their work. This feature is a must-have, but we wanted to ensure the approach did not conflict with Experience Builder's approach to the same problem.

HTML email sending

So users can send nicely formatted emails without additional configuration. See #3480680: Handle sending email in Drupal CMS

Coming... sometime?

Some things we would like to include, but have some blockers:

What

Why

Better select lists

The default select list experience is suboptimal, however, there is not currently a viable non-jQuery solution for this. We would like to use the Accessible Autocomplete Element/Widget based on the Accessible Autocomplete library but there are technical limitations around managing front-end dependencies.

Sitewide alerts

This is a common feature request, but we don't want to implement something that will conflict with Experience Builder when it comes out, leaving sites with a problem to solve. We also feel it is a nice-to-have for our target person rather than a must-have.

What about [insert feature here]? 

This summary covers the base functionality only. So if there is something extremely obvious that seems like it's missing, it is probably covered in one of the other work tracks! Many of them have not yet completed their work, so there are still lots of exciting things to come. Each of the metas links to their current proposal, if they have one. The final track proposal for the initial release are due by 1 November.

If you've scoured the track proposals and the Drupal CMS issue queue and still feel that we're missing a killer feature that is easily included, and high priority for the marketer types that we are focused on, let us know via Slack, in #starshot, or create an issue in the Drupal CMS project.

A new minor release of the drat package arrived on CRAN today, which is just over a year since the previous release. drat stands for drat R Archive Template, and helps with easy-to-create and easy-to-use repositories for R packages. Since its inception in early 2015 it has found reasonably widespread adoption among R users because repositories with marked releases is the better way to distribute code.

Because for once it really is as your mother told you: Friends don’t let friends install random git commit snapshots. Properly rolled-up releases it is. Just how CRAN shows us: a model that has demonstrated for over two-and-a-half decades how to do this. And you can too: drat is easy to use, documented by six vignettes and just works. Detailed information about drat is at its documentation site. That said, and ‘these days’, if you mainly care about github code then r-universe is there too, also offering binaries its makes and all that jazz. But sometimes you just want to, or need to, roll a local repository and drat can help you there.

This release contains a small PR (made by Arne Holmin just after the previous release) adding support for an ‘OSflacour’ variable (helpful for macOS). We also corrected an issue with one test file being insufficiently careful of using git2r only when installed, and as usual did a round of maintenance for the package concerning both continuous integration and documentation.

The NEWS file summarises the release as follows:

Changes in drat version 0.2.5 (2024-10-21)

• Function insertPackage has a new optional argument OSflavour (Arne Holmin in #142)

• A test file conditions correctly about git2r being present (Dirk)

• Several smaller packaging updates and enhancements to continuous integration and documentation have been added (Dirk)

Courtesy of my CRANberries, there is a comparison to the previous release. More detailed information is on the drat page as well as at the documentation site.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

All but one of KDE's Google Summer of Code (GSoC) projects are complete. This post will summarize the completed project outcomes. GSoC is a program where people who are students or are new to Free and Open Source software make programming contributions to an open source project.

Projects Arianna

• Port Arianna to Foliate-js: Ajay Chauhan worked on parting Arianna from epub.js to use Foliate-js. The work will hopefully be merged soon.

Frameworks

Python bindings for KDE Frameworks:

Manuel Alcaraz Zambrano, implemented Python bindings for KWidgetAddons, KUnitConversion, KCoreAddons, KGuiAddons, KI18n, KNotifications, and KXmlGUI. This was done using Shiboken. In addition, Manuel wrote a tutorial on how to generate Python bindings using Shiboken. The complicated set of merge requests are still being reviewed, and Manuel continues to interact with the KDE community.

KDE Connect

Update SSHD library in KDE Connect Android app

The main aim of ShellWen Chen's project was to update Apache Mina SSHD from 0.14.0 to 2.12.1. The older version has a few listed vulnerabilities. The newer library required additional code to enable it to work on older Android phones, upto Android API 21.

KDE Games

Implementing a computerized opponent for the Mancala variant Bohnenspiel:

João Gouveia created Mankala engine, a library to enable easy creation of Mancala games. The engine contains implementations for two Mancala games, Bohnenspiel and Oware. Both games contain computerized opponents, João also started on a QtQuick graphical user interface. The games are functional, but additional investigation on computerized opponents may help improve their effectiveness.

Kdenlive

Improved subtitling support for Kdenlive:

Kdenlive has gotten improved subtitling support. Chengkun Chen added support for using the Advanced SubStation (ASS) file format and for converting SubRip files to ASS files. To support this format, Chengkun Chen also made subtitling editor improvements. The work has been merged in the main repository. Documentation has been written, and will hopefully be merged soon.

Krita

Creating Pixel Perfect Tool for Krita:

Ken Lo worked on implementing Pixel Perfect lines in Krita. As explained by Ricky Han, such algorithms remove corner pixels from L shaped blocks and ensure the thinnest possible line is 1 pixel wide. Implementing such algorithms well is of use not only in Krita, but also in rendering web graphics where user screen resolutions can vary significantly. The algorithm was implemented to work in close to real time while lines are drawn, rather than as a post processing step. Ken Lo's work has been merged into Krita.

Labplot

Improve Python Interoperability with LabPlot

Israel Galadima worked on improving Python support in LabPlot. Shiboken was used for this. It is now possible to call some of LabPlot functions from Python and integrate these into other applications.

3D Visualization for LabPlot

Kuntal Bar added 3D graphing abilities to LabPlot. This was done using QtGraphs. The work has yet to be merged, but there are many nice examples of 3D plots, for bar charts, scatter and surface plots.

Snaps

Improving Snap Ecosystem in KDE

Snaps are self contained linux application packging formats. Soumyadeep Ghosh worked on improving the tooling necessary to make KDE applications easily available in the Snap Store. In addition, Soumyadeep improved packaging of a number of KDE Snap packages, and packaged MarkNote. Finally, Soumyadeep created Snap KCM, a graphical user interface to manage permissions that Snaps have when running.

Next Steps

The GSoC period is over, for all but one contributor, Pratham Gandhi. A follow up post will summarize contributions from the remaining project. Contributors have enjoyed participating in GSoC and we look forward to their continuing participation in free and open source software communities and in contributing to KDE.

Tuesday, 22 October 2024. Today KDE releases a bugfix update to KDE Plasma 6, versioned 6.2.2.

Plasma 6.2 was released in October 2024 with many feature refinements and new modules to complete the desktop experience.

This release adds a week's worth of new translations and fixes from KDE's contributors. The bugfixes are typically small but important and include:

• KWin Backends/drm: leave all outputs disabled by default, including VR headsets. Commit. Fixes bug #493148
• KWin Set WAYLAND_DISPLAY before starting wayland server. Commit.
• Plasma Audio Volume Control: Fix text display for auto_null device. Commit. Fixes bug #494324

View full changelog

GNU Parallel 20241022 ('Sinwar Nasrallah') has been released. It is available for download at: lbry://@GnuParallel:4

Quote of the month:

  GNU Parallel is one of the most helpful tools I've been using recently, and it's just something like: parallel -j4 'gzip {}' ::: folder/*.csv
     -- Milton Pividori @miltondp@twitter
 
New in this release:

• No new features. This is a candidate for a stable release.
• Bug fixes and man page updates.

News about GNU Parallel:

• Separate arguments with a custom separator in GNU Parallel https://boxofcuriosities.co.uk/post/separate-arguments-with-a-custom-separator-in-gnu-parallel
• GNU parallel is underrated https://amontalenti.com/2021/11/10/parallel
• Unlocking the Power of Supercomputers: My HPC Adventure with 2800 Cores and GNU Parallel https://augalip.com/2024/03/10/unlocking-the-power-of-supercomputers-my-hpc-adventure-with-2800-cores-and-gnu-parallel/
• Converting WebP Images to PNG Using parallel and dwebp https://bytefreaks.net/gnulinux/bash/converting-webp-images-to-png-using-parallel-and-dwebp

GNU Parallel - For people who live life in the parallel lane.

If you like GNU Parallel record a video testimonial: Say who you are, what you use GNU Parallel for, how it helps you, and what you like most about it. Include a command that uses GNU Parallel if you feel like it.

About GNU Parallel

GNU Parallel is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. A job can also be a command that reads from a pipe. GNU Parallel can then split the input and pipe it into commands in parallel.

If you use xargs and tee today you will find GNU Parallel very easy to use as GNU Parallel is written to have the same options as xargs. If you write loops in shell, you will find GNU Parallel may be able to replace most of the loops and make them run faster by running several jobs in parallel. GNU Parallel can even replace nested loops.

GNU Parallel makes sure output from the commands is the same output as you would get had you run the commands sequentially. This makes it possible to use output from GNU Parallel as input for other programs.

For example you can run this to convert all jpeg files into png and gif files and have a progress bar:

  parallel --bar convert {1} {1.}.{2} ::: *.jpg ::: png gif

Or you can generate big, medium, and small thumbnails of all jpeg files in sub dirs:

  find . -name '*.jpg' |
    parallel convert -geometry {2} {1} {1//}/thumb{2}_{1/} :::: - ::: 50 100 200

You can find more about GNU Parallel at: http://www.gnu.org/s/parallel/

You can install GNU Parallel in just 10 seconds with:

    $ (wget -O - pi.dk/3 || lynx -source pi.dk/3 || curl pi.dk/3/ || \
       fetch -o - http://pi.dk/3 ) > install.sh
    $ sha1sum install.sh | grep 883c667e01eed62f975ad28b6d50e22a
    12345678 883c667e 01eed62f 975ad28b 6d50e22a
    $ md5sum install.sh | grep cc21b4c943fd03e93ae1ae49e28573c0
    cc21b4c9 43fd03e9 3ae1ae49 e28573c0
    $ sha512sum install.sh | grep ec113b49a54e705f86d51e784ebced224fdff3f52
    79945d9d 250b42a4 2067bb00 99da012e c113b49a 54e705f8 6d51e784 ebced224
    fdff3f52 ca588d64 e75f6033 61bd543f d631f592 2f87ceb2 ab034149 6df84a35
    $ bash install.sh

Watch the intro video on http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

Walk through the tutorial (man parallel_tutorial). Your command line will love you for it.

When using programs that use GNU Parallel to process data for publication please cite:

O. Tange (2018): GNU Parallel 2018, March 2018, https://doi.org/10.5281/zenodo.1146014.

If you like GNU Parallel:

• Give a demo at your local user group/team/colleagues
• Post the intro videos on Reddit/Diaspora*/forums/blogs/ Identi.ca/Google+/Twitter/Facebook/Linkedin/mailing lists
• Get the merchandise https://gnuparallel.threadless.com/designs/gnu-parallel
• Request or write a review for your favourite blog or magazine
• Request or build a package for your favourite distribution (if it is not already there)
• Invite me for your next conference

If you use programs that use GNU Parallel for research:

• Please cite GNU Parallel in you publications (use --citation)

If GNU Parallel saves you money:

• (Have your company) donate to FSF https://my.fsf.org/donate/

About GNU SQL

GNU sql aims to give a simple, unified interface for accessing databases through all the different databases' command line clients. So far the focus has been on giving a common way to specify login information (protocol, username, password, hostname, and port number), size (database and table size), and running queries.

The database is addressed using a DBURL. If commands are left out you will get that database's interactive shell.

When using GNU SQL for a publication please cite:

O. Tange (2011): GNU SQL - A Command Line Tool for Accessing Different Databases Using DBURLs, ;login: The USENIX Magazine, April 2011:29-32.

About GNU Niceload

GNU niceload slows down a program when the computer load average (or other system activity) is above a certain limit. When the limit is reached the program will be suspended for some time. If the limit is a soft limit the program will be allowed to run for short amounts of time before being suspended again. If the limit is a hard limit the program will only be allowed to run when the system is below the limit.

Today we are talking about Access Policy API, What it does, and How you can use it with guest Kristiaan Van den Eynde. We’ll also cover Visitors as our module of the week.

For show notes visit: https://www.talkingDrupal.com/472

Topics

• What is the Access Policy API
• Why does Drupal need the Access Policy API
• How did Drupal handle access before
• How does the Access Policy API interact with roles
• Does a module exist that shows a UI
• What is the difference between Policy Based Access Control (PBAC), Attribute Based Access Control (ABAC) and Role Based Access Control (RBAC)
• How does Access Policy API work with PBAC, ABAC and RBAC
• Can you apply an access policy via a recipe
• Is there a roadmap
• What was it like going through pitchburg
• How can people get involved

Resources

• Access Policy API
• Access Policy
• Talking Drupal #226 Group
• Flexible Permissions
• External roles
• Test Super access policy
• Access policy talk at Drupalcon barcelona
• D.o Issue about exception on security issue

Guests

Kristiaan Van den Eynde - kristiaanvandeneynde

Hosts

Nic Laflin - nLighteneddevelopment.com nicxvan John Picozzi - epam.com johnpicozzi Aubrey Sambor - star-shaped.org starshaped

MOTW Correspondent

Martin Anderson-Clutz - mandclu.com mandclu

• Brief description:
  • Have you ever wanted a Drupal-native solution for tracking website visitors and their behavior? There’s a module for that
• Module name/project name:
  • Visitors
• Brief history
  • How old: created in Mar 2009 by gashev, though recent releases are by Steven Ayers (bluegeek9)
  • Versions available: 8.x-2.19, which works with Drupal 10 and 11
• Maintainership
  • Actively maintained
  • Security coverage
  • Test coverage
  • Documentation guide is available
  • Number of open issues: 20 open issues, none of which are bugs against the 8.x branch
• Usage stats:
  • Over 6,000 sites
• Module features and usage
  • A benefit of using a Drupal-native solution is that you retain full ownership over your visitor data. Not sharing that data with third parties can be important for data protection regulations, as well as data privacy concerns.
  • You also have a variety of reports you can access directly within the Drupal UI, including top pages, referrers, and more
  • There is a submodule for geoip lookups using Maxmind, if you also want reporting on what region, country, or city your visitors hail from
  • It provides drush commands to download a geoip database, and then update your data based on geoip lookups using that database
  • It should be mentioned that the downside of using Drupal as your analytics solution is the potential performance impact and also a likely uptick in usage for hosts that charge based on the number of dynamic requests served

List of public mirrors in India. Location discovered basis personal knowledge, traces or GeoIP. Mirrors which aren’t accessible outside their own ASN are excluded.

North India

• Bharat Datacenter - mirror.bharatdatacenter.com (AS151704)
• CSE Department, IIT Kanpur - mirror.cse.iitk.ac.in (AS55479)
• Cyfuture - cyfuture.dl.sourceforge.net (AS55470)
• Extreme IX - repos.del.extreme-ix.org | repo.extreme-ix.org (AS135814)
• Garuda Linux - in-mirror.garudalinux.org (AS133661)
• Hopbox - mirrors.hopbox.net (AS10029)
• IIT Delhi - mirrors.iitd.ac.in (AS132780)
• NKN - debianmirror.nkn.in (AS4758)
• Nxtgen - mirrors.nxtgen.com (AS132717)
• Saswata Sarkar - mirrors.saswata.cc (AS132453)
• Shiv Nadar Institution of Eminence - ubuntu-mirror.snu.edu.in (AS132785)

East India

• NISER Bhubaneshwar - mirror.niser.ac.in (AS141288))

South India

• Cogan Ng - in.mirror.coganng.com (AS31898)
• CUSAT - foss.cusat.ac.in/mirror (AS55824))
• Excell Media - centos-stream.excellmedia.net | excellmedia.dl.sourceforge.net (AS17754)
• IIT Madras - ftp.iitm.ac.in (AS141340)
• NIT Calicut - mirror.nitc.ac.in (AS55824)
• NKN - mirrors-1.nkn.in (AS148003)
• Planet Unix - mirror.planetunix.net | ariel.in.ext.planetunix.net (AS14061)
• Shrirang Kahale - mirror.maa.albony.in (AS24560)

West India

• Abhinav Krishna C K - mirrors.abhy.me (AS31898)
• Arun Mathai - mirrors.arunmathaisk.in (AS141995)
• Balvinder Singh Rawat - mirror.ubuntu.bsr.one (AS31898)
• ICTS - cran.icts.res.in (AS134322)
• Nilesh Patra - mirrors.nileshpatra.info (AS31898)
• PicoNets-WebWerks - mirrors.piconets.webwerks.in (AS133296)
• Ravi Dwivedi - mirrors.ravidwivedi.in (AS141995)
• Sahil Dhiman - mirrors.in.sahilister.net (AS141995)
• Shrirang Kahale - mirror.bom.albony.in | mirror.nag.albony.in (AS24560)
• Starburst Services - almalinux.in.ssimn.org | elrepo.in.ssimn.org | epel.in.ssimn.org | mariadb.in.ssimn.org (AS141995)
• Unknown - mirror.4v1.in (AS24560)
• Utkarsh Gupta - mirrors.utkarsh2102.org (AS31898)

CDN (or behind one)

• Amazon Cloudfront - cdn-aws.deb.debian.org (AS16509)
• Cicku - in.mirrors.cicku.me (AS13335)
• CIQ - rocky-linux-asia-south1.production.gcp.mirrors.ctrliq.cloud | rocky-linux-asia-south2.production.gcp.mirrors.ctrliq.cloud (GeoIP doubtful. Could be behind a CDN or single node) (AS396982)
• Cloudflare - cloudflare.cdn.openbsd.org | kali.download/ (AS13335)
• Edgecast - mirror.edgecast.com (AS15133)
• Fastly - cdn.openbsd.org | deb.debian.org | dlcdn.apache.org | dl-cdn.alpinelinux.org (sponsored?) | images-cdn.endlessm.com (sponsored?) | repo-fastly.voidlinux.org (AS54113)
• Naman Garg - in-mirror.chaotic.cx (AS13335)
• Microsoft - debian-archive.trafficmanager.net (AS8075)
• Niranjan Fartare - arch.niranjan.co | termux.niranjan.co (AS13335)
• Sahil Kokamkar - mirror.sahil.world (AS13335)

Let me know if I’m missing someone or something is amiss.

Check out the important work our volunteers accomplished at today's Free Software Directory (FSD) IRC meeting.

Check out the important work our volunteers accomplished at today's Free Software Directory (FSD) IRC meeting.