FLOSS Project Planets

The Rcpp Core Team is once again thrilled to announce a new release 1.0.12 of the Rcpp package. It arrived on CRAN early today, and has since been uploaded to Debian as well. Windows and macOS builds should appear at CRAN in the next few days, as will builds in different Linux distribution–and of course at r2u should catch up tomorrow. The release was uploaded yesterday, and run its reverse dependencies overnight. Rcpp always gets flagged nomatter what because the grandfathered .Call(symbol) but … we had not single ‘change to worse’ among over 2700 reverse dependencies!

This release continues with the six-months January-July cycle started with release 1.0.5 in July 2020. As a reminder, we do of course make interim snapshot ‘dev’ or ‘rc’ releases available via the Rcpp drat repo and strongly encourage their use and testing—I run my systems with these versions which tend to work just as well, and are also fully tested against all reverse-dependencies.

Rcpp has long established itself as the most popular way of enhancing R with C or C++ code. Right now, 2791 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with 254 in BioConductor. On CRAN, 13.8% of all packages depend (directly) on Rcpp, and 59.9% of all compiled packages do. From the cloud mirror of CRAN (which is but a subset of all CRAN downloads), Rcpp has been downloaded 78.1 million times. The two published papers (also included in the package as preprint vignettes) have, respectively, 1766 (JSS, 2011) and 292 (TAS, 2018) citations, while the the book (Springer useR!, 2013) has another 617.

This release is incremental as usual, generally preserving existing capabilities faithfully while smoothing our corners and / or extending slightly, sometimes in response to changing and tightened demands from CRAN or R standards.

The full list below details all changes, their respective PRs and, if applicable, issue tickets. Big thanks from all of us to all contributors!

Changes in Rcpp release version 1.0.12 (2024-01-08)

• Changes in Rcpp API:

  • Missing header includes as spotted by some recent tools were added in two places (Michael Chirico in #1272 closing #1271).

  • Casts to avoid integer overflow in matrix row/col selections have neem added (Aaron Lun #1281).

  • Three print format correction uncovered by R-devel were applied with thanks to Tomas Kalibera (Dirk in #1285).

  • Correct a print format correction in the RcppExports glue code (Dirk in #1288 fixing #1287).

  • The upcoming OBJSXP addition to R 4.4.0 is supported in the type2name mapper (Dirk and Iñaki in #1293).

• Changes in Rcpp Attributes:

  • Generated interface code from base R that fails under LTO is now corrected (Iñaki in #1274 fixing a StackOverflow issue).

• Changes in Rcpp Documentation:

  • The caption for third figure in the introductory vignette has been corrected (Dirk in #1277 fixing #1276).

  • A small formatting issue was correct in an Rd file as noticed by R-devel (Dirk in #1282).

  • The Rcpp FAQ vignette has been updated (Dirk in #1284).

  • The Rcpp.bib file has been refreshed to current package versions.

• Changes in Rcpp Deployment:

  • The RcppExports file for an included test package has been updated (Dirk in #1289).

Thanks to my CRANberries, you can also look at a diff to the previous release Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. Bugs reports are welcome at the GitHub issue tracker as well (where one can also search among open or closed issues).

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

pre > code.sourceCode { white-space: pre; position: relative; } pre > code.sourceCode > span { display: inline-block; line-height: 1.25; } pre > code.sourceCode > span:empty { height: 1.2em; } .sourceCode { overflow: visible; } code.sourceCode > span { color: inherit; text-decoration: inherit; } div.sourceCode { margin: 1em 0; } pre.sourceCode { margin: 0; } @media screen { div.sourceCode { overflow: auto; } } @media print { pre > code.sourceCode { white-space: pre-wrap; } pre > code.sourceCode > span { text-indent: -5em; padding-left: 5em; } } pre.numberSource code { counter-reset: source-line 0; } pre.numberSource code > span { position: relative; left: -4em; counter-increment: source-line; } pre.numberSource code > span > a:first-child::before { content: counter(source-line); position: relative; left: -1em; text-align: right; vertical-align: baseline; border: none; display: inline-block; -webkit-touch-callout: none; -webkit-user-select: none; -khtml-user-select: none; -moz-user-select: none; -ms-user-select: none; user-select: none; padding: 0 4px; width: 4em; color: #aaaaaa; } pre.numberSource { margin-left: 3em; border-left: 1px solid #aaaaaa; padding-left: 4px; } div.sourceCode { } @media screen { pre > code.sourceCode > span > a:first-child::before { text-decoration: underline; } } code span.al { color: #ff0000; font-weight: bold; } /* Alert */ code span.an { color: #60a0b0; font-weight: bold; font-style: italic; } /* Annotation */ code span.at { color: #7d9029; } /* Attribute */ code span.bn { color: #40a070; } /* BaseN */ code span.bu { color: #008000; } /* BuiltIn */ code span.cf { color: #007020; font-weight: bold; } /* ControlFlow */ code span.ch { color: #4070a0; } /* Char */ code span.cn { color: #880000; } /* Constant */ code span.co { color: #60a0b0; font-style: italic; } /* Comment */ code span.cv { color: #60a0b0; font-weight: bold; font-style: italic; } /* CommentVar */ code span.do { color: #ba2121; font-style: italic; } /* Documentation */ code span.dt { color: #902000; } /* DataType */ code span.dv { color: #40a070; } /* DecVal */ code span.er { color: #ff0000; font-weight: bold; } /* Error */ code span.ex { } /* Extension */ code span.fl { color: #40a070; } /* Float */ code span.fu { color: #06287e; } /* Function */ code span.im { color: #008000; font-weight: bold; } /* Import */ code span.in { color: #60a0b0; font-weight: bold; font-style: italic; } /* Information */ code span.kw { color: #007020; font-weight: bold; } /* Keyword */ code span.op { color: #666666; } /* Operator */ code span.ot { color: #007020; } /* Other */ code span.pp { color: #bc7a00; } /* Preprocessor */ code span.sc { color: #4070a0; } /* SpecialChar */ code span.ss { color: #bb6688; } /* SpecialString */ code span.st { color: #4070a0; } /* String */ code span.va { color: #19177c; } /* Variable */ code span.vs { color: #4070a0; } /* VerbatimString */ code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warning */

• Geographical Data Visualization in Python with GeoPandas
  • GIS data formats
  • Load SHAP data into Python object
  • Stay updated on Python tips

Geographical Data Visualization in Python with GeoPandas

Working with geographical data can often be a bit tricky for the uninitiated. This post aims to shed some light for those who are encountering geo data for the first time and want an easy way to perform initial manipulations and plot it to gain their first insights.

We’re about to dive into a sea of coordinates, shapes, and a whole lot of mapping magic!

GIS data formats

First of all, we want to make a summary about the most used data formats for geo data. These are the unsung heroes in the world of mapping and spatial analysis. Imagine trying to describe the vastness of our planet without them – it’s like trying to paint a masterpiece with just one color. Not happening, right?

First up in our parade of geo data superstars are formats like Shapefile (SHP), the old faithful of the geo world. But you’ll need a complete set of three files that are mandatory to make up a shapefile. The three required files are: .SHP for geometry, .SHX is the shape index position and .DBF is the attribute data.

Developed by ESRI, it’s like the trusty old pickup truck – not the flashiest but gets the job done. Then there’s GeoJSON, the cool kid on the block. It’s all about simplicity and web-friendliness, perfect for those who love to mingle with JavaScript and web mapping.

And let’s not forget about KML (Keyhole Markup Language), the go-to for Google Earth enthusiasts. It’s like having a GPS in your pocket – straightforward and ready to guide you through those virtual globetrotting adventures. Each of these formats has its own quirks and charms, kind of like different types of pasta – some are better suited for hearty meaty sauces, while others are perfect for a light seafood affair.

In short, the world of geo data formats is as diverse and colorful as the world it represents. Whether you’re a GIS guru or a casual map enthusiast, getting to know these formats is like getting the keys to a whole new world of spatial wonders

Yuo can check more info here!

Load SHAP data into Python object

Let’s kick things off by accessing the geo data and geometries. Imagine we are a company with central stations spread across the Spanish territory, responsible for the alerts in their vicinity. Each station acts as a sentinel, vigilantly monitoring and responding to the events unfolding around it. So we will begin with defining the country over which we’re planning to plot the stations and event coordinates.

Now, here comes the fun part: we load the Shape data using GeoPandas. It’s like unlocking a treasure chest of geographical wonders! GeoPandas makes it a breeze, turning what could be a complex task into a walk in the park. Imagine GeoPandas as your trusty GPS, guiding you through the intricate world of geo data with ease and precision. So, grab your data, let’s fire up GeoPandas, and watch as those lines of code magically transform into a map full of possibilities!

import geopandas as gp import pandas as pd def filter_shp_peninsular_data(df: pd.DataFrame) -> pd.DataFrame: return df[df["acom_name"] != "Canarias"] spain_gis_map = gp.read_file('./data/georef-spain-provincia/georef-spain-provincia-millesime.shp') peninsular_gis_map = filter_shp_peninsular_data(spain_gis_map)

We can filter out any layer in the GeoPandas read_file function with the bbox and mask parameters. But let’s keep it simple for the moment and just load and filter out it.

For ease of use, we are going to load and filter also the alerts peninsular data. You can easily apply coordinates filters in order to focus in one specific geography. We will transform our internal data read with Polars to GeoPandas dataframe to show how it could be do, but it is not strictly necessary.

from shapely.geometry import Point import polars as pl def filter_df_peninsular_data(df: pd.DataFrame) -> pd.DataFrame: peninsular_bounds_min = peninsular_gis_map["geometry"].bounds.min() peninsular_bounds_max = peninsular_gis_map["geometry"].bounds.max() return df.filter( ( pl.col("x")>=peninsular_bounds_min["minx"] ) & ( pl.col("x")<=peninsular_bounds_max["maxx"] ) & ( pl.col("y")>=peninsular_bounds_min["miny"] ) & ( pl.col("y")<=peninsular_bounds_max["maxy"] ) ) alerts_geo_df = pl.read_csv("alers_data.csv") alerts_geo_df = gp.GeoDataFrame({ "geometry": alerts_geo_df["coord"].map_elements(lambda x: Point(x)), "alert_solved": alerts_geo_df["alert_solved"].is_not_null(), "x": alerts_geo_df["x"], "y": alerts_geo_df["y"], }) stations_geo_df = pl.read_csv("stations_data.csv") stations_geo_df = gp.GeoDataFrame({ "geometry": stations_geo_df["coord"].map_elements(lambda x: Point(x)), "x": stations_geo_df["x"], "y": stations_geo_df["y"], })

Geopandas plotting with Plotnine

Let’s start with our mission to plot a map sprinkled with stations (in cheerful yellow) and events (in bold red and blue).

Once we have the data both country layers and our project datapoints properly formated as GeoPandas DataFrames

The geom_map function is our stroke of genius, turning geographical data into visual narratives.

( ggplot() + geom_map(peninsular_gis_map, fill=None) + geom_map(alerts_geo_df, aes(fill="alert_solved"), size=2) + geom_map(stations_geo_df, colour="yellow", size=3) + labs( title="Alerts solved y/n, by location + Stations (yellow)", caption = "Data from 2023-01-01 to 2023-02-01", ) )

Geopandas plotting the most representative category by location

Now, we’re not just mapping points; we’re painting a picture of the most representative category by location. Think of it as a data detective story, where each clue (or data point) reveals a part of the bigger picture.

Here’s how we crack the case:

( ggplot() + geom_bin2d(alerts_geo_df, aes(x="x", y="y", fill="alert_solved"), bins = 30) + geom_map(stations_geo_df, colour="yellow", size=3) + labs( title="Alerts solved y/n, by location + Stations (yellow)", caption = "Data from 2023-01-01 to 2023-02-01", ) )

With geom_bin2d, we’re transforming our map into a vibrant tapestry, showcasing the most contacted categories in a kaleidoscope of colors. Each square on this grid is like a pixel, together weaving the story of our data’s journey across the Spanish landscape. And, of course, our stations, marked in sunny yellow, stand out as beacons in this sea of information.

Stay updated on Python tips

Hopefully, this post has helped familiarize you with GeoPandas, GIS data nd Plotnine in Python.

If you want to stay updated…

#mc_embed_signup{background:#fff; clear:left; font:14px Helvetica,Arial,sans-serif; width:100%;} #mc_embed_signup .button { background-color: #0294A5; /* Green */ color: white; transition-duration: 0.4s; } #mc_embed_signup .button:hover { background-color: #379392 !important; } Suscribe for more Python posts! p { word-spacing: 3px; text-indent: 20px; text-align: justify; } .page-subtitle { text-align: left !important; text-indent: 0px !important; } .card-text { text-align: left !important; text-indent: 0px !important; } .hljs-keyword,.hljs-selector-tag,.hljs-subst{color:#2e8516;font-weight:bold}.hljs-comment, .hljs-quote { color: #0e847b; font-style: italic; }.hljs-number, .hljs-literal, .hljs-variable, .hljs-template-variable, .hljs-tag .hljs-attr { color: #008021; }

News and updates

• We have shipped New improvements to redirects, making our redirects much more powerful and flexible.

• We have shipped an updated approach to notifications. Currently there isn’t much UX difference, but as we move forward with this project we will be able to provide more context and control to users.

• We continue to work on improving Addons, our new approach to documentation integrations. New documentation and bug fixing continues to happen.

• We shipped version 2.0 of our Read the Docs Sphinx Theme, which adds support for new Sphinx releases and drops support for many old versions of Sphinx and Python.

You can always see the latest changes to our platforms in our Read the Docs Changelog.

Upcoming changes

• Addons will be made more configurable in our new beta dashboard, starting a trend of moving away from the old dashboard for new features.

• Our beta dashboard continues to be tested in public beta, and new functionality for Addons configuration will only be available in that new interface.

• We continue to work on some business model changes enabled by the new redirects work, including allowing access to Forced Redirects for more users.

Want to follow along with our development progress? View our full roadmap 📍️

Possible issues

• Users need to update their webhooks before January 31, 2024 if they are configured without a secret. All users who need to take action should have received email and site notifications about this.

• We are discussing removing support for all VCS systems except Git, as our userbase is heavily biased towards Git users and it will simplify maintenance and development of features. We stopped developing features for Mercurial, Subversion, and Bazaar years ago, and we are considering removing support for them entirely. We will be reaching out to these users to get feedback on this change.

Questions? Comments? Ideas for the next newsletter? Contact us!

Plasma 6, Frameworks and Gear draw closer

Every few years we port the key components of our software to a new version of Qt, taking the opportunity to remove cruft and leverage the updated features the most recent version of Qt has to offer us.

KDE's megarelease is less than 50 days away. At the end of February 2024 we will publish Plasma 6, Frameworks 6, and a whole new set of applications in a special edition of KDE Gear all in one go.

If you have been following the updates here, here and here, you will know we are making our way through the testing phase and gradually reaching stability. KDE is making available today the first Release Candidate version of all the software we will include in the megarelease.

As with the Alpha and Beta versions, this is a preview intended for developers and testers. The software provided is nearing stability, but is still not 100% safe to use in a production environment. We still recommend you continue using stable versions of Plasma, Frameworks and apps for your everyday work. But if you do use this, watch out for bugs and report them promptly, so we can solve them.

Read on to find out more about KDE's 6th Megarelease, what it covers, and how you can help make the new versions of Plasma, KDE's apps and Frameworks a success now.

Plasma

Plasma is KDE's flagship desktop environment. Plasma is like Windows or macOS, but is renowned for being flexible, powerful, lightweight and configurable. It can be used at home, at work, for schools and research.

Plasma 6 is the upcoming version of Plasma that integrates the latest version of Qt, Qt 6, the framework upon which Plasma is built.

Plasma 6 incorporates new technologies from Qt and other constantly evolving tools, providing new features, better support for the latest hardware, and supports for the hardware and software technologies to come.

You can be part of the new Plasma. Download and install a Plasma 6-powered distribution (like Neon Unstable) to a test machine and start trying all its features. Check the Contributing section below to find out how you can deliver reports of what you find to the developers.

• Plasma 6 RC 1 Source Code Info Page
• Plasma 6 RC 1 Full Changelog
• Plasma 6 packagers' release notes

KDE Gear

KDE Gear is a collection of applications produced by the KDE community. Gear includes file explorers, music and video players, text and video-editors, apps to manage social media and chats, email and calendaring applications, travel assistants, and much more.

Developers of these apps also rely on the Qt toolbox, so most of the software will also be adapted to use the new Qt6 toolset and we need you to help us test them too.

• KDE Gear 24.02 RC 1 Source Code Info Page
• KDE Gear 24.02 RC 1 Full Changelog
• KDE Gear 24.02 packagers' release notes

Frameworks

KDE's Frameworks add tools created by the KDE community on top of those provided by the Qt toolbox. These tools give developers more and easier ways of developing interfaces and functionality that work on more platforms.

Among many other things, KDE Frameworks provide

• widgets (buttons, text boxes, etc.) that make building your apps easier and their looks more consistent across platforms, including Windows, Linux, Android and macOS
• libraries that facilitate storing and retrieving configuration settings
• icon sets, or technologies that make the integration of the translation workflow of applications easier

KDE's Frameworks also rely heavily on Qt and will also be upgraded to adapt them to the new version 6. This change will add more features and tools, enable your applications to work on more devices, and give them a longer shelf life.

• KDE Frameworks 6 RC 1 Source Code Info Page
• KDE Frameworks 6 RC 1 Full Changelog
• KDE Frameworks 6 packagers release notes

Contributing

KDE relies on volunteers to create, test and maintain its software. You can help too by...

• Reporting bugs -- When you come across a bug when testing the software included in this Alpha Megarelease, you can report it so developers can work on it and remove it. When reporting a bug
  • make sure you understand when the bug is triggered so you can give developers a guide on how to check it for themselves
  • check you are using the latest version of the software you are testing, just in case the bug has been solved in the meantime
  • go to KDE's bug-tracker and search for your bug to make sure it does not get reported twice
  • if no-one has reported the bug yet, fill in the bug report, giving all the details you think are significant.
  • keep tabs on the report, just in case developers need more details.
• Solving bugs -- Many bugs are easy to solve. Some just require changing a version number or tweaking the name of a library to its new name. If you have some basic programming knowledge of C++ and Qt, you too can help carry the weight of debugging KDE's software for the grand release in February.
• Joining the development effort -- You may have a deeper knowledge development, and would like to contribute to KDE with your own solutions. This is the perfect moment to get involved in KDE and contribute with your own code.
• Donating to KDE -- Creating, debugging and maintaining the large catalogue of software KDE distributes to users requires a lot of resources, many of which cost money. Donating to KDE helps keep the day-to-day operation of KDE running smoothly and allows developers to concentrate on creating great software. KDE is currently running a drive to encourage more people to become contributing supporters, but you can also give one-time donations if you want.

A note on pre-release software

Pre-release software is only suited for developers and testers. Alpha/Beta/RC software is unfinished, will be unstable and will contain bugs. It is published so volunteers can trial-run it, identify its problems, and report them so they can be solved before the publication of the final product.

The risks of running pre-release software are many. Apart from the hit to productivity produced by instability and the lack of features, using pre-release software can lead to data loss, and, in extreme cases, damage to hardware. That said, the latter is highly unlikely in the case of KDE software.

The version of the software included in KDE's 6th Megarelease is beta software. We strongly recommend you do not use it as your daily driver.

If, despite the above, you want to try the software distributed in KDE's 6th Megarelease, you do so under your sole responsibility, and in the understanding that the main aim, as a tester, you help us by providing feedback and your know-how regarding the software. Please see the Contributing section above.

#611 – JANUARY 9, 2024
View in Browser »

How to Get the Current Time in Python

In this video course, you’ll be getting the current time in Python. You’ll get your hands on a datetime object that represents the current time. You’ll see how to format it according to international standards, and you’ll even check out how computers represent time.
REAL PYTHON course

Using a Markov Chain to Generate Readable Nonsense

Describes a simple Markov chain algorithm to generate reasonable-sounding but utterly nonsensical text, and presents some example outputs as well as a Python implementation in only 20 lines of code.
BEN HOYT

End-to-End Data Science Workflows: ML Workflows With Python

Learn Machine Learning workflow with Python & Posit Team in this insightful video. Starting with an example classification model, through design, development, deployment, & maintenance, learn from our experience with thousands of enterprise customers →
POSIT sponsor

Top Python Libraries of 2023

Tryolab’s 9th annual Top Python libraries list. Loads of LLM stuff this time around, not surprisingly.
DESCOINS & ALFARO

DjangoCon Europe 2024 CFP Now Open

DJANGO SOFTWARE FOUNDATION

Django Bugfix Releases Issued: 4.2.9 and 5.0.1

DJANGO SOFTWARE FOUNDATION

Python GUI Designer “PAGE” 8.0 Released

THEDESIGNATEDGEEK.XYZ • Shared by Greg Walters

Python Jobs Senior Python Architect and Tech Lead (America)

Six Feet Up

Python Tutorial Editor (Anywhere)

Real Python

More Python Jobs >>>

Articles & Tutorials Efficient Inequality Joins in Pandas

Joins in Pandas are equality based. However, there are other scenarios that require joins on inequality or a combination of equi and non-equi joins. Usually in Pandas this is executed via a Cartesian join which can be inefficient as well as memory intensive. This article shows an efficient way to handle inequality joins in Pandas.
SAMUEL ORANYELI • Shared by Samuel oranyeli

2023 Real Python Tutorial & Video Course Wrap-Up

Three members of the Real Python team are joining us this week: Kate Finegan, Tappan Moore, and Philipp Acsany. We wanted to share a year-end wrap-up with tutorials, step-by-step projects, code conversations, and video courses that showcase what our team created in 2023.
REAL PYTHON podcast

Advice for New Devs Who’ve Read Other Advice Essays

After reading some programming advice posts, this author decided a lot of them concentrated on the wrong things. Here is his own take. Associated HN discussion.
HILLEL WAYNE

Building a Bootstrap Styled Form in Vanilla Django

When playing with django-crispy-forms, David wondered what it would take to build a Bootstrap 5 form in Django without any third party libraries. This article outlines the work involved.
DAVID SMITH

Where Have You Installed Your Python Packages?

This article introduces virtual environments and what it means to run pip install. It includes information about the sys module and variables it uses to look for your packages.
KE PI

How to Use Python in Excel Natively

How to use Python in Excel natively using libraries like Pandas, NumPy, Matplotlib, Seaborn and more for analysis and spectacular charts. Includes a cheat sheet.
MYNDA TREACY

My Proof-of-Concept Record Type

As part of a conversation as to whether Python should include a struct-like syntax, Brett has created a project. This post outlines the proof-of-concept.
BRETT CANNON

Python Rgonomics

If you’re coming to Python from R, this article outlines some libraries that have an R-like feel, helping you make the transition to Pythonic workflows.
EMILY RIEDERER

10 Things Software Developers Should Learn About Learning

Developers want and need to perpetually learn. Needing to learn and knowing how are two different things though, and this article highlights key ideas to teach yourself how to learn better.
BROWN, HERMANS, & MARGULIEUX

Don’t Use Python’s @property

A quick opinion piece on the over-use of the @property decorator and how it can lead to misunderstandings in your code.
JAMES BENNETT

10 Noteworthy AI Research Papers of 2023

This article is a compilation of 10 noteworthy AI research papers of 2023.
SEBASTIAN RASCHKA

Projects & Code django-ninja-crud: CRUD Endpoints & Tests With Ninja

GITHUB.COM/HBAKRI

pathway: Framework for Real-Time Data Processing

GITHUB.COM/PATHWAYCOM

cyclopts: Intuitive CLIs Based on Python Type Hints

GITHUB.COM/BRIANPUGH

netchecks: Tool to Validate Assumptions About the Network

GITHUB.COM/HARDBYTE

arrest: Validate REST API Calls Using Pydantic and HTTPX

GITHUB.COM/S-BOSE

Events Building Python Communities Around Python for Kids

January 10 to January 11, 2024
NOKIDBEHIND.ORG

Weekly Real Python Office Hours Q&A (Virtual)

January 10, 2024
REALPYTHON.COM

Python Atlanta

January 12, 2024
MEETUP.COM

PyDelhi User Group Meetup

January 13, 2024
MEETUP.COM

DFW Pythoneers 2nd Saturday Teaching Meeting

January 13, 2024
MEETUP.COM

Inland Empire Python Users Group Monthly Meeting

January 17, 2024
MEETUP.COM

Happy Pythoning!
This was PyCoder’s Weekly Issue #611.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

This post covers Symfony Messenger’s message and message handlers, which are the day to day code developers using features of Symfony Messenger typically will be working on.

by daniel.phin / 10 January 2024

This post is part 2 in a series about Symfony Messenger.

1. Introducing Symfony Messenger integrations with Drupal
2. Symfony Messenger’ message and message handlers, and comparison with @QueueWorker
3. Real-time: Symfony Messenger’ Consume command and prioritised messages
4. Automatic message scheduling and replacing hook_cron
5. Adding real-time processing to QueueWorker plugins
6. Making Symfony Mailer asynchronous: integration with Symfony Messenger
7. Displaying notifications when Symfony Messenger messages are processed
8. Future of Symfony Messenger in Drupal

The Symfony Messenger integration with Drupal provided by the SM project is the only requirement for the following examples.

A message itself is very flexible, as it doesn't require annotations, attributes, or specific class namespace. It only needs to be a class serialisable by Symfony. For simplicity, don’t include any complex objects like Drupal entities. Opt to store entity UUIDs instead.

At its most simple implementation, a message handler is:

• a class at the Messenger\ namespace
• with a #[AsMessageHandler] class attribute
• an __invoke method. Where its first argument is an argument typehinted with the message class.

Example message and message handler:

namespace Drupal\my_module; final class MyMessage { public function __construct(public string $foo) {} } namespace Drupal\my_module\Messenger; use Drupal\Core\State\StateInterface; use Symfony\Component\Messenger\Attribute\AsMessageHandler; #[AsMessageHandler] final class MyMessageHandler { public function __construct(StateInterface $state) {} public function __invoke(\Drupal\my_module\MyMessage $message): void { // Do something with $message. $this->state->set('storage', $message->foo); } }

And dispatch code:

$bus = \Drupal::service(\Symfony\Component\Messenger\MessageBusInterface::class); $bus->dispatch(new MyMessage(foo: 'bar'));

Non-autowirable dependency injection

Message handlers use autowiring by default, so you don’t need ContainerFactoryPluginInterface and friends.

In the rare case that dependencies are not autowirable, you can opt to define a message handler as a tagged service instead of a class with #[AsMessageHandler] attribute and define dependencies explicitly. The same __invoke and argument typehinting semantics apply.

services: my_module.my_message_handler: class: Drupal\my_module\Messenger\MyMessageHandler arguments: - '@my_module.myservice' tags: - { name: messenger.message_handler } Comparison with Legacy Drupal Queues

Typically, when setting up a Drupal queue, you’ll be putting together a rigid class with a verbose annotation. When compared to the functionality of the messenger and handler above, the equivalent @QueueWorker looks like:

namespace Drupal\my_module\Plugin\QueueWorker; use Drupal\Core\Plugin\ContainerFactoryPluginInterface; use Drupal\Core\Queue\QueueWorkerBase; use Drupal\Core\State\StateInterface; use Symfony\Component\DependencyInjection\ContainerInterface; /** * @QueueWorker( * id = "my_module_queue", * title = @Translation("My Module Queue"), * cron = {"time" = 60} * ) */ final class MyModuleQueue extends QueueWorkerBase implements ContainerFactoryPluginInterface { private function __construct( array $configuration, $plugin_id, $plugin_definition, private StateInterface $state, ) { parent::__construct($configuration, $plugin_id, $plugin_definition); } public static function create(ContainerInterface $container, array $configuration, $plugin_id, $plugin_definition): static { return new static( $configuration, $plugin_id, $plugin_definition, $container->get('state'), ); } public function processItem(mixed $data): void { // Do something with $data. $this->state->set('storage', $data['foo']); } }

And dispatch code

\Drupal::service('queue') ->get('my_module_queue') ->createItem(['foo' => 'bar']);

Notice the hard-to-remember annotation, boilerplate dependency injection, and mixed-type processItem argument $data . In comparison, Symfony Messenger messages and message handlers are easier to use thanks to PHP attributes.

Routing messages to transports

All messages will be handled synchronously by default. To route messages to specific transports, routing needs to be configured.

Behind the scenes, routing is a simple map of class/namespaces to transports defined in a container parameter.

parameters: sm.routing: Drupal\my_module\MyMessage: doctrine Drupal\my_module\MyMessage2: synchronous 'Drupal\my_module\*': doctrine '*': doctrine

Keys are either verbatim class names, partial class namespace followed by asterisk, or a standalone asterisk indicating the fallback. The values are the machine name of a transport. SM includes a synchronous transport out of the box, which indicates messages are handled in the same thread as it is dispatched. The doctrine database transport is available as a separate module. I’d recommend always using an asynchronous transport like Doctrine.

Routing configuration UI

SM includes a configuration UI submodule that allows site builders to build a routing map without needing to mess with YAML. The container parameter is set automatically as soon as the form is saved.

Advanced usage of messages and handlersAdding stamps to messages

A common use case for adding stamps to a message is to delay the message for an amount of time. A stamp is created and attached to the envelope containing the message to be processed:

$envelope = new Envelope( message: new MyMessage(foo: 'bar'), stamps: [\Symfony\Component\Messenger\Stamp\DelayStamp::delayUntil(new \DateTimeImmutable('tomorrow'))], ); $bus = \Drupal::service(\Symfony\Component\Messenger\MessageBusInterface::class); $bus->dispatch($envelope);Multiple handlers per message

For more advanced use cases, multiple handlers can be configured for a message. Useful if you want to listen for messages that you do not own. For example, additional handling of the Symfony Mailer email message:

namespace Drupal\my_module\Messenger; use Symfony\Component\Messenger\Attribute\AsMessageHandler; use Drupal\Core\State\StateInterface; #[AsMessageHandler] final class MyMessageHandler { public function __construct(StateInterface $state) {} public function __invoke(\Symfony\Component\Mailer\Messenger\SendEmailMessage $message): void { $this->state->set( 'sent_emails_counter', $this->state->get('sent_emails_counter', 0) + 1, ); } }

Both this custom handler and the original \Symfony\Component\Mailer\Messenger\MessageHandler::__invoke handler will be invoked.

Multiple messages per handler

Handlers can be configured to handle multiple message types. Instead of using the #[AsMessageHandler] attribute on the class, use it with methods.

namespace Drupal\my_module\Messenger; use Drupal\Core\State\StateInterface; use Symfony\Component\Messenger\Attribute\AsMessageHandler; final class MyMessageHandler { #[AsMessageHandler] public function myHandler1(\Drupal\my_module\MyMessage $message): void { // Do something with $message. } #[AsMessageHandler] public function myHandler2(\Drupal\my_module\MyMessage2 $message2): void { // Do something with $message2. } }

The next post covers the worker, the heart of messenger’s real-time capabilities.

Tagged Symfony, Symfony Messenger, QueueWorker

Newer Cisco wireless controllers run IOS-XE instead of AireOS, but they still don't speak ACME (which would let them integrate with Let's Encrypt); they support only SCEP, which I guess is basically Microsoft-and-Cisco only? Something like that.

But it is possible to get it to work nevertheless, and get proper and free certificates for the web administration interface. Unfortunately, I don't think you can use HTTP authentication (since you can't drop arbitrary files into webui:), but assuming you've got some sort of DNS authentication going, you can use that and then autodeploy the certificate using a script as follows:

#! /bin/bash HOST=$1 ENDPOINT=letsencrypt-$( date +%Y%m%d ) PASSWORD=$( pwgen -s 16 ) /usr/bin/certbot certonly --manual --preferred-challenges dns -d $HOST --authenticator manual --key-type rsa echo $PASSWORD | openssl pkcs12 -export -in /etc/letsencrypt/live/$HOST/fullchain.pem -inkey /etc/letsencrypt/live/$HOST/privkey.pem -out $ENDPOINT.p12 -passout stdin scp -O $ENDPOINT.p12 admin@$HOST:flash:$ENDPOINT.p12 ssh admin@$HOST <<EOF crypto pki import $ENDPOINT pkcs12 flash:$ENDPOINT.p12 password $PASSWORD conf t ip http secure-trustpoint $ENDPOINT exit wr EOF

Exercises for the reader:

• Set up non-manual DNS for whatever setup (it's really weird that there's no included nsupdate support in Certbot, only various cloud offerings).
• Make this into a deploy hook instead of a shell script that runs after certonly
• Set up SSH certificates so that you can SSH into the host without passwords

You'll need to have SCP enabled (there's no obvious way of doing un-base64 from the command line?), which you can do with

ip scp enable aaa authentication login default local aaa authorization exec default local none

and probably some other magic stuff if you're using TACACS or similar.

Posted on January 9, 2024 Tags: debian, free software, life

I wanted to report this issue in chromiums issue tracker, but it gave me:

“Something went wrong, please try again later.”

Ok, then at least let me reply to this askubuntu question. But my attempt to signup with my launchpad account gave me:

“Launchpad Login Failed. Please try logging in again.”

I refrain from commenting on this to not violate some code of conduct.

So this is what I wanted to write:

GTK file chooser image preview size should be configurable

The file chooser that appears when uploading a file (e.g. an image to Google Fotos) learned to show a preview in issue 15500.

The preview image size is hard coded to 256x512 in kPreviewWidth and kPreviewHeight in ui/gtk/select_file_dialog_linux_gtk.cc.

Please make the size configurable.

On high DPI screens the images are too small to be of much use.

Yes, I should not use chromium anymore.

Dear Readers,

Have you ever felt the transformative power of a teacher's or mentor's encouragement? Anatole France, the French poet and journalist, once encapsulated this influence by saying, 

"Nine-tenths of education is encouragement."

It's astonishing how teachers can nurture love or sow seeds of positivity, potentially steering an entire future. When a child moves beyond their family circle, teachers often step into a pivotal role. Their impact isn't confined to imparting knowledge; it extends to shaping perspectives, igniting passions, and defining life paths.

Just as many attribute their accomplishments to teachers who unearthed their hidden talents guiding their paths, the Drupal community thrives on mentors who play a pivotal role. Like educators, these mentors guide newcomers, share their expertise, and foster an environment conducive to growth within the Drupal ecosystem. They serve as pillars, offering invaluable guidance and insights enriching the community with their depth of knowledge and experience.

In this context, Michael Anello shines as a beacon within the Drupal realm. His trajectory, from teaching engineering to co-founding DrupalEasy—a hub for Drupal-centric training, career development, and consultancy—mirrors the impact of a guiding force.  The DropTimes had the opportunity to interview Michael Anello. He reflects on his journey, the evolution of Drupal, navigating challenges in the learning process and steering DrupalEasy through adaptations to accommodate various skill levels. In essence, his role echoes the influence of those pivotal teachers who shape destinies within the dynamic and ever-evolving landscape of the Drupal community.

In another enlightening interview, the multifaceted expertise of Lyubomir Filipov, a Group Architect and Deputy Program Manager at FFW, takes centre stage. His journey spans various roles—from organizer to academician to mentor—an embodiment of the diverse skills of an expert Drupal developer.

Lyubomir Filipov operates as a leader by example, orchestrating the convergence of talent with opportunity. He's instrumental in shaping career trajectories for over 800 professionals at FFW, ensuring that the right individuals find their niche within the company. 

Our adept sub-editor, Alka Elizabeth, skillfully conducted these insightful conversations, shedding light on the depth and breadth of Michael Anello's and Lyubomir Filipov's contributions within the Drupal sphere. Alka also delved into a Reddit discussion about the recent changes to not just Drupal alone but the economy, AI, and growth of other CMSs like WordPress, Wix, and Shopify. 

On a side note, I've compiled a thorough overview highlighting TDT's significant milestones of the past year.

Speaking of events, The DropTimes has been announced as the official Media Partner for Drupal Mountain Camp 2024, Switzerland. To learn more about the event, visit the official website. You can also follow our tag Drupal Mountain Camp to catch every update. 

February 06 is DrupalCon Portland's registration kickoff—don't miss out! Find all the essential details right here. Stay tuned for more updates and announcements as the event draws nearer. 

The highly anticipated return of the PHP Conference Kansai 2024—a momentous occasion for PHP engineers in Japan is back after a notable six-year hiatus. Do check out the details here. 

As MidCamp 2024 draws near, it invites organizations to unite and bolster the vibrant Drupal community together. 

FLDC 2024's countdown has commenced! Seize the early bird registration before January 12 and save $25 on your attendance fees. Don't wait—the deadline is approaching fast!  

Fan tickets for DrupalCamp Poland are available until November 1, 2024—grab yours before time runs out.  A list of the Drupal events for the week is published here.

Acquia's latest release, the e-book "Achieving Success with Headless and Hybrid CMSs," provides in-depth insights into the benefits of employing CMS in various architectures—traditional, headless, or hybrid. For more information, dive into this informative resource.

Some notable news highlights include WeebPal's celebration of its 12th anniversary, dubbed the Silk or Fine Linen Jubilee, featuring an enticing offer for website owners. They're offering 12 participants an opportunity to upgrade from Drupal 9 to Drupal 10 free of cost.  

Centarro is rejoicing in its most substantial release yet: Drupal Commerce 2.37. Learn more about it here.

A groundbreaking addition to Drupal's arsenal is the newly launched Error Reporting module, poised to transform code analysis and debugging for effortless operations. Dive deeper into its capabilities here.

While more stories beckon for exploration, constraints compel us to halt further selection. To get timely updates, follow us on LinkedIn, Twitter and Facebook.

Keep enjoying your Drupal journey! 
Thank you,

With warm wishes,
Elma John
Sub-Editor, TheDropTimes

In the second episode of our three-part series on the ETL Migration process, we delve into the most involved stage of Drupal migration: Transformation. This episode features insights from Tag1 Consulting’s experts, including Mike Ryan, co-creator of Drupal Migrate, and notable contributor Benji Fisher. They analyze the Transformation phase in the ETL process, specifically examining Drupal’s unique “row-by-row” approach, and the discussion session revolves around the advantages and challenges of this method, with a strong emphasis on optimizing performance within the transformation pipeline. The episode is a treasure trove for those considering or currently working on Drupal migrations, as it steps into the technical realm while touching on the practical aspects of transforming data during a migration. Benji expresses his fascination with this particular stage, describing it as a playground for innovation and detailing the intricacies that make it the heart of the migration process. This discussion is essential for developers and IT professionals seeking to understand or undertake large-scale Drupal migrations. Part 1 | Part 2 | Part 3 - Coming soon! Please let us know if you have specific migration-related topics you'd like to see us cover. Or, reach out and let us know if we...

Read more janez Tue, 01/09/2024 - 07:33

One of the most popular web frameworks, Django, has adopted the “batteries included” philosophy. This means that you can build a production-ready application using only the vanilla Django with no extensions or other frameworks. Even the database SQLite is available out of the box. SQLite is great for learning and developing small applications, but has […]

As you learned in Python Basics: Functions and Loops, functions serve as the fundamental building blocks in almost every Python program. They’re where the real action happens!

You now know that functions are crucial for breaking down code into smaller, manageable chunks. They enable you to define actions that your program can execute repeatedly throughout your code. Instead of duplicating the same code whenever your program needs to accomplish a particular task, you can simply call the function.

However, there are instances when you need to repeat certain code multiple times in a row. This is where loops become invaluable.

In this Python Basics Exercises video course, you’ll practice:

• Creating user-defined functions
• Implementing for loops
• Getting user input
• Rounding numbers

This video course is part of the Python Basics series, which accompanies Python Basics: A Practical Introduction to Python 3. You can also check out the other Python Basics courses.

Note that you’ll be using IDLE to interact with Python throughout this course.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Qt for Android plugin, introduced more than a decade ago, has been a game-changing change that opened a multitude of possibilities for developers looking to harness the power and flexibility of Qt for Android application development. Since then, many Android, Qt, and plugin changes have been made to support new features. However, neither the overall architecture nor the public Java bindings have changed much. These bindings contain wrappers for the Android Activity. It is about time we did that! 

The advantages of Drupal CMS always include its friendliness to users who cannot write code. In most cases, these users are content managers. How can you give more power to the content manager of your Drupal website? Add the Layout Builder module that makes building web pages as easy as putting Lego bricks together.

<strong>Topics covered in this episode:</strong><br> <ul> <li><a href="https://tonybaloney.github.io/posts/python-gets-a-jit.html"><strong>Python 3.13 gets a JIT</strong></a></li> <li><a href="https://fosstodon.org/@basnijholt/111605262871991435">UniDep - Unified Conda and Pip Dependency Management</a></li> <li><a href="https://hynek.me/articles/pull-requests-branch/"><strong>Don’t Start Pull Requests from Your Main Branch</strong></a></li> <li><a href="https://github.com/pomponchik/instld"><strong>instld: The simplest package management</strong></a></li> <li><strong>Extras</strong></li> <li><strong>Joke</strong></li> </ul><a href='https://www.youtube.com/watch?v=PRaTs3PnJvI' style='font-weight: bold;'>Watch on YouTube</a><br> <p><strong>About the show</strong></p> <p>Sponsored by us! Support our work through:</p> <ul> <li>Our <a href="https://training.talkpython.fm/"><strong>courses at Talk Python Training</strong></a></li> <li><a href="https://courses.pythontest.com/p/the-complete-pytest-course"><strong>The Complete pytest Course</strong></a></li> <li><a href="https://www.patreon.com/pythonbytes"><strong>Patreon Supporters</strong></a></li> </ul> <p><strong>Connect with the hosts</strong></p> <ul> <li>Michael: <a href="https://fosstodon.org/@mkennedy"><strong>@mkennedy@fosstodon.org</strong></a></li> <li>Brian: <a href="https://fosstodon.org/@brianokken"><strong>@brianokken@fosstodon.org</strong></a></li> <li>Show: <a href="https://fosstodon.org/@pythonbytes"><strong>@pythonbytes@fosstodon.org</strong></a></li> </ul> <p>Join us on YouTube at <a href="https://pythonbytes.fm/stream/live"><strong>pythonbytes.fm/live</strong></a> to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too.</p> <p><strong>Brian #1:</strong> <a href="https://tonybaloney.github.io/posts/python-gets-a-jit.html"><strong>Python 3.13 gets a JIT</strong></a></p> <ul> <li>Anthony Shaw</li> <li>Great article that walks through JIT concepts with a small example as if you were writing a parser in Python instead of C.</li> <li>Covers <ul> <li>What is a JIT?</li> <li>What is a copy-and-patch JIT? and Why?</li> <li>How does the Python JIT work?</li> <li>Is it faster?</li> <li>This is a building block to future improvements</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href="https://fosstodon.org/@basnijholt/111605262871991435">UniDep - Unified Conda and Pip Dependency Management</a></p> <ul> <li>🔄 Single requirements.yaml for both #Conda &amp; #Pip.</li> <li>⚙️ Works with pyproject.toml &amp; setup.py.</li> <li>🏢 Perfect for monorepos.</li> <li>🔒 Create consistent conda-lock files for multiple projects.</li> <li>🌍 Platform-specific support.</li> <li>🚀 <code>unidep install</code> for easy setup.</li> <li><a href="https://github.com/basnijholt/unidep">Full source page</a>.</li> </ul> <p><strong>Brian #3:</strong> <a href="https://hynek.me/articles/pull-requests-branch/"><strong>Don’t Start Pull Requests from Your Main Branch</strong></a></p> <ul> <li>Hynek Schlawack</li> <li>When contributing to other users’ repositories, always start a new branch in your fork.</li> <li>Reasons to not use main <ul> <li>Forces you to only have one change in progress</li> <li>Merges will generate conflicts and you can’t pull from that branch anymore. Need to kill the fork and start over</li> <li>If the target repo has branch protection on, then maintainers can’t push to your branch.</li> </ul></li> <li>Hynek also provides a way to fix things if you’ve already started your changes on a main branch fork.</li> </ul> <p><strong>Michael #4:</strong> <a href="https://github.com/pomponchik/instld"><strong>instld: The simplest package management</strong></a></p> <ul> <li>Thanks to this package, it is very easy to manage the lifecycle of packages.</li> <li>⚡ Run your code without installing libraries.</li> <li>⚡ You can use 2 different versions of the same library in the same program.</li> <li>⚡ You can use incompatible libraries in the same project, as well as libraries with incompatible/conflicting dependencies.</li> <li>⚡ It's easy to share written scripts. The script file becomes self-sufficient - the user does not need to install the necessary libraries.</li> <li>⚡ The library does not leave behind "garbage". After the end of the program, no additional files remain in the system.</li> </ul> <p><strong>Extras</strong> </p> <p>Brian: </p> <ul> <li><a href="https://courses.pythontest.com/p/complete-pytest-course">The Complete pytest Course</a> is now actually complete <ul> <li>Although updates will happen when and if necessary as pytest/Python changes.</li> <li>To celebrate, <strong><em>use code 2024 in January for 10% off</em></strong> any pricing option.</li> </ul></li> <li>More episodes of <a href="https://pythonpeople.fm">Python People</a> and <a href="https://podcast.pythontest.com">Python Test</a> on the way now <ul> <li>That course took up a lot of my time in late 2023</li> <li>Just released an <a href="https://pythonpeople.fm/episodes/will-vincent-django-writing-technical-books">episode with Will Vincent</a> and Python Test will have a new episode this week and for the foreseeable future.</li> <li>Let me know if you want to be on Python People or Python Test</li> </ul></li> </ul> <p>Michael:</p> <ul> <li><em>Hatch follow up: Great coverage of Hatch v1.8.0! One small correction: only the binaries for Hatch are signed with the certificate from the PSF.</em> - <strong>Ofek</strong></li> <li><a href="https://status.python.org/incidents/0mld3fml68nd">PyPI new user registration temporarily suspended</a></li> <li>Pagefind and <a href="https://fosstodon.org/@mkennedy/111637520985150159">how I’m using it</a></li> <li><a href="https://www.youtube.com/watch?v=KeegA_uzzSo">Talk Python Live: Data Doodles event</a> coming early Feb</li> <li>New essay: <a href="https://mkennedy.codes/posts/ai-features-a-waste-of-time/">AI Features a Waste of Time?</a></li> </ul> <p><strong>Joke:</strong> <a href="https://workchronicles.com/put-it-in-the-backlog/">Put it in the backlog</a></p>

The restaurant sector has faced challenges amidst economic downturns, prompting a search for recovery through enhanced business agility. In fact, a staggering 71% of food establishments acknowledge the central role of digital transformation in achieving this objective.

In response to the growing influx of restaurants into the online domain, the demand for an efficient Restaurant CMS has become increasingly evident. A well-suited Restaurant website CMS, such as Drupal, proves instrumental in optimizing business operations.

What Makes Drupal the Right Choice for Restaurants? 

Drupal has over 20 years of experience in managing restaurant content and has become the top choice for many big companies and restaurants. 

Did you know? 

Around 89.2% of users believe that Drupal will become even more popular among different types of businesses in the next three years, not just in the restaurant industry. 

This means the demand for a Sports CMS and an Education CMS will also increase.

Looking for an assistant to handle all your routine Drupal tasks including checking for updates and clean-ups, without any manual intervention? We'd like to introduce you to Cron Jobs! Now let’s dive into the world of this behind-the-scenes helper, specifically in the context of task automation within Drupal. In this blog, we will delve into the fascinating realm of automation, explore the fundamental concept of cron jobs, provide an overview of cron jobs in Drupal, and discuss various essential aspects of crafting your first custom Drupal cron job. What is Automation Automation is the practice of utilizing technology, machinery, or software to perform tasks, processes, or operations with minimal or no human intervention. The primary objective of automation is to streamline and optimize workflows, reduce manual effort, improve efficiency, enhance accuracy, and increase productivity. In simpler terms, automation entails using technology to minimize the human effort required for task execution. As technology continues to advance, we can expect to see automated machines, technology, and software playing a more significant role in many industries and everyday tasks. Now, keeping all these things in mind and asking ourselves how we can automate something, we move to the next topic. What are Cron Jobs? The term "cron" is frequently encountered in the domain of computers, especially within operating systems like Linux. It serves as a tool for scheduling tasks based on time. In Unix systems, such as Linux, "cron" is a scheduling utility. A "cron job" is an automated task scheduled using the "cron" utility. Cron is a daemon – a background process that handles non-interactive tasks. In Unix-like operating systems, a daemon is a component, or a type of program, that silently operates in the background without requiring human interaction. It's like having a robot that wakes up and starts working when it hears a specific sound or when something specific happens. Cron jobs provide users with a powerful tool to automate system maintenance, monitor disk space, and manage backups at regular intervals. While cron job scheduling is widely favored by system administrators, it also offers significant benefits to web developers. It functions as a built-in alarm clock for your website tasks, ensuring that you can automate routine actions with ease. Cron is typically pre-installed on the system. However, in cases where it's not available, you can easily install it using the following commands: Update the package repository sudo apt updateInstall cron sudo apt install cronEnable the cron service to ensure it runs as expected sudo systemctl enable cronThe question arises: How can we activate cron or configure a customized cron to meet our specific needs? In this scenario, crontab comes to the rescue. Crontab A crontab is like a to-do list of tasks you want your computer to do at specific times, and it's also the name of the command you use to create and manage that list. Crontab stands for “cron table”. Each user in Linux has their own crontab, which may be used to schedule jobs that will be executed as that user. You can view, and update the crontab file using the crontab command. Example: To update the crontab file crontab -eTo list down or show the commands stored in the crontab file crontab -lCron Expression In simple terms, a cron expression in Linux is like setting an alarm for your computer to do specific tasks regularly. You have five fields to play with: minutes, hours, days of the month, months, and days of the week. By adjusting these fields, you can tell your computer exactly when to do things. There are also special characters like asterisks and slashes that let you be flexible with your schedule. For instance, if you set a cron expression like "0 * * * *," it means your computer will perform a task at the beginning of every hour. It's a handy tool that makes your computer do routine jobs automatically, like maintenance, backups, and running scripts. A job is represented by each line of a crontab file, which looks like this SYNTAX: # ┌────── minute (0 - 59)# │ ┌────── hour (0 - 23)# │ │ ┌────── day of the month (1 - 31)# │ │ │ ┌────── month (1 - 12)# │ │ │ │ ┌────── day of the week (0 - 6) (Sunday to Saturday;# │ │ │ │ │                7 is also Sunday on some systems)# │ │ │ │ │# │ │ │ │ │# *  *  *  *  *  command_to_execute Examples: At every 1 minute: * * * * * At every 2 minute:  * /2 *  *  *  * At every 15 minute: * /15  *  *  *  * At every 1 hour: 0  *  *  *  * At every 3 hours: 0  * /3  *  *  * At every 8 hours: 0  * /8  *  *  * Everyday (At 00:00):  0  0  *  *  * Hour range: Every hour between 11 am to 2 pm: 0  11-14  *  *  * Thanks to crontab Guru which is a simple editor for cron schedule expressions. For more examples, you can refer to this page. Drupal Cron Job Overview Drupal has a handy feature called Automated Cron. The Drupal automated cron job is an automatic task scheduler that handles regular website upkeep, including tasks like optimizing the database and clearing the cache, all at predetermined time intervals. It guarantees the website's smooth operation and data precision, minimizing the necessity for manual involvement. Some key features of automated cron: Scheduled Maintenance Tasks: In simple terms, Drupal's automated cron is like a behind-the-scenes helper that does regular chores for your website. It cleans up the website's storage, checks for important updates, and takes care of other routine jobs without bothering you. This helps keep your website running smoothly without you having to do these tasks manually. Configurable Frequency: If you're in charge of a Drupal website, you can decide how often the automated cron job should do its work. By default, it does its tasks every three hours, but you can change that to better suit your website's requirements. You can set this schedule by going into the Drupal admin settings, and it's kind of like adjusting the timing on a sprinkler system for your garden. Background Process: Think of automated cron in Drupal like a silent worker in the background. It gets things done without disturbing the part of your website that visitors see. It doesn't need outside tools to know when to do its tasks; it uses its internal clock, much like an alarm clock that wakes you up at a specific time without needing help from another device. Limitations of Drupal cron: Simultaneous Execution of hook_cron Modules and Resource Challenges: When several modules use the hook_cron function, they all run simultaneously, either in alphabetical order or based on the weight assigned to each module. This can make a webpage request more resource-intensive all at once and has the potential to exceed memory limits on a complex site. Handling Errors in Cron Tasks and Identifying Culprits for Smooth Execution: If there's an error in one cron task, it can halt the entire cron process, preventing the execution of the remaining tasks until the issue is fixed. It's also challenging to identify which task is causing the problem. Optimizing Cron Execution by Analyzing Task Durations: To improve the efficiency of cron execution and to optimize the cron run time. It's important to examine which cron tasks consumed more time during their run. Now to overcome all these limitations, Drupal provides us with modules like Ultimate Cron, Super Cron (abandoned module), Elysia Cron (Drupal 7), etc.. All these modules are one or the same with more added features. Now Super Cron is unsupported, the module is abandoned due to some security issues. The module is no longer developed by the maintainer whereas Elysia cron is for Drupal 7. How does Ultimate Cron Help? Parallel Execution of Cron Tasks: Error Isolation and Independence: Tasks are no longer executed sequentially; they now run in parallel. This means that if one cron task encounters an error, it won't affect the independent execution of other tasks. Individual Configuration for Cron Jobs with Differing Schedules: You can configure each cron job separately. For example, you can set up your cron job to run every hour and another cron job (cron job B) to run every two hours. Independent Logging for Each Cron Job: Each has separate logs. Let's install the Ultimate cron module and create our first custom cron job. So, to install the module you can visit https://www.drupal.org/project/ultimate_cron and download the module using composer. Use this command: composer require 'drupal/ultimate_cron:^2.0@alpha'This will install the Ultimate cron module, the next step is to enable it. You can enable the module via the site or by drush command. Use this drush command to enable the module: drush en ultimate_cronAfter installing Ultimate cron, we get some predefined cron jobs that the module provides, for example, cleanups (cache, temp files), removing expired log messages, and history deletion. Path: xyz.domain.name/admin/config/system/cron/jobs How to Configure Custom Ultimate Cron Job Create a config/install directory in the root of your custom module’s directory. Make a new file called ultimate_cron.job.jobname.yml (replace jobname with the name of your job). In our case we are creating a custom cron job to publish all the unpublished article contents, therefore let's name our job as publish_article_job. 3. Next step is to edit the ultimate_cron.job.publish_article_job.yml file. Inside ultimate_cron.job.publish_article_job.yml, we’ll add the details which will include a callback function, cron job ID, and rules for scheduling the task. langcode: en status: true dependencies: module: - custom_cron_example title: 'Custom job for publishing articless' id: publish_article_job module: custom_cron_example callback: custom_cron_example_publish_article_job scheduler: id: simple configuration: rules: - '* * * * *'4. Once editing the configuration file, we will now define a callback function inside the custom_cron_example.module file. <?php use Drupal\node\Entity\Node; /** * Custom ultimate cron callback function. */ function custom_cron_example_publish_article_job() { $message = "Voila ! Ultimate cron works."; \Drupal::logger("custom_cron_example")->notice($message); // Query to fetch all article unpublished nodes. $query = \Drupal::entityQuery('node'); $query->condition('status', 0); $query->condition('type', 'article'); $entity_ids = $query->accessCheck(FALSE)->execute(); foreach ($entity_ids as $entity_id) { $node = Node::load($entity_id); $node->setPublished(TRUE)->save(); } }Once all steps have been completed, it's time to activate the module in order to ensure that the new configurations we've created are applied to the website. Once the module is enabled, you will be able to view your custom cron job on the cron job dashboard.   Now this cron job will run every 1 minute since we added the cron expression as * * * * *. You can alter the rule as per your requirement. Feel free to use crontab guru for references. I believe there’s no harm in that ;) Results: Before the cron run, the article was unpublished.   Once the scheduled task (cron) is completed, the article gets published on the website. Additionally, there's a record in the database log that displays our unique message, which we included in the custom callback function we created.   Enabling Cron using External Calls By using crontab on the server. External cron job services like easycron or cronless. You can configure the time as per your requirement.  Easycron Cronless It may be beneficial to disable the automated cron system for performance reasons or to ensure that cron can only ever run from an external trigger. To disable the automated cron: Change the value of "Run cron every" to "Never."                    (OR) Add the following line to your settings.php: (so that no other user can enable it). $config['automated_cron.settings']['interval'] = 0;Final Thoughts In today's world, Drupal's cron job system remains essential for automated site maintenance. However, its design limitations and potential resource impact are better addressed with modern alternatives like scheduled tasks in server or external cron services, ensuring smoother website performance and management. Automated technology and machines are fascinating aspects of the future that hold the potential to revolutionize various aspects of our lives. However, adopting automation will also require careful consideration of its impact on the workforce and society as a whole Is your online presence ready for an upgrade? Let's discuss how Specbee can be your strategic partner in Drupal development.  

I ended 2022 with a musical retrospective and very much enjoyed writing that blog post. As such, I have decided to do the same for 2023! From now on, this will probably be an annual thing :)

Albums

In 2023, I added 73 new albums to my collection — nearly 2 albums every three weeks! I listed them below in the order in which I acquired them.

I purchased most of these albums when I could and borrowed the rest at libraries. If you want to browse though, I added links to the album covers pointing either to websites where you can buy them or to Discogs when digital copies weren't available.

Once again this year, it seems that Punk (mostly Oï!) and Metal dominate my list, mostly fueled by Angry Metal Guy and the amazing Montréal Skinhead/Punk concert scene.

Concerts

A trend I started in 2022 was to go to as many concerts of artists I like as possible. I'm happy to report I went to around 80% more concerts in 2023 than in 2022! Looking back at my list, April was quite a busy month...

Here are the concerts I went to in 2023:

• March 8th: Godspeed You! Black Emperor
• April 11th: Alexandra Stréliski
• April 12th: Bikini Kill
• April 21th: Brigada Flores Magon, Union Thugs
• April 28th: Komintern Sect, The Outcasts, Violent Way, Ultra Razzia, Over the Hill
• May 3rd: First Fragment
• May 12th: Rhapsody of Fire, Wind Rose
• May 13th: Aeternam
• June 2nd: Mortier, La Gachette
• June 17th: Ultra Razzia, Total Nada, BLEMISH
• June 30th: Avishai Cohen Trio
• July 9th: Richard Galliano
• August 18th: Gojira, Mastodon, Lorna Shore
• September 14th: Jinjer
• September 22nd: CUIR, Salvaje Punk, Hysteric Polemix, Perestroika, Ultra Razzia, Ilusion, Over the Hill, Asbestos
• October 6th: Rancoeur, Street Code, Tenaz, Mortimer, Guernica, High Anxiety

Although metalfinder continues to work as intended, I'm very glad to have discovered the Montréal underground scene has departed from Facebook/Instagram and adopted en masse Gancio, a FOSS community agenda that supports ActivityPub. Our local instance, askapunk.net is pretty much all I could ask for :)

That's it for 2023!