Feeds

Happy to announce a new package: RcppMagicEnum. It arrived on CRAN yesterday following the resumption of normal service following the CRAN summer break. RcppMagicEnum brings the magicenum library by Daniil Goncharov to R.

Modern C++ is powerful, but still lacks reflection. This may change with C++26 but until then this library can help. A simple example, also shown on the README is as follows (and can be called from R via Rcpp::sourceCpp() if the RcppMagicEnum package is installed):

// [[Rcpp::depends(RcppMagicEnum)]] #include <RcppMagicEnum> // define a simple enum class, it uses optional typing as well as optional assigned values enum class Color : int { RED = -10, BLUE = 0, GREEN = 10 }; // [[Rcpp::export]] void example() { // instantiate an enum value in variable 'val' auto val = Color::RED; // show the current value on stdout Rcpp::Rcout << "Name of enum: " << magic_enum::enum_name(val) << std::endl; Rcpp::Rcout << "Integer value of enum: " << magic_enum::enum_integer(val) << std::endl; } /*** R example() */

It produces the following output (where the ‘meta-comment’ at the end ensure the included and created-by-sourcing function example() is also called):

> Rcpp::sourceCpp("miniex.cpp") > example() Name of enum: RED Integer value of enum: -10 >

The plan to experiment some more with this and then see if we could possible make factor variables map to such enums and vice versa. Help and discussion input is always welcome, and could be submitted either on the rcpp-devel list or as an issue at the repo.

The short NEWS entry follows.

Changes in version 0.0.1 (2024-07-31)

• Initial version and CRAN upload

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

It has been a few months since I wrote about Rye here last. You might remember that in February I passed over stewardship of my Rye packaging too to Astral. The folks over there have been super busy in building a lot of amazing tooling for Python packaging in the last few months. If you have been using Rye in the last few months you will have noticed that the underlying resolver and installer uv got a lot better and faster.

As of the most recent release, uv also gained a lot of functionality that previously required Rye such as manipulating pyproject.toml files, workspace support, local package references and script installation. It now also can manage Python installations for you so it's getting much closer.

If you are using Rye today, consider this blog post as a reminder that you should probably starting having a closer look at uv and give feedback to the Astral folks.

I gave a talk just recently in Prague at EuroPython about my current view of the Python packaging, the lessons I learned when creating Rye and one of the things I mentioned there is that the goal of a packaging tool has to be that it will dominate the space. The tool that absolutely everybody uses has to be the best tool: it's the thing any new person to Python gets to see when they start their programming journey. After that talk a lot of people walked up to me and had a lot of questions about that in particular.

Python in the last two years has become an incredibly hot and popular platform for many new developers. That has in part been fueled by all the investments and interest that went into AI and ML. I really want everybody who gets to learn and experience Python not to remember it as an old language with bad tooling, but as an amazing language with a stellar developer experience. Unfortunately that's not the case today because there is so much choice, so many tools that are not quite compatible, and by the inconsistency everywhere. I have seen people walk down one tool, just to re-emerge moving their entire stack to conda and back because they hit some wall.

Domination is a goal because it means that most investment will go into one stack. I can only re-iterate my wish and desire that Rye (and with it a lot of other tools in the space) should cease to exist once the dominating tool has been established. For me uv is poised to be that tool. It's not quite there today yet for all cases, but it will be in no time, and now is the moment to step up as a community and start to start to rally around it. That doesn't mean that this tool will be the tool forever. Things come and go and maybe there is a future for some other tool.

But today I'm looking forward to the moment when there will be a final release of Rye that is no remaining functionality other than to just largely alias to uv, that retires Rye specific functionality and migrates you over to uv.

However I only have the power to retire one tool, and that won't be enough. Today we are using so many other package managing solutions for Python and we should be advertising fewer. I understand how much time and effort went into many of those, and everybody's contributions are absolutely appreciated. Software like Rye and uv were built on the advancements of the ecosystem underneath it. They leverage years and years of work that went into migrating the Python ecosystems from setup.py files to eggs and finally wheels. From not having a metadata standard to having one. From coupled to decoupled build systems. Much of what makes Rye so enjoyable were individuals that worked towards making redistributable and downloadable Python binaries a possibility. There was a lot of work that was put into building out an amazing ecosystem of Rust crates and Python libraries needed to make these tools work. All of that brought us to that point where we are today.

But it is my believe that we need to take the next step and be willing to say as a community that some tools are no longer recommended. Maybe not today, but that moment will come quicker than we think. I remember a time when many of us who maintained Python libraries pointed new developers to using ez_setup.py and easy_install in our onboarding guides. Years later we removed the mentions of ez_setup.py from our guides to replace them with pip. Some of us have pointed developers at pip-tools, at poetry or PDM. Many projects today even show 5 different installation guides because of that wild variety of tools available because they no longer feel like they can recommend one.

If you maintain an important Python project I would ask you to give uv a try and ask yourself if you would consider pointing people towards it. I think that this is our best shot in the community at finding ourselves in a much better position than we have ever been.

Have a look at the blog post that Charlie from Astral wrote about what uv can do today. It's a true accomplishment worth celebrating and enjoying.

Postscriptum: there is an elephant in the room which is that Astral is a VC funded company. What does that mean for the future of these tools? Here is my take on this: for the community having someone pour money into it can create some challenges. For the PSF and the core Python project this is something that should be considered. However having seen the code and what uv is doing, even in the worst possible future this is a very forkable and maintainable thing. I believe that even in case Astral shuts down or were to do something incredibly dodgy licensing wise, the community would be better off than before uv existed.

Ever wished you could take an Intro to Python training with me, but you don’t work for a company with a generous training budget? I’m running a Python-learning program just for this situation.

Python High Five is a 10-week Python jumpstart program that starts this September.

Set aside the time to learn ⌚

One of the biggest problems for folks starting to learn Python is setting aside the time. And even if you do manage to set aside the time, you’ll often hit a roadblock where you feel confused.

Python High Five is a way to keep a daily learning habit and to get help when find yourself stuck.

This program is based around daily practice. Monday through Friday you’ll pick 30 minutes from your schedule, at any time that works you. During those 30 minutes, you’ll watch a 5 minute video, work on the day’s exercise, and reflect on your progress.

The most effective learning is hands-on 🖐️

Python High Five is all about learning through writing Python code. Each week we’ll dive deeper into Python, building upon what we’ve learned so far.

When you find yourself stuck you can get help through an asynchronous group chat and weekly office hour sessions. In addition to our weekly office hours together, I’ll check the chat each day, respond to questions, and provide guidance.

Proven learning techniques behind the scenes 📝

The daily check-ins allow for daily accountability. The group chat also provides both a community of peers to rely on, and guidance from an experienced Python trainer (me).

We’ll also be using proven learning techniques behind the scenes:

• Retrieval practice: you don’t learn by putting information into your head, but by trying to take it out; for Python learning, that means writing code.
• Spaced repetition: cramming is less effective than learning spaced out over time, which is why we’ll spend 30 minutes each weekday instead of spending a few hours every week.
• Interleaving: each day’s exercise isn’t predictably themed because a bit of unpredictability can be really improve learning outcomes.
• Elaboration: your daily check-in isn’t just about reflection: it’s also a helpful learning tool!

Plus, we’ll be working through curriculum I’ve been developing and iterating on for many years. I have taught these topics in many different settings to folks from many different backgrounds.

Form a daily learning habit 🔁

Any 10-week program will be just the start of a Python learning habit. You’ll need to keep up your Python after Python High Five ends, either by promptly applying your skills to a new project or diving deeper into Python with continued daily practice.

That’s why I’m offering an 80% discount for High Five attendees on one year of Python Morsels, which is my skill-building service designed to help deepen your Python skills every week. You can see more details on that here.

Ready to start your Python journey? ⛰️

Are you ready to start your Python journey with a solid foundation?

Read more about Python High Five and decide whether this is for you.

Keep in mind that while the program begins on September 9, enrollment closes on August 31. So check the FAQs and if you have additional questions, be sure to email me soon!

Python programmers typically check for empty lists by relying on truthiness.

Table of contents

1. Checking the length of a list
2. Evaluating the truthiness of a list
3. Comparing for equality with an empty list
4. Truthiness checks are non-emptiness checks on lists

Checking the length of a list

One way to check whether a list is empty is to check the length of that list. If the length is 0, the list must be empty:

>>> numbers = [] >>> if len(numbers) == 0: ... print("The list is empty.") ... The list is empty.

Or if we wanted to check for non-empty lists, we could make sure that the length is greater than 0:

>>> if len(numbers) > 0: ... print("The list is NOT empty.") ...

But this is actually not the most typical way to check for an empty list in Python.

Evaluating the truthiness of a list

Many Python users prefer to …

Read the full article: https://www.pythonmorsels.com/checking-for-an-empty-list-in-python/

#643 – AUGUST 20, 2024
View in Browser »

Using a Custom Scrapy Extension to Log to a DB

The Scrapy crawl stat logs are useful for tracking and monitoring the performance of a spider. If you want to keep them longer rather than just see the console printout, you can have them written to a database.
XIEGERTS.COM • Shared by Stephen

Interacting With REST APIs and Python

In this video course, you’ll learn how to use Python to communicate with REST APIs. You’ll learn about REST architecture and how to use the requests library to get data from a REST API. You’ll also explore different Python tools you can use to build REST APIs.
REAL PYTHON course

Posit Connect - Help Your Analytics Team Share and Collaborate

Tired of tediously send files and trying to use general-purpose collaboration tools? Posit Connect makes it easy to share, collaborate, and get feedback on your data science work including Jupyter notebooks, Plotly dashboards, Streamlit, Quarto, Shiny or other interactive analytics applications →
POSIT sponsor

Django: Create Management Sub-Commands

argparse, the standard library module that Django uses for parsing command line options, supports sub-commands. These are pretty neat for providing an expansive API without hundreds of individual commands. This article shows you how to write your own.
ADAM JOHNSON

PEP 752: Package Repository Namespaces (Added)

PYTHON

Quiz: Python Protocols: Leveraging Structural Subtyping

REAL PYTHON

Quiz: Logging in Python

REAL PYTHON

Discussions How Do You Motivate Yourself to Sit Everyday & Code?

REDDIT

Articles & Tutorials Crawling Infinite Scroll Pages With Playwright

When crawling websites with Scrapy you’ll quickly come across all sorts of scenarios that require you to get creative or interact with the page that you’re trying to scrape. One of these scenarios is when you need to crawl an infinite scroll page. This type of website page loads more content as you scroll down the page like a social media feed.
STEPHEN SIEGERT • Shared by Stephen Siegert

Automating Boolean-Based SQL Injection With Python

SQL injection is the process of tricking a database into doing unintended things by modifying the input values to a query. Boolean-based blind injection is a subset that reveals structural information about the database. These can be hard to craft by hand, this article shows you how to automate the process to help do penetration testing.
TREBLEDJ

Automatic Speech Recognition with AssemblyAI

Experience near-human accuracy, low-latency performance, and advanced Speech AI capabilities with AssemblyAI’s Speech-to-Text API. Sign up today and receive free API credits—No credit card required. Get $50 Credit →
ASSEMBLY AI sponsor

Logging in Python

If you use Python’s print() function to get information about the flow of your programs, then logging is the natural next step for you. This tutorial will guide you through creating your first logs and show you ways to curate them to grow with your projects.
REAL PYTHON

Recent CPython Function Call Performance Improvements

How costly it is to call functions and builtins in your python code? Does inlining help? How have the recent CPython releases improved performance in these areas? This article dives deep on function performance.
ABHINAV UPADHYAY

Debugging Running Python Scripts With PDB via GDB

On *nix systems with GDB installed, you can attach to already running processes. This article shows you how to combine that with Python’s PDB debugger to then add breakpoints to Python in a running script.
DOMINIK CZARNOTA

Approximating Sum Types in Python With Pydantic

This is a deeper dive into types and Pydantic around how to build “correct by construction” design patterns. Building your objects so that validation becomes a single call.
WILLIAM WOODRUFF

PyTorch vs TensorFlow for Your Python Deep Learning Project

PyTorch vs Tensorflow: Which one should you use? Learn about these two popular deep learning libraries and how to choose the best one for your project.
REAL PYTHON

How to Plot in the Terminal With Python and Textualize

Have you ever wanted to create a plot or graph in your terminal? Learn how with the textual-plotext package.
MIKE DRISCOLL

It Works on My Machine. Why?

A list of things to check when something works on your computer but not on someone else’s.
MATHEUS RICHARD

Python Packaging Is Great Now: uv Is All You Need

Juan talks about his love for the uv tool and how it has simplified Python packaging.
JUAN LUIS CANO RODRIGUEZ

Projects & Code TTS: Deep Learning Toolkit for Text-to-Speech

GITHUB.COM/COQUI-AI

pyawaitable: CPython API for Asynchronous Functions

GITHUB.COM/ZEROINTENSITY

audiosample: NumPy-Like Audio Manipulation Library

GITHUB.COM/DEEPDUB-AI

django-http-debug: Endpoints That Log and Return Mock Data

GITHUB.COM/SIMONW

emval: Speeding Up Email Validation 1000x

GITHUB.COM/BNKC • Shared by lev ostatnigrosh

Events Weekly Real Python Office Hours Q&A (Virtual)

August 21, 2024
REALPYTHON.COM

PyCon Somalia 2024

August 21 to August 23, 2024
PYCON.ORG.SO

Kiwi PyCon XIII

August 23 to August 26, 2024
KIWIPYCON.NZ

PyDelhi User Group Meetup

August 24, 2024
MEETUP.COM

EuroSciPy 2024

August 26 to August 31, 2024
EUROSCIPY.ORG

PyCon PL 2024

August 29 to September 1, 2024
PYCON.ORG

Happy Pythoning!
This was PyCoder’s Weekly Issue #643.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

An HTML email utilizes HTML code for presentation. Its design is heavy and looks like a modern web page, rich with visual elements like images, videos, etc., to emphasize different parts of an email's content.

Building email templates tailored to your brand is useful for various email marketing purposes such as welcoming new customers, order confirmation, and so on. Email template customization allows you to save time by not having to create emails from scratch each time. You can also include an email link in HTML to automatically compose emails in your email client.

In this step-by-step guide, you'll learn how to build an HTML email template, add a CSS email design to it, and send it to your target audience.

Setting Up Your Template Directory and Jinja2

Follow the steps below to set up your HTML email template directory and Jinja2 for Python email automation:

• Create a Template Directory: To hold your HTML email templates, you will need to set up a template directory inside your project module. Let's name this directory - html_emailtemp.

• Install Jinja2: Jinja is a popular templating engine for Python that developers use to create configuration files, HTML documents, etc. Jinja2 is its latest version. It lets you create dynamic content via loops, blocks, variables, etc. It's used in various Python projects, like building websites and microservices, automating emails with Python, and more.

  Use this command to install Jinja2 on your computer:

  pip install jinja2

Creating an HTML Email Template

To create an HTML email template, let's understand how to code your email step by step. If you want to modify your templates, you can do it easily by following the steps below:

Step 1: Structure HTML

A basic email will have a proper structure - a header, a body, and a footer.

• Header: Used for branding purposes (in emails, at least)
• Body: It will house the main text or content of the email
• Footer: It's at the end of the email if you want to add more links, information, or call-to-actions (CTA)

Begin by creating your HTML structure, keeping it simple since email clients are less compatible than web browsers. For example, using tables is preferable for custom email layouts.

Here's how you can create a basic HTML mail with a defined structure:

<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>HTML Email Template</title> <style type="text/css"> /* Add your CSS here */ </style> </head> <body> <table width="100%" cellpadding="0" cellspacing="0"> <tr> <td align="center"> <table width="600" cellpadding="0" cellspacing="0"> <!-- Header --> <tr> <td style="background-color: #1c3f60; color: #ffffff; text-align: center; padding: 20px;"> <h1>Your order is confirmed</h1> </td> </tr> <!-- Body --> <tr> <td style="padding: 20px; font-size: 16px; line-height: 1.6; color:#ffffff;"> <p>The estimated delivery date is 22nd August 2024.</p> </td> </tr> <!-- Footer --> <tr> <td style="background-color: #ff6100; color: #000000; text-align: center; padding: 20px;"> <p>For additional help, contact us at support@domain.com</p> </td> </tr> </table> </td> </tr> </table> </body> </html>

Explanation:

• <!DOCTYPE html>: This declares HTML as your document type.
• <html>: This is an HTML page's root element.
• <head>: This stores the document's metadata, like CSS styles.
• <style>: CSS styles are defined here.
• <body>: This stores your email's main content.
• <table>: This tag defines the email layout, giving it a tabular structure with cells and rows, which makes rendering easier for email clients.
• <tr>: This tag defines the table's row, allowing vertical content stacking.
• <td>: This tag is used to define a cell inside a row. It contains content like images, text, buttons, etc.

Step 2: Structure Your Email

Now, let's create the structure of your HTML email. To ensure it's compatible with different email clients, use tables to generate a custom email layout, instead of CSS.

<table width="100%" cellpadding="0" cellspacing="0"> <tr> <td align="center"> <table width="600" cellpadding="0" cellspacing="0" style="border: 1px solid #1c3f60; padding: 20px;"> <tr> <td align="center"> <h1 style="color: #7ed957;">Hi, Jon!</h1> <p style="font-size: 16px; color: #ffde59;">Thank you for being our valuable customer!</p> </td> </tr> </table> </td> </tr> </table> Styling the Email with CSS

Once you've defined your email structure, let's start designing emails with HTML and CSS:

Inline CSS

Use inline CSS to ensure different email clients render CSS accurately and preserve the intended aesthetics of your email style.

<p style="font-size: 16px; color: blue;">Styled paragraph.</p> Adjusting Style

Users might use different devices and screen sizes to view your email. Therefore, it's necessary to adapt the style to suit various screen sizes. In this case, we'll use media queries to achieve this goal and facilitate responsive email design.

<style type="text/css"> @media screen and (max-width: 600px) { .container { width: 100% !important; padding: 10px !important; } } </style> <table class="container" width="600"> <!-- Content --> </table>

Explanation:

• @media screen and (max-width: 600px) {....}: This is a media query that targets device screens of up to 600 pixels, ensuring the style applies only to these devices, such as tablets and smartphones.
• width: 100% !important;: This style changes the width of the table - .container. The code instructs that the table width be set to full screen, not 600px.
• !important: This rule overrides other styles that may conflict with it.
• padding: 10px !important;: Inside the .container table, a padding of 10px is added to the table.

Adding CTA Button and Links

Here, we are adding a call to action (CTA) link at the button - "Get a 30-day free trial" that points to this page - https://www.mydomain.com.

<table cellpadding="0" cellspacing="0" style="margin: auto;"> <tr> <td align="center" style="background-color: #8c52ff; padding: 10px 20px; border-radius: 5px;"> <a href="https://www.mydomain.com" target="_blank" style="color: #ffffff; text-decoration: none; font-weight: bold;">Get a 30-day free trial</a> </td> </tr> </table>

Let's Now Look at the Complete HTML Email Template:

<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>HTML Email Template</title> <style type="text/css"> /* Adding the CSS */ body { margin: 0; padding: 0; background-color: #f4f4f4; font-family: Arial, sans-serif; } table { border-collapse: collapse; } .mailcontainer { width: 100%; max-width: 600px; margin: auto; background-color: #ffffff; } .header { background-color: #1c3f60; color: #ffffff; text-align: center; padding: 20px; } .body { padding: 20px; font-size: 16px; line-height: 1.6; background-color: #1c3f60; color: #7ed957; } .footer { background-color: #ff6100; color: #000000; text-align: center; padding: 20px; } .cta { background-color: #8c52ff; padding: 10px 20px; border-radius: 5px; color: #ffffff; text-decoration: none; font-weight: bold; } @media screen and (max-width: 600px) { .container { width: 100% !important; padding: 10px !important; } } </style> </head> <body> <table width="100%" cellpadding="0" cellspacing="0"> <tr> <td align="center"> <table class="container" width="600" cellpadding="0" cellspacing="0"> <!-- Header --> <tr> <td class="header"> <h1>Your order is confirmed</h1> </td> </tr> <!-- Body --> <tr> <td class="body"> <p>The estimated delivery date is 22nd August 2024.</p> <p style="font-size: 16px; color: blue;">Styled paragraph.</p> <table width="100%" cellpadding="0" cellspacing="0" style="border: 1px solid #1c3f60; padding: 20px;"> <tr> <td align="center"> <h1 style="color: #7ed957;">Hi, Jon!</h1> <p style="font-size: 16px; color: #ffde59;">Thank you for being our valuable customer!</p> </td> </tr> </table> <table cellpadding="0" cellspacing="0" style="margin: auto;"> <tr> <td align="center" style="background-color: #8c52ff; padding: 10px 20px; border-radius: 5px;"> <a href="https://www.mydomain.com" target="_blank" rel="noopener noreferrer" style="color: #ffffff; text-decoration: none; font-weight: bold;">Get a 30-day free trial</a> </td> </tr> </table> </td> </tr> <!-- Footer --> <tr> <td style="background-color: #ff6100; color: #000000; text-align: center; padding: 20px;"> <p>For additional help, contact us at support@domain.com</p> </td> </tr> </table> </td> </tr> </table> </body> </html>

Explanation:

• .mailcontainer: This is a class that you can use to style your email content's main section. It's given a set width, margin, border, and color.
• .header, .footer, .body: These are classes used to style your email's header, footer, and body, respectively.
• .cta: This class allows you to style your buttons, such as CTA buttons, with a specified color, border design, padding, etc.

Bringing Everything Together With Jinja2

Having created our HTML template, it's now time to bring everything together using the Jinja2 templating engine.

Import Project Modules

You've already set up your template directory - html_emailtemp. Now you can find and render templates using code. But before you do that, import the relevant project modules using the code below:

from jinja2 import Environment, PackageLoader, select_autoescape env = Environment(loader=PackageLoader('email_project', 'html_emailtemp'), autoescape=select_autoescape(['html', 'xml']))

Explanation:

• Environment: Jinja2 utilizes a central object, the template Environment. Its instances store global objects and configurations, and load your email templates from a file.

• PackageLoader: This configures Jinja2 to load email templates.

• autoescape: To mitigate security threats such as cross-site scripting (XSS) attacks and protect your code, you can escape values (that are passed to the email template) while rendering HTML using the command autoescape. Or, you can validate user inputs to reject malicious code.

  For security, autoescape is set to True to enable escaping values. If you turn it to False, Jinja2 won't be able to escape values, and XSS attacks may occur. To enable autoescape, set autoescape to True:

  env = Environment(loader=PackageLoader("myapp"), autoescape=True)

Load Your Template

Once done, a template environment will be created with a template loader to find email templates created inside your project module's template folder.

Next, load your HTML email template using the method - get_template(). This function will return your loaded template. It also offers several benefits such as enabling email template inheritance, so you can reuse the template in multiple scenarios.

template1 = env.get_template("myemailtemplate.html")

Render the Template

To render your email template, use the method - render()

html1 = template1.render()

As these HTML email templates are dynamic, you can pass keyworded arguments (kwargs) with Jinja2 to the render function. The kwargs will then be passed to your email template. Here's how you can render your templates using the destined user's name - "Jon Doe" - in your email.

html1 = template1.render(name="Jon Doe")

Let's look at the complete code for this section:

from jinja2 import Environment, PackageLoader, select_autoescape env = Environment(loader=PackageLoader("email_project", "html_emailtemp"), autoescape=select_autoescape(["html", "xml"])) template1 = env.get_template("myemailtemplate.html") html1 = template1.render() Sending the Email

To send an email, you can use the application-level, straightforward protocol - Simple Mail Transfer Protocol (SMTP). This protocol streamlines the email sending process and determines how to format, send, and encrypt your emails between the source and destination mail servers.

In this instance, we'll send emails in Python via SMTP since Python offers a built-in module for email sending. To send emails, Python provides a library, 'smtplib', to facilitate effortless interaction with the SMTP protocol.

To get started:

Install 'smtplib': Ensure you have installed Python on your system. Now, import 'smtplib' to set up connectivity with the mail server.

import smtplib

Define your HTML parameter: Define your HTML parameter for the mail object where you'll keep your HTML template. It will instruct email clients to render the template.

Here's the full code for this section:

import smtplib from email.mime.text import MIMEText # MIMEText is a class from the email package from jinja2 import Template # Let's use Template class for our HTML template sender = "<a href='mailto:sender1@gmail.com' target='_blank' rel='noopener noreferrer'>sender1@gmail.com</a>" recipient = "<a href='mailto:recipient1@gmail.com' target='_blank' rel='noopener noreferrer'>recipient1@gmail.com</a>" subject = "Your order is confirmed!" with open('myemailtemplate.html', 'r') as f: template1 = Template(f.read()) # Enter the HTML template html_emailtemp = """ <!DOCTYPE html> <html lang='en'> <head> <meta charset='UTF-8'> <meta name='viewport' content='width=device-width, initial-scale=1'> <title>HTML Email Template</title> <style type='text/css'> # Adding the CSS body { margin: 0; padding: 0; background-color: #f4f4f4; font-family: Arial, sans-serif; } table { border-collapse: collapse; } .mailcontainer { width: 100%; max-width: 600px; margin: auto; background-color: #ffffff; } .header { background-color: #1c3f60; color: #ffffff; text-align: center; padding: 20px; } .body { padding: 20px; font-size: 16px; line-height: 1.6; background-color: #1c3f60; color: #7ed957; } .footer { background-color: #ff6100; color: #000000; text-align: center; padding: 20px; } .cta { background-color: #8c52ff; padding: 10px 20px; border-radius: 5px; color: #ffffff; text-decoration: none; font-weight: bold; } @media screen and (max-width: 600px) { .container { width: 100% !important; padding: 10px !important; } } </style> </head> <body> <table width='100%' cellpadding='0' cellspacing='0'> <tr> <td align='center'> <table class='container' width='600' cellpadding='0' cellspacing='0'> <!-- Header --> <tr> <td class='header'> <h1>Your order is confirmed</h1> </td> </tr> <!-- Body --> <tr> <td class='body'> <p>The estimated delivery date is 22nd August 2024.</p> <p style='font-size: 16px; color: blue;'>Styled paragraph.</p> <table width='100%' cellpadding='0' cellspacing='0' style='border: 1px solid #1c3f60; padding: 20px;'> <tr> <td align='center'> <h1 style='color: #7ed957;'>Hi, Jane!</h1> <p style='font-size: 16px; color: #ffde59;'> Thank you for being our valuable customer! </p> </td> </tr> </table> <table cellpadding='0' cellspacing='0' style='margin: auto;'> <tr> <td align='center' style='background-color: #8c52ff; padding: 10px 20px; border-radius: 5px;'> <a href='https://www.mydomain.com' target='_blank' rel='noopener noreferrer' style='color: #ffffff; text-decoration: none; font-weight: bold;'>Get a 30-day free trial</a> </td> </tr> </table> </td> </tr> <!-- Footer --> <tr> <td style='background-color: #ff6100; color: #000000; text-align: center; padding: 20px;'> <p>For additional help, contact us at support@domain.com</p> </td> </tr> </table> </td> </tr> </table> </body> </html> """ template1 = Template(html_emailtemp) html1 = template1.render(name="Jon Doe") # Attach your MIMEText objects for HTML message = MIMEText(html1, 'html') message['Subject'] = subject message['From'] = sender message['To'] = recipient # Send the HTML email with smtplib.SMTP_SSL('smtp.gmail.com', 465) as server: server.login(username, password) server.sendmail(sender, recipient, message.as_string())

Explanation:

• sender: The sender's email address
• recipient: The recipient's email address
• from email.mime.text import MIMEText: This is used to import the class MIMEText, enabling you to attach your HTML template in the email.
• smtplib.SMTP_SSL('smtp.gmail.com', 465) as server:: This establishes a connection with your email provider's (Gmail's) SMTP server using port 465. If you are using another SMTP provider, use their domain name, such as smtp.domain.com, with an appropriate port number. The connection is secured with SSL.
• server.login(username, password): This function allows you to log in to the email server using your username and password.
• server.sendemail(sender, recipient, message.as_string()): This command sends the HTML email.

Testing

Before sending your HTML email, test it to understand how different email clients render CSS and HTML. Testing tools like Email on Acid, Litmus, etc. can assist you.

Conclusion

To build custom email templates with HTML and CSS in Python, follow the above instructions. First, begin structuring your HTML email template, style emails with CSS, and then send them to your recipients. Always check your email template's compatibility with different email clients and ensure to keep your HTML simple using tables. Adding an email link in HTML will also allow you to compose an email automatically in your email client and send it to a specific email address.

Drupal CMS is the official name for Drupal Starshot. We officially have the Drupal CMS project on Drupal.org, where the previous prototype has been converted into the official codebase. This monolithic repository will contain the Composer project and packages (recipes) that makeup Drupal CMS. 

A while ago a colleague of mine asked about our crash infrastructure in Plasma and whether I could give some overview on it. This seems very useful to others as well, I thought. Here I am, telling you all about it!

Our crash infrastructure is comprised of a number of different components.

• KCrash: a KDE Framework performing crash interception and prepartion for handover to…
• coredumpd: a systemd component performing process core collection and handover to…
• DrKonqi: a GUI for crashes sending data to…
• Sentry: a web service and UI for tracing and presenting crashes for developers

We will look at them in turn. This post introduces KCrash.

KCrash

KCrash, as the name suggests, is our KDE framework for crash handling. While it is a mid-tier framework and could be used by outside projects, it mostly doesn’t make sense to, because some behavior is very KDE-specific.

It installs POSIX signal handlers to intercept crash signals and then prepares the crashed process for handover to coredumpd and DrKonqi. More on these two in another post. Once prepared it sends the crash signal into the next higher level crash handler until the signal eventually reaches the default handler and cause the kernel to invoke the core pattern.

Before that can happen, a bunch of work needs doing inside KCrash. Most of it quite boring, but also somewhat challenging.

You see, when handling a signal you need to only use signal-safe functions. The manpage explains very well why. This proves quite challenging at the level we usually are at (i.e. Qt) because it is entirely unclear what is and isn’t ultimately signal-safe under the hood. Additionally, since we are dealing with crash scenarios, we must not trigger new memory allocation, because the heap management may have had an accident.

To that end, KCrash has to use fairy low-level API. To make that easier to work with, there are actually two parts to KCrash:

• The Initialization Stage
• The Crash Stage

The Initialization Stage

Initialization is generally triggered by calling KCrash::initialize. You may already wonder what kind of initialization KCrash could possibly need. Well, the obvious one is setting up the signal handling. But beyond that the init stage is also used to prepare us for the crash stage. I’ve already mentioned the serious constraints we will encounter once the signal hits, so we had best be prepared for that. In particular we’ll do as much of the work as possible during initialization. This most important includes copying QString content into pre-allocated char * instances such that we later only need to read existing memory. The second most important aspect is the metadata file preparation for use in…

The Crash Stage

Once initialization has happened, we are ready for crashes. Ideally the application doesn’t crash, of course. 😉

But if it does the biggest task is rescuing our data!

Metadata

Inside KCrash we have the concept of Metadata: everything we know about the crashed application: the signal, process ID, executable, used graphics device… and so on and so forth. All this data is collected into a metadata file on-disk in ~/.cache/kcrash-metadata at the time of crash.

Here’s an example file:

[KCrash] exe=/usr/bin/kwin_wayland glrenderer= platform=wayland appname=kwin_wayland apppath=/usr/bin signal=11 pid=1353 appversion=6.1.80 programname=KWin bugaddress=submit@bugs.kde.org

The actual fields vary depending on what is available for any given application, but it’s generally more or less what is shown in the example.

This metadata file will later be consumed by DrKonqi in an effort to obtain information that only existed at runtime inside the application - such as the version that was running, or whether it was running in legacy X11 mode.

Handoff

Once the metadata is safely saved to disk, KCrash simply calls raise(). This re-raises the signal into the default handler, and through that causes a core dump.

What happens next is up to the system configuration as per the core manpage.

The recommended setup for distributions is that a crash handler be configured as core_pattern and that this handler consumes the crash. We recommend an implementation of the coredumpd and journald interfaces as that will then allow our crash handler to come in and log the crash with KDE.

So that was KCrash, the first in our four-step crash-handling pipeline. In the next blog post I’ll tell you all about the next one: coredumpd.

This course uses three problems often covered in introductory astro-physics courses to play in Python. Along the way you’ll learn some astronomy, and how to use a variety of datascience libraries like NumPy, Matplotlib, pandas, and pint.

In this video course you’ll learn about:

• Introductory astrophysics topics
• Working with dataframes in pandas
• Writing code that uses scientific units
• Visualizing information with Matplotlib

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

As a Drupal 7 user, you might feel the pressure of the impending end-of-life (EOL) announcement. While Drupal 7 has served us well over the years, preparing for the future is essential. However, The Drupal Association has partners who offer D7 Extended Security Support.  you can confidently extend the life of your Drupal 7 site while strategically planning your migration to newer versions of Drupal. Find a partner here.

The Importance of Planning Your Migration

Migrating from Drupal 7 is crucial for staying current with new features, security updates, and performance improvements. There are significant advantages to moving to newer versions of Drupal:

• Enhanced Security: Newer versions of Drupal come with advanced security measures that protect your site from emerging threats.

• Modern Features: Newer Drupal versions introduce features that streamline content management, enhance user experience, and improve site performance.

• Better Performance: Newer versions are optimized for speed and efficiency, providing a better experience for your users.

• Community Support: With most of the Drupal community moving forward, staying on an outdated version might limit your access to community resources and modules.

To find a Drupal 7 Migration Partner to lead your migration, reach out to our Certified Migrations Partners that offer services in these categories.

• Enterprise Grade D7 Migrations

• Mid-Scale D7 Migrations

• Freelancers and small shops for smaller D7 Migrations

• D7 Extended Security Support

by por Andrew Gonçalves

Debian Day in Santa Maria - RS 2024 was held after a 5-year hiatus from the previous version of the event. It took place on the morning of August 16, in the Blue Hall of the Franciscan University (UFN) with support from the Debian community and the Computing Practices Laboratory of UFN.

The event was attended by students from all semesters of the Computer Science, Digital Games and Informational Systems, where we had the opportunity to talk to the participants.

Around 60 students attended a lecture introducing them to Free and Open Source Software, Linux and were introduced to the Debian project, both about the philosophy of the project and how it works in practice and the opportunities that have opened up for participants by being part of Debian.

After the talk, a packaging demonstration was given by local DD Francisco Vilmar, who demonstrated in practice how software packaging works in Debian.

I would like to thank all the people who helped us:

• Debian Project
• Professor Ana Paula Canal (UFN)
• Professor Sylvio André Garcia (UFN)
• Laboratory of Computing Practices
• Francisco Vilmar (local DD)

And thanks to all the participants who attended this event asking intriguing questions and taking an interest in the world of Free Software.

Photos:

por Andrew Gonçalves

O Debian Day em Santa Maria - RS 2024 foi realizado após 5 anos de hiato, foi feito durante a manhã do dia 16/08/2024 no Salão Azul da Universidade Franciscana (UFN) com apoio da comunidade Debian e do Laboratório de Práticas da Computação da UFN.

O evento contou com alunos de todos os semestres dos cursos de Ciência da Computação, Jogos Digitais e Sistemas de Informação, fizemos um coffee break onde tivemos a oportunidade de conversar com os participantes.

Cerca de 60 alunos prestigiaram uma palestra de introdução ao Software Livre e de Código Aberto, Linux e foram introduzidos ao projeto Debian, tanto sobre a filosofia do projeto, até como ele acontece na prática e oportunidades que se abriram para participantes do projeto por fazerem parte do Debian.

Após a palestra foi feita uma demonstração de empacotamento pelo DD local Francisco Vilmar, que demonstrou na prática como funciona o empacotamento de software no Debian.

Gostaria de agradecer a todas as pessoas que nos ajudaram:

• Projeto Debian
• Professora Ana Paula Canal (UFN)
• Professor Sylvio André Garcia
• Laboratório de Práticas da Computação
• Francisco Vilmar (DD local)

E um muito obrigado a todos os participantes que nos prestigiaram neste evento fazendo perguntas intrigantes e se interessando pelo mundo do Software Livre.

Algumas fotos:

Faceted search offers users with a superior search experience by displaying filters against their search results. It is particularly useful for websites having large catalogues and listings. Once the user types in their search query, they will be presented with a list of relevant filter options to further narrow down their search. These filtering elements are facets.Previously Facet API in Drupal 9, the Facet module in Drupal 10 enables your website with faceted searching abilities. Let’s look at configuring and implementing Faceted search with Drupal’s own search server, Search API. What is Faceted Search? If your users are finding it hard to see what they are searching for even after keying in their search query, they are bound to get frustrated. Faceted search provides users with multiple filters at the same time for the various attributes of the content. The facets provided are based on the search query the user has executed. Facets will also display the number of matched results (usually within brackets) next to it. Let’s take a look at this below screenshot to understand Facets better. In one of our recent Drupal projects, a quick search for Homes in Columbia on this website presents to you with facets like Communities, Hot Deals, Quick Move-ins and more. You will also see the count of the results next to each facet. So, a query with “Columbia” keyword is sent to the search server to retrieve the already configured and indexed categories (Communities, Hot Deals, etc.) Installing the Facets Module for Drupal 10 As previously discussed, we will be implementing Faceted search using Drupal’s Search API module.  Step 1: Enabling the modules Install and enable these modules •    The Facet Module •    Search API module Step 2: Creating Content Types Create the content you would like to include in the faceted search by adding Content types as shown below. You can also use the default content types provided by Drupal. Step 3: Configuring the Search server Navigate to Configuration -> Search and metadata -> Search-API from the admin interface to configure your search server. Give a name to your search server (here - data server).  Step 4: Configuring the Search Index Next, configure the search index to improve the search performance. Navigate to Configuration -> Search and metadata -> Search-API -> Index -> data_index. Give a name to your index and then select Content as your Datasources since we will be indexing the Content entities here. You can then move on to the next section - Configuring the Datasource (here – Content). Here you can choose to select all the bundles or only select a few from the below list to index. Next, select your server that you had already created (here - data server). Select the “Index items immediately” option to begin the indexing process. Click on Save. Step 5: Adding Fields for Indexing Next, we need to add Fields to be indexed. Navigate to Configuration -> Search and metadata ->Search API -> data index and select the Fields tab. Click on the Add fields button to create fields according to your requirement. Step 6: Indexing the Content Under the same location, click on the View tab to start the process of indexing your content. In the Start Indexing Now section, click on the Index Now button. It will then show you a progress bar with the status of the number of items that have been indexed. Step 7: Creating a View Now we will be creating a view for the data that needs to be indexed and displayed to your users. Navigate to Structure -> Views -> Add View. Give a name for the View.Under View Settings dropdown list, select the index that you have created in Step 4.Create a page for your search results by clicking on the Create a page checkbox under the Page Settings tab. Give a name and a path for the same. Under Items to Display, select 0 if you want to display all the results in one page. Else, select a number of results to be displayed. Under Page Display settings, you can select the format in which you want to display your results – Table, Grid, HTML list or Unformatted list. We have selected Unformatted list here. Click on Save. Step 8: Adding Fields to the View Here we will be adding fields that we have indexed earlier to the View.Go to Views, click on Add button next to the Fields section. Select the Fields, click on Add and Configure. Under Render Settings, select the Link to Content checkbox so that the results displayed are clickable.Click Save. Step 9: Configuring the Facets Now let’s begin configuring and enabling the facets. Navigate to Configuration -> Search and meta data -> FacetsClick on the Add Facet button. Select the Facet Source – This will be your View that you created previously.Select the Field – This will display the fields you had added for indexing in Step 5.Give a name to the Facet.Click on Save. Next, you will then see more configuration options for displaying the facets (as shown in the below image). Widgets will list out a number of options like List of links, array, dropdown, etc. You can choose what suits your website the best.Select the “Transform entity ID to label” to avoid displaying the machine name of the content type.Click on Save. Step 10: Placing the Facet blocks in the chosen page regions  Next, place the Facets you created as blocks in a page region of your choice. Navigate to Structure -> Block Layout. Select the region of the page where you would like to place the block containing the Facets.Here, we are selecting Sidebar. Click on the Place Block button next to the Sidebar.In the next dialog box, search for the Facet name and click on Place Block. In the Configure Block section, mention the Search page path that you had previously created. Here -“site-search” is our page we had created. Give a display name for your Block and select the Display title checkbox if you want the block name to be displayed (here – Type).Click on Save Block. The Result And just like that, your faceted search page and functionality is ready! Notice the Facet called Type (display name) that has Basic page and Article listed as content types to filter against.(Note: Since there is no content available under ‘Basic Page’, it will not show up in the filter).   Select Type as ‘Person’   Select Type as ‘Article’ Final Thoughts Implementing faceted search provides users with an easy and fast search experience. In this tutorial, we have used Search API as the search engine. You could also use search servers like Apache Solr, Search API Solr search or Elasticsearch with Drupal 8, whatever suits your requirements. As a leading Drupal development company, Specbee’s Drupal experts can help you build exemplary search experiences. Contact us to know more.

As part of The DropTimes&#039; &quot;Meet the Candidate&quot; campaign for the ongoing Drupal Association Board Elections, Kevin Quillen, Practice Lead at Velir, shares his vision for making Drupal more accessible and appealing worldwide. With over 16 years of experience in the community, Kevin discusses his plans to modernize Drupal.org, attract new developers, and enhance the platform&#039;s global impact. Voting runs until 5 September.

<strong>Topics covered in this episode:</strong><br> <ul> <li><strong><a href="https://github.com/ZeroIntensity/pyawaitable?featured_on=pythonbytes">pyawaitable</a></strong></li> <li><strong><a href="https://nrennie.rbind.io/blog/plotnine-annotated-area-chart/?featured_on=pythonbytes">Annotated area charts with plotnine</a></strong></li> <li><strong><a href="https://github.com/uname-n/deltabase?featured_on=pythonbytes">DeltaDB</a></strong></li> <li><strong><a href="https://pycon.blogspot.com/2024/08/pycon-us-2024-recap-and-recording.html?featured_on=pythonbytes">PyCon US 2024 Recap + Videos are up</a></strong></li> <li><strong>Extras</strong></li> <li><strong>Joke</strong></li> </ul><a href='https://www.youtube.com/watch?v=67F3kv6jCEU' style='font-weight: bold;'data-umami-event="Livestream-Past" data-umami-event-episode="397">Watch on YouTube</a><br> <p><strong>About the show</strong></p> <p>Sponsored by us! Support our work through:</p> <ul> <li>Our <a href="https://training.talkpython.fm/?featured_on=pythonbytes"><strong>courses at Talk Python Training</strong></a></li> <li><a href="https://courses.pythontest.com/p/the-complete-pytest-course?featured_on=pythonbytes"><strong>The Complete pytest Course</strong></a></li> <li><a href="https://www.patreon.com/pythonbytes"><strong>Patreon Supporters</strong></a></li> </ul> <p><strong>Connect with the hosts</strong></p> <ul> <li>Michael: <a href="https://fosstodon.org/@mkennedy"><strong>@mkennedy@fosstodon.org</strong></a></li> <li>Brian: <a href="https://fosstodon.org/@brianokken"><strong>@brianokken@fosstodon.org</strong></a></li> <li>Show: <a href="https://fosstodon.org/@pythonbytes"><strong>@pythonbytes@fosstodon.org</strong></a></li> </ul> <p>Join us on YouTube at <a href="https://pythonbytes.fm/stream/live"><strong>pythonbytes.fm/live</strong></a> to be part of the audience. Usually Tuesdays at 10am PT. Older video versions available there too.</p> <p>Finally, if you want an artisanal, hand-crafted digest of every week of the show notes in email form? Add your name and email to <a href="https://pythonbytes.fm/friends-of-the-show">our friends of the show list</a>, we'll never share it.</p> <p><strong>Michael #1:</strong> <a href="https://github.com/ZeroIntensity/pyawaitable?featured_on=pythonbytes">pyawaitable</a></p> <ul> <li>CPython API for asynchronous functions.</li> <li>by Peter Bierma</li> <li>It was originally designed to be directly part of CPython - you can read the <a href="https://gist.github.com/ZeroIntensity/8d32e94b243529c7e1c27349e972d926?featured_on=pythonbytes">scrapped PEP</a> about it. </li> <li>Since this library only uses the public ABI, it's better fit outside of CPython, as a library.</li> </ul> <p><strong>Brian #2:</strong> <a href="https://nrennie.rbind.io/blog/plotnine-annotated-area-chart/?featured_on=pythonbytes">Annotated area charts with plotnine</a></p> <ul> <li>Nicola Rennie</li> <li>This is a marvelous, very professional looking plot, and a tutorial for how to achieve it.</li> <li>Uses <a href="https://plotnine.org?featured_on=pythonbytes">plotline</a>, which is “.. an implementation of a <em>grammar of graphics</em> in Python based on ggplot2” <ul> <li>I actually didn’t know the gg in ggplot came from “grammar of graphics”. TIL</li> </ul></li> </ul> <p><strong>Michael #3:</strong> <a href="https://github.com/uname-n/deltabase?featured_on=pythonbytes">DeltaDB</a></p> <ul> <li>A lightweight, comprehensive solution for managing delta tables built on polars and deltalake.</li> <li><a href="https://github.com/delta-io/delta-rs?featured_on=pythonbytes">Deltalake</a>: Delta Lake is an open-source storage format that runs on top of existing data lakes. </li> <li><a href="https://github.com/pola-rs/polars?featured_on=pythonbytes">Polars</a>: Dataframes powered by a multithreaded, vectorized query engine, written in Rust (aka fluent, rust-based pandas)</li> <li>See <a href="https://uname-n.github.io/deltabase/?featured_on=pythonbytes">the docs</a>.</li> </ul> <p><strong>Brian #4:</strong> <a href="https://pycon.blogspot.com/2024/08/pycon-us-2024-recap-and-recording.html?featured_on=pythonbytes">PyCon US 2024 Recap + Videos are up</a></p> <ul> <li>95 countries attended</li> <li>total attendance of 2,991 <ul> <li>2,551 in person</li> <li>440 remote</li> </ul></li> <li>Videos available <a href="https://www.youtube.com/@PyConUS">PyConUS</a> <ul> <li>I recommend <a href="https://www.youtube.com/playlist?list=PL2Uw4_HvXqvYhjub9bw4uDAmNtprgAvlJ">Playlist → 2024 → view full playlist</a>, as it’s easier to see the talk titles.</li> <li>I’ve got Paul Gannsle’s pytest for unittesters and Amitosh Swain’s Testing Data Pipelines queued up</li> </ul></li> </ul> <p><strong>Extras</strong> </p> <p>Brian:</p> <ul> <li><a href="https://courses.pythontest.com/hello-pytest?featured_on=pythonbytes">Hello, pytest!</a> course available as of last Friday. <ul> <li>Now the fastest way to get started using pytest. </li> <li>16 lessons (really 12 + intro, outro, code download, pytest flag cheat sheet)</li> <li>The whole shebang is about 90 min. (faster if you bump up the video speed. :)</li> </ul></li> </ul> <p>Michael:</p> <ul> <li>Cutting back on digital distractions, trying <a href="https://apps.apple.com/us/app/dumb-phone/id6504743503?featured_on=pythonbytes">Dumb Phone</a> for iPhone. <ul> <li>See <a href="https://python-bytes-static.nyc3.digitaloceanspaces.com/dumb-phone-experiment.jpeg?featured_on=pythonbytes">screenshot</a></li> </ul></li> <li><a href="https://www.codeinacastle.com/python-zero-to-hero-2024?featured_on=pythonbytes">Code in a Castle Event</a></li> </ul> <p><strong>Joke:</strong> <a href="https://www.talisman.org/tao/?featured_on=pythonbytes">The Tao of Programming: 4.3</a></p> <p>A master was explaining the nature of Tao of to one of his novices, "The Tao is embodied in all software -- regardless of how insignificant," said the master.</p> <p>"Is the Tao in a hand-held calculator?" asked the novice.</p> <p>"It is," came the reply.</p> <p>"Is the Tao in a video game?" continued the novice.</p> <p>"It is even in a video game," said the master.</p> <p>"And is the Tao in the DOS for a personal computer?"</p> <p>The master coughed and shifted his position slightly. "The lesson is over for today," he said.</p>

Simon Morvan discusses EvolvingWeb, Drupal development, and CKEditor 5 migration at DrupalCon Portland 2024.

Release 0.6.37 of the digest package arrived at CRAN today and has also been uploaded to Debian.

digest creates hash digests of arbitrary R objects. It can use a number different hashing algorithms (md5, sha-1, sha-256, sha-512, crc32, xxhash32, xxhash64, murmur32, spookyhash, blake3,crc32c, xxh3_64 and xxh3_128), and enables easy comparison of (potentially large and nested) R language objects as it relies on the native serialization in R. It is a mature and widely-used package (with 70.8 million downloads just on the partial cloud mirrors of CRAN which keep logs) as many tasks may involve caching of objects for which it provides convenient general-purpose hash key generation to quickly identify the various objects.

This release updates one of the different hashing source functions which, to remain close to their upstream, used Free() and Calloc() (uppercased to use the R allocator) but not the prefixed stricter versions R_Free() and R_Calloc(). R will switch to enforcing these in the next release next year. Kevin had noticed (while doing some other testing) that this now fails under R-devel (with a switch set), and prepares a very nice and clean PR to take care of it. As of today, CRAN is now sending ‘please fix, or else …’ notes so it was a good time to send this to CRAN. We also updated some remaining http URLs in the README.md to https, and switched to Author/Maintainer field to the now also mandatory Authors@R.

My CRANberries provides a summary of changes to the previous version. For questions or comments use the issue tracker off the GitHub repo. For documentation (including the changelog) see the documentation site.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

This article looks at how to limit the content types in a Django model.

In the KDE world, famously there was that weekend where DCOP (Desktop Communicating Objects Protocol) was created, setting the stage for things like KParts. GNOME picked the CORBA object model, and much later the Free Desktop world settled on DBus as a message-passing API. But even at the time, there were other message-passing APIs. At work-work I use one, called SIMPL, which is kind of shout-out to the late ’90s of DCOP.

Please note that my presentation of “history” is just what I remember now of events that were already “tales told ‘round the campfire” 15 years ago. Corrections welcome (by email).

At work-work SIMPL is just a given, and it’s got wrappers and abstractions so that there’s a decent C++ style API around it. At its heart it is a point-to-point message-passing API with no policy at all about the payload of messages. That is both a blessing – no policy means it can be used for whatever kind of messages you think is necessary – and a curse – no policy means that you end up writing a bunch of abstractions to represent the messages that you actually use.

Code at the wrapped-in-C++ level looks something like this (effectively obscuring the underlying transport):

auto reply = CoordinatorTask().Send(requests::GetCurrentUser()); if(reply.has_value()) { do_something(reply.value().username); ...

There does not seem to be much online about SIMPL anymore. There is a LinuxDevices article from 2000 when SIMPL was under some active development as an Open Source project, there’s a LWN comment from 2015 that mentions it still, and there’s a Wikipedia article on it, as a historical note. Note that the Website link on wikipedia goes to something that is now a spam domain. I can’t quickly find sources anymore.

I do wonder at the chances of history that made desktop developers at the time entirely miss out on this existing message-passing mechanism – perhaps the SIMPL authors were too focused on industrial automation and not visible in the X11 desktop space, and not visible on SunOS and other platforms where a fair bit of KDE development happened at the time.

One thing that the SIMPL library developers emphasize repeatedly in documentation (there’s a book) is a philosophy of doing one thing and doing it well. So serialization and message payloads are not part of this – no policy. So security and access control are not part of his – no policy. Those concerns are things that can go into a different API layer on top of SIMPL.

Something else that the SIMPL authors emphasize is doing one thing and doing it well. That applies to the applications that use SIMPL, in particular. And that translates, in the depths of the library, to being able to register only one name for an application (for purposes of discovering what other applications there are, each application has a unique name in the system). It translates into a lack of thread safety, due to the use of global state. It translates into a contextless API, so all an application can do is call “give me the next incoming message”. There’s no concept of an event-loop, and no obvious mechanisms for integrating SIMPL message-handling into another event loop (like an X11 loop).

For me personally this means that I need to re-tool my brain during the trip between home and work, switching of Qt wrappers around DBus to work-work wrappers around SIMPL and the subtleties of each. In practice, that means mostly cursing QtDBus (because of the relative amount of time I put into both).

(The issues described in this post have been fixed, I have not exhaustively researched whether any other issues exist)

Feeld is a dating app aimed largely at alternative relationship communities (think "classier Fetlife" for the most part), so unsurprisingly it's fairly popular in San Francisco. Their website makes the claim:

Can people see what or who I'm looking for?
No. You're the only person who can see which genders or sexualities you're looking for. Your curiosity and privacy are always protected.

which is based on you being able to restrict searches to people of specific genders, sexualities, or relationship situations. This sort of claim is one of those things that just sits in the back of my head worrying me, so I checked it out.

First step was to grab a copy of the Android APK (there are multiple sites that scrape them from the Play Store) and run it through apk-mitm - Android apps by default don't trust any additional certificates in the device certificate store, and also frequently implement certificate pinning. apk-mitm pulls apart the apk, looks for known http libraries, disables pinning, and sets the appropriate manifest options for the app to trust additional certificates. Then I set up mitmproxy, installed the cert on a test phone, and installed the app. Now I was ready to start.

What became immediately clear was that the app was using graphql to query. What was a little more surprising is that it appears to have been implemented such that there's no server state - when browsing profiles, the client requests a batch of profiles along with a list of profiles that the client has already seen. This has the advantage that the server doesn't need to keep track of a session, but also means that queries just keep getting larger and larger the more you swipe. I'm not a web developer, I have absolutely no idea what the tradeoffs are here, so I point this out as a point of interest rather than anything else.

Anyway. For people unfamiliar with graphql, it's basically a way to query a database and define the set of fields you want returned. Let's take the example of requesting a user's profile. You'd provide the profile ID in question, and request their bio, age, rough distance, status, photos, and other bits of data that the client should show. So far so good. But what happens if we request other data?

graphql supports introspection to request a copy of the database schema, but this feature is optional and was disabled in this case. Could I find this data anywhere else? Pulling apart the apk revealed that it's a React Native app, so effectively a framework for allowing writing of native apps in Javascript. Sometimes you'll be lucky and find the actual Javascript source there, but these days it's more common to find Hermes blobs. Fortunately hermes-dec exists and does a decent job of recovering something that approximates the original input, and from this I was able to find various lists of database fields.

So, remember that original FAQ statement, that your desires would never be shown to anyone else? One of the fields mentioned in the app was "lookingFor", a field that wasn't present in the default profile query. What happens if we perform the incredibly complicated hack of exporting a profile query as a curl statement, add "lookingFor" into the set of requested fields, and run it?

Oops.

So, point 1 is that you can't simply protect data by having your client not ask for it - private data must never be released. But there was a whole separate class of issue that was an even more obvious issue.

Looking more closely at the profile data returned, I noticed that there were fields there that weren't being displayed in the UI. Those included things like "ageRange", the range of ages that the profile owner was interested in, and also whether the profile owner had already "liked" or "disliked" your profile (which means a bunch of the profiles you see may already have turned you down, but the app simply didn't show that). This isn't ideal, but what was more concerning was that profiles that were flagged as hidden were still being sent to the app and then just not displayed to the user. Another example of this is that the app supports associating your profile with profiles belonging to partners - if one of those profiles was then hidden, the app would stop showing the partnership, but was still providing the profile ID in the query response and querying that ID would still show the hidden profile contents.

Reporting this was inconvenient. There was no security contact listed on the website or in the app. I ended up finding Feeld's head of trust and safety on Linkedin, paying for a month of Linkedin Pro, and messaging them that way. I was then directed towards a HackerOne program with a link to terms and conditions that 404ed, and it took a while to convince them I was uninterested in signing up to a program without explicit terms and conditions. Finally I was just asked to email security@, and successfully got in touch. I heard nothing back, but after prompting was told that the issues were fixed - I then looked some more, found another example of the same sort of issue, and eventually that was fixed as well. I've now been informed that work has been done to ensure that this entire class of issue has been dealt with, but I haven't done any significant amount of work to ensure that that's the case.

You can't trust clients. You can't give them information and assume they'll never show it to anyone. You can't put private data in a database with no additional acls and just rely on nobody ever asking for it. You also can't find a single instance of this sort of issue and fix it without verifying that there aren't other examples of the same class. I'm glad that Feeld engaged with me earnestly and fixed these issues, and I really do hope that this has altered their development model such that it's not something that comes up again in future.

(Edit to add: as far as I can tell, pictures tagged as "private" which are only supposed to be visible if there's a match were appropriately protected, and while there is a "location" field that contains latitude and longitude this appears to only return 0 rather than leaking precise location. I also saw no evidence that email addresses, real names, or any billing data was leaked in any way)

comments