Feeds
Gunnar Wolf: A new RISC-V toy... requiring almost no tinkering
Shortly before coming back from Argentina, I got news of a very interesting set of little machines, the MilkV Duo. The specs looked really interesting and fun to play with, particularly those of the “bigger” model, Milk-V DUO S Some of the highlights:
- The SG2000
SoC
is a Dual-architecture beast. A hardware switch controls whether the CPU is an
ARM or a RISC-V.
- Not only that: It has a second (albeit lesser) RISC-V core that can run independently. They mention this computer can run simultaneously Linux and FreeRTOS!
- 512MB RAM
- Sweet form factor (4.2×4.2cm)
- Peeking around their Web site, it is one of the most open and well documented
computers in their hardware range.
- Schematics at different levels of detail
- Datasheet. It’s preliminary, and it’s written in Mandarin, but the information it contains is quite descriptive and clear.
- Full specifications (all 710 pages of it!) Again, it has a lot of Mandarin in it, but it conveys a lot of useful information.
Naturally, for close to only US$12 (plus shipping) for the configuration I wanted… I bought one, and got it delivered in early May. The little box sat on my desk for close to six weeks until I had time to start tinkering with it…
I must say I am surprised. Not only the little bugger delivers what it promises, but it is way more mature than what I expected: It can be used right away without much tinkering with! I mean, I have played with it for less than an hour by now, and I’ve even managed to get (almost) regular Debian working.
Milk-V distributes a simple, 58.9MB compressed Linux image, based on Buildroot, a simple Linux image generator mostly used for embedded applications, as well as its source tree. I thought that would be a good starting point to work on setting up a minimal Debian filesystem, as I did with the CuBox-i4Pro ten years ago, and maybe even to grow towards a more official solution, akin to what we currently have for the Raspberry Pi family…
…Until I discovered what looks like a friendly and very active online community of Milk-V users! I haven’t yet engaged in it, but I stumbled across a thread announcing the availability of Debian images for the Milk-V family.
And yes, it feels like a very normal Debian system. /etc/apt/sources.list does point to a third-party repository, but it’s for only four packages, all related to pinmux controlfor CVITEK chips. It does feel like a completely normal Debian system! It is not as snappy and fast to load as Buildroot, but given Debian’s generality, that’s completely as expected. Even the wireless network, one of the usual pain points, works just out of the box! The Debian images can be built or downloaded from this Git repository.
In case you wonder how is this system booting or what hardware it detects, I captured two boot logs:
Krita Monthly Update – Edition 16
Welcome to the latest development and community news curated for you by the Krita-promo team.
Development report Krita is 25 years old!Artwork by David Revoy (CC BY-SA)
May 31, 2024 marks Krita’s 25th birthday. As one would expect, there have been many changes over the years – even the name changed several times. You can get a look inside Krita’s history in this blog post written by @Halla, Krita’s Maintainer for more than 20 years.
In honour of this milestone, @RamonM prepared a special treat for all Krita users: a video interview with @Halla.
Your feedback is requested-
5.2.3-Beta1 was released June 5th. This release represents a complete rework of the build system and numerous fixes by the core Krita developer team as well as freyalupen, Grum999, NabilMaghfurUsman, Deif_Lou, Alvin Wong, Rasyuqa A. H. and Mathias Wein. There are a number of first-time contributors whose names appear next to their contribution in the release notes.
-
Testing packages for every platform are provided on the release notes page. Please report your findings and feedback in the Testers Wanted thread.
-
Text property editing: Merge request 2092 is almost finished, awaiting review. Testing builds are now available on the CI. @Wolthera is requesting user feedback on the UX. Please read this post and share your comments there.
-
Google Summer of Code (GSOC) @Ken_Lo is seeking input on the Pixel Perfect project https://krita-artists.org/t/pixel-perfect-line-setting-for-pixel-art-brushes/42629/16
- Free transform bounding box rotation by Stuffin has been merged. This completes the feature request Adjusting the transform box to match the object angle in the drawing and can be tested in the 5.3.0-prealpha nightly. (Note to testers: Adjusting the bounding box is activated with Ctrl+Alt.) Thanks stuffin!
-
Grum999 is improving the python API so that it is more robust for python developers and they can access more of krita’s internal features through python. There is a work-in-progress MR to add new scripting functions for accessing Grids, Guides, and Mirror Axes from the document, and signals for changes in the document and view. Check the MR
-
@Ralek has added lossless transformation conditions - Rotations in increments of 90 degrees, and perfect x and y mirrors should now be lossless. This should greatly help out pixel artists, who I believe previously could not use these functions at all. Check the MR
Community Report May 2024 Monthly Art Challenge
And the winner is… Cat Reflection by Elixiah.
For the June Art Challenge, Elixiah has chosen Magnificent Dragon with an interesting optional challenge for any who care to give themselves an added stretch.
Featured artworkTen images were submitted to the Best of Krita-Artists Nominations thread which was open from April 14th to May 11th. When voting closed on May 14th, these five had the most votes and were added to the Krita-Artists featured artwork banner.
Quiet Morning by @Gurkirat_Singh.
Pollinatrix Terrae by @jimplex.
The Lone Rider-2 by @rohithela.
005 (Spider in the web) by @HappyBuket.
Challenge Horn by @MangooSalade.
In addition to their place of honour on the banner, all five will be entered into the Best of Krita-Artists 2024 competition next January. The Best of Krita-Artists May/June Nominations thread will be open for submissions until June 11, 2024. You are invited to join in by nominating your favourite piece of Krita artwork!
Noteworthy PluginCreate a New View as Window and Topped by Cliscylla saves steps by opening a new view and setting it to always stay on top.
Tutorial of the monthHow to record video directly from Krita and post to social media by Deevad is a comprehensive tutorial for beginner and intermediate Krita users. It takes the viewer through the initial screen set up and recommended canvas dimensions right through to the export process.
Ways to help KritaKrita is a Free and Open Source application, mostly developed by an international team of enthusiastic volunteers. Donations from Krita users to support maintenance and development is appreciated.
Visit Krita’s funding page to see how donations are used and explore a one-time or monthly contribution.
Notable Changes in the codeThis section has been compiled by freyalupen. (May 6 - June 6, 2024)
Stable branch (5.2.3-beta1):Bugfixes:
- General Don't waste memory generating empty animation frames on images with no animation. This was a regression in 5.2.x. (commit, Dmitry Kazakov)
- Storyboard Docker Fix reordering storyboard scenes causing all frame data to be deleted while still appearing to be present. (BUG:476440) (merge request, Freya Lupen)
- Android: Animation Fix crash when attempting to load audio on Android, a regression present in 5.2.2.1. (merge request, Dmitry Kazakov)
Stable branch (5.2.3-beta1+):
Bugfixes:
- Animation Fix crash when adding a keyframe column with a locked layer selected. (BUG:486893) (commit, Dmitry Kazakov)
- Keyboard shortcuts While continuing making a Selection, ignore other modifier shortcuts to avoid conflicts. (merge request, Aqaao)
- File Formats: TIFF Ask to use PSD data in TIFF only if any was found. (BUG:488024) (commit, Freya Lupen)
- General, macOS Fix update of "read-only" state of the document when loading and saving. Fixes a crash on macOS when loading TIFF or JPEG-XL recent file icons (which load a temporary document). (BUG:487544) (commit, Dmitry Kazakov)
- Android, Recorder Docker Fix saving Recorder frames as JPEG on Android, a regression present in 5.2.2.1. (BUG:487667) (commit, Dmitry Kazakov)
- Android, General Improve Krita's apk icons to follow Android design guidelines. (BUG:463043) (merge request, Jesse 205)
- Scripting Generate a Python type stub file for Krita's API, which can be used to setup type auto-completion in IDEs, located inside the Krita package at /lib/krita-python-libs/PyKrita/krita.pyi. (merge request, Kate Corcoran)
Stable branch (5.2.3-beta1+) backports from Unstable: Bugfixes:
- Recorder Docker Reworked default recorder docker FFmpeg profiles. If canvas size changes during recording, the export profiles now keep aspect instead of stretching (BUG:429326). Issues with resize, result preview, and extend result are avoided (BUG:455006, BUG:450790, BUG:485515, BUG:485514). For MP4, detect whether openh264 or libx264 is present instead of using separate profiles. Also, prevent an error when using FFmpeg 7. (merge request, Ralek Kolemios)
- Selection Tools Fix issue making selections on color-labeled reference selections. (BUG:486419) (commit, Deif Lou)
Features:
- Transform Tool Allow rotating the free transform bounding box with Ctrl+Alt, in order to make transformations along an arbitrary axis. (WISHBUG:383587) (merge request, Stuffins)
Bugfixes:
- Transform Tool Make sure perfect mirrors and 90-degree rotations are transformed losslessly. (merge request, Ralek Kolemios)
- Shortcuts Fix Sample Screen Color getting stuck if activated multiple times without completing. (BUG:485739) (merge request, Deif Lou)
- Scripting Fix setting Color Adjustment (perchannel) and Cross-Channel filters from Python scripts. (merge request, Deif Lou)
These changes are made available for testing in the following Nightly builds:
- Stable "Krita Plus" (5.2.3-beta1+): Linux - Windows - macOS [unsigned currently] - Android (arm64-v8a / arm32-v7a / x86_64)
- Unstable "Krita Next" (5.3.0-prealpha): Linux - Windows - macOS [unsigned currently] - Android (arm64-v8a / arm32-v7a / x86_64)
Krita is a free and open source project. Please consider supporting the project with donations or by buying training videos or the artbook! With your support, we can keep the core team working on Krita full-time.
Donate Buy somethingSeth Michael Larson: CPython vulnerability data infrastructure (CVE and OSV)
Published 2024-06-21 by Seth Larson
Reading time: minutes
Let's talk about some vulnerability data infrastructure for the Python Software Foundation (PSF). In the recent past, most of the vulnerability data processes were manual. This worked okay because the PSF processes a low double-digit number of vulnerabilities each year (2023 saw 12 published vulnerabilities).
However, almost all of this manual processing was being done by me as full-time staff. Imagining this work being done by either someone else on staff or a volunteer isn't great, because it's a non-zero amount of extra work. Automation to the rescue!
How the vulnerability data flowsThe PSF uses the CVE database as its “source of truth” for vulnerability data which then gets imported into our Open Source Vulnerability (OSV) database by translating CVE records into OSV records.
We manually update CVE information in a program called Vulnogram which provides a helpful UI for CVE services.
So what is the minimum amount of information we need to manually create to automatically generate the rest? This is the current list of data the PSF CVE Numbering Authority team creates manually:
- Advisory text and description
- CVE reference to the advisory
- GitHub issue (as a CVE reference)
GitHub RepositoryGitHub Reposito...CVE
ServicesCVE...PSF OSV DatabasePSF OSV DatabaseGitHub
IssueGitHub...GitHub
Merged PRGitHub...git commitgit commitgit tag
(v3.12.4)git tag...CVE RecordCVE RecordCVE Affected VersionsCVE Affected...CVE ReferencesCVE ReferencesOSV RecordOSV RecordOSV Affected CommitsOSV Affected...OSV Affected TagsOSV Affected...OSV ReferencesOSV ReferencesText is not SVG - cannot display
Blue items are manually created, green items are automatically generated.
Advisories are sent to the security-announce@python.org mailing list and then the description is reused as the CVE record's description. The linkages between a GitHub pull request and a GitHub issue is maintained by Bedevere which automatically updates metadata as new pull requests are opened for an issue.
From this information we can use scripts to generate the rest in two stages:
- CVE affected versions and references are populated by finding git tags that contain git commits.
- All OSV record information is generated from CVE records. New OSV records are automatically assigned their IDs by OSV tooling. The central OSV database calculates affected git tags on our behalf.
The PSRT publishes advisories and patches once they're available in the latest CPython versions. For low-severity vulnerabilities there typically isn't an immediate release of all bugfix and security branches (ie 3.12, 3.11, 3.10, 3.9, etc). This means that many vulnerability records will be need to be updated over time as fixes are merged and released into other CPython versions so these scripts run periodically to avoid tracking these updates manually.
Other items- This week marks my one-year anniversary in the role of Security Developer-in-Residence. Woohoo! 🥳
- Advisories and records were published for CVE-2024-0397 and CVE-2024-4032.
- Wrote the draft for "Trusted Publishers for All Package Repositories" for the OpenSSF Securing Software Repositories WG. This document would be used by other package repositories looking to implement Trusted Publishers like PyPI has.
- Working on documenting more of PSRT processes like membership.
- Triaging reports to the PSRT.
- Reviewing PEP 740 and other work for generating publish provenance from William Woodruff.
- Google Summer of Code contributor, Nate Ohlson has been making excellent progress on the effort for CPython to adopt the "Hardened Compiler Options Guide" from the OpenSSF Best Practices WG. You can follow along with his progress on Mastodon and on GitHub.
That's all for this post! 👋 If you're interested in more you can read the last report.
Thanks for reading! ♡ Did you find this article helpful and want more content like it?
Get notified of new posts by subscribing to the RSS feed or the email newsletter.
This work is licensed under CC BY-SA 4.0
The Python Show: 45 - Computer Vision and Data Science with Python
In this episode, we welcome Kyle Stratis from Real Python to the show to chat with us about computer vision and Python.
We chatted about several different topics including:
Writing about Python on Real Python
Data science
Artificial intelligence
Python packaging
and much more!
Paolo Melchiorre: Django 5 by Example preface
The story of my experience in writing the preface of the book “Django By Example” by Antonio Melé.
The Drop is Always Moving: Drupal 10.3.0 is out! The third and final feature release of Drupal 10 ships with a new experimental Navigation UI, stable Workspaces and Single-Directory Components, simplified menu editing, taxonomy moderation, new recipe...
Drupal 10.3.0 is out! The third and final feature release of Drupal 10 ships with a new experimental Navigation UI, stable Workspaces and Single-Directory Components, simplified menu editing, taxonomy moderation, new recipe and access policy APIs and more. https://www.drupal.org/blog/drupal-10-3-0
Drupal blog: Drupal 10.3 is now available
The third and final feature release of Drupal 10 ships with a new experimental Navigation user interface, stable Workspaces functionality, stable Single-Directory Components support, simplified menu editing, taxonomy moderation support, new recipe and access policy APIs, performance improvements and more.
New experimental Navigation moduleThe new Navigation module provides a redesigned collapsible, vertical navigation sidebar for the administrative user interface. Sub-menus open on a full height drawer that can accommodate deeper navigation levels. On smaller viewports, the toolbar is placed on top of the content, and opens with an overlay.
The Navigation module allows multiple types of customization, like adding new custom menus or changing the default Drupal logo provided. It also uses the Layout Builder module, so that site builders can easily add or reorder these menu blocks.
The Navigation module includes a new content creation and management menu, which allows quick access to content-related tasks to increase usability for content users.
Stable Workspaces moduleThe Workspaces module allows Drupal sites to have multiple work environments, enabling site owners to stage multiple content changes to be deployed to the live site all at once. It has long been available in Drupal core as an experimental module. Following the module's use in live projects, the remaining stable blocking issues have been resolved, so now it is available to all!
Workspaces are sets of content changes that are prepared and reviewed separately from the live site. This is a differentiating feature for Drupal that is important for many large organizations' websites. An organization might use Workspaces to ensure all relevant content goes live simultaneously for a new product launch, or with the outcomes of sporting or election events.
Stable Single-Directory ComponentsSingle-Directory Components (SDCs) are Drupal core’s implementation of a user interface components system. Within SDC, all files necessary to render the user interface component are grouped together in a single directory. This includes Twig, YAML, and optional CSS and JavaScript. SDC support was added to Drupal core in 10.1 as an experimental module. The solution has been very well-received and is now part of the base system. No need to enable a module to use this feature.
Simplified content organizationMenu item editing is now simplified. Advanced options are displayed in a sidebar to help content editors focus on what is most important for the menu item. Taxonomy terms also now have both a dedicated user interface to edit earlier revisions and content moderation support.
New Recipes and Default Content APIsDrupal recipes allow the automation of Drupal module installation and configuration. Drupal recipes are easy to share, and can be composed from other Drupal recipes. For example, Drupal 10.3 includes a Standard recipe providing the same functionality as the Standard install profile. It is a combination of 16 component recipes that can be reused in other recipes.
Recipes provide similar functionality to install profiles but are more flexible. With install profiles only one can be installed on a site. With recipes, multiple recipes can be applied after each other.
Install profiles/distributions Recipes Lock-in Not possible to uninstall (until Drupal 10.3) No lock-in Inheritance Cannot extend other profiles or distributions Can be based on other recipes Composability Cannot install multiple profiles or distributions Multiple recipes can be applied on the site and be the basis of another recipeThe recently announced Starshot Initiative will rely heavily on recipes to provide composable features.
The added APIs include Configuration Actions, Configuration Checkpoints and Default Content.
Additionally, it is now possible to install Drupal without an install profile, or to uninstall an install profile after Drupal is already set up.
More flexible access management with the new Access Policy APIThe new Access Policy API supports the implementation of access management solutions that go beyond permissions and user roles. Other conditions and contexts may be taken into account, like whether the user used two-factor authentication, or whether they reached a rate limit of an activity. Drupal's existing permission- and role-based access control has been converted to the new API, and custom or contributed projects can add more access policies.
The future of Drupal 10Drupal 10.3 is the final feature release of Drupal 10. Drupal 11 is scheduled to be released the week of July 29th. With that, Drupal 10 goes into long-term support. While more minor releases will be made available of Drupal 10, they will not contain new features, only functionality backported to support security and a smoother upgrade to Drupal 11. Drupal 10's future minor releases will be supported until mid- to late 2026, when Drupal 12 is released and Drupal 11 enters long-term support.
Core maintainer team updatesCristina Chumillas (at Lullabot), Sally Young (also at Lullabot) and Théodore Biadala (at Très Bien Tech) were all promoted from provisional to full Drupal Core Frontend Framework Managers.
Alex Pott (at Acro Commerce and Thunder), Adam Globus-Hoenich (at Acquia) and Jim Birch (at Kanopi Studios) are the maintainers of the new Default Content and Recipes subsystems.
Andrei Mateescu (at Tag1 Consulting) is the maintainer of the newly stable Workspaces module.
Ivan Berdinsky (at Skilld) became a co-maintainer of the Umami demo.
Daniel Veza (at PreviousNext) is a new co-maintainer of Layout Builder.
Mateu Aguiló Bosch (at Lullabot) and Pierre Dureau are new co-maintainers of the Theme API, focusing on Single Directory Components.
Want to get involved?If you are looking to make the leap from Drupal user to Drupal contributor, or you want to share resources with your team as part of their professional development, there are many opportunities to deepen your Drupal skill set and give back to the community. Check out the Drupal contributor guide, or join us at DrupalCon Barcelona and attend sessions, network, and enjoy mentorship for your first contributions.
mark.ie: My LocalGov Drupal contributions for week-ending June 21th, 2024
Here's what I've been working on for my LocalGov Drupal contributions this week. Thanks to Big Blue Door for sponsoring the time to work on these.
PyBites: Introducing eXact-RAG: the ultimate local Multimodal Rag
Exact-RAG is a powerful multimodal model designed for Retrieval-Augmented Generation (RAG). It seamlessly integrates text, visual and audio information, allowing for enhanced content understanding and generation.
In the rapidly evolving landscape of the Large language model (LLM), the quest for more efficient and versatile models continues unabated.
One of the latest advancements in this realm is the emergence of eXact-RAG, a multimodal RAG (Retrieval Augmented Generation) system that leverages state-of-the-art technologies to deliver powerful results.
eXact-RAG stands out for its integration of LangChain and Ollama for backend and model serving, FastAPI for REST API service, and its adaptability through the utilization of ChromaDB or Elasticsearch.
Coupled with an intuitive user interface built on Streamlit, eXact-RAG represents a significant leap forward in LLMs capabilities.
A Step back: what is a RAG?Retrieval Augmented Generation, or RAG, is an architectural approach that can improve the efficacy of Large Language Model (LLM) applications by leveraging custom data. This is done by retrieving data/documents relevant to a question or task and providing them as context for the LLM. RAG has shown success in support chatbots and Q&A systems that need to maintain up-to-date information or access domain-specific knowledge.
RAG process schema. Source: neo4j.comAs the name suggests, RAG has two phases: retrieval and content generation. In the retrieval phase, algorithms search for and retrieve snippets of information relevant to the user’s prompt or question. In an open-domain, consumer setting, those facts can come from indexed documents on the internet; in a closed-domain, enterprise setting, a narrower set of sources are typically used for added security and reliability.
Understanding eXact-RAGAt its core, eXact-RAG combines the principles of RAG, which focuses on retrieval-based conversational agents, with multimodal capabilities, enabling it to process and generate responses from various modalities such as text, images, and audio.
This versatility makes eXact-RAG well-suited for a wide range of applications, from chatbots to content recommendation systems and beyond.
Technologies Powering eXact-RAG- LangChain and Ollama: LangChain and Ollama serve as the backbone of eXact-RAG, providing robust infrastructure for model development, training, and serving.
LangChain offers a comprehensive suite of tools for natural language understanding and processing, while Ollama specializes in multimodal learning, enabling eXact-RAG to seamlessly integrate and process diverse data types. - FastAPI for REST API Service: FastAPI, known for its high performance and simplicity, serves as the interface for eXact-RAG, facilitating seamless communication between the backend system and external applications.
Its asynchronous capabilities ensure rapid response times, crucial for real-time interactions. - ChromaDB or Elasticsearch: eXact-RAG offers flexibility in data storage and retrieval by supporting both ChromaDB and Elasticsearch.
ChromaDB provides a lightweight solution suitable for simpler tasks, while Elasticsearch caters to more complex operations involving vast amounts of data. This versatility enables users to tailor eXact-RAG to their specific needs, balancing performance and scalability accordingly.
The user interface of eXact-RAG is built on Streamlit, a popular framework for creating interactive web applications with Python. Streamlit’s intuitive design and seamless integration with Python libraries allow users to interact with eXact-RAG effortlessly.
Through the interface, users can input queries, explore results, and interact with generated content across various modalities, enhancing the overall user experience.
Related article: From concepts to MVPs: Validate Your Idea in few Lines of Code with Streamlit
Applications of eXact-RAGThe versatility of eXact-RAG opens up a myriad of applications across different domains:
- Conversational Agents: eXact-RAG can power chatbots and virtual assistants capable of engaging users in natural and meaningful conversations, leveraging both text and multimedia inputs.
- Content Recommendation: By analyzing user preferences and behavior, eXact-RAG can recommend personalized content, including articles, videos, and images, tailored to individual tastes and interests.
- Information Retrieval: eXact-RAG excels at retrieving relevant information from large datasets, making it invaluable for tasks such as question answering, document summarizing, and knowledge base retrieval.
The first step to use eXact-RAG is to run your preferred LLM model using Ollama or get an OpenAI token. Both are supported by the RAG and to be sure to configure them rightly, it’s necessary to fill the settings.toml with the preferred options. Here an example:
[embedding] type = "openai" api_key = "" chat.model_name = "gpt-3.5-turbo" ... [database] type = "chroma" persist_directory = "persist" ...In this example it’s possible to configure the LLM model and the vector database in which store the embeddings of your local data.
eXact-RAG is a multimodal RAG, for this reason it can ingest different kind of data like audio files or images. It is possible to choose, in the installation phase, which “backends” to install:
poetry install # -E audio -E image- audio extra will install openai-whisper for speech-to-text
- image extra will install transformers and pillow for image captioning*
* this feature gives the possibility to process images even if the user has not the (hardware) possibility to run a Vision model like llava locally but still wants to pass images as data.
Now the job is done! The following command
poetry run python exact_rag/main.pystarts the server and at http://localhost:8080/docs it is showed the OpenAPI document (swagger) with all the available endpoints.
DemoeXact-RAG was built as a server for multimodal RAGs but we provide also a user interface just for demo purposes to test all the features.
To run the UI just use the command:
Now, the page at http://localhost:8501 will show a chat interface like in the following example:
ConclusioneXact-RAG represents a significant advancement in the field of multimodal RAG systems, offering unparalleled versatility and performance through its integration of cutting-edge technologies.
With its robust backend powered by LangChain and Ollama, flexible data storage options, and user-friendly interface built on Streamlit, eXact-RAG is poised to revolutionize various applications of natural language processing and multimodal learning.
As the demand for sophisticated LLM solutions continues to grow, eXact-RAG stands ready to meet the challenges of tomorrow’s digital landscape.
C.J. Collier: Signed NVIDIA drivers on Google Cloud Dataproc 2.2
Hello folks,
I’ve been working this year on better integrating NVIDIA hardware with the Google Cloud Dataproc product (Hadoop on Google Cloud) running the default cluster node image. We have an open bug[1] in the initialization-actions repo regarding creation failures upon enabling secure boot. This is because with secure boot, kernel driver code has its signature verified before insmod places the symbols into kernel memory. The verification process involves reading trust root certificates from EFI variables, and validating that the signatures on the kernel driver either a) were made directly by one of the certificates in the boot sector or b) were made by certificates which chain up to one of them.
This means that Dataproc disk images must have a certificate installed into them. My work on the internals will likely start producing images which have certificates from Google in them. In the meantime, however, our users are left without a mechanism to have both secure boot enabled and install out-of-tree kernel modules such as the NVIDIA GPU drivers. To that end, I’ve got PR #83[2] open with the GoogleCloudDataproc/custom-images github repository. This PR introduces a new argument to the custom image creation script, `–trusted-cert`, the argument of which is the path to a DER-encoded certificate to be included in the certificate database in the EFI variables of the disk’s boot sector.
I’ve written up the instructions on creating a custom image with a trusted certificate here:
https://github.com/cjac/custom-images/blob/secure-boot-custom-image/examples/secure-boot/README.md
Here is a set of commands that can be used to create a general purpose GCE disk image with the certificate inserted into EFI.
wget 'https://github.com/cjac/custom-images/raw/secure-boot-custom-image/examples/secure-boot/create-key-pair.sh' bash create-key-pair.sh cacert_der=tls/db.der # The Microsoft Corporation UEFI CA 2011 ms_uefi_ca="tls/MicCorUEFCA2011_2011-06-27.crt" test -f "${ms_uefi_ca}" || \ curl -L -o ${ms_uefi_ca} 'https://go.microsoft.com/fwlink/p/?linkid=321194' JSON="$(gcloud compute images --project debian-cloud list --format json)" SRC_IMAGE_NAME="$(echo ${JSON} | jq -r '.[] | .name' | grep -i ^debian-12-bookworm-v)" NEW_IMAGE_NAME="debian12-with-db-key-list-${USER}-$(date +%F)" SRC_DISK_ZONE="${ZONE}" gcloud -q compute images create "${NEW_IMAGE_NAME}" \ --source-disk "${SRC_IMAGE_NAME}" \ --source-disk-zone "${SRC_DISK_ZONE}" \ --signature-database-file="${cacert_der},${ms_uefi_ca}" \I’d love to hear your feedback!
[1] https://github.com/GoogleCloudDataproc/initialization-actions/issues/1058
[2] https://github.com/GoogleCloudDataproc/custom-images/pull/83
Daniel Lange: Fixing esptool read_flash above 2MB on some cheap ESP32 boards
esptool, the Espressif SoC serial bootloader utility, tends to dislike cheap Flash chips attached to the various incarnations of the ESP32 chip family. And it seems to dislike them even more when running esptool on Linux than on other OSs.
The common error mode is seeing it break at the 2MB barrier when trying to dump (esptool read_flash) a 4MB flash configuration.
esptool -p /dev/ttyUSB0 -b 921600 read_flash 0 0x400000 flash_dump.binwill fail with
esptool.py v4.7.0 Serial port /dev/ttyUSB0 Connecting.... Detecting chip type... ESP32 Chip is ESP32-D0WD-V3 (revision v3.1) Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None Crystal is 40MHz [..] Detected flash size: 4MB [..] 2097152 (50 %) A fatal error occurred: Failed to read flash block (result was 01090000: CRC or checksum was invalid)typically at the 2MB barrier.
I found the solution in a rather unrelated esptool Github issue:
Create an esptool.cfg file in the project directory (from where you will run esptool):
[esptool]timeout = 30
max_timeout = 240
erase_write_timeout_per_mb = 40
mem_end_rom_timeout = 0.2
serial_write_timeout = 10
The timeout = 30 is the setting that fixed reading flash memory via esptool read_flash for me.
When your esptool.cfg is read, esptool will tell you so in its second line of output:
$ esptool flash_id esptool.py v4.7.0 Loaded custom configuration from /home/dl/[..]/Embedded_dev/ESP-32_Wemos/esptool.cfg Found 1 serial ports Serial port /dev/ttyUSB0 Connecting...... [..]Thank you Radim Karnis and wibbit from the Github issue linked above.
Kdenlive 24.05.1 released
The first maintenance release of the 24.05 series is out fixing issues in the spacer tool, effects and compositions, subtitle management and project settings to name a few. We addressed recently introduced crashes and freezes, including fixing the undo/redo track insertion and multiple track insertion issues. This version also improves AppImage packaging and enables notarization for macOS.
Full changelog
- Don’t try renaming sequence on double click in empty area of timeline tab bar. Commit.
- Fix deletion of wrong effect wihh multiple instances of an effect and group effects enabled. Commit.
- Fix single selected clip disappearing from timeline when dragging a new clip in timeline. Commit.
- [cmd rendering] Ensure proper kdenlive_render path for AppImage. Commit.
- Fix freeze/crash on undo/redo track insertion. Commit.
- Fix crash on undo/redo multiple track insertion. Commit.
- Project settings: don’t list embedded title clips as empty files in the project files tab. Commit.
- Fix undo move effect up/down. On effect move, also move the active index, increase margins between effects. Commit.
- Fix removing a composition from favorites. Commit.
- Properly activate effect when added to a timeline clip. Commit.
- Fix spacer tool can move backwards and overlap existing clips. Commit.
- Fix crash deleting subtitle when the file url was selected. Commit. Fixes bug #487872.
- Fix build when using openGLES. Commit. Fixes bug #483425.
- Fix possible crash on project opening. Commit.
- Fix extra dash added to custom clip job output. Commit. See bug #487115.
- Fix usage of QUrl for LUT lists. Commit. See bug #487375.
- Fix default keyframe type referencing the old deprecated smooth type. Commit.
- Be more clever splitting custom ffmpeg commands around quotes. Commit. See bug #487115.
- Fix effect name focus in save effect. Commit. See bug #486310.
- Fix tests. Commit.
- Fix selection when cutting an unselected clip under mouse. Commit.
- Fix loading timeline clip with disabled stack should be disabled. Commit.
- Fix crash trying to save effect with slash in name. Commit. Fixes bug #487224.
- Remove quotes in custom clip jobe, fix progress display. Commit. See bug #487115.
- Fix setting sequence thumbnail from clip monitor. Commit.
- Fix locked track items don’t have red background on project open. Commit.
- Fix spacer tool doing fake moves with clips in locked tracks. Commit.
- Hide timeline clip status tooltip when mouse leaves. Commit
The post Kdenlive 24.05.1 released appeared first on Kdenlive.
PyBites: What is the Repository Pattern and How to Use it in Python?
The repository pattern is a design pattern that helps you separate business logic from data access code.
It does so by providing a unified interface for interacting with different data sources, bringing the following advantages to your system:
- Flexibility: You can swap out your data source without changing business logic.
- Maintainability: It’s easier to manage and test different storage backends.
- Decoupling: You reduce tight coupling between your application and data access layer.
Let’s use Python and sqlmodel (PyPI) to demonstrate this pattern (code here):
from abc import ABC, abstractmethod from sqlmodel import SQLModel, create_engine, Session, Field, select # Define the model class Item(SQLModel, table=True): id: int = Field(default=None, primary_key=True) name: str # Repository Interface class IRepository(ABC): @abstractmethod def add(self, item: Item): pass @abstractmethod def get(self, name: str) -> Item | None: pass # SQLModel implementation class SQLModelRepository(IRepository): def __init__(self, db_string="sqlite:///todo.db"): self.engine = create_engine(db_string) SQLModel.metadata.create_all(self.engine) self.session = Session(self.engine) def add(self, item: Item): self.session.add(item) self.session.commit() def get(self, name: str) -> Item | None: statement = select(Item).where(Item.name == name) return self.session.exec(statement).first() # CSV implementation class CsvRepository(IRepository): def __init__(self, file_path="todo.csv"): self._file_path = file_path def add(self, item: Item): with open(self._file_path, "a") as f: f.write(f"{item.id},{item.name}\n") def get(self, name: str) -> Item | None: with open(self._file_path, "r") as f: return next( ( Item(id=int(id_str), name=item_name) for line in f if (id_str := line.strip().split(",", 1)[0]) and (item_name := line.strip().split(",", 1)[1]) == name ), None, ) if __name__ == "__main__": repo = SQLModelRepository() repo.add(Item(name="Buy Milk")) sql_item = repo.get("Buy Milk") # Swap out the repository implementation csv_repo = CsvRepository() csv_repo.add(Item(id=1, name="Buy Milk")) csv_item = csv_repo.get("Buy Milk") print(f"{sql_item=}, {csv_item=}, {sql_item == csv_item=}") # outputs: # sql_item=Item(name='Buy Milk', id=1), csv_item=Item(id=1, name='Buy Milk'), sql_item == csv_item=True- First we define the Item model using SQLModel. You can also use SQLAlchemy or any other ORM, but I find SQLModel a bit easier to use and I like its integration with Pydantic.
- Next we define the IRepository interface with the add and get methods, making them required for any class that inherits from it. This is done using Python’s Abstract Base Classes (ABCs) and applying the @abstractmethod decorator. This is a way to “enforce a contract” and a crucial part of the repository pattern (see also the tests). If you don’t implement these methods in a subclass, Python will raise an error. This is a way to ensure that all repository classes have the same interface, even if they use different storage. To learn more about ABCs, check out our article).
- Then we implement the SQLModelRepository and CsvRepository subclasses, which inherit from IRepository. These classes implement the add and get methods, which are required by the IRepository interface. This is where we define the data access logic for each of the storage. The SQLModelRepository uses SQLModel to interact with a SQLite database, while the CsvRepositoryinteracts with a CSV file. Same interface, different storage backends.
- Finally, we demonstrate how to use the repository pattern by adding an item to both the SQL and CSV repositories and then retrieving it.
Note: In this implementation, we did not add error handling to keep the example relatively small. You should ensure that if something goes wrong (e.g., database connection issues or file access errors), the program can handle the error gracefully and provide useful feedback.
The example might be a bit contrived, but it shows how we can leverage the repository pattern in Python.
Again, the advantage is that we have a flexible design that allows us to swap out the data access implementation without changing the business logic. This makes our code easier to test and maintain.
Have you used this pattern yourself and if so, how? Hit us up on social media and let us know: @pybites …
DrupalEasy: Visual Debugger module: a modern take on an old idea
If you've been a Drupal developer since before the time of modern Drupal (pre-Drupal 8,) then you probably remember the Theme Developer module. This module is/was used to figure out theme template usage and possible overrides in Drupal 7.
With modern Drupal, this same information is available directly in the HTML source code of a file when Twig development mode is enabled.
The Visual Debugger module by Marcos Hollanda combines the best of the present with the best of the past to surface template information directly in the user interface without having to dig into the page's HTML.
Pre-impressionsI will admit that before I installed the module, I was doubtful that I'd be interested in using it. Marcos had pinged me on Slack asking me to take a look at it, and I figured it would be a nice exercise for DrupalEasy Office Hours, which it was.
Much to my surprise though, I really like using it.
Basic usageThere's not much to say about how to use this module - it works just like you'd expect. Use Composer to get the code, enable the module, then enable Twig development mode and you're done. By default it will appear on each non-admin page of your site, with a handy activate/deactivate icon. There's really not too much to say about its usage 😀.
Current statusThe module is in alpha status with several planned features not yet implemented. What is implemented so far appears to be solid and useful. The basic functionality that I expected is there. Click on any section of the page, and the module will display the template file that is used to render that section as well as a list of possible template file override names. This is the same information that Twig developer mode displays in HTML comments.
Marcos let me know that other planned features include theme hook suggestions and displaying caching information; (similar to the Cache review module, perhaps?)
There are other challenges to overcome as well, including how to handle selecting overlapping elements. I did make one very minor suggestion to relocate the module's configuration page which Marcos quickly implemented 🤘🏼.
Who is this module for?I would definitely put this module in the category of a developer tool for folks who are either new to Drupal theme development or people who prefer a UI instead of scrolling through raw HTML.
I plan on following the progress of this module closely, in hopes that it will be suitable to use during the next semester of Drupal Career Online, where we have a theming lesson that could definitely benefit from this module.
1xINTERNET blog: 10 React use cases at 1xINTERNET
React is one of the primary technologies in our technology stack. This article explores our React use cases, outlining their purpose, technical aspects and implementation examples.
The Drop Times: Driving Drupal Forward: Suzanne Dergacheva on the Strategic Rebranding of Drupal
Tag1 Consulting: Migrating Your Data from Drupal 7 to Drupal 10: Generating migrations with Migrate Upgrade
Series Overview & ToC | Previous Article | Next Article - coming June 27th In the previous article we performed an automated migration via the administration interface with tools provided by Drupal core out of the box. This gave us the opportunity to familiarize ourselves with the different modules that assist with the task, settings that affect the migration, and the results when there are no customizations in place. Today we will leverage the Migrate Plus and Migrate Upgrade modules to generate migrations from the command line, which we can then pick from and customize. Automated migration using the Migrate Upgrade module Let’s start by listing the modules that we will use to run the automated migration from the command line: * Migrate: consists of the core API that can be used to import data from any source. * Migrate Drupal: allows connecting to a Drupal 6 or 7 site to import configuration and content. * Migrate Plus: provides extra source, process, and destination plugins; handles migrations as configuration entities; allows reusing of configuration via migration groups; and more. It is a dependency of Migrate Upgrade and a very useful module on its own. * Migrate Upgrade: provides a Drush...
Read more mauricio Thu, 06/20/2024 - 05:15Talk Python to Me: #467: Data Science Panel at PyCon 2024
Matt Layman: Password Resets and Signal Handling - Building SaaS #194
Mario Hernandez: Managing image embeds with Drupal media
Allowing your content creators to embed images in text fields is a big risk if you don't have the right measures in place to get properly rendered images without jeopardizing your site's performance. We faced this issue first-hand with embedded images due to not using the right configuration and this lead to extremely large images being rendered. In this post I'll go over the techniques I took for addressing those issues and set up a system for image embeds that is solid and performant.
I started by writing a seven-part guide on how to setup responsive images. In this post I'll focus on image embeds. If you followed or read the responsive images guide, you should be able to take advantage of some of the work we did there in this post. The guidelines covered here include:
- Defining requirements
- Image styles
- Media view modes
- Text format configuration
Before you write the first line of code or set the first drupal configuration for this issue, you need to have a clear understanding of your requirements. Here is a summary of my requirements:
-
Only certain user roles can embed images
This means we need to figure out if the text formats used in our site will allow us to set the restrictions we need. Otherwise we may need to create or edit a text format for our target user roles.
-
Users need to be able to choose the image size and aspect ratio when embedding images
We defined the image sizes and aspect ratios and assigned names that were user-friendly for non-technical users. We came up with name options we think our users will find easy to work with such as:
- Small square, Small portrait, Small rectangular
- Medium square, Medium portrait, Medium rectangular, Medium rectangular wide
- Large square, Large rectangular, Large rectangular wide
- Extra large square, Extra large rectangular, Extra large rectangular wide
-
If no option is selected by users, set a default image size
For the default option when no selection is made by the user, we decided to use the Medium rectangular option. This has an aspect ratio of 3:2 and it measures about 720x480.
-
Existing Media items need to be available for embedding
This was a tricky one because my original inclination was to create a new Media type so we can isolate all configuration for its view modes and not overpopulate our default Media type. However, this ended up not working for us because when you limit your image embeds to only use a new Media type, you don't get access to any of the media items (images), that have already been uploaded to the Media library using other media types. Ultimately we ended up using Drupal core's Media type, Image, and our dev team had to compromise on having a very busy list of view modes for this media type.
-
Images need the ability to be cropped wihin the Media page
Since most of our images already provide the ability to be cropped at different aspect ratios, using the core Media type in the previous bullet point made this an easy solution.
It all starts with image styles. I'm not going to go over how to create image styles, you can read my post Image styles in Drupal. The one thing I am going to repeat however is the importance of creating reusable image styles. Reusable image styles can help you reduce the number of image styles you create while providing the flexibility you need with each use case.
Image styles are key as each of the size options we defined above translate into image styles. So Small square for example, is an image style that is defined as 1:1 (250px). Medium rectangular would be something like 3:2 (720x480), etc. You may be wondering, how do you plan on using fiendly names for your content editors when your image styles names are not very friendly? Great question. Since we are using Drupal's Media, content editors do not interact directly with image styles, they do with Media view modes and this is where we will use more friendly names.
Media view modesView modes are one of Drupal's powerful features. Being able to display content is different ways with little effort can turn a simple website into a dynamic content hub. The example I always give when someone asks me what view modes are or how do they work is the Amazon website. When you are viewing a product in amazon.com, or most retail websites for that matter, you will notice that the same product or similar ones appear all over the page but in slightly different ways, with different fields or styles. See the page below for an example.
The image above shows many ways in which a product can be displayed. I've numbered each display.
In Drupal, every entity such as content types, media types, blocks, etc., offer the ability to create view modes. For the purpose of image embeds, we will create a Media type view mode for each image style we plan on using. The view modes is what content editors will interact with when choosing an image size or aspect ratio during the image embed process. This is where we will use the user-friendly names we defined earlier. Let's go over how this relationship between view modes and image styles works for image embeds.
Configure view modes for the Image media type-
In your Drupal site, create an image style for each image size option you wish to provide to users when embedding images.
-
Next, create a Media view mode for each image style (/admin/structure/display-modes/view). Very iimportant: Remember the view mode's label (name) is where we are going to use the friendly name (i.e. Medium rectangular (720x480)). I like to keep the machine name similar to the label so it's easier to debug or identify in code (i.e. medium_rectangular_720x480).
-
Now, let's tie 1 & 2 together:
- Go to the media type you plan on using for media embeds (/admin/structure/media/manage/image/display). I am using Drupal core's Image media type.
- Scroll down and expand the Custom display settings fieldset.
- Check each of the view modes you created in step 2 and click Save.
-
Now click each of the view modes and update the image field to use the respective/matching image style.
View modes and image styles are all configured. Now let's configure the Text format that authorized users will use to embed images.
- Go to the Text formats and editors page (/admin/config/content/formats)
- Click Configure next to the text format you plan on using (i.e. Full HTML)
- Ensure the right user roles are selected
- Within the Toolbar configuration section, drag the Drupal media button from the Available buttons options to the Active toolbar section. You could probably remove the original insert image button since you won't be using it.
- Scroll to the Enabled filters section and check the Embed media checkbox
- Scroll to the Filter settings section and set the following:
-
Default view mode: This is the default display that will be used if content editors don't pick an option when embedding images. Select any of the view modes that represents the image size you want to use as default.
-
Media types selectable in the Media Library: Select the Media type you plan on using. In my case is Image.
-
View modes selectable in the 'Edit media' dialog: Finally, select each of the view modes you created in the previous section. FYI: View modes will be sorted in alpha order by their machine name. In my case I had to prefix some of the machine names with either "a" or "b" so the list of options for the users to choose from would be nicely organized by their label name. See screnshot below.
-
Click Save configuration
-
Now that we've completed all the configuration we should be able to take it for test drive.
- Go to any page where there is a text field with a WYSIWYG editor
- Make sure you are using the right text format by selecting it at the bottom of the text field where you want to embed an image
- Click the Insert media button from the editor's toolbar
- Select or upload the image you'd like to embed
- When the image has been inserted, click on it and you should see several options of actions you can do with the image. Things like align the image, add a caption, link it, and you should also see a selection box listing all the view modes you created.
- After making your selection you should immediately see the image size/aspect ratio change to the one you selected. When you are happy with your selection, click the Save button to save your page.
Important: Depending on your site's configuration, the options for changing your image size may look different than mine. In my case, I am only using Drupal's core modules and this is how the options look for me:
In closingPutting a system like this for your image embeds will give you the piece of mind that content editors have options to choose how big or small they would like images to be displayed, and from a performance point of view, if your image styles are done properly, you can rest assurred that bloated images will never be rendered because you have put the guard rails in place to avoid this from happening.
I hope you found this article useful and can put these techniques to use in your own Drupal project. Happy New Year! 🎉 🎊 🎆 👋