Feeds
ComputerMinds.co.uk: My text filter's placeholder content disappeared!
When I upgraded a site from Drupal 10.1 to 10.2, I discovered a particularly serious bug: the login form on our client's site vanished ... which was pretty serious for this site which hid all content behind a login!
We had a custom text format filter plugin to render the login form in place of a custom token in text that editors set, on one of the few pages that anonymous users could access. Forms can have quite different cacheability to the rest of a page, and building them can be a relatively expensive operation anyway, so we used placeholders which Drupal can replace 'lazily' outside of regular caching:
class MymoduleLoginFormFilter extends FilterBase implements TrustedCallbackInterface { public function process($text, $langcode) { $result = new FilterProcessResult($text); $needle = '[login_form]'; // No arguments needed as [login_form] is always to be replaced with the same form. $arguments = []; $replace = $result->createPlaceholder(self::class . '::renderLoginForm', $arguments); return $result->setProcessedText(str_replace($needle, $replace, $text)); } public static function renderLoginForm() { // Could be any relatively expensive operation. return \Drupal::formBuilder()->getForm(UserLoginForm::class); } public static function trustedCallbacks() { return ['renderLoginForm']; } }But our text format also had core's "Correct faulty and chopped off HTML" filter enabled - which completely removed the placeholder, and therefore the form went missing from the final output!
Debugging this to investigate was interesting - it took me down the rabbit hole of learning more about PHP 8 Fibers, as Drupal 10.2 uses them to replace placeholders. Initially, I thought the problem could be there, but it turned out that the placeholder itself was the problem. Drupal happily generated the form to go in the right place, but couldn't find the placeholder. Here's what a placeholder, created by FilterProcessResult::createPlaceholder() should look like:
<drupal-filter-placeholder callback="Drupal\mymodule\Plugin\Filter\MymoduleLoginFormFilter::renderLoginForm" arguments="" token="hqdY2kfgWm35IxkrraS4AZx6zYgR7YRVmOwvWli80V4"></drupal-filter-placeholder>Looking very carefully, I spotted that the arguments="" attribute in the actual markup was just arguments - i.e. it had been turned into a 'boolean' HTML attribute:
<drupal-filter-placeholder callback="Drupal\mymodule\Plugin\Filter\MymoduleLoginFormFilter::renderLoginForm" arguments token="hqdY2kfgWm35IxkrraS4AZx6zYgR7YRVmOwvWli80V4"></drupal-filter-placeholder>There is a limited set of these, and yet the masterminds/html5 component that Drupal 10.2 now uses to process HTML 5 requires an explicit list of the attributes that should not get converted to boolean attributes when they are set to an empty string.
At this point, I should point out that this means a simple solution could be to just pass some arguments so that the attribute isn't empty! That is a nice immediate workaround that avoids the need for any patch, so is an obvious maintainable solution:
// Insert your favourite argument; any value will do. $arguments = [42];At least that ensures our login form shows again!
But I don't see any documentation saying there must be arguments, and it would be easy for someone to write this kind of code again elsewhere, especially if we're trying to do The Right Thing by using placeholders in filters.
So I decided to contribute a fix back to Drupal core. I've worked on core before. Sometimes it's a joy to find or fix something that affects thousands of people, other times the contribution process can be soul-destroying. At least in this case, I found an existing test in core that could be easily extended to demonstrate the bug. Then I wrote a surgical fix... but I can see that it tightly couples the filter system to Drupal's HtmlSerializerRules class. That class is within the \Drupal\Component namespace, which is described as:
Drupal Components are independent libraries that do not depend on the rest of Drupal in order to function.
Components MAY depend on other Drupal Components or external libraries/packages, but MUST NOT depend on any other Drupal code.
So perhaps it needs configuration in order to be decoupled; and/or a factory service; or maybe modules should subscribe to an event to be able to inject their own rules .... and very quickly perfection feels like the enemy of good, as I can imagine the scope of a solution ballooning in size and complexity.
I'm all for high standards in core, but fulfilling them to produce solutions can still be a slow and frustrating experience. I'm already involved in enough long-running issues that just bounce around between reviewers, deprecations and changes in standards. I risk just ranting here rather than providing answers - and believe me, I'm incredibly grateful for the work that reviewers and committers have put into producing Drupal - but surely the current process must be putting so many potential contributors off. We worry about attracting talent to the Drupal ecosystem, and turning Takers into Makers, but what are they going to find when they arrive? Contributing improvements of decent size is hard and can require perseverance over years. Where can we adjust the balance to make contribution easier for anyone, even seasoned developers?
As I suggested, perhaps this particular bug needs any of a factory pattern, event subscriber, or injected configuration... but what would my next step be? I'm reluctant to put effort into writing a more complex solution when I know from experience that reviewers might just suggest doing something different anyway. At least I have that simple (if unsatisfying) workaround for the filter placeholder method: always send an argument, even if it might be ignored. I guess that reflects the contribution experience itself sometimes!
Sahil Dhiman: First Iteration of My Free Software Mirror
As I’m gearing towards setting up a Free Software download mirror in India, it occurred to me that I haven’t chronicled the work and motivation behind setting up the original mirror in the first place. Also, seems like it would be good to document stuff here for seeing the progression, as the mirror is going multi-country now. Right now my existing mirror i.e., mirrors.de.sahilister.net, which was mirrors.sahilister.in, hosted in Germany serves traffic for Termux, NomadBSD, Blender, BlendOS and GIMP. For a while in between, I hosted OSMC project mirror as well.
To explain what is a Free Software download mirror thing is first, I’ll quote myself from work blog -
As most Free Software doesn’t have commercial backing and require heavy downloads, the concept of software download mirrors helps take the traffic load off of the primary server, leading to geographical redundancy, higher availability and faster download in general.
So whenever someone wants to download a particular (mirrored) software and click download, upstream redirects the download to one of the mirror server which is geographical (or in other parameters) nearby to the user, leading to faster downloads and load sharing amongst all mirrors.
Since the time I got into Linux and servers, I always wanted to help the community somehow, and mirroring seemed to be the most obvious thing. India seems to be a country which has traditionally seen less number of public download mirrors. IITB, TiFR, and some of the public institutions used to host them for popular Linux and Free Softwares, but they seem to be diminishing these days.
In the last months of 2021, I started using Termux and saw that it had only a few mirrors (back then). I tried getting a high capacity, high bandwidth in budget was hard in India in 2021-22. So after much deliberation I decided to go where it’s available and chose a German hosting provider with the thought helping where possible and adding India node which conditions are favorable here (thankfully that happened, and India node is live too now.). Termux required only 29 GB of storage, so went ahead and started mirroring it. I raised this issue in Termux’s GitHub repository in January 2022. This blog post chronicles the start of the mirror.
Termux has high request counts from a mirror point of view. Each Termux client, usually check each mirror in selected group for availability before randomly selecting one for download (only other case is when client has explicitly selected a single mirror using termux-repo-change). The mirror stared getting thousands of requests daily, but only a small percentage would actually get my mirror in selection, so download traffic was lower. Similar thing happened with OSMC too (which I started mirroring later).
With this start, I started exploring various project that would be benefit from additional mirrors. Public information from Academic Computer Club in Umeå’s mirror and Freedif’s mirror stats helped to figure out storage and bandwidth requirements for potential projects.
Later, I migrated to a different provider for better speeds and added LibreSpeed test on the mirror server. Those were fun times. Between OSMC, Termux and LibreSpeed, I was getting almost 1.2 millions hits/day on the server at its peak, crossing for the first time a TB/day traffic number.
Next came Blender, which took the longest time to set up, around 9–10 months. Blender had a push-trigger requirement for rsync from upstream that took quite some back and forth. It now contributes the most amount of traffic on my mirror. On release days, mirror does more than 3 TB/day and normal days, it hovers around 2 TB/day. Gimp is the latest addition to the mirror.
At one time, the mirror traffic touched 4.97 TB/day. That’s when I decided on dropping LibreSpeed server to solely focus on mirroring for now, keeping the bandwidth allotment for serving downloads for now.
The mirror project selection grew organically. I used to reach out many projects discussing the need of for additional mirrors. Some projects outright denied mirroring request as Germany already has good academic mirrors boosting 20-25 Gbits/s speeds from FTP era, which seems fair. Finding the niche was essential to only add softwares, which truly required additional capacity. There were months when nothing much would happen with the mirror, rsync would continue to update the mirror while nginx would keep on serving the traffic. Nowadays, the mirror pushes around 70 TB/month. I occasionally check logs, vnstat, add new security stuff here and there and pay the bills. The mirror now saturates the Gigabit link sometimes and goes beyond that, peaking around 1.42 Gbits/s (the hosting provider seems to be upping their game). The plan is to upgrade the link to better speeds.
Yearly traffic stats (through `vnstat -y`)On the way, learned quite a few things like -
- IPv6 and exposing rsync module due to OSMC requirement.
- Implementing user with restricted access to only trigger rsync, basically make rsync pull trigger based due to Blender.
- Iterating over right client response size for LibreSpeed test project.
- Mistakenly identifying torrent traffic for BlendOS as DDoS and blocking it for quite a few months. BlendOS added loads of hits for torrent traffic, making my mirror also serve as web seed. Web seeds in conjunction with normal seeds surely is a good combination for serving downloads as it combines best of both world, general availability of web seed/mirror and benefit of traditional seeds to maximize download speeds at user end.
- Handling abusive traffic (a lot of it to be frank) and how to handle it. The approach is more of a whack a mole right now, which I want to improve and automate.
- Most of the traffic on non-Linux/BSDs operating system serving mirrors (like mine) is for people on Windows and Mac asking for EXEs and DMGs. Mostly because package repositories carry software distribution load for Linux/BSDs OS and partly because the number of Windows/Mac users are quite high compared to other OSs.
- Load balancing through DNS and HTTP redirection (which I would implement in my India mirror now) to better maximize available resources.
Generated from IPinfo.io
Fun fact, Academic Computer Club in Umeå which runs mirror.accum.se (one of the prominent Debian, Ubuntu etc.) mirror, now has 200 Gbits/s uplink to the internet through SUNET.
In hindsight, the stats look amazing, hundreds of TBs of traffic served from the mirror. That does show that there’s still quite an appetite for public mirrors in time of commercially “donated” CDNs and GitHub. The world could have done with one less mirror, but it saved some time, lessened the burden for others, while providing redundancy and traffic localization with one additional mirror. And it’s fun for someone like me who’s into infrastructure that powers the Internet. Now, I’ll try focusing and expanding the India mirror, which in itself started pushing almost half a TB/day. Long live Free Software and public download mirrors.
PyCoder’s Weekly: Issue #634 (June 18, 2024)
#634 – JUNE 18, 2024
View in Browser »
Python’s use of semantic style versioning numbers causes confusion, as breaking changes can be present in the “minor” number position. This proposal given at the Python Language Summit is to switch to using calendar based versioning. A PEP is forthcoming.
PYTHON SOFTWARE FOUNDATION
In this tutorial, you’ll learn the basic characteristics and operations of Python mappings. You’ll explore the abstract base classes Mapping and MutableMapping and create a custom mapping.
REAL PYTHON
Data scientists use Posit Connect to get their work into the hands of decision-makers. Securely deploy python analytic work & distribute that across teams. Publish data apps, documents, notebooks, and dashboards. Deploy models as APIs & configure reports to run & get distributed on a custom schedule →
POSIT sponsor
The long awaited 2.0 release of NumPy landed this week. Not all the docs are up to date yet, but this final draft of the release notes shows you what is included.
NUMPY.ORG
In this quiz, you’ll test your understanding of the basic characteristics and operations of Python mappings. By working through this quiz, you’ll revisit the key concepts and techniques of creating a custom mapping.
REAL PYTHON
As part of the upcoming board election, three new bylaws are also being proposed for your consideration. The first makes it easier to qualify for membership for Python-related volunteer work, the second makes it easier to vote, and the third gives the board more options around the code of conduct.
PYTHON SOFTWARE FOUNDATION
In this video course, you’ll explore how to use a Python interface. You’ll come to understand why interfaces are so useful and learn how to implement formal and informal interfaces in Python. You’ll also examine the differences between Python interfaces and those in other programming languages.
REAL PYTHON course
Rest easy with Judoscale’s web & worker autoscaling for Heroku, Render, and Amazon ECS. Traffic spike? Scaled up. Quiet night? Scaled down. Work queue backlog? No problem →
JUDOSCALE sponsor
In this video course, you’ll be examining a couple of methods to get a list of files and folders in a directory with Python. You’ll also use both methods to recursively list directory contents. Finally, you’ll examine a situation that pits one method against the other.
REAL PYTHON course
Logging levels allow you to control which messages you record in your logs. Think of log levels as verbosity levels. How granular do you want your logs to be? This article teaches you how to control your logging.
MIKE DRISCOLL
You may get paid for 8 hours a day, but that doesn’t necessarily mean you’re coding that whole time. This article touches on the variety of the job and what you should expect if you are new to the field.
BITE CODE!
A summary of the six lightning talks given at the 2024 Python Language Summit. Topics include Python for iOS, improving asserts in 3.14, sharing data between sub-interpreters, and more.
PYTHON SOFTWARE FOUNDATION
Learn how to start and stop uvicorn in the background using a randomly selected free port number. Useful for running test suites that require live-webservers.
CHRISTOPH SCHIESSL • Shared by Christoph Schiessl
This article describes Nicolò’s project to build a bot that retrieves images from Wikimedia, selecting the best ones, and deploying it to the cloud.
NICOLÒ GISO • Shared by Nicolò Giso
This article shows real life techniques and fixtures needed to make the test suite of your MongoDB and AWS-based application usable and performant.
HANDMADESOFTWARE • Shared by Thorin Schiffer
Thibaud shares some of the best moments of DjangoConEU 2024. He highlights some of the talks, workshops, and the outcome of the sprints.
THIBAUD COLAS
This tutorial shows how to configure Django to load and serve up static and media files, public and private, via DigitalOcean Spaces.
MICHAEL HERMAN
A detailed code walkthrough of how CPython implements memory management, including reference counting and garbage collection
ABHINAV UPADHYAY
“Software has never looked cooler, but user interface design and user experience have taken a sharp turn for the worse.”
NICK HODGES
This article dives into the structure of the popular ruff Python linter written in Rust.
ABDUR-RAHMAAN JANHANGEER • Shared by Abdur-Rahmaan Janhangeer
June 19, 2024
REALPYTHON.COM
June 20 to June 23, 2024
WAGTAIL.SPACE
June 20, 2024
MEETUP.COM
June 20, 2024
PYLADIES.COM
June 21 to June 22, 2024
MEETUP.COM
June 22 to June 24, 2024
BARCAMPS.EU
Happy Pythoning!
This was PyCoder’s Weekly Issue #634.
View in Browser »
[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]
PyBites: Learn Python From Scratch: We Extended Our Newbie Bite Exercises From 25 to 50 🐍 📈
We are excited to announce that we’ve extended our Newbie Bites from 25 to 50 Python exercises!
The importance of exercising when learning how to codeWe’re passionate about this new batch of exercises because they require active engagement, which is crucial for learning how to code. Passive methods like reading books or watching videos don’t help concepts click or stick.
Our exercises involve writing code that is validated with pytest. This immediate feedback helps you understand mistakes and learn more effectively. You’ll encounter errors, re-think your approach, and practice deliberately.
Why double the number of Newbie exercises?The first 25 exercises taught the fundamentals well, but many found it hard to tackle the intro and beginner Bites afterward. The extra 25 Newbie Bites (#26-50) bridge the gap between complete beginners and intermediate Python programmers.
These new exercises cover essential concepts like error handling, type hints, default arguments, special characters, working with dates, classes (!), list comprehensions, constants, exception handling, and more.
We believe these challenges will provide a deeper understanding and more robust skill set to tackle the regular Bites and become more proficient in Python.
Get full access to the Newbie Bites Overview of the new exercisesReading Errors: Learn to read and understand error messages in Python. These messages provide valuable information for debugging and fixing issues efficiently.
Failing Tests: Practice reading and interpreting failing test outputs with the pytest framework. This skill is crucial for resolving Bites and any Python development.
Type Hints: Explore type hints introduced in Python 3.5, which help you write more readable and maintainable code by specifying expected data types.
Default Arguments: Understand how to define functions with default values, making your functions more flexible and easier to use.
Special Chars: Learn about special characters in Python strings, such as \n and \t, for better formatting and readability.
Word Count: Use string methods like .split() and .splitlines() to manipulate and process text data effectively.
Dict Retrieval – Part 2: Explore advanced techniques for retrieving values from dictionaries to enhance your data handling skills.
Dict Retrieval – Part 3: Learn safer methods to retrieve values from dictionaries, providing defaults if keys are not present.
Random Module: Use Python’s random module to write a number guessing game, showcasing the practical use of standard library modules.
Working With Dates – Part 1: Explore the datetime module, focusing on the date class and the .weekday() method to work with dates.
Working With Dates – Part 2: Continue working with the datetime module, focusing on importing specific objects versus entire modules.
Make a Class: Learn about Python classes, which serve as blueprints for creating objects, starting from the basics.
Class With Str: Build upon the previous exercise by learning special methods like __str__ for adding string representations to classes.
Make Dataclass: Simplify class creation with Python dataclasses, introduced in Python 3.7.
Scope: Understand variable scope to write clearer and less error-prone code. Scope determines the visibility and lifespan of variables.
String Manipulations: Practice fundamental string manipulations, essential for processing and transforming text data.
List Comprehension: Learn to write concise and efficient list comprehensions, a powerful feature in Python for creating new lists.
Named Tuple: Explore namedtuples, which allow attribute access to elements and support type hints, enhancing your data handling capabilities.
Constants: Learn to assign and use constants, which are fixed values that improve code readability and maintainability.
Exceptions: Master exception handling to write resilient code and establish clear boundaries for function callers.
For Loop With Break And Continue: Control loop flow using break and continue statements to manage iterations based on conditions.
In Operator: Use the in operator to check for item presence in collections, a common practice in Python programming.
String Module: Combine list comprehensions with the string module to check and manipulate characters, then join them back into strings.
Formatting Intro: Learn string interpolation using the .format() method to insert variables into strings dynamically.
Formatting Intro: Learn string interpolation using the .format() method to insert variables into strings dynamically.
Buy the Newbie exercise bundleThe exercises build upon each other, so they must be done in order. After completing them, you’ll earn your Pybites Newbie Certificate.
We’re also working on a series of screencasts to further explain the second half of the Newbie Bites. Stay tuned by subscribing to our YouTube channel.
About the Pybites PlatformWe offer a rich platform for learning Python through practical exercises.
Our Bite exercises are designed to challenge you and help you apply Python in real-world scenarios, ranging from basic syntax to advanced topics.
Whether you’re a beginner or an experienced coder, our exercises aim to improve your coding skills in an effective and fun manner.
Start coding today: https://codechalleng.es/
And join our growing community of passionate Pythonistas: https://pybites.circle.so/
Wim Leers: XB week 5: chaos theory
We started the week off with a first MVP config entity: the component config entity. We have to start somewhere and then iterate — and iterate we can: thanks to Felix “f.mazeikis”, #3444417 landed! Most crucially this means we now have both a low-level unit test and a kernel test that ensures the validation for this config entity is thorough — of course I’m bringing config validation to Experience Builder (XB) from the very start!
It doesn’t do much: it allows “opting in” single directory components (SDCs) to XB, and now only those are available in the PoC admin UI. Next up is #3452397: expanding the admin UI and the config entity to allow defining default values for its SDC props. This will allow the UI to immediately generate a preview of the component, which otherwise may (depending on the component) not render anything visible (imagine: a <h2> without text, an <img> without an image, etc.).
But literally everything about this config entity is subject to change:
- its name (issue: #3455036)
- the whole “opting in” principle (issue comment: #3444424-7
- what it does exactly (issue: #3446083)
- whether it should handle only exposing SDCs 1:1 in XB, or whether it should also handle other component types (issue: #3454519)
Which brings me to the main subject for this week: I have a theory that explains the chaos that I think now many of you see.
We rushed to open the 0.x branch to invite contribution, because Dries announced XB at DrupalCon. This wrongly gave the impression that there was a very detailed plan, with a precise architecture in mind. That is not the case, and mea culpa for giving that impression. That branch was simply the combination of different PoCs merged together.
Some additional context: back in March, from one day to the next, I along with Ben “bnjmnm”, Ted “tedbow”, Alex “effulgentsia”, Tim and Lauri, plus Felix and Jesse (both from the Site Studio team, both very experienced) I were asked to stop everything we were doing (for me: dropping config validation) and start estimating the product requirements.
~4 weeks of non-stop grueling meetings 1 to assess and estimate the 64 requirements, with Lauri providing visual examples and additional context for the product requirements he crafted (after careful research: #3452440). We looked at multiple radical options and evaluated which of these was viable:
- building on top of one of the page building solutions that already exist in the Drupal ecosystem
- partnering with one of the existing page building systems, if they were willing to relicense under GPL-2+ and we’d be able to fit them onto Drupal’s data modeling strengths
- building something new (but reusing the parts in the Drupal ecosystem that we can build on top of, such as SDC, field types for data modeling, etc.)
The result: 60 pages worth of notes 2, with for each of the 42 “Required for MVP” product requirements (out of a total of 64), there now was an outline of how it could be implemented, a best+realistic+worst case estimate (week-level accuracy), which skillsets needed and how many people. It’s during this process that it became clear that only one choice made sense: the last one. That was the conclusion we reached because the existing solutions in the ecosystem fall short (see #3452440), Site Studio does not meet all of XB’s requirements and some of its architectural choices conflict with XB’s requirements and we did not end up finding a magical unicorn that meshed perfectly with Drupal.
It’s during this time that I made my biggest mistake yet: because the request for estimating this was coming from within Acquia, I assumed this was intended to be built by a team of Acquians. Because … why else create estimates?! If it’s built by a mix of paid full-time, paid part-time and volunteer community members, there’s little point in estimating, right?!
Well, I was wrong: turns out the intent was to get a sense of how feasible it was to achieve in roughly which timeline. But I and my fellow team members were so deep into this enormous estimation exercise based on very high-level requirements that thinking about capturing this information in a more shareable form was a thought that simply did not occur…
So, choice 3 it was, with people that have deep Layout Builder knowledge (Ted & Tim), while bringing in expertise from the Site Studio team (Jesse & Felix) and strong front-end expertise (Ben & Harumi “hooroomoo”) … because next we were asked to narrow the estimates for the highest-variance estimates, to bring it down to a higher degree of confidence. That’s where the “FTEs”, “Best Case Size (weeks)”, “Realistic Case Size (weeks)”, “Worst Case Size (weeks)”, “Most Probable Case Size (calculated, FTE weeks)”, “Variance” columns in the requirements spreadsheet come in.
For the next ~4 weeks, we built PoCs for the requirements where our estimates had either the highest variance or the highest number, to both narrow and reduce the estimates. That’s where all the branches on the experience_builder project come from! For example:
- Harumi’s poc-codemirror for editing component styles (not in 0.x)
- Felix’ research__component_export_poc to verify we could actually transport default file content as part of the regular config import/export system (not in 0.x)
- Ben’s cypress_starter
- Ben’s easyblocks_backend (not in 0.x)
- Jesse’s poc_reactrouter (not in 0.x)
- Ted’s research_sb_config for config-defined SDCs (not in 0.x)
- my research__design_versions (not in 0.x)
- …
After the ~8 chaotic weeks above, it was DrupalCon (Dries also wrote about this, just before DrupalCon, based on the above work!), and then … it was week 1.
In that first week, we took a handful of branches that seemed to contain the pieces most sensible to start iterating on an actual implementation, threw that into a witch pot, and called that 0.x!
Now that all of you know the full origin story, and many of you have experienced the subsequent ~4 equally chaotic weeks, you’re all caught up! :D
Missed a prior week? See all posts tagged Experience Builder.
Goal: make it possible to follow high-level progress by reading ~5 minutes/week. I hope this empowers more people to contribute when their unique skills can best be put to use!
For more detail, join the #experience-builder Slack channel. Check out the pinned items at the top!
Going forward: order from chaosSo, how are we going to improve things?
- Tuesday: issue queue 100% triaged — each issue that is for a later phase is now Postponed and each issue that needs input from someone has that person assigned.
- Friday: adopt Architecture Decision Records (ADRs): #3454669) — many thanks to Lullabot for leading the way
- Friday: the first high-level diagrams landed, and thanks to Kyle “ctrlADel”, they follow the C4 model
- More to come in week 6!
So besides the backstory and attempts to bring more order & overview, what happened?
The UI gained panning support (#3452623), which really makes it feel like it’s coming alive:
Try it yourself locally if you like, but there’s not much you can do yet.
Install the 0.x branch — the “Experience Builder PoC” toolbar item takes you there!
… you can also try an imperfect/limited version of the current UI on the statically hosted demo UI thanks to #3450311 by Lee “larowlan”.
We also got two eslint CI jobs running now (#3450307): one for the typical Drupal stuff, one for the XB UI (which is written in TypeScript) and reduced the overhead of PHPStan Level 8 (again thanks to Lee, in #3454501) … and a bunch more low-level things that were improved.
Various things are in progress … expect an update for those next week :)
Thanks to Lauri for reviewing this!
Nonprofit Drupal posts: June Drupal for Nonprofits Chat
Join us THURSDAY, June 20 at 1pm ET / 10am PT, for our regularly scheduled call to chat about all things Drupal and nonprofits. (Convert to your local time zone.)
We don't have anything specific on the agenda this month, so we'll have plenty of time to discuss anything that's on our minds at the intersection of Drupal and nonprofits. Got something specific you want to talk about? Feel free to share ahead of time in our collaborative Google doc: https://nten.org/drupal/notes!
All nonprofit Drupal devs and users, regardless of experience level, are always welcome on this call.
This free call is sponsored by NTEN.org and open to everyone.
-
Join the call: https://us02web.zoom.us/j/81817469653
-
Meeting ID: 818 1746 9653
Passcode: 551681 -
One tap mobile:
+16699006833,,81817469653# US (San Jose)
+13462487799,,81817469653# US (Houston) -
Dial by your location:
+1 669 900 6833 US (San Jose)
+1 346 248 7799 US (Houston)
+1 253 215 8782 US (Tacoma)
+1 929 205 6099 US (New York)
+1 301 715 8592 US (Washington DC)
+1 312 626 6799 US (Chicago) -
Find your local number: https://us02web.zoom.us/u/kpV1o65N
-
- Follow along on Google Docs: https://nten.org/drupal/notes
Real Python: Rounding Numbers in Python
With many businesses turning to Python’s powerful data science ecosystem to analyze their data, understanding how to avoid introducing bias into datasets is absolutely vital. If you’ve studied some statistics, then you’re probably familiar with terms like reporting bias, selection bias, and sampling bias. There’s another type of bias that plays an important role when you’re dealing with numeric data: rounding bias.
Understanding how rounding works in Python can help you avoid biasing your dataset. This is an important skill. After all, drawing conclusions from biased data can lead to costly mistakes.
In this video course, you’ll learn:
- Why the way you round numbers is important
- How to round a number according to various rounding strategies
- How to implement each strategy in pure Python
- How rounding affects data and which rounding strategy minimizes this effect
- How to round numbers in NumPy arrays and pandas DataFrames
- When to apply different rounding strategies
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
Mike Driscoll: How to Publish a Python Package to PyPI
Do you have a Python package that you’d like to share with the world? You should publish it on the Python Package Index (PyPI). The vast majority of Python packages are published there. The PyPI team has also created extensive documentation to help you on your packaging journey. This article does not aim to replace that documentation. Instead, it is just a shorter version of it using ObjectListView as an example.
The ObjectListView project for Python is based on a C# wrapper .NET ListView but for wxPython. You use ObjectListView as a replacement for wx.ListCtrl because its methods and attributes are simpler. Unfortunately, the original implementation died out in 2015 while a fork, ObjectListView2 died in 2019. For this article, you will learn how I forked it again and created ObjectListView3 and packaged it up for PyPI.
Creating a Package StructureWhen you create a Python package, you must follow a certain type of directory structure to build everything correctly. For example, your package files should go inside a folder named src. Within that folder, you will have your package sub-folder that contains something like an __init__.py and your other Python files.
The src folder’s contents will look something like this:
package_tutorial/ | -- src/ | -- your_amazing_package/ | -- __init__.py | -- example.pyThe __init__.py file can be empty. You use that file to tell Python that the folder is a package and is importable. But wait! There’s more to creating a package than just your Python files!
You also should include the following:
- A license file
- The pyproject.toml file which is used for configuring your package when you build it, among other things
- A README.md file to describe the package, how to install it, example usage, etc
- A tests folder, if you have any (and you should!)
Go ahead and create these files and the tests folder. It’s okay if they are all blank right now. At this point, your folder structure will look like this:
package_tutorial/ | -- LICENSE | -- pyproject.toml | -- README.md | -- src/ | -- your_amazing_package/ | -- __init__.py | -- example.py | -- tests/ Picking a Build BackendThe packaging tutorial mentions that you can choose various build backends for when you create your package. There are examples for the following:
- Hatchling
- setuptools
- Flit
- PDM
You add this build information in your pyproject.toml file. Here is an example you might use if you picked setuptools:
[build-system] requires = ["setuptools>=61.0"] build-backend = "setuptools.build_meta"This section in your config is used by pip or build. These tools don’t actually do the heavy lifting of converting your source code into a wheel or other distribution package. That is handled by the build backend. You don’t have to add this section to your pyproject.toml file though. You will find that pip will default to setuptools if there isn’t anything listed.
Configuring MetadataAll the metadata for your package should go into a pyproject.toml file. In the case of ObjectListView, it didn’t have one of these files at all.
Here’s the generic example that the Packaging documentation gives:
[project] name = "example_package_YOUR_USERNAME_HERE" version = "0.0.1" authors = [ { name="Example Author", email="author@example.com" }, ] description = "A small example package" readme = "README.md" requires-python = ">=3.8" classifiers = [ "Programming Language :: Python :: 3", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", ] [project.urls] Homepage = "https://github.com/pypa/sampleproject" Issues = "https://github.com/pypa/sampleproject/issues"Using this as a template, I created the following for the ObjectListView package:
[project] name = "ObjectListView3" version = "1.3.4" authors = [ { name="Mike Driscoll", email="mike@somewhere.org" }, ] description = "An ObjectListView is a wrapper around the wx.ListCtrl that makes the list control easier to use." readme = "README.md" requires-python = ">=3.9" classifiers = [ "Programming Language :: Python :: 3", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", ] [project.urls] Homepage = "https://github.com/driscollis/ObjectListView3" Issues = "https://github.com/driscollis/ObjectListView3/issues"Now let’s go over what the various parts are in the above metadata:
- name – The distribution name of your package. Make sure your name is unique!
- version – The package version
- authors – One or more authors including emails for each
- description – A short, one-sentence summary of your package
- readme – A path to the file that contains a detailed description of the package. You may use a Markdown file here.
- requires-python – Tells which Python versions are supported by this package
- classifiers – Gives an index and some metadata for pip
- URLs – Extra links that you want to show on PyPI. In general, you would want to link to the source, documentation, issue trackers, etc.
You can specify other information in your TOML file if you’d like. For full details, see the pyproject.toml guide.
READMEs and LicensesThe README file is almost always a Markdown file now. Take a look at other popular Python packages to see what you should include. Here are some recommended items:
- How to install the package
- Basic usage examples
- Link to a guide or tutorial
- An FAQ if you have one
There are many licenses out there. Don’t take advice from anyone; buy a lawyer on that unless you know a lot about this topic. However, you can look at the licenses for other popular packages and use their license or one similar.
Generating a PackageNow you have the files you need, you are ready to generate your package. You will need to make sure you have PyPA’s build installed.
Here’s the command you’ll need for that:
python3 -m pip install --upgrade buildNext, you’ll want to run the following command from the same directory that your pyproject.toml file is in:
python3 -m buildYou’ll see a bunch of output from that command. When it’s done, you will have a dist A folder with a wheel (*.whl) file and a gzipped tarball inside it is called a built distribution. The tarball is a source distribution , while the wheel file is called a built distribution. When you use pip, it will try to find the built distribution first, but pip will fall back to the tarball if necessary.
Uploading / Publishing to PyPIYou now have the files you need to publish to share your package with the world on the Python Package Index (PyPI). However, you need to register an account on TestPyPI first. TestPyPI is a separate package index intended for testing and experimentation, which is perfect when you have never published a package before. To register an account, go to https://test.pypi.org/account/register/ and complete the steps on that page. It will require you to verify your email address, but other than that, it’s a very straightforward signup.
To securely upload your project, you’ll need a PyPI API token. Create one from your account and make sure to set the “Scope” to the “Entire account”. Don’t close the page until you have copied and saved your token or you’ll need to generate another one!
The last step before publishing is to install twine, a tool for uploading packages to PyPI. Here’s the command you’ll need to run in your terminal to get twine:
python3 -m pip install --upgrade twineOnce that has finished installing, you can run twine to upload your files. Make sure you run this command in your package folder where the new dist folder is:
python3 -m twine upload --repository testpypi dist/*You will see a prompt for your TestPyPI username and/or a password. The password is the API token you saved earlier. The directions in the documentation state that you should use __token__ as your username, but I don’t think it even asked for a username when I ran this command. I believe it only needed the API token itself.
After the command is complete, you will see some text stating which files were uploaded. You can view your package at https://test.pypi.org/project/example_package_name
To verify you can install your new package, run this command:
python3 -m pip install --index-url https://test.pypi.org/simple/ --no-deps example-package-nameIf you specify the correct name, the package should be downloaded and installed. TestPyPI is not meant for permanent storage, though, so it will likely delete your package eventually to conserve space.
When you have thoroughly tested your package, you’ll want to upload it to the real PyPI site. Here are the steps you’ll need to follow:
- Pick a memorable and unique package name
- Register an account at https://pypi.org. You’ll go through the same steps as you did on TestPyPI
- Be sure to verify your email
- Create an API key and save it on your machine or in a password vault
- Use twine to upload the dist folder like this: python -m twine upload dist/*
- Install the package from the real PyPI as you normally would
That’s it! You’ve just published your first package!
Wrapping UpCreating a Python package takes time and thought. You want your package to be easy to install and easy to understand. Be sure to spend enough time on your README and other documentation to make using your package easy and fun. It’s not good to look up a package and not know which versions of Python it supports or how to install it. Make the process easy by uploading your package to the Python Package Index.
The post How to Publish a Python Package to PyPI appeared first on Mouse Vs Python.
The Drop Times: Christina Lockhart on Elevating Drupal: The Essential Role of Women in Tech
DrupalEasy: Two very different European Drupal events in one week
I was fortunate enough to attend two very different European Drupal events recently, and wanted to take a few minutes to share my experience as well as thank the organizers for their dedication to the Drupal community.
The events couldn't have been more different - one being a first-time event in a town I've never heard of and the other celebrating their 20th anniversary in a city that can be considered the crossroads of the Netherlands.
DrupalCamp CemaesFirst off - it is pronounced KEM-ice (I didn't learn this until I actually arrived). Cemaes is a small village at the very northern tip of Wales with a population of 1,357. Luckily, one of those 1,357 people is Jennifer Tehan, an independent Drupal developer who decided to organize a Drupal event in a town that (I can only assume) has more sheep than people.
When this event was first announced a few months back, I immediately knew that this was something I wanted to attend - mostly because I've always wanted to visit Wales to do some hiking ("walking," if you're not from the United States.) I used the camp as an excuse for my wife and I to take a vacation and explore northern Wales. I could write an entire other blog post on how amazeballs the walking was…
DrupalCamp Cemaes was never going to be a large event, and it turns out there wound up being only nine of us (ten if you count Frankie, pictured below.) The advantage of such a small event was easy to see - we all got to know each other quite well. There's something really nice - less intimidating - about small Drupal events that I realized that I've really missed since the decline of Drupal meetups in many localities.
The event itself was an unconference, with different people presenting on Drupal contributions (James Shields), GitHub Codespaces (Rachel Lawson), remote work (Jennifer Tehan), the LocalGov distribution (Andy Broomfield), and Starshot (myself.)
The setting of the event couldn't have been more lovely - an idyllic seaside community where everything was within walking distance, including the village hall where the event took place. If Jennifer decides to organize DrupalCamp Cemaes 2025, I'm pretty sure Gwendolyn and I will be there.
Read a brief wrap-up of DrupalCamp Cemaes from Jennifer Tehan.
This camp was proof that anyone, anywhere can organize a successful Drupal event with a minimum of fuss.
DrupaljamFour days later, I was in Utrecht, Netherlands, at the 20th anniversary of Drupaljam, the annual main Dutch Drupal event.
I had previously attended 2011 Drupaljam and recall two things about that event: the building it took place in seemed like it was from the future, and many of the sessions were in Dutch.
This was a very different event from the one I had been at a few days earlier, in a city of almost 400,000 people with over 300 attendees (I was told this number anecdotally) in one of the coolest event venues I've ever been to. Defabrique is a renovated linseed oil and compound feed factory that is a bit over 100 years old that is (IMHO) absolutely perfect for a Drupal event. Each and every public space has character, yet is also modern enough to support open-source enthusiasts. On top of all that, the food was amazing.
Attending Drupal Jam was a bit of a last minute decision for me, but I'm really glad that I made the time for it. I was able to reconnect with members of the European Drupal community that I don't often see, and make new connections with more folks that I can count.
I spent the day the way I spend most of my time at Drupal events - alternating between networking and attending sessions that attracted me. There were a number of really interesting AI-related sessions that I took in; it's pretty obvious to me that the Drupal community is approaching AI integrations in an unsurprisingly thoughtful manner.
The last week reinforced to me how fortunate I am to be able to attend so many in-person Drupal events. The two events I was able to participate in couldn't have been more different in scale and scope, but don't ask me to choose my favorite, because I'm honestly not sure I could!
Real Python: Quiz: Build a Guitar Synthesizer
In this quiz, you’ll test your understanding of what it takes to build a guitar synthesizer in Python. By working through this quiz, you’ll revisit a few key concepts from the music theory and sound synthesis.
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
CodersLegacy: Adding Data Files in Nuitka (Python Files, Images, etc)
Nuitka is a Python-to-C compiler that converts Python code into executable binaries. While Nuitka efficiently compiles Python scripts, incorporating data files such as images, audio, video, and additional Python files requires can be a little tricky. This guide outlines the steps to include various data files in a Nuitka standalone executable.
Before delving into the inclusion of data files, ensure you have Nuitka installed. You can install Nuitka via pip:
pip install nuitkaFor a more detailed starter guide on Nuitka and its various options, you can follow this link to our main Nuitka Tutorial. If you already familiar with Nuitka, then proceed ahead in this tutorial to learn how to add data files in our Nuitka EXE.
Assuming you have a Python script named main.py, the basic command to compile it into a standalone executable using Nuitka is:
python -m nuitka --standalone main.pyThis command creates a standalone directory containing the executable and all necessary dependencies. However, if your program relies on external data files (images, audio, video, other Python files), you need to explicitly include these files.
To include individual files such as images, use the --include-data-files option. The syntax is:
--include-data-files=<source-location>=<target-location>For example, to include an image file favicon.ico, the command is:
--include-data-files=./favicon.ico=favicon.icoThis command tells Nuitka to include the favicon.ico file from the current directory and place it in the same location within the output directory.
To include entire directories, use the --include-data-dir option. The syntax is the same as earlier:
--include-data-dir=<source-location>=<target-location>For instance, to include an entire directory named Images, the command is:
--include-data-dir=./Images=ImagesThis will copy the Images directory from the current location to the output directory, preserving its structure.
Suppose you have a project with the following files and directories:
- main.py: The main Python script.
- favicon.ico: An icon file.
- Images/: A directory containing images.
- audio.mp3: An audio file.
- videos/: A directory containing video files.
- utils.py: An additional Python file used in the project.
To compile this project with all necessary data files included, the command would be:
python -m nuitka --standalone --include-data-files=./favicon.ico=favicon.ico --include-data-files=./audio.mp3=audio.mp3 --include-data-dir=./Images=Images --include-data-dir=./videos=videos --include-data-files=./utils.py=utils.py --disable-console main.pyPretty confusing isn’t it?
Having to write out this command over and over again each time you want to re-build your executable is a massive pain. Instead, I recommend you use Nuitka Configuration Files, which allow you write out all your dependencies/datafiles/imports in a single file, which can then be executed as many times as you like.
By using the --include-data-files and --include-data-dir options, you can include all necessary data files in your Nuitka-compiled executable. This ensures that your standalone application has all the resources it needs to run successfully, whether they are images, audio, video files, or additional Python scripts. Combining these options with plugins and other configurations allows for the creation of robust and self-sufficient executables.
Happy coding!
This marks the end of the Adding Data Files in Nuitka Tutorial. Any suggestions or contributions for CodersLegacy are more than welcome. Questions regarding the tutorial content are more than welcome.
The post Adding Data Files in Nuitka (Python Files, Images, etc) appeared first on CodersLegacy.
Python Bytes: #388 Don't delete all the repos
Specbee: Getting started with integrating Drupal and Tailwind CSS
Week 3 recap - points on a canvas
Plasma 6.1
Plasma 6 hits its stride with version 6.1. While Plasma 6.0 was all about getting the migration to the underlying Qt 6 frameworks correct (and what a massive job that was), 6.1 is where developers start implementing the features that will take you desktop to a new level.
In this release, you will find features that go far beyond subtle changes to themes and tweaks to animations (although there is plenty of those too), as you delve into interacting with desktops on remote machines, become more productive with usability and accessibility enhancements galore, and discover customizations that will even affect the hardware of your computer.
These features and more are being built directly into Plasma's Wayland version natively, avoiding the need for third party software and hacky extensions required by similar solutions implemented in X.
Things will only get more interesting from here. But meanwhile enjoy what will land on your desktop with your next update.
Note: Due to unforeseen circumstances, we have been unable to ship the new wallpaper, "Reef", with this version of Plasma. However, there will be a new wallpaper coming soon in the next 6.2 version.
If you can't wait, you can download "Reef" here.
We apologise for this delay and the inconvenience this may cause.
What's New Access Remote Plasma DesktopsOne of the more spectacular (and useful) features added in Plasma 6.1 is that you can now start up a remote desktop directly from the System Settings app. This means that if you are sysadmin who needs to troubleshoot users' machines, or simply need to work on a Plasma-enabled computer that is out of reach, setting up a connection is just a few clicks away.
Once enabled, you can connect to the remote desktop using a client such as KRDC. You will see the remote machine's Plasma desktop in a window and be able to interact with it from your own computer.
Customization made (more) VisualWe all love customizing our Plasma desktops, don't we? One of the quickest ways to do this is by entering Plasma's Edit Mode (right-click anywhere on the desktop background and select Enter Edit Mode from the menu).
In version 6.1, the visual aspect of Edit Mode has been overhauled and you will now see a slick animation when you activate it. The entire desktop zooms out smoothly, giving you a better overview of what is going on and allowing you to make your changes with ease.
Persistent AppsPlasma 6.1 on Wayland now has a feature that "remembers" what you were doing in your last session like it did under X11. Although this is still work in progress, If you log off and shut down your computer with a dozen open windows, Plasma will now open them for you the next time you power up your desktop, making it faster and easier to get back to what you were doing.
Sync your Keyboard's Colored LEDsIt wouldn't be a new Plasma release without at least one fancy aesthetic customization features. This time, however, we give you the power to reach beyond the screen, all the way onto your keyboard, as you can now synchronize the LED colours of your keys to match the accent colour on your desktop.
The ultimate enhancement for the fashion-conscious user.
Please note that this feature will not work on unsupported keyboards — support for additional keyboards is on the way!
And all this too...-
We have simplified the choices you see when you try to exit Plasma by reducing the number of confusing options. For example, when you press Shutdown, Plasma will only list Shutdown and Cancel, not every single power option.
-
Screen Locking gives you the option of configuring it to behave like a traditional screensaver, as you can choose it not to ask you for a password to unlock it.
-
Two visual accessibility changes make it easier to use the cursor in Plasma 6.1:
-
Shake Cursor makes the cursor grow when you "shake" it. This helps you locate that tiny little arrow on your large, cluttered screens when you lose it among all those windows.
-
Edge Barrier is useful if you have a multi-monitor setup and want to access things on the very edge of your monitor. The "barrier" is a sticky area for your cursor near the edge between screens, and it makes it easier to click on things (if that is what you want to do), rather than having the cursor scooting across to the next display.
-
-
Two major Wayland breakthroughs will greatly improve your Plasma experience:
-
Explicit Sync eliminates flickering and glitches traditionally experienced by NVidia users.
-
Triple Buffering support in Wayland makes animations and screen rendering smoother.
-
Talking Drupal: Talking Drupal #455 - Top 5 uses of AI for Drupal
Today we are talking about AI Tips for Drupal Devs, AI Best Practices, and Drupal Droid with guest Mike Miles. We’ll also cover AI interpolator as our module of the week.
For show notes visit: www.talkingDrupal.com/455
Topics- Top 5 tips
- Idea Generation (Ideation)
- Code Generation
- Debugging
- Content Generation
- Technical Explanations
- How do you suggest people use AI for Ideation
- Is MIT Sloan using AI to help with Drupal Development
- Does that code get directly inserted into your sites
- What are some common pitfalls
- Is your team using AI for debugging
- Any best practices you have found helping when working with AI
- Is MIT Sloan using AI for content generation
- What is an example of how you use AI for technical explanations
- What is your view ont he future of AI in Drupal, do you think AI will replace Drupal developers
- Drupal Droid
- Talking Drupal - Drupal Droid
- Workflows of ai
- AI guidelines
- Using AI as a Development Tool
- Drupal Starshot: Delivering on a promise to Ambitious Site Builders
Michael Miles - mike-miles.com mikemiles86
HostsNic Laflin - nLighteneddevelopment.com nicxvan John Picozzi - epam.com johnpicozzi Randy Fay - rfay
MOTW CorrespondentMartin Anderson-Clutz - mandclu.com mandclu
- Brief description:
- Have you ever wanted to use AI to help populate entity fields that were left blank by Drupal content authors? There’s a module for that.
- Module name/project name:
- Brief history
- How old: created in Sep 2023 by Marcus Johansson of FreelyGive
- Versions available: 1.0.0-rc4
- Maintainership
- Actively maintained, recent release in the past month
- Security coverage - opted in, needs stable release
- Test coverage
- Documentation - User guide
- Number of open issues: 18 open issues, none of which are a bugs
- Usage stats:
- 94 sites
- Module features and usage
- In scientific fields, interpolation is the process of using known data to extrapolate or estimate unknown data points. In a similar way this module helps your Drupal site provide values for fields that didn’t receive input, based on the information that was provided.
- Fundamentally Interpolator AI provides a framework and an API, and then relies on companion modules for processing, either by leveraging third-party services like AI LLMs, or PHP-based scripting.
- There are existing integrations with a variety of AI services, including OpenAI, Dreamstudio, Hugging Face, and more.
- You can add retrievers to help extract and normalize the content you’re processing, for example photos from an external site, and other tools to help normalize and optimize content and media, and optimize any prompts you will be using with AI services.
- You can also extend the workflow capabilities of AI Interpolator, for example using the popular and powerful ECA module that we’ve talked about before on this show.
PyBites: Deploying a FastAPI App as an Azure Function: A Step-by-Step Guide
In this article I will show you how to deploy a FastAPI app as a function in Azure. Prerequisites are that you have an Azure account and have the Azure CLI installed (see here).
Setup AzureFirst you need to login to your Azure account:
az loginIt should show your subscriptions and you select the one you want to use.
Create a resource groupLike this:
az group create --name myResourceGroup --location eastusA resource group is a container that holds related resources for an Azure solution (of course use your own names throughout this guide).
It comes back with the details of the resource group you just created.
Create a storage accountUsing this command:
az storage account create --name storage4pybites --location eastus --resource-group myResourceGroup --sku Standard_LRSNote that for me, to get this working, I had to first make an MS Storage account, which I did in the Azure portal.
The storage account name can only contain lowercase letters and numbers. Also make sure it’s unique enough (e.g. mystorageaccount was already taken).
Upon success it comes back with the details of the storage account you just created.
Create a function appNow, create a function app.
az functionapp create --resource-group myResourceGroup --consumption-plan-location eastus --runtime python --runtime-version 3.11 --functions-version 4 --name pybitesWebhookSample --storage-account storage4pybites --os-type Linux Your Linux function app 'pybitesWebhookSample', that uses a consumption plan has been successfully created but is not active until content is published using Azure Portal or the Functions Core Tools. Application Insights "pybitesWebhookSample" was created for this Function App. You can visit <URL> to view your Application Insights component App settings have been redacted. Use `az webapp/logicapp/functionapp config appsettings list` to view. { ...JSON response... }A bit more what these switches mean:
- --consumption-plan-location is the location of the consumption plan.
- --runtime is the runtime stack of the function app, here Python.
- --runtime-version is the version of the runtime stack, here 3.11 (3.12 was not available at the time of writing).
- --functions-version is the version of the Azure Functions runtime, here 4.
- --name is the name of the function app (same here: make it something unique / descriptive).
- --storage-account is the name of the storage account we created.
- --os-type is the operating system of the function app, here Linux (Windows is the default but did not have the Python runtime, at least for me).
Go to your FastAPI app directory and create a requirements.txt file with your dependencies if not done already:
To demo this from scratch, let’s create a simple FastAPI webhook app (the associated repo is here).
mkdir fastapi_webhook cd fastapi_webhook mkdir webhookFirst, create a requirements.txt file:
fastapi azure-functionsThe azure-functions package is required to run FastAPI on Azure Functions.
Then create an __init__.py file inside the webhook folder with the FastAPI app code:
import azure.functions as func from fastapi import FastAPI app = FastAPI() @app.post("/webhook") async def webhook(payload: dict) -> dict: """ Simple webhook that just returns the payload. Normally you would do something with the payload here. And add some security like secret key validation (HMAC for example). """ return {"status": "ok", "payload": payload} async def main(req: func.HttpRequest, context: func.Context) -> func.HttpResponse: return await func.AsgiMiddleware(app).handle_async(req, context)This took a bit of trial and working to get it working, because it was not evident that I had to use AsgiMiddleware to run FastAPI on Azure Functions. This article helped with that.
Next, create a host.json file in the root of the project for the Azure Functions host:
{ "version": "2.0", "extensions": { "http": { "routePrefix": "" } } }A local.settings.json file for local development:
{ "IsEncrypted": false, "Values": { "AzureWebJobsStorage": "UseDevelopmentStorage=true", "FUNCTIONS_WORKER_RUNTIME": "python" } }And you’ll need a function.json file in the webhook directory that defines the configuration for the Azure Function:
{ "scriptFile": "__init__.py", "bindings": [ { "authLevel": "anonymous", "type": "httpTrigger", "direction": "in", "name": "req", "methods": ["post"] }, { "type": "http", "direction": "out", "name": "$return" } ] }I also included a quick test script in the root of the project:
import sys import requests url = "http://localhost:7071/webhook" if len(sys.argv) < 2 else sys.argv[1] data = {"key": "value"} resp = requests.get(url) print(resp) resp = requests.post(url, json=data) print(resp) print(resp.text)If you run this script, you should see a 404 for the GET request (not implemented) and a 200 for the POST request. For remote testing I can run the script with the live webhook URL (see below).
Run the function locallyTo verify that the function works locally run it with:
func startThen run the test script:
$ python test.py <Response [404]> <Response [200]> {"status":"ok","payload":{"key":"value"}}Awesome!
Deploy it to AzureTime to deploy the function to Azure:
(venv) √ fastapi_webhook (main) $ func azure functionapp publish pybitesWebhookSample Getting site publishing info... [2024-06-17T16:06:08.032Z] Starting the function app deployment... Creating archive for current directory... ... ... Remote build succeeded! [2024-06-17T16:07:35.491Z] Syncing triggers... Functions in pybitesWebhookSample: webhook - [httpTrigger] Invoke url: https://pybiteswebhooksample.azurewebsites.net/webhookLet’s see if it also works on Azure:
$ python test.py https://pybiteswebhooksample.azurewebsites.net/webhook <Response [404]> <Response [200]> {"status":"ok","payload":{"key":"value"}}Cool, it works!
If something goes wrong remotely, you can check the logs:
az webapp log tail --name pybitesWebhookSample --resource-group myResourceGroupAnd when you make changes, you can redeploy with:
func azure functionapp publish pybitesWebhookSampleThat’s it, a FastAPI app running as an Azure Function! I hope this article helps you if you want to do the same. If you have any questions, feel free to reach out to me on X | Fosstodon | LinkedIn -> @bbelderbos.
Open Source AI Definition – Weekly update June 17
- After much debate regarding training data, @stefano published a summary of the positions expressed and some clarifications about the terminology included in draft v.0.0.8. You can read the rationale about it and share your thoughts on the forum.
- Initial thoughts:
- @Senficon (Felix Reda) adds that while the discussion has highlighted the case for data information, it’s crucial to understand the implications of copyright law on AI, particularly concerning access to training data. Open Source software relies on a legal element (copyright licenses) and an access element (availability of source code). However, this framework does not seamlessly apply to AI, as different copyright regimes allow text and data mining (TDM) for AI training but not the redistribution of datasets. This discrepancy means that requiring the publication of training datasets would make Open Source AI models illegal, despite TDM exceptions that facilitate AI development. Also, public domain status is not consistent internationally, complicating the creation of legally publishable datasets. Consequently, a definition of Open Source AI that imposes releasing datasets would impede collaborative improvements and limit practical significance. Emphasizing data innovation can help maintain Open Source principles without legal pitfalls.
- @amcasari expresses concern about the usability and neutrality of the “Model Openness Framework” (MOF) for identifying AI systems, suggesting it doesn’t align well with current industry practices and isn’t ready for practical application without further feedback and iteration.
- @shujisado points out that the MOF’s classification of components doesn’t depend on the specific IP laws applied, but rather on a general legal framework, and highlights that Japan’s IP law system differs from the US and EU, yet finds discussions based on the OSD consistent.
- @stefano emphasizes the importance of having well-thought-out, timeless principles in the Open Source AI Definition document, while viewing the Checklist as a more frequently updated working document. He also supports the call to see practical examples of the framework in use and proposes separating the Checklist from the main document to reduce confusion.
- Reviews of eleven different AI systems have been published. We do these review to check existing systems compatibility with our current definition. These are the systems in question: Arctic, BLOOM, Falcon, Grok, Llama 2, Mistral, OLMo, OpenCV, Phy-2, Pythia, and T5.
- @mer has set up a review sheet for the Viking model upon request from @merlijn-sebrechts.
- @anatta8538 asks if MLOps is considered within the topic of the Model Openness Framework and whether CLIP, an LMM, would be consistent with the OSAID.
- @nick clarifies that the evaluation focuses on components as described in the Model Openness Framework, which includes development and deployment aspects but does not cover MLOps as a whole.
- @Alek_Tarkowski agrees that certification of open-source AI will be crucial under the AI Act and highlights the importance of defining what constitutes an Open Source license. He points out the confusion surrounding terms like “free and open source license” and suggests that the issue of responsible AI licensing as a form of Open Source licensing needs resolution. Notes that some restrictive licenses are gaining traction and may need consideration for exemption from regulation, thus urging for a consensus.
Slides and the recording of our previous townhall meeting can be found here.