Feeds
Real Python: Rounding Numbers in Python
With many businesses turning to Python’s powerful data science ecosystem to analyze their data, understanding how to avoid introducing bias into datasets is absolutely vital. If you’ve studied some statistics, then you’re probably familiar with terms like reporting bias, selection bias, and sampling bias. There’s another type of bias that plays an important role when you’re dealing with numeric data: rounding bias.
Understanding how rounding works in Python can help you avoid biasing your dataset. This is an important skill. After all, drawing conclusions from biased data can lead to costly mistakes.
In this video course, you’ll learn:
- Why the way you round numbers is important
- How to round a number according to various rounding strategies
- How to implement each strategy in pure Python
- How rounding affects data and which rounding strategy minimizes this effect
- How to round numbers in NumPy arrays and pandas DataFrames
- When to apply different rounding strategies
[ Improve Your Python With đ Python Tricks đ â Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
Mike Driscoll: How to Publish a Python Package to PyPI
Do you have a Python package that you’d like to share with the world? You should publish it on the Python Package Index (PyPI). The vast majority of Python packages are published there. The PyPI team has also created extensive documentation to help you on your packaging journey. This article does not aim to replace that documentation. Instead, it is just a shorter version of it using ObjectListView as an example.
The ObjectListView project for Python is based on a C# wrapper .NET ListView but for wxPython. You use ObjectListView as a replacement for wx.ListCtrl because its methods and attributes are simpler. Unfortunately, the original implementation died out in 2015 while a fork, ObjectListView2 died in 2019. For this article, you will learn how I forked it again and created ObjectListView3 and packaged it up for PyPI.
Creating a Package StructureWhen you create a Python package, you must follow a certain type of directory structure to build everything correctly. For example, your package files should go inside a folder named src. Within that folder, you will have your package sub-folder that contains something like an __init__.py and your other Python files.
The src folder’s contents will look something like this:
package_tutorial/ | -- src/ | -- your_amazing_package/ | -- __init__.py | -- example.pyThe __init__.py file can be empty. You use that file to tell Python that the folder is a package and is importable. But wait! There’s more to creating a package than just your Python files!
You also should include the following:
- A license file
- The pyproject.toml file which is used for configuring your package when you build it, among other things
- A README.md file to describe the package, how to install it, example usage, etc
- A tests folder, if you have any (and you should!)
Go ahead and create these files and the tests folder. It’s okay if they are all blank right now. At this point, your folder structure will look like this:
package_tutorial/ | -- LICENSE | -- pyproject.toml | -- README.md | -- src/ | -- your_amazing_package/ | -- __init__.py | -- example.py | -- tests/ Picking a Build BackendThe packaging tutorial mentions that you can choose various build backends for when you create your package. There are examples for the following:
- Hatchling
- setuptools
- Flit
- PDM
You add this build information in your pyproject.toml file. Here is an example you might use if you picked setuptools:
[build-system] requires = ["setuptools>=61.0"] build-backend = "setuptools.build_meta"This section in your config is used by pip or build. These tools don’t actually do the heavy lifting of converting your source code into a wheel or other distribution package. That is handled by the build backend. You don’t have to add this section to your pyproject.toml file though. You will find that pip will default to setuptools if there isn’t anything listed.
Configuring MetadataAll the metadata for your package should go into a pyproject.toml file. In the case of ObjectListView, it didn’t have one of these files at all.
Here’s the generic example that the Packaging documentation gives:
[project] name = "example_package_YOUR_USERNAME_HERE" version = "0.0.1" authors = [ { name="Example Author", email="author@example.com" }, ] description = "A small example package" readme = "README.md" requires-python = ">=3.8" classifiers = [ "Programming Language :: Python :: 3", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", ] [project.urls] Homepage = "https://github.com/pypa/sampleproject" Issues = "https://github.com/pypa/sampleproject/issues"Using this as a template, I created the following for the ObjectListView package:
[project] name = "ObjectListView3" version = "1.3.4" authors = [ { name="Mike Driscoll", email="mike@somewhere.org" }, ] description = "An ObjectListView is a wrapper around the wx.ListCtrl that makes the list control easier to use." readme = "README.md" requires-python = ">=3.9" classifiers = [ "Programming Language :: Python :: 3", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", ] [project.urls] Homepage = "https://github.com/driscollis/ObjectListView3" Issues = "https://github.com/driscollis/ObjectListView3/issues"Now let’s go over what the various parts are in the above metadata:
- name – The distribution name of your package. Make sure your name is unique!
- version – The package version
- authors – One or more authors including emails for each
- description – A short, one-sentence summary of your package
- readme – A path to the file that contains a detailed description of the package. You may use a Markdown file here.
- requires-python – Tells which Python versions are supported by this package
- classifiers – Gives an index and some metadata for pip
- URLs – Extra links that you want to show on PyPI. In general, you would want to link to the source, documentation, issue trackers, etc.
You can specify other information in your TOML file if you’d like. For full details, see the pyproject.toml guide.
READMEs and LicensesThe README file is almost always a Markdown file now. Take a look at other popular Python packages to see what you should include. Here are some recommended items:
- How to install the package
- Basic usage examples
- Link to a guide or tutorial
- An FAQ if you have one
There are many licenses out there. Don’t take advice from anyone; buy a lawyer on that unless you know a lot about this topic. However, you can look at the licenses for other popular packages and use their license or one similar.
Generating a PackageNow you have the files you need, you are ready to generate your package. You will need to make sure you have PyPAâs build installed.
Here’s the command you’ll need for that:
python3 -m pip install --upgrade buildNext, you’ll want to run the following command from the same directory that your pyproject.toml file is in:
python3 -m buildYou’ll see a bunch of output from that command. When it’s done, you will have a dist A folder with a wheel (*.whl) file and a gzipped tarball inside it is called a built distribution. The tarball is a source distribution , while the wheel file is called a built distribution. When you use pip, it will try to find the built distribution first, but pip will fall back to the tarball if necessary.
Uploading / Publishing to PyPIYou now have the files you need to publish to share your package with the world on the Python Package Index (PyPI). However, you need to register an account on TestPyPI first. TestPyPI is a separate package index intended for testing and experimentation, which is perfect when you have never published a package before. To register an account, go to https://test.pypi.org/account/register/ and complete the steps on that page. It will require you to verify your email address, but other than that, it’s a very straightforward signup.
To securely upload your project, youâll need a PyPIÂ API token. Create one from your account and make sure to set the “Scope” to the “Entire account”. Don’t close the page until you have copied and saved your token or you’ll need to generate another one!
The last step before publishing is to install twine, a tool for uploading packages to PyPI. Here’s the command you’ll need to run in your terminal to get twine:
python3 -m pip install --upgrade twineOnce that has finished installing, you can run twine to upload your files. Make sure you run this command in your package folder where the new dist folder is:
python3 -m twine upload --repository testpypi dist/*You will see a prompt for your TestPyPI username and/or a password. The password is the API token you saved earlier. The directions in the documentation state that you should use __token__ as your username, but I don’t think it even asked for a username when I ran this command. I believe it only needed the API token itself.
After the command is complete, you will see some text stating which files were uploaded. You can view your package at https://test.pypi.org/project/example_package_name
To verify you can install your new package, run this command:
python3 -m pip install --index-url https://test.pypi.org/simple/ --no-deps example-package-nameIf you specify the correct name, the package should be downloaded and installed. TestPyPI is not meant for permanent storage, though, so it will likely delete your package eventually to conserve space.
When you have thoroughly tested your package, you’ll want to upload it to the real PyPI site. Here are the steps you’ll need to follow:
- Pick a memorable and unique package name
- Register an account at https://pypi.org. You’ll go through the same steps as you did on TestPyPI
- Be sure to verify your email
- Create an API key and save it on your machine or in a password vault
- Use twine to upload the dist folder like this:Â python -m twine upload dist/*
- Install the package from the real PyPI as you normally would
That’s it! You’ve just published your first package!
Wrapping UpCreating a Python package takes time and thought. You want your package to be easy to install and easy to understand. Be sure to spend enough time on your README and other documentation to make using your package easy and fun. It’s not good to look up a package and not know which versions of Python it supports or how to install it. Make the process easy by uploading your package to the Python Package Index.
The post How to Publish a Python Package to PyPI appeared first on Mouse Vs Python.
The Drop Times: Christina Lockhart on Elevating Drupal: The Essential Role of Women in Tech
DrupalEasy: Two very different European Drupal events in one week
I was fortunate enough to attend two very different European Drupal events recently, and wanted to take a few minutes to share my experience as well as thank the organizers for their dedication to the Drupal community.
The events couldn't have been more different - one being a first-time event in a town I've never heard of and the other celebrating their 20th anniversary in a city that can be considered the crossroads of the Netherlands.
DrupalCamp CemaesFirst off - it is pronounced KEM-ice (I didn't learn this until I actually arrived). Cemaes is a small village at the very northern tip of Wales with a population of 1,357. Luckily, one of those 1,357 people is Jennifer Tehan, an independent Drupal developer who decided to organize a Drupal event in a town that (I can only assume) has more sheep than people.
When this event was first announced a few months back, I immediately knew that this was something I wanted to attend - mostly because I've always wanted to visit Wales to do some hiking ("walking," if you're not from the United States.) I used the camp as an excuse for my wife and I to take a vacation and explore northern Wales. I could write an entire other blog post on how amazeballs the walking wasâŠ
DrupalCamp Cemaes was never going to be a large event, and it turns out there wound up being only nine of us (ten if you count Frankie, pictured below.) The advantage of such a small event was easy to see - we all got to know each other quite well. There's something really nice - less intimidating - about small Drupal events that I realized that I've really missed since the decline of Drupal meetups in many localities.
The event itself was an unconference, with different people presenting on Drupal contributions (James Shields), GitHub Codespaces (Rachel Lawson), remote work (Jennifer Tehan), the LocalGov distribution (Andy Broomfield), and Starshot (myself.)
The setting of the event couldn't have been more lovely - an idyllic seaside community where everything was within walking distance, including the village hall where the event took place. If Jennifer decides to organize DrupalCamp Cemaes 2025, I'm pretty sure Gwendolyn and I will be there.
Read a brief wrap-up of DrupalCamp Cemaes from Jennifer Tehan.
This camp was proof that anyone, anywhere can organize a successful Drupal event with a minimum of fuss.
DrupaljamFour days later, I was in Utrecht, Netherlands, at the 20th anniversary of Drupaljam, the annual main Dutch Drupal event.
I had previously attended 2011 Drupaljam and recall two things about that event: the building it took place in seemed like it was from the future, and many of the sessions were in Dutch.
This was a very different event from the one I had been at a few days earlier, in a city of almost 400,000 people with over 300 attendees (I was told this number anecdotally) in one of the coolest event venues I've ever been to. Defabrique is a renovated linseed oil and compound feed factory that is a bit over 100 years old that is (IMHO) absolutely perfect for a Drupal event. Each and every public space has character, yet is also modern enough to support open-source enthusiasts. On top of all that, the food was amazing.
Attending Drupal Jam was a bit of a last minute decision for me, but I'm really glad that I made the time for it. I was able to reconnect with members of the European Drupal community that I don't often see, and make new connections with more folks that I can count.
I spent the day the way I spend most of my time at Drupal events - alternating between networking and attending sessions that attracted me. There were a number of really interesting AI-related sessions that I took in; it's pretty obvious to me that the Drupal community is approaching AI integrations in an unsurprisingly thoughtful manner.
The last week reinforced to me how fortunate I am to be able to attend so many in-person Drupal events. The two events I was able to participate in couldn't have been more different in scale and scope, but don't ask me to choose my favorite, because I'm honestly not sure I could!
Real Python: Quiz: Build a Guitar Synthesizer
In this quiz, you’ll test your understanding of what it takes to build a guitar synthesizer in Python. By working through this quiz, you’ll revisit a few key concepts from the music theory and sound synthesis.
[ Improve Your Python With đ Python Tricks đ â Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
CodersLegacy: Adding Data Files in Nuitka (Python Files, Images, etc)
Nuitka is a Python-to-C compiler that converts Python code into executable binaries. While Nuitka efficiently compiles Python scripts, incorporating data files such as images, audio, video, and additional Python files requires can be a little tricky. This guide outlines the steps to include various data files in a Nuitka standalone executable.
Before delving into the inclusion of data files, ensure you have Nuitka installed. You can install Nuitka via pip:
pip install nuitkaFor a more detailed starter guide on Nuitka and its various options, you can follow this link to our main Nuitka Tutorial. If you already familiar with Nuitka, then proceed ahead in this tutorial to learn how to add data files in our Nuitka EXE.
Assuming you have a Python script named main.py, the basic command to compile it into a standalone executable using Nuitka is:
python -m nuitka --standalone main.pyThis command creates a standalone directory containing the executable and all necessary dependencies. However, if your program relies on external data files (images, audio, video, other Python files), you need to explicitly include these files.
To include individual files such as images, use the --include-data-files option. The syntax is:
--include-data-files=<source-location>=<target-location>For example, to include an image file favicon.ico, the command is:
--include-data-files=./favicon.ico=favicon.icoThis command tells Nuitka to include the favicon.ico file from the current directory and place it in the same location within the output directory.
To include entire directories, use the --include-data-dir option. The syntax is the same as earlier:
--include-data-dir=<source-location>=<target-location>For instance, to include an entire directory named Images, the command is:
--include-data-dir=./Images=ImagesThis will copy the Images directory from the current location to the output directory, preserving its structure.
Suppose you have a project with the following files and directories:
- main.py: The main Python script.
- favicon.ico: An icon file.
- Images/: A directory containing images.
- audio.mp3: An audio file.
- videos/: A directory containing video files.
- utils.py: An additional Python file used in the project.
To compile this project with all necessary data files included, the command would be:
python -m nuitka --standalone --include-data-files=./favicon.ico=favicon.ico --include-data-files=./audio.mp3=audio.mp3 --include-data-dir=./Images=Images --include-data-dir=./videos=videos --include-data-files=./utils.py=utils.py --disable-console main.pyPretty confusing isn’t it?
Having to write out this command over and over again each time you want to re-build your executable is a massive pain. Instead, I recommend you use Nuitka Configuration Files, which allow you write out all your dependencies/datafiles/imports in a single file, which can then be executed as many times as you like.
By using the --include-data-files and --include-data-dir options, you can include all necessary data files in your Nuitka-compiled executable. This ensures that your standalone application has all the resources it needs to run successfully, whether they are images, audio, video files, or additional Python scripts. Combining these options with plugins and other configurations allows for the creation of robust and self-sufficient executables.
Happy coding!
This marks the end of the Adding Data Files in Nuitka Tutorial. Any suggestions or contributions for CodersLegacy are more than welcome. Questions regarding the tutorial content are more than welcome.
The post Adding Data Files in Nuitka (Python Files, Images, etc) appeared first on CodersLegacy.
Python Bytes: #388 Don't delete all the repos
Specbee: Getting started with integrating Drupal and Tailwind CSS
Week 3 recap - points on a canvas
Plasma 6.1
Plasma 6 hits its stride with version 6.1. While Plasma 6.0 was all about getting the migration to the underlying Qt 6 frameworks correct (and what a massive job that was), 6.1 is where developers start implementing the features that will take you desktop to a new level.
In this release, you will find features that go far beyond subtle changes to themes and tweaks to animations (although there is plenty of those too), as you delve into interacting with desktops on remote machines, become more productive with usability and accessibility enhancements galore, and discover customizations that will even affect the hardware of your computer.
These features and more are being built directly into Plasma's Wayland version natively, avoiding the need for third party software and hacky extensions required by similar solutions implemented in X.
Things will only get more interesting from here. But meanwhile enjoy what will land on your desktop with your next update.
Note: Due to unforeseen circumstances, we have been unable to ship the new wallpaper, "Reef", with this version of Plasma. However, there will be a new wallpaper coming soon in the next 6.2 version.
If you can't wait, you can download "Reef" here.
We apologise for this delay and the inconvenience this may cause.
What's New Access Remote Plasma DesktopsOne of the more spectacular (and useful) features added in Plasma 6.1 is that you can now start up a remote desktop directly from the System Settings app. This means that if you are sysadmin who needs to troubleshoot users' machines, or simply need to work on a Plasma-enabled computer that is out of reach, setting up a connection is just a few clicks away.
Once enabled, you can connect to the remote desktop using a client such as KRDC. You will see the remote machine's Plasma desktop in a window and be able to interact with it from your own computer.
Customization made (more) VisualWe all love customizing our Plasma desktops, don't we? One of the quickest ways to do this is by entering Plasma's Edit Mode (right-click anywhere on the desktop background and select Enter Edit Mode from the menu).
In version 6.1, the visual aspect of Edit Mode has been overhauled and you will now see a slick animation when you activate it. The entire desktop zooms out smoothly, giving you a better overview of what is going on and allowing you to make your changes with ease.
Persistent AppsPlasma 6.1 on Wayland now has a feature that "remembers" what you were doing in your last session like it did under X11. Although this is still work in progress, If you log off and shut down your computer with a dozen open windows, Plasma will now open them for you the next time you power up your desktop, making it faster and easier to get back to what you were doing.
Sync your Keyboard's Colored LEDsIt wouldn't be a new Plasma release without at least one fancy aesthetic customization features. This time, however, we give you the power to reach beyond the screen, all the way onto your keyboard, as you can now synchronize the LED colours of your keys to match the accent colour on your desktop.
The ultimate enhancement for the fashion-conscious user.
Please note that this feature will not work on unsupported keyboards â support for additional keyboards is on the way!
And all this too...-
We have simplified the choices you see when you try to exit Plasma by reducing the number of confusing options. For example, when you press Shutdown, Plasma will only list Shutdown and Cancel, not every single power option.
-
Screen Locking gives you the option of configuring it to behave like a traditional screensaver, as you can choose it not to ask you for a password to unlock it.
-
Two visual accessibility changes make it easier to use the cursor in Plasma 6.1:
-
Shake Cursor makes the cursor grow when you "shake" it. This helps you locate that tiny little arrow on your large, cluttered screens when you lose it among all those windows.
-
Edge Barrier is useful if you have a multi-monitor setup and want to access things on the very edge of your monitor. The "barrier" is a sticky area for your cursor near the edge between screens, and it makes it easier to click on things (if that is what you want to do), rather than having the cursor scooting across to the next display.
-
-
Two major Wayland breakthroughs will greatly improve your Plasma experience:
-
Explicit Sync eliminates flickering and glitches traditionally experienced by NVidia users.
-
Triple Buffering support in Wayland makes animations and screen rendering smoother.
-
Talking Drupal: Talking Drupal #455 - Top 5 uses of AI for Drupal
Today we are talking about AI Tips for Drupal Devs, AI Best Practices, and Drupal Droid with guest Mike Miles. Weâll also cover AI interpolator as our module of the week.
For show notes visit: www.talkingDrupal.com/455
Topics- Top 5 tips
- Idea Generation (Ideation)
- Code Generation
- Debugging
- Content Generation
- Technical Explanations
- How do you suggest people use AI for Ideation
- Is MIT Sloan using AI to help with Drupal Development
- Does that code get directly inserted into your sites
- What are some common pitfalls
- Is your team using AI for debugging
- Any best practices you have found helping when working with AI
- Is MIT Sloan using AI for content generation
- What is an example of how you use AI for technical explanations
- What is your view ont he future of AI in Drupal, do you think AI will replace Drupal developers
- Drupal Droid
- Talking Drupal - Drupal Droid
- Workflows of ai
- AI guidelines
- Using AI as a Development Tool
- Drupal Starshot: Delivering on a promise to Ambitious Site Builders
Michael Miles - mike-miles.com mikemiles86
HostsNic Laflin - nLighteneddevelopment.com nicxvan John Picozzi - epam.com johnpicozzi Randy Fay - rfay
MOTW CorrespondentMartin Anderson-Clutz - mandclu.com mandclu
- Brief description:
- Have you ever wanted to use AI to help populate entity fields that were left blank by Drupal content authors? Thereâs a module for that.
- Module name/project name:
- Brief history
- How old: created in Sep 2023 by Marcus Johansson of FreelyGive
- Versions available: 1.0.0-rc4
- Maintainership
- Actively maintained, recent release in the past month
- Security coverage - opted in, needs stable release
- Test coverage
- Documentation - User guide
- Number of open issues: 18 open issues, none of which are a bugs
- Usage stats:
- 94 sites
- Module features and usage
- In scientific fields, interpolation is the process of using known data to extrapolate or estimate unknown data points. In a similar way this module helps your Drupal site provide values for fields that didnât receive input, based on the information that was provided.
- Fundamentally Interpolator AI provides a framework and an API, and then relies on companion modules for processing, either by leveraging third-party services like AI LLMs, or PHP-based scripting.
- There are existing integrations with a variety of AI services, including OpenAI, Dreamstudio, Hugging Face, and more.
- You can add retrievers to help extract and normalize the content youâre processing, for example photos from an external site, and other tools to help normalize and optimize content and media, and optimize any prompts you will be using with AI services.
- You can also extend the workflow capabilities of AI Interpolator, for example using the popular and powerful ECA module that weâve talked about before on this show.
PyBites: Deploying a FastAPI App as an Azure Function: A Step-by-Step Guide
In this article I will show you how to deploy a FastAPI app as a function in Azure. Prerequisites are that you have an Azure account and have the Azure CLI installed (see here).
Setup AzureFirst you need to login to your Azure account:
az loginIt should show your subscriptions and you select the one you want to use.
Create a resource groupLike this:
az group create --name myResourceGroup --location eastusA resource group is a container that holds related resources for an Azure solution (of course use your own names throughout this guide).
It comes back with the details of the resource group you just created.
Create a storage accountUsing this command:
az storage account create --name storage4pybites --location eastus --resource-group myResourceGroup --sku Standard_LRSNote that for me, to get this working, I had to first make an MS Storage account, which I did in the Azure portal.
The storage account name can only contain lowercase letters and numbers. Also make sure it’s unique enough (e.g. mystorageaccount was already taken).
Upon success it comes back with the details of the storage account you just created.
Create a function appNow, create a function app.
az functionapp create --resource-group myResourceGroup --consumption-plan-location eastus --runtime python --runtime-version 3.11 --functions-version 4 --name pybitesWebhookSample --storage-account storage4pybites --os-type Linux Your Linux function app 'pybitesWebhookSample', that uses a consumption plan has been successfully created but is not active until content is published using Azure Portal or the Functions Core Tools. Application Insights "pybitesWebhookSample" was created for this Function App. You can visit <URL> to view your Application Insights component App settings have been redacted. Use `az webapp/logicapp/functionapp config appsettings list` to view. { ...JSON response... }A bit more what these switches mean:
- --consumption-plan-location is the location of the consumption plan.
- --runtime is the runtime stack of the function app, here Python.
- --runtime-version is the version of the runtime stack, here 3.11 (3.12 was not available at the time of writing).
- --functions-version is the version of the Azure Functions runtime, here 4.
- --name is the name of the function app (same here: make it something unique / descriptive).
- --storage-account is the name of the storage account we created.
- --os-type is the operating system of the function app, here Linux (Windows is the default but did not have the Python runtime, at least for me).
Go to your FastAPI app directory and create a requirements.txt file with your dependencies if not done already:
To demo this from scratch, let’s create a simple FastAPI webhook app (the associated repo is here).
mkdir fastapi_webhook cd fastapi_webhook mkdir webhookFirst, create a requirements.txt file:
fastapi azure-functionsThe azure-functions package is required to run FastAPI on Azure Functions.
Then create an __init__.py file inside the webhook folder with the FastAPI app code:
import azure.functions as func from fastapi import FastAPI app = FastAPI() @app.post("/webhook") async def webhook(payload: dict) -> dict: """ Simple webhook that just returns the payload. Normally you would do something with the payload here. And add some security like secret key validation (HMAC for example). """ return {"status": "ok", "payload": payload} async def main(req: func.HttpRequest, context: func.Context) -> func.HttpResponse: return await func.AsgiMiddleware(app).handle_async(req, context)This took a bit of trial and working to get it working, because it was not evident that I had to use AsgiMiddleware to run FastAPI on Azure Functions. This article helped with that.
Next, create a host.json file in the root of the project for the Azure Functions host:
{ "version": "2.0", "extensions": { "http": { "routePrefix": "" } } }A local.settings.json file for local development:
{ "IsEncrypted": false, "Values": { "AzureWebJobsStorage": "UseDevelopmentStorage=true", "FUNCTIONS_WORKER_RUNTIME": "python" } }And you’ll need a function.json file in the webhook directory that defines the configuration for the Azure Function:
{ "scriptFile": "__init__.py", "bindings": [ { "authLevel": "anonymous", "type": "httpTrigger", "direction": "in", "name": "req", "methods": ["post"] }, { "type": "http", "direction": "out", "name": "$return" } ] }I also included a quick test script in the root of the project:
import sys import requests url = "http://localhost:7071/webhook" if len(sys.argv) < 2 else sys.argv[1] data = {"key": "value"} resp = requests.get(url) print(resp) resp = requests.post(url, json=data) print(resp) print(resp.text)If you run this script, you should see a 404 for the GET request (not implemented) and a 200 for the POST request. For remote testing I can run the script with the live webhook URL (see below).
Run the function locallyTo verify that the function works locally run it with:
func startThen run the test script:
$ python test.py <Response [404]> <Response [200]> {"status":"ok","payload":{"key":"value"}}Awesome!
Deploy it to AzureTime to deploy the function to Azure:
(venv) â fastapi_webhook (main) $ func azure functionapp publish pybitesWebhookSample Getting site publishing info... [2024-06-17T16:06:08.032Z] Starting the function app deployment... Creating archive for current directory... ... ... Remote build succeeded! [2024-06-17T16:07:35.491Z] Syncing triggers... Functions in pybitesWebhookSample: webhook - [httpTrigger] Invoke url: https://pybiteswebhooksample.azurewebsites.net/webhookLet’s see if it also works on Azure:
$ python test.py https://pybiteswebhooksample.azurewebsites.net/webhook <Response [404]> <Response [200]> {"status":"ok","payload":{"key":"value"}}Cool, it works!
If something goes wrong remotely, you can check the logs:
az webapp log tail --name pybitesWebhookSample --resource-group myResourceGroupAnd when you make changes, you can redeploy with:
func azure functionapp publish pybitesWebhookSampleThat’s it, a FastAPI app running as an Azure Function! I hope this article helps you if you want to do the same. If you have any questions, feel free to reach out to me on X |Â Fosstodon | LinkedIn -> @bbelderbos.
Open Source AI Definition â Weekly update June 17
- After much debate regarding training data, @stefano published a summary of the positions expressed and some clarifications about the terminology included in draft v.0.0.8. You can read the rationale about it and share your thoughts on the forum.Â
- Initial thoughts:
- @Senficon (Felix Reda) adds that while the discussion has highlighted the case for data information, it’s crucial to understand the implications of copyright law on AI, particularly concerning access to training data. Open Source software relies on a legal element (copyright licenses) and an access element (availability of source code). However, this framework does not seamlessly apply to AI, as different copyright regimes allow text and data mining (TDM) for AI training but not the redistribution of datasets. This discrepancy means that requiring the publication of training datasets would make Open Source AI models illegal, despite TDM exceptions that facilitate AI development. Also, public domain status is not consistent internationally, complicating the creation of legally publishable datasets. Consequently, a definition of Open Source AI that imposes releasing datasets would impede collaborative improvements and limit practical significance. Emphasizing data innovation can help maintain Open Source principles without legal pitfalls.
- @amcasari expresses concern about the usability and neutrality of the “Model Openness Framework” (MOF) for identifying AI systems, suggesting it doesn’t align well with current industry practices and isn’t ready for practical application without further feedback and iteration.
- @shujisado points out that the MOF’s classification of components doesn’t depend on the specific IP laws applied, but rather on a general legal framework, and highlights that Japan’s IP law system differs from the US and EU, yet finds discussions based on the OSD consistent.
- @stefano emphasizes the importance of having well-thought-out, timeless principles in the Open Source AI Definition document, while viewing the Checklist as a more frequently updated working document. He also supports the call to see practical examples of the framework in use and proposes separating the Checklist from the main document to reduce confusion.
- Reviews of eleven different AI systems have been published. We do these review to check existing systems compatibility with our current definition. These are the systems in question: Arctic, BLOOM, Falcon, Grok, Llama 2, Mistral, OLMo, OpenCV, Phy-2, Pythia, and T5.
- @mer has set up a review sheet for the Viking model upon request from @merlijn-sebrechts.
- @anatta8538 asks if MLOps is considered within the topic of the Model Openness Framework and whether CLIP, an LMM, would be consistent with the OSAID.
- @nick clarifies that the evaluation focuses on components as described in the Model Openness Framework, which includes development and deployment aspects but does not cover MLOps as a whole.
- @Alek_Tarkowski agrees that certification of open-source AI will be crucial under the AI Act and highlights the importance of defining what constitutes an Open Source license. He points out the confusion surrounding terms like “free and open source license” and suggests that the issue of responsible AI licensing as a form of Open Source licensing needs resolution. Notes that some restrictive licenses are gaining traction and may need consideration for exemption from regulation, thus urging for a consensus.
Slides and the recording of our previous townhall meeting can be found here.
ADCI Solutions: How to add a live chat to a website using Mercure
Real Python: Ruff: A Modern Python Linter for Error-Free and Maintainable Code
Linting is essential to writing clean and readable code that you can share with others. A linter, like Ruff, is a tool that analyzes your code and looks for errors, stylistic issues, and suspicious constructs. Linting allows you to address issues and improve your code quality before you commit your code and share it with others.
Ruff is a modern linter thatâs extremely fast and has a simple interface, making it straightforward to use. It also aims to be a drop-in replacement for many other linting and formatting tools, such as Flake8, isort, and Black. Itâs quickly becoming one of the most popular Python linters.
In this tutorial, youâll learn how to:
- Install Ruff
- Check your Python code for errors
- Automatically fix your linting errors
- Use Ruff to format your code
- Add optional configurations to supercharge your linting
To get the most from this tutorial, you should be familiar with virtual environments, installing third-party modules, and be comfortable with using the terminal.
Ruff cheat sheet: Click here to get access to a free Ruff cheat sheet that summarizes the main Ruff commands youâll use in this tutorial.
Take the Quiz: Test your knowledge with our interactive âRuff: A Modern Python Linterâ quiz. Youâll receive a score upon completion to help you track your learning progress:
Interactive Quiz
Ruff: A Modern Python LinterIn this quiz, you'll test your understanding of Ruff, a modern linter for Python. By working through this quiz, you'll revisit why you'd want to use Ruff to check your Python code and how it automatically fixes errors, formats your code, and provides optional configurations to enhance your linting.
Installing RuffNow that you know why linting your code is important and how Ruff is a powerful tool for the job, itâs time to install it. Thankfully, Ruff works out of the box, so no complicated installation instructions or configurations are needed to start using it.
Assuming your project is already set up with a virtual environment, you can install Ruff in the following ways:
Shell $ python -m pip install ruff Copied!In addition to pip, you can also install Ruff with Homebrew if youâre on macOS or Linux:
Shell $ brew install ruff Copied!Conda users can install Ruff using conda-forge:
Shell $ conda install -c conda-forge ruff Copied!If you use Arch, Alpine, or openSUSE Linux, you can also use the official distribution repositories. Youâll find specific instructions on the Ruff installation page of the official documentation.
Additionally, if youâd like Ruff to be available for all your projects, you might want to install Ruff with pipx.
You can check that Ruff installed correctly by using the ruff version command:
Shell $ ruff version ruff 0.4.7 Copied!For the ruff command to appear in your PATH, you may need to close and reopen your terminal application or start a new terminal session.
Linting Your Python CodeWhile linting helps keep your code consistent and error-free, it doesnât guarantee that your code will be bug-free. Finding the bugs in your code is best handled with a debugger and adequate testing, which wonât be covered in this tutorial. Coming up in the next sections, youâll learn how to use Ruff to check for errors and speed up your workflow.
Checking for ErrorsThe code below is a simple script called one_ring.py. When you run it, it gets a random Lord of the Rings character name from a tuple and lets you know if that character bore the burden of the One Ring. This code has no real practical use and is just a bit of fun. Regardless of the size of your code base, the steps are going to be the same:
Read the full article at https://realpython.com/ruff-python/ »[ Improve Your Python With đ Python Tricks đ â Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
Real Python: Quiz: Ruff: A Modern Python Linter
In this quiz, you’ll test your understanding of Ruff, a modern linter for Python.
By working through this quiz, you’ll revisit why you’d want to use Ruff to check your Python code and how it automatically fixes errors, formats your code, and provides optional configurations to enhance your linting.
[ Improve Your Python With đ Python Tricks đ â Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
Robin Wilson: Accessing Planetary Computer STAC files in DuckDB
Microsoft Planetary Computer is a wonderful archive of geospatial datasets (primarily raster images of various types), provided with a STAC catalog to enable them to be easily searched through an API. That’s fine for normal usage where you want to find a selection of images and access the images themselves, but less useful when you want to do some analysis of the metadata itself.
For that use case, Planetary Computer provide the STAC metadata in bulk, stored as GeoParquet files. Their documentation page explains how to use this with geopandas and do some large-ish scale processing with Dask. I wanted to try this with DuckDB – a newer tool that is excellent at accessing and processing Parquet files, including efficiently accessing files available via HTTP, and only downloading the relevant parts of the file. So, this post explains how I managed to do this – showing various different approaches I tried and how (or if!) each of them worked.
Getting URLsPlanetary Computer URLs are basically just URLs to files on Azure Blob Storage. However, the URLs require signing with a Shared Access Signature (SAS) key. Luckily, there is a Planetary Computer Python module that will use an API to generate the correct keys for us.
In the docs for using the STAC GeoParquet files, they give this code:
catalog = pystac_client.Client.open( "https://planetarycomputer.microsoft.com/api/stac/v1/", modifier=planetary_computer.sign_inplace, ) asset = catalog.get_collection("io-lulc-9-class").assets["geoparquet-items"]This gets a collection, and the geoparquet-items collection-level asset, having used a ‘modifier’ to sign the URLs as they are acquired from the STAC API. The URL is then stored in asset.href.
Using the DuckDB CLI – with HTTP URLsWhen using GeoPandas, the docs show that you can pass a storage_options parameter to the read_parquet method and it will ‘just work’:
df = geopandas.read_parquet( asset.href, storage_options=asset.extra_fields["table:storage_options"] )However, to use the file in DuckDB we need a full URL. The SQL code we’re going to use in DuckDB for a quick test looks like this:
SELECT * FROM <URL> LIMIT 1This just gets the first row of whatever URL is given.
Unfortunately, though, the URL provided by the STAC catalog looks like this:
abfs://items/io-lulc-9-class.parquetIf we try that with DuckDB, we find it doesn’t know how to deal with it. So, we need to convert this to a standard HTTP URL that can be downloaded as if it were just typed into a web browser (or used with cURL or wget). After a bit of playing around, I found I could create a full URL with this Python code:
url = f'https://{asset.extra_fields["table:storage_options"] ["account_name"]}.dfs.core.windows.net/{asset.href[7:]}? {asset.extra_fields["table:storage_options"]["credential"]}'That’s taking the account_name and sticking it in as part of the host part of the UK, then extracting the path from the original URL and adding that, and then finishing with the SAS token (stored as credential in the storage options) as a URL query parameter.
That results in a URL that looks like this (split over multiple lines for readability):
https://pcstacitems.dfs.core.windows.net/items/io-lulc-9-class.parquet ?st=2024-06-14T19%3A04%3A20Z&se=2024-06-15T19%3A49%3A20Z&sp=rl&sv=2024-05-04& sr=c&skoid=9c8ff44a-6a2c-4dfb-b298-1c9212f64d9a &sktid=72f988bf-86f1-41af-91ab-2d7cd011db47&skt=2024-06-15T09%3A00%3A14Z &ske=2024-06-22T09%3A00%3A14Z&sks=b &skv=2024-05-04 &sig=A0e%2BihbAagHmIZ%2Bg8gzH71TavRYQMZiHWJ/Uk9j0Who%3D(note: that specific URL won’t work for you, as the SAS tokens are time-limited).
If we insert that into our DuckDB SQL statement then we find it works (click to enlarge):
Great! Now let’s try with a different collection on Planetary Computer – in this case the Sentinel 2 collection. We can make the URL in the same way as before, by changing how we define asset:
asset = catalog.get_collection("sentinel-2-l2a").assets["geoparquet-items"]and then using the same query in DuckDB with the new URL. Unfortunately, this time we get an error:
Invalid Input Error: File '<URL>' too small to be a Parquet fileThe problem here is that for larger collections, the STAC metadata is spread out over a series of GeoParquet files inside a folder, and the URL we’ve been given is just to the folder (even though it ends in .parquet). As we’re just using HTTP, there is no way to do things like list the files in a folder, so DuckDB has no way to find out what files are within the folder and start reading them. We need to find another way.
Using the DuckDB CLI – with the Azure extensionConveniently, there is an Azure extension for DuckDB, and it lets you use URLs like 'abfss://âšmy_filesystemâ©/âšpathâ©/âšmy_fileâ©.âšparquet_or_csvâ©'. That’s a slightly different URL scheme to the one we’ve been given (abfss as opposed to abfs), but we can easily sort that.
Looking at the authentication docs though, it seems to require you to specify either a connection string, a service principal or using the ‘Azure Credential Chain’ (which, I think, works with the az command line and various environment variables that you may have set up). We don’t have any of those – they’re all a far broader scope than what we’ve got, which is just a SAS token for a specific file or folder. It looks like the Azure extension doesn’t support this, so we’ll have to look for another way.
Using the DuckDB Python library – with an Azure connectorAs well as the command-line interface, DuckDB can also be used from Python. To set this up, just run pip install duckdb. While you’re at it, you might want to install pandas and the adlfs library for connecting to Azure Storage.
Using this is actually quite simple, first import the libraries:
import duckdb from adlfs.spec import AzureBlobFileSystemthen set up the Azure connection:
a = AzureBlobFileSystem(account_name='pcstacitems', sas_token=asset.extra_fields["table:storage_options"]['credential'])Note how here we’re using the sas_token parameter to provide a SAS token. We could use a different parameter here to provide a connection string or some other credential type. I couldn’t find much real-world use of this sas_token parameter when looking online – so this is probably the key ‘less well documented’ bit to take away from this article.
Continuing, we then connect to DuckDB and ‘register’ this Azure connection:
connection = duckdb.connect() connection.register_filesystem(a)From there, we can run a SQL query like this:
query = connection.sql(f""" SELECT * FROM 'abfs://items/sentinel-2-l2a.parquet/*.parquet' LIMIT 1; """) query.to_df()The call to to_df at the end converts the result into a Pandas DataFrame for easy viewing and manipulation. The results are shown below (click to enlarge):
Note that we’re passing a URL of abfs://items/sentinel-2-l2a.parquet/*.parquet – this is the URL from the STAC catalog with /*.parquet added to the end to ensure DuckDB picks up the large number of Parquet files stored there. I’d have thought this would have worked without that, but if I miss that out I get a error saying:
TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'I suspect this is something to do with how things are passed to the Azure connection that we registered with DuckDB, but I’m not entirely sure. If you do know, then please leave a comment!
A few fun queriesSo, now we’ve got this working, what can we do? Well, we can – fairly efficiently – run queries across all the STAC metadata for Sentinel 2 images. I’ll just give a few examples of queries I’ve run below.
We can find out how many images there are in the collection in total:
SELECT COUNT(*) FROM 'abfs://items/sentinel-2-l2a.parquet/*.parquet'We can do the same for just the images acquired in 2020 – by using a fact we know about how the individual parquet files are named (from the Planetary Computer docs):
SELECT COUNT(*) FROM 'abfs://items/sentinel-2-l2a.parquet/*2020*.parquet'And we can start looking at how many scenes were captured with different amounts of cloud cover:
SELECT COUNT(*) / (SELECT COUNT(*) FROM 'abfs://items/sentinel-2-l2a.parquet/*2020*.parquet'), CASE WHEN "eo:cloud_cover" between 0 and 5 then '0-5%' WHEN "eo:cloud_cover" between 5 and 10 then '5-10%' WHEN "eo:cloud_cover" between 10 and 15 then '10-15%' END AS category FROM 'abfs://items/sentinel-2-l2a.parquet/*2020*.parquet' GROUP BY category,This tells us that 20% of scenes had between 0 and 5% cloud cover (quite a high number, I thought – but then again, I am used to living in the UK!), and around 4-5% in each of the 5-10% and 10-15% categories.
There are plenty of other analyses that you could do with these Parquet files, of course. At some point I might even get around to the task which initially made me look into this: that is, trying to find Landsat and Sentinel 2 scenes that were acquired over the same location at very similar times. I think I’ll leave that for another day though…
Greg Casamento: Keysight laid me off in January!
 I think it happened for several reasons:Â
- Economic - This is what was explained to me, but I am not sure I believe itÂ
- Politics - I think this part is because I expressed my opinions HONESTLY about the direction of the company given that they wanted to make the application into a VSCode plugin.
- Perception - I am 54 years old... so I think that they believed that Objective-C was my one and only talent, it's not... I know many other languages and have many other skills.Â
Keysight is and will remain a major contributor to GNUstep.
That being said, I recently ran into something rather disturbing at another company.  I have been working with a company based out of New Mexico that is interested in space applications. They have been using GNUstep and have been awaiting funding.
The lead of this effort expressed something during a meeting saying "We will work on the GNUstep side of this because there is no reason we should have to pay for any of this."  This hit a sour note with me to say the very least.  As it turns out he was under the mistaken impression that, because the work was on GNUstep, it was for free... which is WRONG.
I wonder if the same impression was present at Keysight or if other companies believe this. The saying, according to RMS, is "Free as in freedom, not as in beer."  If you are a manager at a company who is under the mistaken impression that work on any Free Software or Open Source project is free when your product depends on it, please correct your thinking.  Just because it is someone's passion project does NOT mean that they are going to do that work for free and prioritize the things that need to be done for your organization.
All of that being said the positive sides are this:
- More time to code on GNUstep without interruption
- More time to work on my own projects
- Time to rest and relax
The Drop Times: Tech Talks, Brand Strategies, and the Future of Drupal: Highlights from The DropTimes
An enterprise web solution is inherently complex. Its endless integrations with various applications that are added or loaded off as per the changing requirements, multiple front-ends that provide varied and personalised digital experiences, the cultural nuances catered to in its l10n editions, the time-zone differences that are to be taken care of, etc., etc., are the most basic parts that add to the complexity.
Continuous investment, development and deployment are part and parcel of such a solution. Even so, it becomes amiable only if the complexity is masked out in the simplicity of the consumer touch points.
Two separate sets of consumers should be considered there. One is the enterprise consumers, the content creators and marketers, who are the immediate benefactors of the solution. The second, a more prominent set, is the customers to whom the enterprises cater.
Drupal, the platform for building a digital experience, invested heavily in satisfying the end customers. That was what Drupal agencies were doing all these years. They got an exoskeleton on which they built the end-user experiences. But what about the enterprise consumers themselves? Werenât they the ones who ultimately decided to ditch or continue using the services? How good were their experiences sans the involvement of a hired developer?
The decision to simplify the touch points of enterprise consumers is a development and design decision. These content creators and marketers will benefit the most from the Starshot Initiative. It supports the marketing team by reducing the go-to-market time and eases the content creators by providing the most valuable tools out of the box. A complex solution is now draped in a simple cassock.
The first story Iâm sharing today concerns Drupalâs new brand strategy and future directions. The Starshot Initiative manifests this strategy and the direction the project is taking. An interview with Shawn Perritt, Senior Director of Brand & Creative at Acquia, shines a light on this brand refresh. Read Shawnâs conversation with Alka Elizabeth, our sub-editor.
Montreal became the hub for tech talks in the past week. We conducted one-question interviews with three featured speakers at Evolve Drupal Montreal. Kazima Abbas, our sub-editor, spoke with Josh Koenig, co-founder of Pantheon, Joe Kwan, Manager of Web Development at the University of Waterloo, and the formidable Mike Herchel, Senior Front-end Developer at Agileana. Their answers form part of the feature published here.
Kazima also conducted an exclusive interview with BaddĂœ Sonja Breidert, CEO and Co-Founder of 1xINTERNET. The interview focused on the agencyâs support for the Starshot Initiative based on its experience with âTry Drupalâ, which implements a default Drupal deployment for enterprises. It explains why they did not wait or hesitate to offer all-out support for the new initiative.
Alka Elizabeth and Ben Peter Mathew, our community manager, had jointly covered the Starshot Session on âStrategic Milestones in Product Definitionâ led by Dries Buytaert and Cristina Chumillas on June 7. The report can be read here.
The next two stories are our signature stories, written by our founder and lead, Anoop John, CTO of Zyxware Technologies. From the start, Anoop was vocal about âDrupalCollabâ, an idea he had been nurturing for years. As part of the initiative, he has published a list of the largest 500+ cities in the world with a sizable population of people tagged âDrupal developersâ on LinkedIn. You can read the analysis here. The second story published with this list is, On Using LinkedIn to Analyze the Size of the Drupal Community, an article on the methodology for this analysis using LinkedIn, associated errors and justifications. Based on these stories, Anoop is trying to build momentum to restart local Drupal meetups in as many cities as possible. Also, read his LinkedIn article on the initiative here.
Mautic 5.1 Andromeda Edition was released last week with major updates. Read our report here. We published a quick review of the Drupal Quick Exit Module developed by Oomph Inc. Ben Peter Mathew spoke with Alyssa Varsanyi of Oomph to produce this report.
To ensure brevity, I shall list the rest of the important stories below without much ado.
That is for the week, dear readers. I wish you all a very sacred Eid Ul Adha. Let the festival of sacrifice be a time to ponder the things we are ready to give up for the better common good.
To get timely updates, follow us on LinkedIn, Twitter and Facebook. Also, join us on Drupal Slack at #thedroptimes.
Thanks and regards.
Sincerely,
Sebin A. Jacob,
Editor-in-Chief, The DropTimes
GSoC'24 Okular | Week 2-3 Recap
While working on keystroke events, I realized my improvements to the event.change property were still inconsistent for certain Unicode characters. This led me to delve into code units, code points, graphemes, and other cool Unicode concepts. I found this blog post to be very enlightening.
Hereâs an update on my progress over the past two weeks:
MRs merged:- event.change : The change property of the event object now correctly handles Unicode, with adjustments to selStart and selEnd calculations. !MR998
- cursor position and undo/redo fix : Fixed cursor position calculations to account for rejected input text and resolved merging issues with undo/redo commands in text fields. !MR1011
- DocOpen Event implementation : Enabled document-level scripts to access the event object by implementing the DocOpen event. !MR1003
- Executing validation events correctly : Fixed a bug where validation scripts wouldnât run after KeystrokeCommit scripts in Okular. !MR999
- Widget refresh functions for RadioButton, ListEdit and ComboEdit : Added refresh functions as slots for RadioButton, ListEdit, and ComboEdit widgets, aiding in reset functionality and script updates. !MR1012
- Additional document actions in Poppler : Implemented reading additional document actions (CloseDocument, SaveDocumentStart, SaveDocumentFinish, PrintDocumentStart, PrintDocumentFinish) in the qt5 and qt6 frontends for Poppler. !MR1561 (in Poppler)
- Reset form implementation in qt6 frontend for Okular : Working on the reset form functionality in Okular, currently focusing on qt6 frontend details. !MR1564 (in Poppler)
- Reset form in Okular : Using the Poppler API to reset forms within Okular. !MR1007
- Fixing order of execution of events for text form fields : Addressing the incorrect execution order of certain events (e.g., calculation scripts) and ensuring keystroke commit, validation, and format scripts are evaluated correctly when setting fields via JavaScript. !MR1002
For the coming weeks, my focus will be on implementing reset forms, enhancing keystroking and formatting scripts, and possibly starting on submit forms. Letâs see how it goes.
See you next time. Cheers!