Feeds
eGenix.com: Python Meeting Düsseldorf - 2024-07-03
The following text is in German, since we're announcing a regional user group meeting in Düsseldorf, Germany.
Das nächste Python Meeting Düsseldorf findet an folgendem Termin statt:
03.07.2024, 18:00 Uhr
Raum 1, 2.OG im Bürgerhaus Stadtteilzentrum Bilk
Düsseldorfer Arcaden, Bachstr. 145, 40217 Düsseldorf
- Charlie Clark:
Reflections on the importance of APIs in Python
- Klaus Bremer:
Nox
- Marc-Andre Lemburg:
Shipping ready-to-run Python apps without the need to install Python
- Jens Diemer:
Django Find My Device
Wir treffen uns um 18:00 Uhr im Bürgerhaus in den Düsseldorfer Arcaden.
Das Bürgerhaus teilt sich den Eingang mit dem Schwimmbad und befindet
sich an der Seite der Tiefgarageneinfahrt der Düsseldorfer Arcaden.
Über dem Eingang steht ein großes "Schwimm’ in Bilk" Logo. Hinter der Tür
direkt links zu den zwei Aufzügen, dann in den 2. Stock hochfahren. Der
Eingang zum Raum 1 liegt direkt links, wenn man aus dem Aufzug kommt.
>>> Eingang in Google Street View
Das Python Meeting Düsseldorf ist eine regelmäßige Veranstaltung in Düsseldorf, die sich an Python Begeisterte aus der Region wendet.
Einen guten Überblick über die Vorträge bietet unser PyDDF YouTube-Kanal, auf dem wir Videos der Vorträge nach den Meetings veröffentlichen.Veranstaltet wird das Meeting von der eGenix.com GmbH, Langenfeld, in Zusammenarbeit mit Clark Consulting & Research, Düsseldorf:
Das Python Meeting Düsseldorf nutzt eine Mischung aus (Lightning) Talks und offener Diskussion.
Vorträge können vorher angemeldet werden, oder auch spontan während des Treffens eingebracht werden. Ein Beamer mit HDMI und FullHD Auflösung steht zur Verfügung.(Lightning) Talk Anmeldung bitte formlos per EMail an info@pyddf.de
KostenbeteiligungDas Python Meeting Düsseldorf wird von Python Nutzern für Python Nutzer veranstaltet.
Da Tagungsraum, Beamer, Internet und Getränke Kosten produzieren, bitten wir die Teilnehmer um einen Beitrag in Höhe von EUR 10,00 inkl. 19% Mwst. Schüler und Studenten zahlen EUR 5,00 inkl. 19% Mwst.
Wir möchten alle Teilnehmer bitten, den Betrag in bar mitzubringen.
AnmeldungDa wir nur 25 Personen in dem angemieteten Raum empfangen können, möchten wir bitten, sich vorher anzumelden.
Meeting Anmeldung bitte per Meetup
Weitere Informationen finden Sie auf der Webseite des Meetings:
https://pyddf.de/
Viel Spaß !
Marc-Andre Lemburg, eGenix.com
Specbee: SAML and OAuth2 - What’s the difference and how to implement in Drupal
Niels de Feyter: Maximize Productivity and Success as a Drupal Contractor: 6 Expert Tips
Niels de Feyter: Is Drupal Still the Leading CMS in 2022 and 2023?
Niels de Feyter: Retrospective Drupal Dev Days April 2022 in Gent Belgium
Niels de Feyter: Drupal 8 survey Feeds Migrate usage and functionality
Niels de Feyter: Watch Drupalcon Now Orleans sessions on Youtube
That is also true for the now ongoing DrupalCon New Orleans 2016.
An excellent option for learning and understanding Drupal. (and following the event!)
Niels de Feyter: I am getting excited about Drupal 8
In a conceptual way, Drupal 8 is not so different from Drupal 7. The Administration-UI’s are the same and you probably will also use the same modules for a project. Site builders will feel really comfortable.
All-in-all I am getting excited about Drupal 8!
Niels de Feyter: Watch Drupalcon Barcelona 2015 sessions on Youtube
https://www.youtube.com/user/DrupalAssociation/videos
Posted by the Drupal Association.
Hope you can learn from it too!
Niels de Feyter: Watch Drupalcon Barcelona 2015 sprinting on drupal.org
Niels de Feyter: Setup Entity Translation the right way
Entity Translation is part of Drupal 8 core and its approach is to translate fields instead of full nodes/entities.
Goal of this tutorial is to set up a multilingual website that can be navigated in multiple languages by visitors and to enable the content to be easily manageable by editors / cms administrators.
To get multilingual right, it’s critical that you configure your content-types and fields with care and precision and upfront, because if content is already in your database it is almost impossible to change these configurations.
So let's go.
Niels de Feyter: Why I use Display Suite, Entity Reference and Bootstrap
Russ Allbery: INN 2.7.2
This is a bug fix and minor feature release over INN 2.7.1, and the upgrade should be painless. You can download the new release from ISC or my personal INN pages. The latter also has links to the full changelog and the other INN documentation.
For the full list of changes, see the INN 2.7.2 NEWS file.
As always, thanks to Julien ÉLIE for preparing this release and doing most of the maintenance work on INN!
Tellico 3.5.5 Released
Tellico 3.5.5 is available, with a few important fixes.
Improvements and Bug Fixes- Fixed the XSLT file loading to work correctly with libxml2 >= 2.13 (Bug 488707).
- Fixed bug for showing entries with large content (Bug 487079).
- Improved the SRU fetcher to allow user-defined search indices (Bug 488931).
Fixing KWin’s performance on old hardware
KWin had a very long standing bug report about bad performance of the Wayland session on older Intel integrated graphics. There have been many investigations into what’s causing this, with a lot of more specific performance issues being found and fixed, but none of them managed to fully fix the issue… until now.
The source of the problemUnderstanding the issue requires some minimal understanding about how displays work. My earlier post about gaming on Wayland goes into more depth about them and the different presentation modes, but the TL;DR is that most displays today require frames to be sent to it in fixed intervals if you want them to be shown without tearing artifacts.
With the vast majority of displays, that interval is 16.67ms. In order to show everything as smooth as possible, the compositor thus has to render a new frame every 16.67ms as well; it must not miss a single deadline or the user will see stutter as some frames are shown twice and others are skipped.
This problem is not unique to Intel of course, when the deadline is missed, that causes the same stutter on every system. It’s just an often reported issue on old Intel processors because they’re used a lot, because both CPUs and GPUs in them are pretty slow, and laptop manufacturers too often paired them with high resolution screens, requiring the GPU to render for a long time each frame.
How KWin deals with this deadlineIn the past, KWin would just start compositing immediately once the last frame was presented, to have as much time as possible for rendering; this worked pretty well but meant that it almost always rendered too early. On desktop GPUs, compositing can take as little as a few hundred microseconds; if we start compositing 16ms before the deadline, that means we’re also unnecessarily increasing latency by roughly 16ms, which makes the system feel less responsive.
For KWin 5.21, Vlad Zahorodnii implemented a scheduling mechanism to do this better: Instead of assuming we always need the whole frame for rendering, KWin measures how long rendering takes and could start compositing closer to the deadline, which reduced latency. However, this strategy could still not get very close to the deadline by default, because it only measured how long rendering took on the CPU, but not how long it took on the GPU.
For KWin 6.0, I implemented the missing part, recording GPU render times. This meant that we could reduce latency more, without causing stutter… Or at least that was the idea. It turns out, render times are very volatile at times; KWin’s rendering can be delayed by other apps using the CPU and GPU, by the hardware changing power management states, by additional windows opening, by KWin effects starting to render something heavy, by input events taking CPU time, and so on.
As a result, taking a simple average of recent render times wasn’t enough, and even taking the maximum wasn’t good enough to prevent all the noticeable stutter. Instead, KWin now analyzes past render times for how volatile they are, and starts compositing much earlier if they’re volatile, and only moves closer to the deadline when render times are stable and predictable. This will likely be tweaked a few more times as I can collect more render time data from different PCs and optimize that algorithm with it, but so far it works pretty well, and gets us the best of both worlds: High latency when necessary to prevent stutter, and low latency when possible.
So, with these old Intel processors, KWin should now detect that rendering takes long and render times are not very stable, and start rendering as early as possible, and that should fix everything, right? Unfortunately, that’s not the whole story.
Old Intel processors are just too damn slow!On these old processors, especially when paired with a 4k screen, rendering doesn’t just take long, it often takes too long for a frame to be completed after 16.67ms! All the previous improvements were useful, but can’t make the hardware faster.
In the very beginning of the post I hinted that this is a Wayland only problem though, so what’s going on on Xorg? kwin_x11 has a trick that makes this problem less severe: On the start of each refresh cycle, it starts rendering a frame, even if the last frame isn’t done rendering yet. This is called “triple buffering”1 because it uses up to three buffers at the same time (two for rendering, one for displaying) and it has two big benefits:
- while the GPU is still working on the last frame, the CPU can already prepare rendering commands for next one. As long as both CPU and GPU individually take less than 16.67ms, you can still get one image rendered for each frame the display can present
- because more frames are being rendered, the driver may increase CPU and GPU clock speeds, which makes rendering faster and might allow for hitting the full refresh rate
However, it also has some caveats:
- it increases latency in general, as rendering is started earlier than necessary
- when rendering takes more than one refresh duration, latency is increased by a whole refresh duration - even if it would only need a single millisecond more for rendering
- to avoid increasing latency for dedicated GPUs, it’s only active on Intel GPUs and never used elsewhere
- it’s active even on Intel GPUs that have good enough performance to not need it
- when the driver increases GPU clocks because of triple buffering, that may be enough for rendering to be fast enough to not need triple buffering anymore… which means the GPU clocks will reduce again, and frames will be dropped until triple buffering is active again, and that repeats in a cycle. This can, in some situations (like video playback), be more noticeable than a reduced but constant refresh rate.
The goal then was to implement a form of triple buffering that would come with the same benefits, without also having the same shortcomings. First, a few things needed to be patched up to allow for triple buffering to work.
Fixing prerequisitesThe DRM/KMS kernel API currently only allows a single frame to be queued for presentation at a time. When you submit a frame to the kernel, you have to wait until it’s done rendering and shown on the screen, before you’re allowed to commit the next frame - but for triple buffering we need to queue two frames. Luckily I had already implemented a queue for other functionality with a drm commit thread (check out my previous post about cursor updates for details), so this one was mostly taken care of already and only needed minor changes.
The queue wasn’t good enough yet though. If KWin’s render time prediction is too pessimistic and it starts rendering much earlier than necessary, it could end up rendering two frames that are meant for consecutive refresh cycles, and complete rendering both during the same refresh cycle… which means that GPU power is wasted. Worse, as the refresh rate of apps is coupled to KWin’s, apps would try to render twice as fast too! To fix that, frames are now simply delayed until the time they’re intended to be displayed.
In order to figure out how long compositing takes on the GPU, KWin uses OpenGL query objects. These are quite useful, but they have three issues for triple buffering:
- query objects only hold a single result. If you start two queries, the last one gets dropped
- if you query a timestamp for commands that haven’t finished executing yet, OpenGL will do a blocking wait until that’s done
- they’re bound to an OpenGL context, which especially complicates things with multiple GPUs
To avoid the last problem, I did a bunch of OpenGL refactors that removed unnecessary global state from OpenGL code, and encapsulated what was left in OpenGL context objects. Render time queries in KWin now store a reference to the OpenGL context object they were created with and handle the context switching for fetching the result themselves, so code using them doesn’t have to care a lot about OpenGL anymore. With that done, the other two issues were fixed by simply creating a new query for each frame, which gets queried after the frame is done rendering, and never gets reused.
Actually implementing triple bufferingAfter these prerequisites were taken care of, I extended the existing infrastructure for frame tracking, OutputFrame objects, to encapsulate more properties of presentation requests and handle render time tracking as well as presentation feedback directly. With all information and feedback for each frame being tracked in a single object, a lot of presentation logic was simplified and allowing multiple pending frames ended up being a relatively simple change in the drm backend.
To make use of that capability, I extended the render scheduling logic to allow render times of up to two frames. It computes a target presentation timestamp by calculating how many refresh cycles have gone by since the last presented frame and how many refresh cycles rendering will take. If there’s already a frame pending, it just ensures that the refresh cycle after that is targeted instead of the same one… and that’s almost it.
Remember how I wrote that displays refresh in a fixed time interval? Well, it’s not that simple after all. Timings can fluctuate quite a bit, and that could make KWin sometimes schedule two frames for the same refresh cycle, just for the older one to get dropped again and be a complete waste of energy and even cause stutter. As a fix, when the last frame completes and provides a more accurate timestamp about when the next refresh cycle begins, KWin now reschedules rendering to match it.
This is the state of things in the 6.1.0 release; since then issues on a few systems were reported and more adjustments may still be added - in particular, some sort of hysteresis to not make KWin switch between double- and triple buffering too often.
The resultWith all those changes implemented in Plasma 6.1, triple buffering on Wayland
- is only active if KWin predicts rendering to take longer than a refresh cycle
- doesn’t add more latency than necessary even while triple buffering is active, at least as long as render time prediction is decent
- works independently of what GPU you have
In practice, on my desktop PC with a dedicated GPU, triple buffering is effectively never active, and latency is the same as before. On my AMD laptop it’s usually off as well, only kicking in once in a while… But on some older Intel laptops with high resolution screens, it’s always active and I’ve been told it’s like having a brand new laptop - KWin goes from doing stuttery 30-40fps to a solid 60fps.
It’s not just old or slow processors that benefit though, I also tested this on a laptop with an integrated Intel and a dedicated NVidia GPU. With an external display connected to the NVidia GPU, due to some issues in the NVidia driver, multi gpu copies are quite slow, and without triple buffering, the external monitor was limited to 60fps. Triple buffering can’t do magic, but KWin now at least reaches around 100-120fps on that setup, which is likely the best that can be done until the driver issue is resolved and feels a lot smoother already.
-
Keep in mind that “triple buffering” means a few different things to different people and in different contexts. I won’t go deeper into that mess here; just be aware that it’s a loaded term. ↩
KDE Plasma 6.1.1, Bugfix Release for June
Tuesday, 25 June 2024. Today KDE releases a bugfix update to KDE Plasma 6, versioned 6.1.1.
Plasma 6.1 was released in June 2024 with many feature refinements and new modules to complete the desktop experience.
This release adds a week's worth of new translations and fixes from KDE's contributors. The bugfixes are typically small but important and include:
- KScreenLocker Greeter: Fix Shader Wallpaper plugin and possibly others. Commit.
- Increase minimum plasma wayland protocols version to 1.13. Commit.
- Use snap:// URLs rather than appstream:// ones on snap only distros. Commit.
Kommit 1.6.0
Kommit 1.6.0 is a feature and bugfix release of our Git repo app which now builds with Q 5 or 6.
Improvements:
- build without kdbusaddons on windows
- Add flatpak support
- Fix show date (using QLocale for it)
- Fix mem leak
- Reactivate open file in external apps in qt6
- Add zoom support in Report Chart Widget
- Replace a QTableWidget by QTreeWidget in report page
- Fix crash when we didn't open git repository
- Fix load style/icon on windows (KF >= 6.3)
- Implement a gravatar cache
- Fix i18n
URL: https://download.kde.org/stable/kommit
Source: kommit-1.6.0.tar.xz
SHA256: 4091126316ab0cd2d4a131facd3cd8fc8c659f348103b852db8b6d1fd4f164e2
Signed by: E0A3EB202F8E57528E13E72FD7574483BB57B18D Jonathan Esk-Riddell jr@jriddell.org
https://jriddell.org/esk-riddell.gpg
Open Source AI Definition – Weekly update June 24
Following @stefano’s publication regarding why the OSI considers training data to be “optional” under the checklist in Open Source AI Definition, the debate has continued. Here are the main points:
- Preferred Form of Modification
- @hartmans states finding an agreement on the meaning of “preferred form of modification” depends on the user’s objectives. The disagreement may stem from different priorities in ranking the freedoms associated with open source AI, though they emphasize prioritizing model weights for practical modifications. He suggested that data information could be more beneficial than raw data for understanding models and urged flexibility in AI definitions.
- @shujisado highlighted that training data for machine learning models is a preferred form of modification but questioned if it is the most preferred. He further emphasized the need for a flexible definition for preferred forms of modification in AI.
- @quaid supported the idea of conducting controlled experiments to determine if data information alone is sufficient to recreate AI models accurately. Suggested practical steps for testing the effectiveness of data information and encouraged community participation in such experiments.
- @jberkus raised concerns about the practical assessment of data information and its ability to facilitate the recreation of AI systems. He questioned how to evaluate data information without recreating the AI system.
- Practical Applications and Community Insights
- @hartmans proposed practical scenarios where data information could suffice for modifying AI models and suggested that the community’s flexibility in defining the preferred form of modification has been valuable for Debian.
- @quaid shared insights from his research on the OpenVLA project, noting its compliance with OSAID requirements. He further proposed conducting controlled experiments to verify if data information is enough to recreate models with fidelity.
- General observations
- @shujisado emphasized the need for flexible definitions in AI, drawing from open-source community experiences. Agreed on the complexity of training data issues and supported the flexible approach of OSI in defining the preferred form of modification.
- @quaid suggested practical approaches for evaluating data information and its adequacy for recreating AI models and proposed further experiments and community involvement to refine the understanding and application of data information in open-source AI.
- @jberkus asked whether OSAID will apply to licenses or systems, noting that current drafts focus on systems. He questioned if a certification program for reviewing systems as open source or proprietary is the intended direction.
- @shujisado confirmed that discussions are moving towards certifying AI systems and pointed at an existing thread. He emphasized the need for evaluating individual components of AI systems and expressed concern about OSI’s capacity to establish a certification mechanism, highlighting that it would significantly expand OSI’s role.
parallel @ Savannah: GNU Parallel 20240622 ('34 counts') released
GNU Parallel 20240622 ('34 counts') has been released. It is available for download at: lbry://@GnuParallel:4
Quote of the month:
The most glorious 15,000 lines of Perl ever written.
-- @nibblrrr7124@YouTube
New in this release:
- Bug fixes and man page updates.
News about GNU Parallel:
- Howto - Parallel: lanciare comandi in simultanea https://github.com/linuxhubit/linuxhub.it/blob/main/_posts/2024-06-14-howto-parallel-per-lanciare-comandi-in-simultanea.md
- Implementing Concurrency in Shell Scripts https://dev.to/siddhantkcode/implementing-concurrency-in-shell-scripts-521o
GNU Parallel - For people who live life in the parallel lane.
If you like GNU Parallel record a video testimonial: Say who you are, what you use GNU Parallel for, how it helps you, and what you like most about it. Include a command that uses GNU Parallel if you feel like it.
GNU Parallel is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. A job can also be a command that reads from a pipe. GNU Parallel can then split the input and pipe it into commands in parallel.
If you use xargs and tee today you will find GNU Parallel very easy to use as GNU Parallel is written to have the same options as xargs. If you write loops in shell, you will find GNU Parallel may be able to replace most of the loops and make them run faster by running several jobs in parallel. GNU Parallel can even replace nested loops.
GNU Parallel makes sure output from the commands is the same output as you would get had you run the commands sequentially. This makes it possible to use output from GNU Parallel as input for other programs.
For example you can run this to convert all jpeg files into png and gif files and have a progress bar:
parallel --bar convert {1} {1.}.{2} ::: *.jpg ::: png gif
Or you can generate big, medium, and small thumbnails of all jpeg files in sub dirs:
find . -name '*.jpg' |
parallel convert -geometry {2} {1} {1//}/thumb{2}_{1/} :::: - ::: 50 100 200
You can find more about GNU Parallel at: http://www.gnu.org/s/parallel/
You can install GNU Parallel in just 10 seconds with:
$ (wget -O - pi.dk/3 || lynx -source pi.dk/3 || curl pi.dk/3/ || \
fetch -o - http://pi.dk/3 ) > install.sh
$ sha1sum install.sh | grep 883c667e01eed62f975ad28b6d50e22a
12345678 883c667e 01eed62f 975ad28b 6d50e22a
$ md5sum install.sh | grep cc21b4c943fd03e93ae1ae49e28573c0
cc21b4c9 43fd03e9 3ae1ae49 e28573c0
$ sha512sum install.sh | grep ec113b49a54e705f86d51e784ebced224fdff3f52
79945d9d 250b42a4 2067bb00 99da012e c113b49a 54e705f8 6d51e784 ebced224
fdff3f52 ca588d64 e75f6033 61bd543f d631f592 2f87ceb2 ab034149 6df84a35
$ bash install.sh
Watch the intro video on http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1
Walk through the tutorial (man parallel_tutorial). Your command line will love you for it.
When using programs that use GNU Parallel to process data for publication please cite:
O. Tange (2018): GNU Parallel 2018, March 2018, https://doi.org/10.5281/zenodo.1146014.
If you like GNU Parallel:
- Give a demo at your local user group/team/colleagues
- Post the intro videos on Reddit/Diaspora*/forums/blogs/ Identi.ca/Google+/Twitter/Facebook/Linkedin/mailing lists
- Get the merchandise https://gnuparallel.threadless.com/designs/gnu-parallel
- Request or write a review for your favourite blog or magazine
- Request or build a package for your favourite distribution (if it is not already there)
- Invite me for your next conference
If you use programs that use GNU Parallel for research:
- Please cite GNU Parallel in you publications (use --citation)
If GNU Parallel saves you money:
- (Have your company) donate to FSF https://my.fsf.org/donate/
GNU sql aims to give a simple, unified interface for accessing databases through all the different databases' command line clients. So far the focus has been on giving a common way to specify login information (protocol, username, password, hostname, and port number), size (database and table size), and running queries.
The database is addressed using a DBURL. If commands are left out you will get that database's interactive shell.
When using GNU SQL for a publication please cite:
O. Tange (2011): GNU SQL - A Command Line Tool for Accessing Different Databases Using DBURLs, ;login: The USENIX Magazine, April 2011:29-32.
GNU niceload slows down a program when the computer load average (or other system activity) is above a certain limit. When the limit is reached the program will be suspended for some time. If the limit is a soft limit the program will be allowed to run for short amounts of time before being suspended again. If the limit is a hard limit the program will only be allowed to run when the system is below the limit.
The Last 2 Weeks in my GSoC II
The first month of the coding period of GSoC has already passed! Since the last update, I added Python support for the remaining classes of KWidgetsAddons. It was only recently when I discovered that apart from the C++ classes, the libraries also have namespaces which I didn’t even know about. So it turned out that it wasn’t actually completed. But anyway, there were only a few so that is now done. I also added support to automatically build a Python wheel for the bindings.
Last week I improved some Python demos and added bindings for KCoreAddons. That was quicker than I expected, so I might end up adding support for 5-7 more libraries that it was initially planned for. Here’s a list of the libraries that I plan to add during the rest of the summer:
- KI18n
- KGuiAddons
- KNotifications
- KUnitConversion
- KXMLGui