Feeds
Peter Bengtsson: Converting Celsius to Fahrenheit with Python
The Drop is Always Moving: 📢 Drupal 11.0.0-rc1 is now available! Release candidates are not for production sites, they are intended for widespread testing in preparation for the upcoming stable release. Test now! 11.0.0 is planned for the week of July...
📢 Drupal 11.0.0-rc1 is now available! Release candidates are not for production sites, they are intended for widespread testing in preparation for the upcoming stable release. Test now! 11.0.0 is planned for the week of July 29, 2024. https://www.drupal.org/project/drupal/releases/11.0.0-rc1
The Drop is Always Moving: Open source Drupal Starshot slide deck from Lauri Timmanee and Gábor Hojtsy with live recording from Drupal Developer Days is now available at https://www.drupal.org/about/starshot/blog/open-source-drupal-starshot-slide-deck...
Open source Drupal Starshot slide deck from Lauri Timmanee and Gábor Hojtsy with live recording from Drupal Developer Days is now available at https://www.drupal.org/about/starshot/blog/open-source-drupal-starshot-slide-deck-with-recording-is-now-available Present this at your company, meetup, Drupal 11 launch party, and so on!
Drupal Starshot blog: Open source Drupal Starshot slide deck with recording is now available
Lauri Timmanee and myself presented a keynote on Drupal Starshot at Drupal Developer Days 2024 two weeks ago in Burgas, Bulgaria. We got requests from many people that more content on Starshot would be great, so we decided to open source our slide deck! The organizers just published the video recording as well, so while we did not put in speaker notes to these slides, you can get the details from the recording.
Feel free to make a copy of the slides and present this at your company, meetup, Drupal 11 launch party, and so on!
Check out the Drupal Starshot marketing page for further marketing resources, logos and backgrounds.
Promet Source: Scaling Government: Open Source vs Proprietary CMS
Python Software Foundation: Announcing Our New PyPI Support Specialist!
We are thrilled to announce that our first-ever search for a dedicated PyPI Support Specialist has concluded with the hire of Maria Ashna, the newest member of the Python Software Foundation (PSF) staff. Reporting to Ee Durbin, Director of Infrastructure, Maria joins us from a background in academic research, technical consulting, and theatre.
Maria will help the PSF to support one of our most critical services, the Python Package Index (PyPI). Over the past 23 years, PyPI has seen essentially exponential growth in traffic and users, relying for the most part on volunteers to support it. With the addition of requirements to keep all Python maintainers and users safe, our support load has outstretched our support resources for some time now. The Python Software Foundation committed to hiring to increase this capacity in April and we’re excited to have Maria on board to begin providing crucially needed support.
From Maria, “I am a firm believer in democratizing tech. The Open Source community is the lifeblood of such democratization, which is why I am excited to be part of PSF and to serve this community.”
As you see Maria around the PyPI support inbox, issue tracker, and discuss.python.org in the future we hope that you’ll extend a warm welcome! We’re eager to get her up and running to reduce the stress that users have been experiencing around PyPI support and further our work to improve and extend PyPI sustainably.
The Drop is Always Moving: For the first time, Drupal Starshot adds a team of advisors to the leadership team. This council will provide strategic input and feedback to help ensure Starshot meets the needs of key stakeholders and end-users. Members are...
For the first time, Drupal Starshot adds a team of advisors to the leadership team. This council will provide strategic input and feedback to help ensure Starshot meets the needs of key stakeholders and end-users. Members are announced now at https://www.drupal.org/about/starshot/blog/announcing-the-drupal-starshot-advisory-council
Drupal Starshot blog: Announcing the Drupal Starshot Advisory Council
I'm excited to announce the formation of the Drupal Starshot Advisory Council. When I announced Starshot's Leadership Team, I explained that we are innovating on the leadership model by adding a team of advisors. This council will provide strategic input and feedback to help ensure Drupal Starshot meets the needs of key stakeholders and end-users.
The Drupal Starshot initiative represents an ambitious effort to expand Drupal's reach and impact. To guide this effort, we've established a diverse Advisory Council that includes members of the Drupal Starshot project team, Drupal Association staff and Board of Directors, representatives from Drupal Certified Partners, Drupal Core Committers, and last but not least, individuals representing the target end-users for Drupal Starshot. This ensures a wide range of perspectives and expertise to inform the project's direction and decision-making.
The initial members include:
- Imre Gmelig Meijling, React Online / Drupal Association Board - Fundraising Committee
- Suzanne Dergachev, Evolving Web / Drupal Association Board - Marketing Committee
- Mike Herchel, Agileana / Drupal Association Board - Innovation Committee
- Tim Doyle, CEO at the Drupal Association
- Tim Lehnen, CTO at the Drupal Association
- Kristen Pol, Salsa Digital - Drupal Certified Partner representative
- Chris Yates, Pantheon - Drupal Certified Partner representative
- Luis Ribeiro, CI&T - Drupal Certified Partner representative
- Kathryn Carruthers, alakasam - End User representative
- Emma Horrell, University of Edinburgh - End User representative
- Nathaniel Catchpole (catch), Third and Grove / Tag1 Consulting - Drupal Core Committer
- Théodore Biadala (nod_), Très Bien Tech - Drupal Core Committer
The council has been meeting monthly to receive updates from myself and the Drupal Starshot Leadership Team. Members will provide feedback on project initiatives, offer recommendations, and share insights based on their diverse experiences and areas of expertise.
In addition to guiding the strategic direction of Drupal Starshot, the Advisory Council will play a vital role in communication and alignment between the Drupal Starshot team, the Drupal Association, Drupal Core, and the broader Drupal community.
I'm excited to be working with this accomplished group to make the Drupal Starshot vision a reality. Together we can expand the reach and impact of Drupal, and continue advancing our mission to make the web a better place.
This blog has been re-posted and edited with permission from Dries Buytaert's blog.
File attachments: starshot-council-1920w.jpg
Real Python: The Real Python Podcast – Episode #212: Digging Into Graph Theory in Python With David Amos
Have you wondered about graph theory and how to start exploring it in Python? What resources and Python libraries can you use to experiment and learn more? This week on the show, former co-host David Amos returns to talk about what he's been up to and share his knowledge about graph theory in Python.
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
The Drop is Always Moving: Announced in https://www.drupal.org/drupalorg/blog/ending-packagesdrupalorg-support-for-composer-1, Drupal Composer 1 support is being phased out! 1️⃣ New Drupal.org packages/releases will not be available for Composer 1...
Announced in https://www.drupal.org/drupalorg/blog/ending-packagesdrupalorg-support-for-composer-1, Drupal Composer 1 support is being phased out!
1️⃣ New Drupal.org packages/releases will not be available for Composer 1 after Aug 12, 2024.
2️⃣ Composer 1 support for older packages will be dropped after Oct 1, 2024.
Akademy 2024 T-Shirt orders open
Pre-orders are now open for the Akademy 2024 T-shirt, this is only for those who will be attending Akademy, in person, in Würzburg
Pre-orders will close on 31st July
We will be opening in a few weeks separate online orders for t-shirts for those who aren't attending in person
Web Review, Week 2024-28
Let’s go for my web review for the week 2024-28.
Take action to stop chat control now!Tags: tech, politics, law, privacy
It’s time to push European governments to abandon this nonsense.
https://www.patrick-breyer.de/en/take-action-to-stop-chat-control-now/
Tags: tech, ai, machine-learning, gpt, computer-vision
Those brand new models keep failing at surprisingly simple tasks.
https://vlmsareblind.github.io/
Tags: tech, browser, security
This is a concerning finding. One can escape from the browser to the system with such chaining.
https://spaceraccoon.dev/universal-code-execution-browser-extensions/
Tags: tech, security, supply-chain, dependencies
Good tour of all the way dependencies might get compromised in your supply chain. Getting this easy to detect is needed.
https://kerkour.com/supply-chain-attacks-and-backdoored-dependencies
Tags: tech, ubuntu, security
The title is a bit pushing it. Still, I didn’t realize some of the fine prints of the Ubuntu support schemes.
https://gld.mcphail.uk/posts/ubuntu-security-updates-are-a-confusing-mess/
Tags: tech, linux, kernel, rust
It’s nice to have a balanced view on the matter. It’s not just roses and rainbows. This gives a good overview of the current limitations and where Rust can give most benefits in the kernel.
https://www.usenix.org/conference/atc24/presentation/li-hongyu
Tags: tech, linux, system, systemd
Interesting approach to test system changes. Especially welcome on immutable systems.
https://www.codethink.co.uk/articles/2024/A-new-way-to-develop-on-Linux/
Tags: tech, linux, uefi, boot, system
Looks like GRUB days as the standard bootloader are counted. Booting straight using the Linux kernel could bring interesting benefits. Probably not doable on every hardware platform though.
https://fizuxchyk.wordpress.com/2024/06/13/nmbl-we-dont-need-a-bootloader/
Tags: tech, linux, packaging
An interesting puzzle to pursue. Is it possible to rebuild exactly the same binary distribution packages?
https://blog.josefsson.org/2024/07/10/towards-idempotent-rebuilds/
Tags: tech, simd, performance
Another interesting algorithm to handle using SIMD.
https://clement-jean.github.io/simd_binary_search_tree/
Tags: tech, linux, system
It’s really a good reminder of how powerful ptrace is. You can nicely intercept and change the behavior of syscalls with it.
https://healeycodes.com/making-python-less-random
Tags: tech, cloud, storage, cryptography, security, tools
Looks like a nice tool indeed. Might be handy.
https://www.andreagrandi.it/posts/cryptomator-end-to-end-encrypt-files-in-cloud/
Tags: tech, databases
A simple explanation about dirty writes during database transactions.
https://surfingcomplexity.blog/2024/07/05/dirty-writes/
Tags: tech, databases, uuid, performance
Forced to use UUID as primary key in a table? Then make sure to use them properly to not kill the performance more than necessary. Ideally use something else though.
https://maciejwalkowiak.com/blog/postgres-uuid-primary-key/
Tags: tech, programming, python
Ever wondered how attributes work in Python under the hood? Here is how.
https://snarky.ca/unravelling-attribute-access-in-python/
Tags: tech, programming, python
An interesting Python construct to make iterators based on a simple function.
https://mathspp.com/blog/til/making-an-iterator-out-of-a-function
Tags: tech, gui, fonts
Very long read but will be an essential resource to have a fine understanding of text rendering in its current form.
Tags: tech, quality, craftsmanship, engineering, complexity, history
Interesting musing about the “software crisis” which was declared in the late 60s. We’re coping with it by piling levels of abstractions but clearly we’re still not out of it. Our craft still needs to grow.
https://wryl.tech/log/2024/the-software-crisis.html
Tags: tech, team, organization, agile
Good reminder that teams are made out of people. It’s good to look at the daily standups less as a technical management tool and more as a need to get into the work.
https://tidyfirst.substack.com/p/standups-individual-teammate
Bye for now!
Talk Python to Me: #470: Python in Medicine and Patient Care
Russ Allbery: Review: The Splinter in the Sky
Review: The Splinter in the Sky, by Kemi Ashing-Giwa
Publisher: Saga Press Copyright: July 2023 ISBN: 1-6680-0849-1 Format: Kindle Pages: 372The Splinter in the Sky is a stand-alone science fiction political thriller. It is Kemi Ashing-Giwa's first novel.
Enitan is from Koriko, a vegetation-heavy moon colonized by the Vaalbaran empire. She lives in the Ijebu community with her sibling Xiang and has an on-again, off-again relationship with Ajana, the Vaalbaran-appointed governor. Xiang is studying to be an architect, which requires passing stringent entrance exams to be allowed to attend an ancillary imperial school intended for "primitives." Enitan works as a scribe and translator, one of the few Korikese allowed to use the sacred Orin language of Vaalbara. In her free time, she grows and processes tea.
When Xiang mysteriously disappears while she's at work, Enitan goes to Ajana for help. Then Ajana dies, supposedly from suicide. The Vaalbaran government demands a local hostage while the death is investigated, someone who will be held as a diplomatic "guest" on the home world and executed if there is any local unrest. This hostage is supposed to be the child of the local headwoman, a concept that the Korikese do not have. Seeing a chance to search for Xiang, Enitan volunteers, heading into the heart of imperial power with nothing but desperate determination and a tea set.
The empire doesn't stand a chance.
Admittedly, a lot of the reason why the empire doesn't stand a chance is because the author is thoroughly on Enitan's side. Before she even arrives on Gondwana, Vaalbara's home world, Enitan is recruited as a spy by the other Gondwana power and Vaalbara's long-standing enemy. Her arrival in the Splinter, the floating arcology that serves as the center of Vaalbaran government, is followed by a startlingly meteoric rise in access. Some of this is explained by being a cultural curiosity for bored nobles, and some is explained by political factors Enitan is not yet aware of, but one can see the author's thumb resting on the scales.
This was the sort of book that was great fun to read, but whose political implausibility provoked "wait, that didn't make sense" thoughts afterwards. I think one has to assume that the total population of Vaalbara is much less than first comes to mind when considering an interplanetary empire, which would help explain the odd lack of bureaucracy. Enitan is also living in, effectively, the palace complex, for reasonably well-explained political reasons, and that could grant her a surprising amount of access. But there are other things that are harder to explain away: the lack of surveillance, the relative lack of guards, and the odd political structure that's required for the plot to work.
It's tricky to talk about this without spoilers, but the plot rests heavily on a conspiratorial view of how government power is wielded that I think strains plausibility. I'm not naive enough to think that the true power structure of a society matches the formal power structure, but I don't think they diverge as much as people think they do. It's one thing to say that the true power brokers of society can be largely unknown to the general population. In a repressive society with a weak media, that's believable. It's quite another matter for the people inside the palace to be in the dark about who is running what.
I thought that was the biggest problem with this book. Its greatest feature is the characters, and particularly the character relationships. Enitan is an excellent protagonist: fascinating, sympathetic, determined, and daring in ways that make her success more believable. Early in the book, she forms an uneasy partnership that becomes the heart of the book, and I loved everything about that relationship. The politics of her situation might be a bit too simple, but the emotions were extremely well-done.
This is a book about colonialism. Specifically, it's a book about cultural looting, appropriation, and racist superiority. The Vaalbarans consider Enitan barely better than an animal, and in her home they're merciless and repressive. Taken out of that context into their imperial capital, they see her as a harmless curiosity and novelty. Enitan exploits this in ways that are entirely believable. She is also driven to incandescent fury in ways that are entirely believable, and which she only rarely can allow herself to act on. Ashing-Giwa drives home the sheer uselessness of even the more sympathetic Vaalbarans more forthrightly than science fiction is usually willing to be. It's not a subtle point, but it is an accurate one.
The first two thirds of this book had me thoroughly engrossed and unable to put it down. The last third unfortunately turns into a Pokémon hunt of antagonists, which I found less satisfying and somewhat less believable. I wish there had been more need for Enitan to build political alliances and go deeper into the social maneuverings of the first part of the book, rather than gaining some deus ex machina allies who trivially solve some otherwise-tricky plot problems. The setup is amazing; the resolution felt a bit like escaping a maze by blasting through the walls, which I don't think played to the strengths of the characters and relationships that Ashing-Giwa had constructed. The advantage of that approach is that we do get a satisfying resolution and a standalone novel.
The central relationship of the book is unfortunately too much of a spoiler to talk about in a review, but I thought it was the best part of the story. This is a political thriller on the surface, but I think it's heart is an unexpected political alliance with a fascinatingly tricky balance of power. I was delighted that Ashing-Giwa never allows the tension in that relationship to collapse into one of the stock patterns it so easily could have become.
The Splinter in the Sky reminded me a little of Arkady Martine's A Memory Called Empire. It's not as assured or as adroitly balanced as that book, and the characters are not quite as memorable, but that's a very high bar. The political point is even sharper, and it has some of the same appeal.
I had so much fun reading this book. You may need to suspend your disbelief about some of the politics, and I wish the conclusion had been a bit less brute-force, but this is great stuff. Recommended when you're in the mood for a character story in the trappings of a political thriller.
Rating: 8 out of 10
Freexian Collaborators: Monthly report about Debian Long Term Support, June 2024 (by Roberto C. Sánchez)
Like each month, have a look at the work funded by Freexian’s Debian LTS offering.
Debian LTS contributorsIn June, 18 contributors have been paid to work on Debian LTS, their reports are available:
- Adrian Bunk did 47.0h (out of 74.25h assigned and 11.75h from previous period), thus carrying over 39.0h to the next month.
- Arturo Borrero Gonzalez did 6.0h (out of 6.0h assigned).
- Bastien Roucariès did 20.0h (out of 20.0h assigned).
- Ben Hutchings did 15.5h (out of 16.0h assigned and 8.0h from previous period), thus carrying over 8.5h to the next month.
- Chris Lamb did 18.0h (out of 18.0h assigned).
- Daniel Leidert did 4.0h (out of 8.0h assigned and 2.0h from previous period), thus carrying over 6.0h to the next month.
- Emilio Pozuelo Monfort did 23.25h (out of 49.5h assigned and 10.5h from previous period), thus carrying over 36.75h to the next month.
- Guilhem Moulin did 4.5h (out of 13.0h assigned and 7.0h from previous period), thus carrying over 15.5h to the next month.
- Lee Garrett did 17.0h (out of 25.0h assigned and 35.0h from previous period), thus carrying over 43.0h to the next month.
- Lucas Kanashiro did 5.0h (out of 10.0h assigned and 10.0h from previous period), thus carrying over 15.0h to the next month.
- Markus Koschany did 40.0h (out of 40.0h assigned).
- Ola Lundqvist did 10.0h (out of 6.5h assigned and 17.5h from previous period), thus carrying over 14.0h to the next month.
- Roberto C. Sánchez did 5.25h (out of 7.75h assigned and 4.25h from previous period), thus carrying over 6.75h to the next month.
- Santiago Ruano Rincón did 22.5h (out of 14.5h assigned and 8.0h from previous period).
- Sean Whitton did 6.5h (out of 6.0h assigned and 0.5h from previous period).
- Stefano Rivera did 0.5h (out of 0.0h assigned and 10.0h from previous period), thus carrying over 9.5h to the next month.
- Sylvain Beucler did 9.0h (out of 24.5h assigned and 35.5h from previous period), thus carrying over 51.0h to the next month.
- Thorsten Alteholz did 14.0h (out of 14.0h assigned).
In June, we have released 31 DLAs.
Notable security updates in June included:
- git: multiple vulnerabilities, which may result in privilege escalation, denial of service, and arbitrary code execution
- sendmail: SMTP smuggling allowed remote attackers bypass SPF protection checks
- cups: arbitrary remote code execution
Looking further afield to the broader Debian ecosystem, LTS contributor Bastien Roucariès also patched sendmail in Debian 12 (bookworm) and 11 (bullseye) in order to fix the previously mentioned SMTP smuggling vulnerability. Furthermore, LTS contributor Thorsten Alteholz provided fixes for the cups packages in Debian 12 (bookworm) and 11 (bullseye) in order to fix the aforementioned arbitrary remote code execution vulnerability.
Additionally, LTS contributor Ben Hutchings has commenced work on an updated backport of Linux kernel 6.1 to Debian 11 (bullseye), in preparation for bullseye transitioning to the responsibility of the LTS (and the associated closure of the bullseye-backports repository). LTS Lucas Kanashiro also began the preparatory work of backporting parts of the rust/cargo toolchain to Debian 11 (bullseye) in order to make future updates of the clamav virus scanner possible.
June was the final month of LTS for Debian 10 (as announced on the debian-lts-announce mailing list). No additional Debian 10 security updates will be made available on security.debian.org.
However, Freexian and its team of paid Debian contributors will continue to maintain Debian 10 going forward for the customers of the Extended LTS offer. Subscribe right away if you sill have Debian 10 which must be kept secure (and which cannot yet be upgraded).
Thanks to our sponsorsSponsors that joined recently are in bold.
- Platinum sponsors:
- TOSHIBA (for 105 months)
- Civil Infrastructure Platform (CIP) (for 73 months)
- VyOS Inc (for 37 months)
- Gold sponsors:
- Roche Diagnostics International AG (for 115 months)
- Linode (for 109 months)
- Babiel GmbH (for 99 months)
- Plat’Home (for 98 months)
- CINECA (for 73 months)
- University of Oxford (for 55 months)
- Deveryware (for 42 months)
- EDF SA (for 27 months)
- Dataport AöR
- Silver sponsors:
- Domeneshop AS (for 120 months)
- Nantes Métropole (for 114 months)
- Univention GmbH (for 106 months)
- Université Jean Monnet de St Etienne (for 106 months)
- Ribbon Communications, Inc. (for 100 months)
- Exonet B.V. (for 90 months)
- Leibniz Rechenzentrum (for 84 months)
- Ministère de l’Europe et des Affaires Étrangères (for 68 months)
- Cloudways by DigitalOcean (for 57 months)
- Dinahosting SL (for 55 months)
- Bauer Xcel Media Deutschland KG (for 49 months)
- Platform.sh SAS (for 49 months)
- Moxa Inc. (for 43 months)
- sipgate GmbH (for 41 months)
- OVH US LLC (for 39 months)
- Tilburg University (for 39 months)
- GSI Helmholtzzentrum für Schwerionenforschung GmbH (for 30 months)
- Soliton Systems K.K. (for 27 months)
- THINline s.r.o. (for 3 months)
- Bronze sponsors:
- Evolix (for 120 months)
- Seznam.cz, a.s. (for 120 months)
- Intevation GmbH (for 117 months)
- Linuxhotel GmbH (for 117 months)
- Daevel SARL (for 116 months)
- Bitfolk LTD (for 115 months)
- Megaspace Internet Services GmbH (for 115 months)
- Greenbone AG (for 114 months)
- NUMLOG (for 114 months)
- WinGo AG (for 113 months)
- Ecole Centrale de Nantes - LHEEA (for 109 months)
- Entr’ouvert (for 104 months)
- Adfinis AG (for 102 months)
- Tesorion (for 97 months)
- GNI MEDIA (for 96 months)
- Laboratoire LEGI - UMR 5519 / CNRS (for 96 months)
- Bearstech (for 88 months)
- LiHAS (for 88 months)
- Catalyst IT Ltd (for 83 months)
- Supagro (for 78 months)
- Demarcq SAS (for 77 months)
- Université Grenoble Alpes (for 63 months)
- TouchWeb SAS (for 55 months)
- SPiN AG (for 52 months)
- CoreFiling (for 48 months)
- Institut des sciences cognitives Marc Jeannerod (for 43 months)
- Observatoire des Sciences de l’Univers de Grenoble (for 39 months)
- Tem Innovations GmbH (for 34 months)
- WordFinder.pro (for 33 months)
- CNRS DT INSU Résif (for 32 months)
- Alter Way (for 25 months)
- Institut Camille Jordan (for 15 months)
Matt Layman: Trial Banner Inclusion Tag - Building SaaS #195
KDE Ships Frameworks 6.4.0
Friday, 12 July 2024
KDE today announces the release of KDE Frameworks 6.4.0.
KDE Frameworks are 72 addon libraries to Qt which provide a wide variety of commonly needed functionality in mature, peer reviewed and well tested libraries with friendly licensing terms. For an introduction see the KDE Frameworks release announcement.
This release is part of a series of planned monthly releases making improvements available to developers in a quick and predictable manner.
New in this version Attica- Gitignore: add VS Code dir. Commit.
- Revert "[Extractor] Change to QCoreApplication". Commit. Fixes bug #487628
- Port QML modules to declarative type registration. Commit.
- CI: update clang-format image. Commit.
- Add core/primary services for LE Audio: CSIS, MCS, BASS, PACS, CAS. Commit.
- Add comments to improve readability and maintainability. Commit.
- Manager: Add list property for connectedDevices. Commit.
- Manager: Sort out property shorthand in tests. Commit.
- Use newly introduced ColorScheme-Accent. Commit. Fixes bug #446468
- Improve excalamation point placement in data-warning/dialog-warning icon. Commit. Fixes bug #487626
- Add generic translate icon, symlink crow-translate-tray. Commit.
- Add non-symbolic base version of network-wireless-bluetooth icons. Commit.
- Mark as non gui executables to have output on Windows. Commit.
- Avoid spurious message about appstreamtest. Commit.
- Gitignore: add VS Code dir. Commit.
- Add missing QtCore dependency to QML module. Commit.
- Add QML bindings for CalendarListModel and CalendarPluginLoader. Commit.
- Add CalendarListModel. Commit.
- Plugins: Use help-about icons instead of dialog-information. Commit.
- SettingHighlighterPrivate: Optimize connections. Commit.
- Plugins: Use symbolic versions of the icons. Commit.
- Declare missing QML module dependency. Commit.
- Properly declare KQuickConfigModule as having attached properties. Commit.
- Support USE_DBUS option to enable/disable dbus use. Commit.
- Gitignore: add VS Code dir. Commit.
- Ensure KCM load function is called only once. Commit. Fixes bug #487388
- Remove unused static tables. Commit.
- Gitignore: add VS Code dir. Commit.
- Fix two cmakelint types of issue. Commit.
- Fix KCONFIG_USE_QML=OFF by moving ECMQmlModule behind the conditional. Commit.
- Kconfig_compiler: Don't generate .moc include. Commit.
- Kconfig_compiler: Allow generating QML type registration macros. Commit.
- Kconfig_compiler: Use automoc for GENERATE_MOC. Commit.
- Avoid unused variable definition. Commit.
- Allow desktop files to be a symbolic link. Commit.
- Fix compile. Commit.
- KStandardActions: Use C++11 for loop. Commit.
- Fix compile dependencies for windows. Commit.
- Register KCoreConfigSkeleton to QML. Commit.
- Provide a generic named USE_DBUS option to allow to choose if to use it. Commit.
- Don't warn if no dbus enabled. Commit.
- Support USE_DBUS option to enable/disable dbus use. Commit.
- Rename deprecated argument CLASSNAME to CLASS_NAME in ecm_add_qml_module() for KCoreAddonsPlugin. Commit.
- Kurlmimedata: warn about problems while retrieving files from portal. Commit.
- Decode os-release as UTF-8. Commit.
- Use REQUIRED_QT_VERSION for dbus, too. Commit.
- Provide USE_DBUS option to allow to turn on/off dbus parts. Commit.
- Add missing QtOpenGL dependency. Commit.
- Gitignore: add VS Code dir. Commit.
- Add missing copyright to KeySequenceItem. Commit.
- KColorSchemeWatcherMac: update name in copyright. Commit.
- Clipboard: Add roundtrip when setting clipboard. Commit. Fixes bug #466414
- Support USE_DBUS option to enable/disable dbus use. Commit.
- Add holidays for Germany, Bremen and Germany, Hamburg. Commit.
- Add public holidays for Uzbekistan. Commit.
- Holiday_pe_es - add Peru holidays per "Decreto Legislativo 713". Commit. Fixes bug #488513
- Update russian holidays substitution for 2021, 2022, 2023, 2024. Commit.
- Update holiday_ph_en - fix syntax error "of august" should be "in august". Commit.
- Change optional name to Philippines. Commit.
- Introduce a holidays file for Philippines. Commit.
- Make country name detection substring matching more strict. Commit.
- Flip logic when to use Accent/HighlightedText. Commit.
- Support USE_DBUS option to enable/disable dbus use. Commit.
- Fixed wrong plugin options behaviour. Commit.
- EXR: Full support for gray image/colorspace. Commit.
- Disable JXR plugin due to crashes in JXRLIB. Commit.
- PFM: Portable FloatMap read only support. Commit.
- PXR: Pixar raster read only support. Commit.
- Sanity checks (fuzzer). Commit.
- XCF: fixed wrong composite on Grayscale images. Commit. See bug #476755
- Fix compilation. Commit.
- Use of metadata macro definitions. Commit.
- JXR: added CMYK image to read test. Commit.
- XCF: Increased maximum property size. Commit. See bug #426222
- JXR support. Commit. See bug #451584
- PSD: support native CMYK introduced by Qt 6.8. Commit.
- Core: use QString to format number. Commit.
- Remove unnecessary space character in empty text file template. Commit. Fixes bug #487768
- Port DropJob to KJobWidgets::windowHandle(). Commit. See bug #488515
- Use resolved executable in KProcessRunner::fromExecutable. Commit.
- Handle using parent environment correctly in SystemdProcessRunner. Commit.
- Prepare environment for systemd in SystemdProcessRunner. Commit.
- Enable CommandLauncherJob tests (modulo _KDE_APPLICATIONS_AS_SERVICE on CI). Commit.
- Kmountpoint: use advised call params when possible for libmount. Commit.
- Remove unused includes. Commit.
- Drop unused modifiers support from user agent code. Commit.
- [kprotocolmanager] Remove unused http_config. Commit.
- Remove unused includes. Commit.
- Don't pass proxy config to workers. Commit.
- Drop protocol proxying. Commit.
- Deprecate KProtocolInfo::proxiedBy. Commit.
- [ftp] Drop ProxiedBy=http. Commit.
- Kfileitem.cpp: If fileMode is not unknown, set InitCalled to true in readUDSEntry. Commit. Fixes bug #485771
- Support USE_DBUS option to enable/disable dbus use. Commit.
- [webdav] emit ERR_UNSUPPORTED_ACTION when performing fileSystemFreeSpace on a file. Commit.
- Add desktop file for kiod. Commit. Fixes bug #459054
- Gitignore: add VS Code dir. Commit.
- Kfileitemtest: testBasicDirectory relax size assertion. Commit.
- Commandlauncherjob: emit error when no executable. Commit.
- Fix: install KirigamiPrivate soversion. Commit.
- Add Kirigmai.Action unit tests. Commit.
- Fix static build. Commit.
- Allow initializing a Kirigami.Action from a QAction. Commit.
- NavigationTabButton: improve text legibility. Commit. Fixes bug #489573
- Dialog: Add horizontal padding to prevent content from leaking onto borders. Commit.
- ActionTextField: de-duplicate action icon code. Commit.
- Dialogs/Dialog.qml: removed colorSet. Commit.
- ActionTextField: fix QML errors. Commit.
- Update build.gradle in application template to Qt 6. Commit.
- Remove unused KItemModels import. Commit.
- Fix: The issue of abnormal touch events. Commit.
- Relax property types to QtQuick.Templates counterparts. Commit. See bug #487904
- Units: decouple grid unit from font metrics. Commit.
- Fix typo. Commit.
- Support USE_DBUS option to enable/disable dbus use. Commit.
- PlaceholderMessage: Announce the text and description to a11y. Commit. Fixes bug #482753
- Kdescendantsproxymodel: fix nested signals on source model reset. Commit.
- Kdescendantsproxymodeltest: use model tester in the tests where it passes. Commit.
- Port kdescendantsproxymodeltest to std::unique_ptr. Commit.
- Relicense files from LGPL-2.0-only to LGPL-2.0-or-later. Commit.
- Gitignore: add VS Code dir. Commit.
- KWidgetItemDelegateEventListener: remove useless destroy event sending. Commit.
- Remove Icons view. Commit.
- Remove user-facing view switcher. Commit.
- Use Tiles view by default. Commit.
- Add punctuation to warning. Commit.
- BigPreviewDelegate: Fix "Update" action. Commit.
- Action: Remove dead code. Commit.
- Gitignore: add VS Code dir. Commit.
- Fix missing HAVE_DBUS usages. Commit.
- Support USE_DBUS option to enable/disable dbus use. Commit.
- Adapt Android default notifications and documentations to KF6 search path. Commit.
- Remove stray lib. Commit.
- Support USE_DBUS option to enable/disable dbus use. Commit.
- Gitignore: add VS Code dir. Commit.
- PartLoader: Allow reading of InitialPreference from KParts object. Commit.
- PartLoader: Add mechanism to provide capabilities standardized in JSON metadata. Commit.
- Add missing send. Commit.
- Remove no longer used var. Commit.
- Fix non-dbus path. Commit.
- Support USE_DBUS option to enable/disable dbus use. Commit.
- Fix cmakelint issues. Commit.
- Fix "Connecting from COMPAT signal" warning. Commit.
- Controls: Don't use visible for hiding text in LegendDelegate. Commit.
- Controls/legenddelegate: Always hide name if its width is 0. Commit.
- Fix build with QT_NO_CAST_FROM_ASCII. Commit.
- Gitignore: add VS Code dir. Commit.
- Fix description in metainfo. Commit.
- Fix tier information. Commit.
- Fix masks of completely black mask frames. Commit.
- Restore proper devicepixelratio when extracting from cache. Commit. Fixes bug #488295
- Fix description in metainfo. Commit.
- Remove explicit maintainer from metainfo. Commit.
- Add a tool to test javascript scripts. Commit.
- Fix JS API result of view.executeCommand(). Commit.
- Kateregexpsearch: fix FAST_DEBUG. Commit.
- API documentation: fix typo on kte_design page. Commit.
- Add action to popy the current file name and line. Commit. Fixes bug #488027
- Prefer even indent sizes. Commit. Fixes bug #474505
- Try different way to read file for digest. Commit. See bug #482800
- Remove gradient, looks ugly. Commit.
- Add missing :, are there for all other entries. Commit.
- Don't remember encoding if known to be broken. Commit. Fixes bug #445015
- Ensure we don't keep old session config keys. Commit. Fixes bug #486648
- Gitignore: add VS Code dir. Commit.
- Gitignore: add VS Code dir. Commit.
- Gitignore: add VS Code dir. Commit.
- Fix the since version of KJobWidgets::windowHandle,setWindowHandle. Commit.
- Kjobwidgets: Force setWindow() creating a window handle. Commit. See bug #488515
- Kjobwidgets: Add functions to associate QWindow with a KJob. Commit. See bug #488515
- KTitleWidget: restore size policies to Preferred & vertical centering. Commit. Fixes bug #488319
- KTitleWidget: do not autoFillBackground by default, drop StyledPanel frame. Commit. Fixes bug #475898
- Make compile fine without qt6.7 deprecated methods. Commit.
- Remove old dead code from KDE4. Commit.
- Use static constexpr for global int value definitions instead of enums. Commit.
- Revert "Avoid a crash in broken Konsole code for now, waiting for fix release". Commit.
- Port away from deprecated signal QCheckBox::stateChanged. Commit.
- Support USE_DBUS option to enable/disable dbus use. Commit.
- Gitignore: add VS Code dir. Commit.
- Add missing find_dependency call on Qt6Network. Commit. Fixes bug #489094
- Fixed occasional lag issue with ActiveConnection. Commit.
- Recognize OWE in AP capabilities and connection security. Commit. See bug #464615
- Add VRF and loopback device types. Commit.
- Recognize the "loopback" connection type. Commit.
- Fix check for device and AP cipher compatibility. Commit.
- Adjust scanner-qt6.qml for newer QtMultimedia. Commit.
- JobView: Fix switch-case: add default branch. Commit.
- Plugins/barcode: Fix up QML. Commit.
- Widgets: Rework type registration and property initialization, improve QML. Commit.
- Don't let the Heading overflow. Commit.
- AlternativesModel: Use logging category. Commit.
- AlternativesModel: Drop unused header. Commit.
- Tests: Rework testing share tool. Commit.
- Tests: Set initial properties instead of introspecting objects later. Commit.
- Tests: Correctly check for additional positional arguments. Commit.
- Tests: Remove duplicating calls to add help and version CLI options. Commit.
- Tests: Add missing translation domain. Commit.
- Drop QML import versions. Commit.
- Fix some QML and C++ code style. Commit.
- Use ellipses instead of triple period. Commit.
- AlternativesModel: Use correct role for a tooltip. Commit.
- Kdeconnect: Don't let the Heading overflow. Commit.
- Namespace embedded resources. Commit.
- Support USE_DBUS option to enable/disable dbus use. Commit.
- Gitignore: add VS Code dir. Commit.
- Combobox: Set focus on text field within combobox. Commit.
- Delegates: Use inset to control spacing around hover/selection highlight. Commit.
- Limit focus workaround to affected versions. Commit.
- Itembranchindicators: Store QPersistentModelIndex instead of QModelIndex. Commit.
- Itemindicators: Don't access thread-unsafe data from the render thread. Commit. Fixes bug #487850
- Implement baselineOffset for more controls. Commit.
- KQuickStyleItem: Mark baselineOffset method as const. Commit.
- Implement SplitView with splitter handles. Commit.
- Support USE_DBUS option to enable/disable dbus use. Commit.
- Fix "Connecting from COMPAT signal" warning. Commit.
- Support USE_DBUS option to enable/disable dbus use. Commit.
- Narrow down types of backend objects to Ifaces::DeviceManager. Commit.
- Doc: Port 0 to nullptr. Commit.
- Drop a reference to non-existent interface class. Commit.
- Add missing forward declaration. Commit.
- Gitignore: add VS Code dir. Commit.
- Dublincore.h "organisation" -> "organization" (prefer en_US). Commit.
- Gitignore: add VS Code dir. Commit.
- Add CustomType highlighting and improve import highlighting. Commit.
- Corrected Gleam syntax. Commit.
- Add Gleam syntax. Commit.
- OrgMode: add header-item folding, properties folding, more todo keywords. Commit.
- Avoid that we try to open files without name. Commit.
- [dracula.theme] Fix "ISO/Delphi Extended" for Pascal. Commit.
- Nginx: update for new directives and variables. Commit.
- Systemd unit: update to systemd v256. Commit.
- Add basic syntax highlighting rules for opsi-script. Commit.
- [dracula.theme] Fix "Special Variable" for PHP. Commit.
- Add Typst highlighter. Commit.
Quansight Labs Blog: Free-threaded CPython is ready to experiment with!
Dries Buytaert: Announcing the Drupal Starshot Advisory Council
I'm excited to announce the formation of the Drupal Starshot Advisory Council. When I announced Starshot's Leadership Team, I explained that we are innovating on the leadership model by adding a team of advisors. This council will provide strategic input and feedback to help ensure Drupal Starshot meets the needs of key stakeholders and end-users.
The Drupal Starshot initiative represents an ambitious effort to expand Drupal's reach and impact. To guide this effort, we've established a diverse Advisory Council that includes members of the Drupal Starshot project team, Drupal Association staff and Board of Directors, representatives from Drupal Certified Partners, Drupal Core Committers, and last but not least, individuals representing the target end-users for Drupal Starshot. This ensures a wide range of perspectives and expertise to inform the project's direction and decision-making.
The initial members include:
- Imre Gmelig Meijling, React Online / Drupal Association Board - Fundraising Committee
- Suzanne Dergachev, Evolving Web / Drupal Association Board - Marketing Committee
- Mike Herchel, Agileana / Drupal Association Board - Innovation Committee
- Tim Doyle, CEO at the Drupal Association
- Tim Lehnen, CTO at the Drupal Association
- Kristen Pol, Salsa Digital - Drupal Certified Partner representative
- Chris Yates, Pantheon - Drupal Certified Partner representative
- Luis Ribeiro, CI&T - Drupal Certified Partner representative
- Kathryn Carruthers, alakasam - End User representative
- Emma Horrell, University of Edinburgh - End User representative
- Nathaniel Catchpole (catch), Third and Grove / Tag1 Consulting - Drupal Core Committer
- Théodore Biadala (nod_), Très Bien Tech - Drupal Core Committer
The council has been meeting monthly to receive updates from myself and the Drupal Starshot Leadership Team. Members will provide feedback on project initiatives, offer recommendations, and share insights based on their diverse experiences and areas of expertise.
In addition to guiding the strategic direction of Drupal Starshot, the Advisory Council will play a vital role in communication and alignment between the Drupal Starshot team, the Drupal Association, Drupal Core, and the broader Drupal community.
I'm excited to be working with this accomplished group to make the Drupal Starshot vision a reality. Together we can expand the reach and impact of Drupal, and continue advancing our mission to make the web a better place.
Dries Buytaert: Building my own temperature and humidity monitor
Last fall, we toured the Champagne region in France, famous for its sparkling wines. We explored the ancient, underground cellars where Champagne undergoes its magical transformation from grape juice to sparkling wine. These cellars, often 30 meters deep and kilometers long, maintain a constant temperature of around 10-12°C, providing the perfect conditions for aging and storing Champagne.
25 meters underground in a champagne tunnel, which often stretches for miles/kilometers.After sampling various Champagnes, we returned home with eight cases to store in our home's basement. However, unlike those deep cellars, our basement is just a few meters deep, prompting a simple question that sent me down a rabbit hole: how does our basement's temperature compare?
Rather than just buying a thermometer, I decided to build my own "temperature monitoring system" using open hardware and custom-built software. After all, who needs a simple solution when you can spend evenings tinkering with hardware, sensors, wires and writing your own software? Sometimes, more is more!
The basic idea is this: track the temperature and humidity of our basement every 15 minutes and send this information to a web service. This web service analyzes the data and alerts us if our basement becomes too cold or warm.
I launched this monitoring system around Christmas last year, so it's been running for nearly three months now. You can view the live temperature and historical data trends at https://dri.es/sensors. Yes, publishing our basement's temperature online is a bit quirky, but it's all in good fun.
A screenshot of my basement temperature dashboard.So far, the temperature in our basement has been ideal for storing wine. However, I expect it will change during the summer months.
In the rest of this blog post, I'll share how I built the client that collects and sends the data, as well as the web service backend that processes and visualizes that data.
Hardware usedFor this project, I bought:
- Adafruit ESP32-S3 Feather: A microcontroller board with Wi-Fi and Bluetooth capabilities, serving as the central processing unit of my project.
- Adafruit SHT4x sensor: A high-accuracy temperature and humidity sensor.
- 3.7v 500mAh battery: A small and portable power source.
- STEMMA QT / Qwiic JST SH 4-pin cable: To connect the sensor to the board without soldering.
The total hardware cost was $32.35 USD. I like Adafruit a lot, but it's worth noting that their products often come at a higher cost. You can find comparable hardware for as little as $10-15 elsewhere. Adafruit's premium cost is understandable, considering how much valuable content they create for the maker community.
An ESP32-S3 development board (middle) linked to a Sensirion SHT41 temperature and humidity sensor (left) and powered by a battery pack (right). Client code for Adafruit ESP32-S3 FeatherI developed the client code for the Adafruit ESP32-S3 Feather using the Arduino IDE, a widely used platform for developing and uploading code to Arduino-compatible boards.
The code measures temperature and humidity every 15 minutes, connects to WiFi, and sends this data to https://dri.es/sensors, my web service endpoint.
One of my goals was to create a system that could operate for a long time without needing to recharge the battery. The ESP32-S3 supports a "deep sleep" mode where it powers down almost all its functions, except for the clock and memory. By placing the ESP32-S3 into deep sleep mode between measurements, I was able to significantly reduce power.
Now that you understand the high-level design goals, including deep sleep mode, I'll share the complete client code below. It includes detailed code comments, making it self-explanatory.
[code c]#include "Adafruit_SHT4x.h" #include "Adafruit_MAX1704X.h" #include "WiFiManager.h" #include "ArduinoJson.h" #include "HTTPClient.h" // The Adafruit_SHT4x sensor is a high-precision, temperature and humidity // sensor with an I2C interface. Adafruit_SHT4x sht4 = Adafruit_SHT4x(); // The Adafruit ESP32-S3 Feather comes with a built-in MAX17048 LiPoly / LiIon // battery monitor. The MAX17048 provides accurate monitoring of the battery's // voltage. Utilizing the Adafruit library, not only helps us obtain the raw // voltage data from the battery cell, but also converts this data into a more // intuitive battery percentage or charge level. We will pass on the battery // percentage to the web service endpoint, which can visualize it or use it to // send notifications when the battery needs recharging. Adafruit_MAX17048 maxlipo; // The setup() function is used to initialize the device's hardware and // communications. It's executed once at startup. Here, we begin serial // communication, initialize sensors, connect to Wi-Fi, and send initial // data. void setup() { Serial.begin(115200); // Wait for the serial connection to establish before proceeding further. // This is crucial for boards with native USB interfaces. Without this loop, // initial output sent to the serial monitor is lost. This code is not // needed when running on battery. //delay(1000); // Generates a unique device ID from a segment of the MAC address. // Since the MAC address is permanent and unchanged after reboots, // this guarantees the device ID remains consistent. To achieve a // compact ID, only a specific portion of the MAC address is used, // specifically the range between 0x10000 and 0xFFFFF. This range // translates to a hexadecimal string of a fixed 5-character length, // giving us roughly 1 million unique IDs. This approach balances // uniqueness with compactness. uint64_t chipid = ESP.getEfuseMac(); uint32_t deviceValue = ((uint32_t)(chipid >> 16) & 0x0FFFFF) | 0x10000; char device[6]; // 5 characters for the hex representation + the null terminator. sprintf(device, "%x", deviceValue); // Use '%x' for lowercase hex letters // Initialize the SHT4x sensor: if (sht4.begin()) { Serial.println(F("SHT4 temperature and humidity sensor initialized.")); sht4.setPrecision(SHT4X_HIGH_PRECISION); sht4.setHeater(SHT4X_NO_HEATER); } else { Serial.println(F("Could not find SHT4 sensor.")); } // Initialize the MAX17048 sensor: if (maxlipo.begin()) { Serial.println(F("MAX17048 battery monitor initialized.")); } else { Serial.println(F("Could not find MAX17048 battery monitor!")); } // Insert a short delay to ensure the sensors are ready and their data is stable: delay(200); // Retrieve temperature and humidity data from SHT4 sensor: sensors_event_t humidity, temp; sht4.getEvent(&humidity, &temp); // Get the battery percentage and calibrate if it's over 100%: float batteryPercent = maxlipo.cellPercent(); batteryPercent = (batteryPercent > 100) ? 100 : batteryPercent; WiFiManager wifiManager; // Uncomment the following line to erase all saved WiFi credentials. // This can be useful for debugging or reconfiguration purposes. // wifiManager.resetSettings(); // This WiFi manager attempts to establish a WiFi connection using known // credentials, stored in RAM. If it fails, the device will switch to Access // Point mode, creating a network named "Temperature Monitor". In this mode, // connect to this network, navigate to the device's IP address (default IP // is 192.168.4.1) using a web browser, and a configuration portal will be // presented, allowing you to enter new WiFi credentials. Upon submission, // the device will reboot and try connecting to the specified network with // these new credentials. if (!wifiManager.autoConnect("Temperature Monitor")) { Serial.println(F("Failed to connect to WiFi ...")); // If the device fails to connect to WiFi, it will restart to try again. // This approach is useful for handling temporary network issues. However, // in scenarios where the network is persistently unavailable (e.g. router // down for more than an hour, consistently poor signal), the repeated // restarts and WiFi connection attempts can quickly drain the battery. ESP.restart(); // Mandatory delay to allow the restart process to initiate properly: delay(1000); } // Send collected data as JSON to the specified URL: sendJsonData("https://dri.es/sensors", device, temp.temperature, humidity.relative_humidity, batteryPercent); // WiFi consumes significant power so turn it off when done: WiFi.disconnect(true); // Enter deep sleep for 15 minutes. The ESP32-S3's deep sleep mode minimizes // power consumption by powering down most components, except the RTC. This // mode is efficient for battery-powered projects where constant operation // isn't needed. When the device wakes up after the set period, it runs // setup() again, as the state isn't preserved. Serial.println(F("Going to sleep for 15 minutes ...")); ESP.deepSleep(15 * 60 * 1000000); // 15 mins * 60 secs/min * 1,000,000 μs/sec. } bool sendJsonData(const char* url, const char* device, float temperature, float humidity, float battery) { StaticJsonDocument<200> doc; // Round floating-point values to one decimal place for efficient data // transmission. This approach reduces the JSON payload size, which is // important for IoT applications running on batteries. doc["device"] = device; doc["temperature"] = String(temperature, 1); doc["humidity"] = String(humidity, 1); doc["battery"] = String(battery, 1); // Serialize JSON to a string: String jsonData; serializeJson(doc, jsonData); // Initialize an HTTP client with the provided URL: HTTPClient httpClient; httpClient.begin(url); httpClient.addHeader("Content-Type", "application/json"); // Send a HTTP POST request: int httpCode = httpClient.POST(jsonData); // Close the HTTP connection: httpClient.end(); // Print debug information to the serial console: Serial.println("Sent '" + jsonData + "' to " + String(url) + ", return code " + httpCode); return (httpCode == 200); } void loop() { // The ESP32-S3 resets and runs setup() after waking up from deep sleep, // making this continuous loop unnecessary. }[/code] Further optimizing battery usageWhen I launched my thermometer around Christmas 2023, the battery was at 88%. Today, it is at 52%. Some quick math suggests it's using approximately 12% of its battery per month. Given its current rate of usage, it needs recharging about every 8 months.
Connecting to the WiFi and sending data are by far the main power drains. To extend the battery life, I could send updates less frequently than every 15 minutes, only send them when there is a change in temperature (which is often unchanged or only different by 0.1°C), or send batches of data points together. Any of these methods would work for my needs, but I haven't implemented them yet.
Alternatively, I could hook the microcontroller up to a 5V power adapter, but where is the fun in that? It goes against the project's "more is more" principle.
Handling web service requestsWith the client code running on the ESP32-S3 and sending sensor data to https://dri.es/sensors, the next step is to set up a web service endpoint to receive this incoming data.
As I use Drupal for my website, I implemented the web service endpoint in Drupal. Drupal uses Symfony, a popular PHP framework, for large parts of its architecture. This combination offers an easy but powerful way for implementing web services, similar to those found across other modern server-side web development frameworks like Laravel, Django, etc.
Here is what my Drupal routing configuration looks like:
[code yaml]sensors.sensor_data: path: '/sensors' methods: [POST] defaults: _controller: '\Drupal\sensors\Controller\SensorMonitorController::postSensorData' requirements: _access: 'TRUE'[/code]The above configuration directs Drupal to send POST requests made to https://dri.es/sensors to the postSensorData() method of the SensorMonitorController class.
The implementation of this method handles request authentication, validates the JSON payload, and saves the data to a MariaDB database table. Pseudo-code:
[code php]public function postSensorData(Request $request) : JsonResponse { $content = $request->getContent(); $data = json_decode($content, TRUE); // Validate the JSON payload: … // Authenticate the request: … $device = DeviceFactory::getDevice($data['device']); if ($device) { $device->recordSensorEvent($data); } return new JsonResponse(['message' => 'Thank you!']); }[/code]For testing your web service, you can use tools like cURL:
[code bash]$ curl -X POST -H "Content-Type: application/json" -d '{"device":"0xdb123", "temperature":21.5, "humidity":42.5, "battery":90.0}' https://localhost/sensors[/code]While cURL is great for quick tests, I use PHPUnit tests for automated testing in my CI/CD workflow. This ensures that everything keeps working, even when upgrading Drupal, Symfony, or other components of my stack.
Storing sensor data in a databaseThe primary purpose of $device->recordSensorEvent() in SensorMonitorController::postSensorData() is to store sensor data into a SQL database. So, let's delve into the database design.
My main design goals for the database backend were:
- Instead of storing every data point indefinitely, only keep the daily average, minimum, maximum, and the latest readings for each sensor type across all devices.
- Make it easy to add new devices and new sensors in the future. For instance, if I decide to add a CO2 sensor for our bedroom one day (a decision made in my head but not yet pitched to my better half), I want that to be easy.
To this end, I created the following MariaDB table:
[code sql]CREATE TABLE sensor_data ( date DATE, device VARCHAR(255), sensor VARCHAR(255), avg_value DECIMAL(5,1), min_value DECIMAL(5,1), max_value DECIMAL(5,1), min_timestamp DATETIME, max_timestamp DATETIME, readings SMALLINT NOT NULL, UNIQUE KEY unique_stat (date, device, sensor) );[/code]A brief explanation for each field:
- date: The date for each sensor reading. It doesn't include a time component as we aggregate data on a daily basis.
- device: The device ID of the device providing the sensor data, such as 'basement' or 'bedroom'.
- sensor: The type of sensor, such as 'temperature', 'humidity' or 'co2'.
- avg_value: The average value of the sensor readings for the day. Since individual readings are not stored, a rolling average is calculated and updated with each new reading using the formula: avg_value = avg_value + new_value - avg_value new_total_readings . This method can accumulate minor rounding errors, but simulations show these are negligible for this use case.
- min_value and max_value: The daily minimum and maximum sensor readings.
- min_timestamp and max_timestamp: The exact moments when the minimum and maximum values for that day were recorded.
- readings: The number of readings (or measurements) taken throughout the day, which is used for calculating the rolling average.
In essence, the recordSensorEvent() method needs to determine if a record already exists for the current date. Depending on this determination, it will either insert a new record or update the existing one.
In Drupal this process is streamlined with the merge() function in Drupal's database layer. This function handles both inserting new data and updating existing data in one step.
[code php]private function updateDailySensorEvent(string $sensor, float $value): void { $timestamp = \Drupal::time()->getRequestTime(); $date = date('Y-m-d', $timestamp); $datetime = date('Y-m-d H:i:s', $timestamp); $connection = Database::getConnection(); $result = $connection->merge('sensor_data') ->keys([ 'device' => $this->id, 'sensor' => $sensor, 'date' => $date, ]) ->fields([ 'avg_value' => $value, 'min_value' => $value, 'max_value' => $value, 'min_timestamp' => $datetime, 'max_timestamp' => $datetime, 'readings' => 1, ]) ->expression('avg_value', 'avg_value + ((:new_value - avg_value) / (readings + 1))', [':new_value' => $value]) ->expression('min_value', 'LEAST(min_value, :value)', [':value' => $value]) ->expression('max_value', 'GREATEST(max_value, :value)', [':value' => $value]) ->expression('min_timestamp', 'IF(LEAST(min_value, :value) = :value, :timestamp, min_timestamp)', [':value' => $value, ':timestamp' => $datetime]) ->expression('max_timestamp', 'IF(GREATEST(max_value, :value) = :value, :timestamp, max_timestamp)', [':value' => $value, ':timestamp' => $datetime]) ->expression('readings', 'readings + 1') ->execute(); }[/code]Here is what the query does:
- It checks if a record for the current sensor and date exists.
- If not, it creates a new record with the sensor data, including the initial average, minimum, maximum, and latest value readings, along with the timestamp for these values.
- If a record does exist, it updates the record with the new sensor data, adjusting the average value, and updating minimum and maximum values and their timestamps if the new reading is a new minimum or maximum.
- The function also increments the count of readings.
For those not using Drupal, similar functionality can be achieved with MariaDB's INSERT ... ON DUPLICATE KEY UPDATE command, which allows for the same conditional insert or update logic based on whether the specified unique key already exists in the table.
Here are example queries, extracted from MariaDB's General Query Log to help you get started:
[code sql]INSERT INTO sensor_data (device, sensor, date, min_value, min_timestamp, max_value, max_timestamp, readings) VALUES ('0xdb123', 'temperature', '2024-01-01', 21, '2024-01-01 00:00:00', 21, '2024-01-01 00:00:00', 1); UPDATE sensor_data SET min_value = LEAST(min_value, 21), min_timestamp = IF(LEAST(min_value, 21) = 21, '2024-01-01 00:00:00', min_timestamp), max_value = GREATEST(max_value, 21), max_timestamp = IF(GREATEST(max_value, 21) = 21, '2024-01-01 00:00:00', max_timestamp), readings = readings + 1 WHERE device = '0xdb123' AND sensor = 'temperature' AND date = '2024-01-01';[/code] Generating graphsWith the data securely stored in the database, the next step involved generating the graphs. To accomplish this, I wrote some custom PHP code that generates Scalable Vector Graphics (SVGs).
Given that is blog post is already quite long, I'll spare you the details. For now, those curious can use the 'View source' feature in their web browser to examine the SVGs on the thermometer page.
ConclusionIt's fun how a visit to the Champagne cellars in France sparked an unexpected project. Choosing to build a thermometer rather than buying one allowed me to dive back into an old passion for hardware and low-level software.
I also like taking control of my own data and software. It gives me a sense of control and creativity.
As Drupal's project lead, using Drupal for an Internet-of-Things (IoT) backend brought me unexpected joy. I just love the power and flexibility of open-source platforms like Drupal.
As a next step, I hope to design and 3D print a case for my thermometer, something I've never done before. And as mentioned, I'm also considering integrating additional sensors. Stay tuned for updates!
Pages
Recent Publications
- Managing Hidden Dependencies in OO Software: a Study Based on Open Source Projects
- Open Source Communities as Liminal Ecosystems
- Investigating developers' email discussions during decision-making in Python language evolution
- Developers, Quality Control and Download Volume in Open Source Software (OSS) Projects
More...