Planet Drupal
Drupal Starshot blog: Join the Drupal Starshot team as a track lead
Drupal Starshot seeks track leads to drive key project components. Apply by July 31st and help bring Drupal Starshot to life!
The Drupal Starshot initiative has been making significant progress behind the scenes, and I'm excited to share some updates with the community.
Leadership team formation and product definitionOver the past few months, we've been working diligently on Drupal Starshot. One of our first steps was to appoint a leadership team to guide the project. With the leadership team in place as well as the new Starshot Advisory Council, we shifted our focus to defining the product. We've made substantial progress on this front and will be sharing more details about the product strategy in the coming weeks.
Introducing Drupal Starshot tracksWe already started to break down the initiative into manageable components, and are introducing the concept of "tracks". Tracks are smaller, focused parts of the Drupal Starshot project that allow for targeted development and contributions. We've already published the first set of tracks on the Drupal Starshot issue queue on Drupal.org.
Example tracks include:
- Creating Drupal Recipes for features like contact forms, advanced search, events, SEO and more.
- Enhancing the Drupal installer to enable Recipes during installation.
- Updating Drupal.org for Starshot, including product marketing and a trial experience.
While many tracks are technical and need help from developers, most of the tracks need contribution from designers, UX experts, marketers, testers and site builders.
Recruiting more track leadsSeveral tracks already have track leads and have made significant progress:
- Matt Glaman is spearheading the development of a trial experience.
- The marketing team, led by Suzanne Dergacheva, is crafting product marketing documentation.
- Martin Anderson-Clutz has been appointed as the track lead for event management.
However, we need many additional track leads to drive our remaining tracks to completion.
We're now accepting applications for track lead positions. Interested individuals and organizations can apply by completing our application form. The application window closes on July 31st, two weeks from today.
Key responsibilities of a track leadTrack leads can be individuals, teams, or organizations, including Drupal Certified Partners. While technical expertise is beneficial, the role primarily focuses on strategic coordination and project management. Key responsibilities include:
- Defining and validating requirements to ensure the track meets the expectations of our target audience.
- Developing and maintaining a prioritized task list, including creating milestones and timelines.
- Overseeing and driving the track's implementation.
- Collaborating with key stakeholders, including the Drupal Starshot leadership team, module maintainers, the marketing team, etc.
- Communicating progress to the community (e.g. blogging).
After the application deadline, the Drupal Starshot Leadership Team will review the applications and appoint track leads. We expect to announce the selected track leads in the first week of August.
While the application period is open, we will be available to answer any questions you may have. Feel free to reach out to us through the Drupal.org issue queue, or join us in an upcoming zoom meeting (details to be announced / figured out).
Looking ahead to DrupalCon BarcelonaOur goal is to make significant progress on these tracks by DrupalCon Barcelona, where we plan to showcase the advancements we've made. We're excited about the momentum building around Drupal Starshot and can't wait to see the contributions from the community.
If you're passionate about Drupal and want to play a key role in shaping its future, consider applying for a track lead position.
Stay tuned for more updates on Drupal Starshot, and thank you for your continued support of the Drupal community.
— Dries Buytaert
Nonprofit Drupal posts: July Drupal for Nonprofits Chat
Join us THURSDAY, July 18 at 1pm ET / 10am PT, for our regularly scheduled call to chat about all things Drupal and nonprofits. (Convert to your local time zone.) We'll be taking August off, so this is the last chat of the summer.
We don't have anything specific on the agenda this month, so we'll have plenty of time to discuss anything that's on our minds at the intersection of Drupal and nonprofits. Got something specific you want to talk about? Feel free to share ahead of time in our collaborative Google doc: https://nten.org/drupal/notes!
All nonprofit Drupal devs and users, regardless of experience level, are always welcome on this call.
This free call is sponsored by NTEN.org and open to everyone.
-
Join the call: https://us02web.zoom.us/j/81817469653
-
Meeting ID: 818 1746 9653
Passcode: 551681 -
One tap mobile:
+16699006833,,81817469653# US (San Jose)
+13462487799,,81817469653# US (Houston) -
Dial by your location:
+1 669 900 6833 US (San Jose)
+1 346 248 7799 US (Houston)
+1 253 215 8782 US (Tacoma)
+1 929 205 6099 US (New York)
+1 301 715 8592 US (Washington DC)
+1 312 626 6799 US (Chicago) -
Find your local number: https://us02web.zoom.us/u/kpV1o65N
-
- Follow along on Google Docs: https://nten.org/drupal/notes
Drupal.org blog: Drupal.org login flow is changing
The Drupal Association engineering team is preparing to switch over to our new single sign-on solution for user login. This is an important step in our work to upgrade Drupal.org, and in the future will give you the ability to use your Drupal.org identity in new ways.
The Drupal Association engineering team is preparing to switch over to our new single sign-on solution for user login. This is an important step in our work to upgrade Drupal.org, and in the future will give you the ability to use your Drupal.org identity in new ways.
The switch-over is scheduled for:
- Thursday 25 July - from 9am to 11am Pacific (4pm-6pm UTC).
During this window you will be unable to login to Drupal.org or edit your profile, and may not be able to access related services which use your Drupal identity, such as git.drupalcode.org.
When we make this change, the login experience will look different.
If you are an existing userWhen you click to login or create an account you will be redirected to: accounts.drupal.org
You will log in with your existing Drupal.org username or email and your current password, and your two factor authentication code if you have TFA enabled.
Once you log in, you will have to change your password.
If you have Two Factor Authentication enabled, you will also have to set up a new seed.
After that, you'll be taken back to Drupal.org as normal. You should be directed back to the path you came from.
If you are creating a new accountWhen you click 'create account' on Drupal.org you will be taken to the new account creation page:
After you complete the basic information, you will be taken to the Drupal.org welcome page to fill out the rest of your user profile.
If you need to change your account informationThe majority of your account information will continue to live in your Drupal.org profile, however, some basic account information will now be stored and updated in the Drupal.org SSO system.
When you click to edit your first and last name, username, password, email address, or enable two factor authentication you'll be taken to the account page:
Setting up Two Factor AuthenticationThis account settings page is also where you can change your Two Factor Authentication settings. You can use the 'Account Security' tab in the sidebar to navigate to the Two Factor setup process:
If you need to reset your passwordIf you have forgotten your password, you can reset your password from the login page:
You will receive a password reset email from noreply@drupal.org allowing you to change your password.
If you encounter any issues with your account, please contact us at help@drupal.org
We want to thank two of our partners for supporting this project.
Cloud-IAM is our SSO partner. Cloud-IAM is a privacy centric provider of hosted solutions for Keycloak, an open source identity management service. They are enthusiastic supporters of the Drupal community, and would like to offer any site owners and agencies who are looking for their own identity and access management service 10% off, with promo code: DRUPAL10.
Our implementation partner on this project was Tag1Consulting. Tag1Consulting is a global team of Drupal experts working with clients from non-profits to the Fortune 500, and is one of the top contributors to Drupal. They have been the Drupal Association's infrastructure partner for many years.
Specbee: Personalizing Experiences in Drupal: CRMs, E-commerce Integration, and UTMs
Drupal Association blog: Drupal Business Survey 2024
The Drupal Business Survey investigates the trends in the digital market, in particular from service providers involved with the open source DXP Drupal. Digital agencies from all over the world participate in the yearly survey. Business insights on market share and growth opportunities are shared in a comprehensive report. The results are presented at DrupalCon Europe, the international Drupal conference. Digital business owners from all over the world are invited to participate.
The Drupal Business Survey shares valuable business insights from Drupal service providers worldwide. Drupal’s open source ecosystem is supported by a strong community of tens of thousands professionals worldwide, working together on the popular digital experience platform. Because Drupal is open source, anyone can work with Drupal or make changes to it. The Drupal Business Survey gives meaningful data for business owners and decision makers to build their next year’s business strategy on.
The Drupal Business Survey results in a comprehensive report on business outlook and customer engagement. It has been a valuable guide for digital service providers, even to those working with other technologies but Drupal. Participants are from all continents, with most of the companies being in business for 10 years or more.
Results in 2023In 2023, the Business Survey underscored Drupal’s powerful selling points including security, flexibility and reliability. Business decision makers choosing Drupal for the digital experiences, found these values to be significantly more important. Drupal’s market share increased notably in 2023 in the finance industry as well as industry and automotive. Education and government remain consistent Drupal users.
Open Source remains one of the most powerful reasons for choosing Drupal. Being vendor independent and using the power is the basis for Drupal’s success. Companies have continued their contribution to the open source ecosystem, in spite of the harder market outlook in 2023.
Also the need for Drupal talent continues to be strong. More than 70% of the companies stated the business growth or need of expertise as their main reason for hiring new staff.
Imre Gmelig Meijling: “We continue to see a strong position of Drupal as valuable solution. Agencies and service providers remain committed to Drupal and pitch it as one of the best content management solutions on the market. The strong commitment to Drupal by so many organizations worldwide proves that Drupal is a solid choice for many brands and organizations across the globe.”
CEO DinnerThe Drupal market is looking forward to how digital service providers are using Drupal and how the Drupal market has evolved this year. The results of the 2024 Drupal Business Survey will be presented at the CEO Dinner at DrupalCon Europe. DrupalCon Europe is the yearly Drupal summit where over 1500 Drupal users and professionals meet to exchange ideas and further evolve Drupal. The conference is held from 24-27 September in Barcelona, Spain. The CEO Dinner will be on Wednesday 25 September, extending the conversation to a BOF (Birds of a Feather) session on 26 September.
About the Business SurveyThe Drupal Business Survey support Drupal businesses worldwide and is organized by a team of industry experts Imre Gmelig Meijling (React Online), Janne Kalliola (Exove) and Michel van Velde (Craftmore) in collaboration with the Drupal Association.
Drupal is the open source Digital Experience Platform used by many organizations worldwide including Nestlé, Lufthansa and WWF.
Participate and share your insightsDrupal experts are invited to share their Drupal business insights through the Business Survey anonymously and come to DrupalCon Europe to review the results together.
You can participate in the Drupal Business Survey anonymously here: The Drupal Business Survey 2024. The survey closes on 4 September.
Talking Drupal: Talking Drupal #459 - Off The Cuff 8
Today we are talking about Config Actions, The Panels Favorite Drupal Modules, and Drupal Contribution. We’ll also cover Transform API as our module of the week.
For show notes visit: www.talkingDrupal.com/459
Topics- New Config Action: Place Block
- Favorite Contrib modules
- Slack channels
- Preparing for Drupal 11
- Drupal events
- Config Action Place Block
- Front End Editing Drupal Module
- Gin Admin Theme
- Migrate Boost
- Keysave
- Navigation
- Matt Glaman Smart Date Code Blog Post
Nic Laflin - nLighteneddevelopment.com nicxvan John Picozzi - epam.com johnpicozzi Martin Anderson-Clutz - mandclu.com mandclu Baddý Sonja Breidert - 1xINTERNET baddysonja
MOTW CorrespondentMartin Anderson-Clutz - mandclu.com mandclu
- Brief description:
- Have you ever wanted to expose your Drupal site’s data as JSON using view modes, formatters, blocks, and more? There’s a module for that.
- Module name/project name:
- https://www.drupal.org/project/transform_api
- Transform API
- Brief history
- How old: created in Sep 2023 by LupusGr3y, aka Martin Giessing of Denmark
- Versions available: 1.1.0-beta4 and 1.0.2 versions available, both of which work with Drupal 9 and 10
- Maintainership
- Actively maintained, in fact the latest commit was earlier today
- Security coverage
- Documentation: in-depth README and a full user guide
- Number of open issues: 14 open issues, 3 of which are bugs, but none against the current branch
- Usage stats:
- 2 sites
- Module features and usage
- After installing Transform API, you should be able to get the JSON for any entities on your site by adding “format=json” as a parameter to the URL
- To get more fields exposed as JSON, you can configure a Transform mode, using a Field UI configuration very similar to view modes
- You can also add transform blocks to globally include specific data in all transformed URLs, in the same way you would use normal blocks to show information on your entity pages. The output of transform blocks is segmented into regions,
- Where Drupal’s standard engine produces render arrays that ultimately become HTML, Transform API replaces it with an engine that produces Transform Arrays that will ultimately become JSON
- Where Drupal’s standard JSON:API supports more or less exposes all information as raw data for the front end to format, Transform API allows for more of the formatting to be managed on the back end, where it will use Drupal’s standard caching mechanisms, permission-based access, and more
- Transform API also supports lazy transformers, which are callbacks that will be called after caching but before the JSON response is sent
- You can also use alter hooks to manipulate the transformed data
The Drop Times: Open is the New Default
Dear Readers,
Governments worldwide are increasingly embracing open-source software (OSS) to enhance their operations' transparency, security, and efficiency. Open source allows anyone to inspect, modify, and improve software, fostering a culture of collaboration and innovation. This shift is transforming the way public sector bodies function, ensuring digital solutions are cost-effective and tailored to the unique needs of each community.
Switzerland's recent mandate for OSS in its public sector is a prime example of this trend. The "Federal Law on the Use of Electronic Means for the Fulfilment of Governmental Tasks" (EMBAG) requires government-developed software to be open source, marking a significant step towards greater transparency and efficiency. However, Switzerland is not alone in this initiative; many countries are adopting similar policies to reduce dependency on costly proprietary software and foster innovation.
One of the notable benefits of open source in government is its potential for increased security and reliability. Continuous scrutiny and improvement by a global community of developers result in robust and adaptable solutions that can quickly respond to changing needs and security threats. Additionally, OSS helps prevent vendor lock-in, allowing governments to choose the best solutions without being tied to specific providers.
A recent study by Veniz Guzman on The DropTimes highlights the growing adoption of the open-source CMS Drupal among government entities of all sizes. Larger governments with complex digital needs are increasingly turning to Drupal for its flexibility and robust features. Interestingly, smaller to mid-sized entities are also recognizing Drupal's benefits, with rising adoption rates among smaller cities and counties.
With that, let's move on to the important stories of last week.
Drupal Starshot has announced the formation of its Advisory Council, a significant step in its efforts to expand the reach and impact of Drupal. This council will provide strategic input and feedback to ensure that the initiative meets the needs of key stakeholders and end users. To learn more about the composition of the Starshot Advisory Council, click here.
In "Using Drupal Migrations to Modify Content Within a Site," Jaymie Strecker demonstrates how to use the Drupal Migrate API for restructuring content within a site. The article provides practical examples and assumes basic knowledge of the Migrate API.
In other news, the Drupal Association has announced HeroDevs as the inaugural partner for the new Drupal 7 Extended Security Support Provider Program. This initiative aims to assist organizations unable to migrate from Drupal 7 by the end-of-life date of January 5, 2025, ensuring their systems remain secure and compliant.
Additionally, Drupal.org has announced that it will end support for Composer 1 on Packages.Drupal.org. This move is part of efforts to prepare its infrastructure for providing automatic updates for Drupal and upgrading Drupal.org itself. New packages and releases will no longer be available for Composer 1 starting August 12, 2024, and full support will be dropped on October 1, 2024.
Doubling the spirit of community, DrupalCon Europe has announced Tomasz Rogalski as the winner of the Merce Drawing Contest. His drawing of Mercè, the celebrated figure of Barcelona's annual festival, will be featured on the official event stickers at DrupalCon Barcelona.
The Women in Drupal award will be presented shortly at DrupalCon Barcelona, an event dedicated to the Drupal community. This award aims to recognize and celebrate the significant contributions made by women within the Drupal ecosystem. To nominate a deserving Drupal artist, attendees can fill out the nomination form provided.
Drupal GovCon 2024 is only a month away, scheduled to happen in August 13 - 15, 2024. It will feature a session titled "What the Heck is ARIA? A Beginner's Guide to ARIA for Accessibility," presented by Kat Shaw, a Lead Engineer and Accessibility Advocate (CPACC) at Lullabot. DrupalCamp Colorado has announced its second Keynote Speaker for this year's event, Matthew Saunders.
The DropTimes has compiled notable Drupal events happening throughout the week of July 15 to July 21. This curated list offers a glimpse into the varied activities taking place within the Drupal community, catering to enthusiasts of all skill levels. Read here.
amazee.io announces the launch of a new cloud region on AWS in Tokyo, Japan, enhancing the company's services in the Asia Pacific region. This expansion provides Japanese customers with reduced latency, improved fault tolerance, and compliance with local data privacy laws.
Roderik Muit and Alexandru Ieremia of drunomics presented the new Custom Elements module at Drupal Developer Days in Burgas, in June 2024. This session, part of an informal "Birds of a Feather" discussion, introduced the latest version of the Custom Elements module, a vital part of drunomics' headless Drupal solutions.
We acknowledge that there are more stories to share. However, due to selection constraints, we must pause further exploration for now.
To get timely updates, follow us on LinkedIn, Twitter and Facebook. Also, join us on Drupal Slack at #thedroptimes.
Thank you,
Sincerely
Alka Elizabeth
Sub-editor, The DropTimes.
The Drop Times: Using Drupal Migrations to Deploy New Content
the floating-point divide: Using Drupal migrations to deploy new content
Following up on my previous article about using Drupal migrations to modify content, this article looks at importing new content into a site. I give examples of migrations that create paragraphs, nodes, URL aliases, and URL redirects.
Wim Leers: XB week 7: Drupal Dev Days Burgas
It’s Drupal Dev Days week! With Lauri presenting a Keynote there and several people out this week (myself included), it was a quiet week for Experience Builder.
Consequently, it was the first week since the 0.x branch was opened that zero merge requests landed…
Even the weekly meeting was fairly quiet.
Missed a prior week? See all posts tagged Experience Builder.
Goal: make it possible to follow high-level progress by reading ~5 minutes/week. I hope this empowers more people to contribute when their unique skills can best be put to use!
For more detail, join the #experience-builder Slack channel. Check out the pinned items at the top!
But on the other side of the planet, undisturbed by things such as vacation or Drupal Dev Days, Lee “larowlan” made a big push forward on [META] Support component types other than SDC, in his MR where he proposes to already remove the current tight coupling to Single-Directory Components (SDC). That tight coupling obviously needs to be removed, but we needed to start somewhere — SDCs were merely the first step.
Still, even at Drupal Dev Days Burgas, important Experience Builder discussions happened: catch, Alex Pott and Dave “longwave” Long discussed the JSON-based data storage model in depth (that was proposed by Alex “effulgentsia” Bronstein). Some important steps forward were made!
Thanks to Lauri for reviewing this!
The Savvy Few: How to dynamically add the view id as a body class for every views page
Here’s a quick guide to add custom CSS classes to the body tag of your page based on the views name for targeted styling of views pages.
Read moreThe Drop is Always Moving: 📢 Drupal 11.0.0-rc1 is now available! Release candidates are not for production sites, they are intended for widespread testing in preparation for the upcoming stable release. Test now! 11.0.0 is planned for the week of July...
📢 Drupal 11.0.0-rc1 is now available! Release candidates are not for production sites, they are intended for widespread testing in preparation for the upcoming stable release. Test now! 11.0.0 is planned for the week of July 29, 2024. https://www.drupal.org/project/drupal/releases/11.0.0-rc1
The Drop is Always Moving: Open source Drupal Starshot slide deck from Lauri Timmanee and Gábor Hojtsy with live recording from Drupal Developer Days is now available at https://www.drupal.org/about/starshot/blog/open-source-drupal-starshot-slide-deck...
Open source Drupal Starshot slide deck from Lauri Timmanee and Gábor Hojtsy with live recording from Drupal Developer Days is now available at https://www.drupal.org/about/starshot/blog/open-source-drupal-starshot-slide-deck-with-recording-is-now-available Present this at your company, meetup, Drupal 11 launch party, and so on!
Drupal Starshot blog: Open source Drupal Starshot slide deck with recording is now available
Lauri Timmanee and myself presented a keynote on Drupal Starshot at Drupal Developer Days 2024 two weeks ago in Burgas, Bulgaria. We got requests from many people that more content on Starshot would be great, so we decided to open source our slide deck! The organizers just published the video recording as well, so while we did not put in speaker notes to these slides, you can get the details from the recording.
Feel free to make a copy of the slides and present this at your company, meetup, Drupal 11 launch party, and so on!
Check out the Drupal Starshot marketing page for further marketing resources, logos and backgrounds.
Promet Source: Scaling Government: Open Source vs Proprietary CMS
The Drop is Always Moving: For the first time, Drupal Starshot adds a team of advisors to the leadership team. This council will provide strategic input and feedback to help ensure Starshot meets the needs of key stakeholders and end-users. Members are...
For the first time, Drupal Starshot adds a team of advisors to the leadership team. This council will provide strategic input and feedback to help ensure Starshot meets the needs of key stakeholders and end-users. Members are announced now at https://www.drupal.org/about/starshot/blog/announcing-the-drupal-starshot-advisory-council
Drupal Starshot blog: Announcing the Drupal Starshot Advisory Council
I'm excited to announce the formation of the Drupal Starshot Advisory Council. When I announced Starshot's Leadership Team, I explained that we are innovating on the leadership model by adding a team of advisors. This council will provide strategic input and feedback to help ensure Drupal Starshot meets the needs of key stakeholders and end-users.
The Drupal Starshot initiative represents an ambitious effort to expand Drupal's reach and impact. To guide this effort, we've established a diverse Advisory Council that includes members of the Drupal Starshot project team, Drupal Association staff and Board of Directors, representatives from Drupal Certified Partners, Drupal Core Committers, and last but not least, individuals representing the target end-users for Drupal Starshot. This ensures a wide range of perspectives and expertise to inform the project's direction and decision-making.
The initial members include:
- Imre Gmelig Meijling, React Online / Drupal Association Board - Fundraising Committee
- Suzanne Dergachev, Evolving Web / Drupal Association Board - Marketing Committee
- Mike Herchel, Agileana / Drupal Association Board - Innovation Committee
- Tim Doyle, CEO at the Drupal Association
- Tim Lehnen, CTO at the Drupal Association
- Kristen Pol, Salsa Digital - Drupal Certified Partner representative
- Chris Yates, Pantheon - Drupal Certified Partner representative
- Luis Ribeiro, CI&T - Drupal Certified Partner representative
- Kathryn Carruthers, alakasam - End User representative
- Emma Horrell, University of Edinburgh - End User representative
- Nathaniel Catchpole (catch), Third and Grove / Tag1 Consulting - Drupal Core Committer
- Théodore Biadala (nod_), Très Bien Tech - Drupal Core Committer
The council has been meeting monthly to receive updates from myself and the Drupal Starshot Leadership Team. Members will provide feedback on project initiatives, offer recommendations, and share insights based on their diverse experiences and areas of expertise.
In addition to guiding the strategic direction of Drupal Starshot, the Advisory Council will play a vital role in communication and alignment between the Drupal Starshot team, the Drupal Association, Drupal Core, and the broader Drupal community.
I'm excited to be working with this accomplished group to make the Drupal Starshot vision a reality. Together we can expand the reach and impact of Drupal, and continue advancing our mission to make the web a better place.
This blog has been re-posted and edited with permission from Dries Buytaert's blog.
File attachments: starshot-council-1920w.jpgThe Drop is Always Moving: Announced in https://www.drupal.org/drupalorg/blog/ending-packagesdrupalorg-support-for-composer-1, Drupal Composer 1 support is being phased out! 1️⃣ New Drupal.org packages/releases will not be available for Composer 1...
Announced in https://www.drupal.org/drupalorg/blog/ending-packagesdrupalorg-support-for-composer-1, Drupal Composer 1 support is being phased out!
1️⃣ New Drupal.org packages/releases will not be available for Composer 1 after Aug 12, 2024.
2️⃣ Composer 1 support for older packages will be dropped after Oct 1, 2024.
Dries Buytaert: Announcing the Drupal Starshot Advisory Council
I'm excited to announce the formation of the Drupal Starshot Advisory Council. When I announced Starshot's Leadership Team, I explained that we are innovating on the leadership model by adding a team of advisors. This council will provide strategic input and feedback to help ensure Drupal Starshot meets the needs of key stakeholders and end-users.
The Drupal Starshot initiative represents an ambitious effort to expand Drupal's reach and impact. To guide this effort, we've established a diverse Advisory Council that includes members of the Drupal Starshot project team, Drupal Association staff and Board of Directors, representatives from Drupal Certified Partners, Drupal Core Committers, and last but not least, individuals representing the target end-users for Drupal Starshot. This ensures a wide range of perspectives and expertise to inform the project's direction and decision-making.
The initial members include:
- Imre Gmelig Meijling, React Online / Drupal Association Board - Fundraising Committee
- Suzanne Dergachev, Evolving Web / Drupal Association Board - Marketing Committee
- Mike Herchel, Agileana / Drupal Association Board - Innovation Committee
- Tim Doyle, CEO at the Drupal Association
- Tim Lehnen, CTO at the Drupal Association
- Kristen Pol, Salsa Digital - Drupal Certified Partner representative
- Chris Yates, Pantheon - Drupal Certified Partner representative
- Luis Ribeiro, CI&T - Drupal Certified Partner representative
- Kathryn Carruthers, alakasam - End User representative
- Emma Horrell, University of Edinburgh - End User representative
- Nathaniel Catchpole (catch), Third and Grove / Tag1 Consulting - Drupal Core Committer
- Théodore Biadala (nod_), Très Bien Tech - Drupal Core Committer
The council has been meeting monthly to receive updates from myself and the Drupal Starshot Leadership Team. Members will provide feedback on project initiatives, offer recommendations, and share insights based on their diverse experiences and areas of expertise.
In addition to guiding the strategic direction of Drupal Starshot, the Advisory Council will play a vital role in communication and alignment between the Drupal Starshot team, the Drupal Association, Drupal Core, and the broader Drupal community.
I'm excited to be working with this accomplished group to make the Drupal Starshot vision a reality. Together we can expand the reach and impact of Drupal, and continue advancing our mission to make the web a better place.
Dries Buytaert: Building my own temperature and humidity monitor
Last fall, we toured the Champagne region in France, famous for its sparkling wines. We explored the ancient, underground cellars where Champagne undergoes its magical transformation from grape juice to sparkling wine. These cellars, often 30 meters deep and kilometers long, maintain a constant temperature of around 10-12°C, providing the perfect conditions for aging and storing Champagne.
25 meters underground in a champagne tunnel, which often stretches for miles/kilometers.After sampling various Champagnes, we returned home with eight cases to store in our home's basement. However, unlike those deep cellars, our basement is just a few meters deep, prompting a simple question that sent me down a rabbit hole: how does our basement's temperature compare?
Rather than just buying a thermometer, I decided to build my own "temperature monitoring system" using open hardware and custom-built software. After all, who needs a simple solution when you can spend evenings tinkering with hardware, sensors, wires and writing your own software? Sometimes, more is more!
The basic idea is this: track the temperature and humidity of our basement every 15 minutes and send this information to a web service. This web service analyzes the data and alerts us if our basement becomes too cold or warm.
I launched this monitoring system around Christmas last year, so it's been running for nearly three months now. You can view the live temperature and historical data trends at https://dri.es/sensors. Yes, publishing our basement's temperature online is a bit quirky, but it's all in good fun.
A screenshot of my basement temperature dashboard.So far, the temperature in our basement has been ideal for storing wine. However, I expect it will change during the summer months.
In the rest of this blog post, I'll share how I built the client that collects and sends the data, as well as the web service backend that processes and visualizes that data.
Hardware usedFor this project, I bought:
- Adafruit ESP32-S3 Feather: A microcontroller board with Wi-Fi and Bluetooth capabilities, serving as the central processing unit of my project.
- Adafruit SHT4x sensor: A high-accuracy temperature and humidity sensor.
- 3.7v 500mAh battery: A small and portable power source.
- STEMMA QT / Qwiic JST SH 4-pin cable: To connect the sensor to the board without soldering.
The total hardware cost was $32.35 USD. I like Adafruit a lot, but it's worth noting that their products often come at a higher cost. You can find comparable hardware for as little as $10-15 elsewhere. Adafruit's premium cost is understandable, considering how much valuable content they create for the maker community.
An ESP32-S3 development board (middle) linked to a Sensirion SHT41 temperature and humidity sensor (left) and powered by a battery pack (right). Client code for Adafruit ESP32-S3 FeatherI developed the client code for the Adafruit ESP32-S3 Feather using the Arduino IDE, a widely used platform for developing and uploading code to Arduino-compatible boards.
The code measures temperature and humidity every 15 minutes, connects to WiFi, and sends this data to https://dri.es/sensors, my web service endpoint.
One of my goals was to create a system that could operate for a long time without needing to recharge the battery. The ESP32-S3 supports a "deep sleep" mode where it powers down almost all its functions, except for the clock and memory. By placing the ESP32-S3 into deep sleep mode between measurements, I was able to significantly reduce power.
Now that you understand the high-level design goals, including deep sleep mode, I'll share the complete client code below. It includes detailed code comments, making it self-explanatory.
[code c]#include "Adafruit_SHT4x.h" #include "Adafruit_MAX1704X.h" #include "WiFiManager.h" #include "ArduinoJson.h" #include "HTTPClient.h" // The Adafruit_SHT4x sensor is a high-precision, temperature and humidity // sensor with an I2C interface. Adafruit_SHT4x sht4 = Adafruit_SHT4x(); // The Adafruit ESP32-S3 Feather comes with a built-in MAX17048 LiPoly / LiIon // battery monitor. The MAX17048 provides accurate monitoring of the battery's // voltage. Utilizing the Adafruit library, not only helps us obtain the raw // voltage data from the battery cell, but also converts this data into a more // intuitive battery percentage or charge level. We will pass on the battery // percentage to the web service endpoint, which can visualize it or use it to // send notifications when the battery needs recharging. Adafruit_MAX17048 maxlipo; // The setup() function is used to initialize the device's hardware and // communications. It's executed once at startup. Here, we begin serial // communication, initialize sensors, connect to Wi-Fi, and send initial // data. void setup() { Serial.begin(115200); // Wait for the serial connection to establish before proceeding further. // This is crucial for boards with native USB interfaces. Without this loop, // initial output sent to the serial monitor is lost. This code is not // needed when running on battery. //delay(1000); // Generates a unique device ID from a segment of the MAC address. // Since the MAC address is permanent and unchanged after reboots, // this guarantees the device ID remains consistent. To achieve a // compact ID, only a specific portion of the MAC address is used, // specifically the range between 0x10000 and 0xFFFFF. This range // translates to a hexadecimal string of a fixed 5-character length, // giving us roughly 1 million unique IDs. This approach balances // uniqueness with compactness. uint64_t chipid = ESP.getEfuseMac(); uint32_t deviceValue = ((uint32_t)(chipid >> 16) & 0x0FFFFF) | 0x10000; char device[6]; // 5 characters for the hex representation + the null terminator. sprintf(device, "%x", deviceValue); // Use '%x' for lowercase hex letters // Initialize the SHT4x sensor: if (sht4.begin()) { Serial.println(F("SHT4 temperature and humidity sensor initialized.")); sht4.setPrecision(SHT4X_HIGH_PRECISION); sht4.setHeater(SHT4X_NO_HEATER); } else { Serial.println(F("Could not find SHT4 sensor.")); } // Initialize the MAX17048 sensor: if (maxlipo.begin()) { Serial.println(F("MAX17048 battery monitor initialized.")); } else { Serial.println(F("Could not find MAX17048 battery monitor!")); } // Insert a short delay to ensure the sensors are ready and their data is stable: delay(200); // Retrieve temperature and humidity data from SHT4 sensor: sensors_event_t humidity, temp; sht4.getEvent(&humidity, &temp); // Get the battery percentage and calibrate if it's over 100%: float batteryPercent = maxlipo.cellPercent(); batteryPercent = (batteryPercent > 100) ? 100 : batteryPercent; WiFiManager wifiManager; // Uncomment the following line to erase all saved WiFi credentials. // This can be useful for debugging or reconfiguration purposes. // wifiManager.resetSettings(); // This WiFi manager attempts to establish a WiFi connection using known // credentials, stored in RAM. If it fails, the device will switch to Access // Point mode, creating a network named "Temperature Monitor". In this mode, // connect to this network, navigate to the device's IP address (default IP // is 192.168.4.1) using a web browser, and a configuration portal will be // presented, allowing you to enter new WiFi credentials. Upon submission, // the device will reboot and try connecting to the specified network with // these new credentials. if (!wifiManager.autoConnect("Temperature Monitor")) { Serial.println(F("Failed to connect to WiFi ...")); // If the device fails to connect to WiFi, it will restart to try again. // This approach is useful for handling temporary network issues. However, // in scenarios where the network is persistently unavailable (e.g. router // down for more than an hour, consistently poor signal), the repeated // restarts and WiFi connection attempts can quickly drain the battery. ESP.restart(); // Mandatory delay to allow the restart process to initiate properly: delay(1000); } // Send collected data as JSON to the specified URL: sendJsonData("https://dri.es/sensors", device, temp.temperature, humidity.relative_humidity, batteryPercent); // WiFi consumes significant power so turn it off when done: WiFi.disconnect(true); // Enter deep sleep for 15 minutes. The ESP32-S3's deep sleep mode minimizes // power consumption by powering down most components, except the RTC. This // mode is efficient for battery-powered projects where constant operation // isn't needed. When the device wakes up after the set period, it runs // setup() again, as the state isn't preserved. Serial.println(F("Going to sleep for 15 minutes ...")); ESP.deepSleep(15 * 60 * 1000000); // 15 mins * 60 secs/min * 1,000,000 μs/sec. } bool sendJsonData(const char* url, const char* device, float temperature, float humidity, float battery) { StaticJsonDocument<200> doc; // Round floating-point values to one decimal place for efficient data // transmission. This approach reduces the JSON payload size, which is // important for IoT applications running on batteries. doc["device"] = device; doc["temperature"] = String(temperature, 1); doc["humidity"] = String(humidity, 1); doc["battery"] = String(battery, 1); // Serialize JSON to a string: String jsonData; serializeJson(doc, jsonData); // Initialize an HTTP client with the provided URL: HTTPClient httpClient; httpClient.begin(url); httpClient.addHeader("Content-Type", "application/json"); // Send a HTTP POST request: int httpCode = httpClient.POST(jsonData); // Close the HTTP connection: httpClient.end(); // Print debug information to the serial console: Serial.println("Sent '" + jsonData + "' to " + String(url) + ", return code " + httpCode); return (httpCode == 200); } void loop() { // The ESP32-S3 resets and runs setup() after waking up from deep sleep, // making this continuous loop unnecessary. }[/code] Further optimizing battery usageWhen I launched my thermometer around Christmas 2023, the battery was at 88%. Today, it is at 52%. Some quick math suggests it's using approximately 12% of its battery per month. Given its current rate of usage, it needs recharging about every 8 months.
Connecting to the WiFi and sending data are by far the main power drains. To extend the battery life, I could send updates less frequently than every 15 minutes, only send them when there is a change in temperature (which is often unchanged or only different by 0.1°C), or send batches of data points together. Any of these methods would work for my needs, but I haven't implemented them yet.
Alternatively, I could hook the microcontroller up to a 5V power adapter, but where is the fun in that? It goes against the project's "more is more" principle.
Handling web service requestsWith the client code running on the ESP32-S3 and sending sensor data to https://dri.es/sensors, the next step is to set up a web service endpoint to receive this incoming data.
As I use Drupal for my website, I implemented the web service endpoint in Drupal. Drupal uses Symfony, a popular PHP framework, for large parts of its architecture. This combination offers an easy but powerful way for implementing web services, similar to those found across other modern server-side web development frameworks like Laravel, Django, etc.
Here is what my Drupal routing configuration looks like:
[code yaml]sensors.sensor_data: path: '/sensors' methods: [POST] defaults: _controller: '\Drupal\sensors\Controller\SensorMonitorController::postSensorData' requirements: _access: 'TRUE'[/code]The above configuration directs Drupal to send POST requests made to https://dri.es/sensors to the postSensorData() method of the SensorMonitorController class.
The implementation of this method handles request authentication, validates the JSON payload, and saves the data to a MariaDB database table. Pseudo-code:
[code php]public function postSensorData(Request $request) : JsonResponse { $content = $request->getContent(); $data = json_decode($content, TRUE); // Validate the JSON payload: … // Authenticate the request: … $device = DeviceFactory::getDevice($data['device']); if ($device) { $device->recordSensorEvent($data); } return new JsonResponse(['message' => 'Thank you!']); }[/code]For testing your web service, you can use tools like cURL:
[code bash]$ curl -X POST -H "Content-Type: application/json" -d '{"device":"0xdb123", "temperature":21.5, "humidity":42.5, "battery":90.0}' https://localhost/sensors[/code]While cURL is great for quick tests, I use PHPUnit tests for automated testing in my CI/CD workflow. This ensures that everything keeps working, even when upgrading Drupal, Symfony, or other components of my stack.
Storing sensor data in a databaseThe primary purpose of $device->recordSensorEvent() in SensorMonitorController::postSensorData() is to store sensor data into a SQL database. So, let's delve into the database design.
My main design goals for the database backend were:
- Instead of storing every data point indefinitely, only keep the daily average, minimum, maximum, and the latest readings for each sensor type across all devices.
- Make it easy to add new devices and new sensors in the future. For instance, if I decide to add a CO2 sensor for our bedroom one day (a decision made in my head but not yet pitched to my better half), I want that to be easy.
To this end, I created the following MariaDB table:
[code sql]CREATE TABLE sensor_data ( date DATE, device VARCHAR(255), sensor VARCHAR(255), avg_value DECIMAL(5,1), min_value DECIMAL(5,1), max_value DECIMAL(5,1), min_timestamp DATETIME, max_timestamp DATETIME, readings SMALLINT NOT NULL, UNIQUE KEY unique_stat (date, device, sensor) );[/code]A brief explanation for each field:
- date: The date for each sensor reading. It doesn't include a time component as we aggregate data on a daily basis.
- device: The device ID of the device providing the sensor data, such as 'basement' or 'bedroom'.
- sensor: The type of sensor, such as 'temperature', 'humidity' or 'co2'.
- avg_value: The average value of the sensor readings for the day. Since individual readings are not stored, a rolling average is calculated and updated with each new reading using the formula: avg_value = avg_value + new_value - avg_value new_total_readings . This method can accumulate minor rounding errors, but simulations show these are negligible for this use case.
- min_value and max_value: The daily minimum and maximum sensor readings.
- min_timestamp and max_timestamp: The exact moments when the minimum and maximum values for that day were recorded.
- readings: The number of readings (or measurements) taken throughout the day, which is used for calculating the rolling average.
In essence, the recordSensorEvent() method needs to determine if a record already exists for the current date. Depending on this determination, it will either insert a new record or update the existing one.
In Drupal this process is streamlined with the merge() function in Drupal's database layer. This function handles both inserting new data and updating existing data in one step.
[code php]private function updateDailySensorEvent(string $sensor, float $value): void { $timestamp = \Drupal::time()->getRequestTime(); $date = date('Y-m-d', $timestamp); $datetime = date('Y-m-d H:i:s', $timestamp); $connection = Database::getConnection(); $result = $connection->merge('sensor_data') ->keys([ 'device' => $this->id, 'sensor' => $sensor, 'date' => $date, ]) ->fields([ 'avg_value' => $value, 'min_value' => $value, 'max_value' => $value, 'min_timestamp' => $datetime, 'max_timestamp' => $datetime, 'readings' => 1, ]) ->expression('avg_value', 'avg_value + ((:new_value - avg_value) / (readings + 1))', [':new_value' => $value]) ->expression('min_value', 'LEAST(min_value, :value)', [':value' => $value]) ->expression('max_value', 'GREATEST(max_value, :value)', [':value' => $value]) ->expression('min_timestamp', 'IF(LEAST(min_value, :value) = :value, :timestamp, min_timestamp)', [':value' => $value, ':timestamp' => $datetime]) ->expression('max_timestamp', 'IF(GREATEST(max_value, :value) = :value, :timestamp, max_timestamp)', [':value' => $value, ':timestamp' => $datetime]) ->expression('readings', 'readings + 1') ->execute(); }[/code]Here is what the query does:
- It checks if a record for the current sensor and date exists.
- If not, it creates a new record with the sensor data, including the initial average, minimum, maximum, and latest value readings, along with the timestamp for these values.
- If a record does exist, it updates the record with the new sensor data, adjusting the average value, and updating minimum and maximum values and their timestamps if the new reading is a new minimum or maximum.
- The function also increments the count of readings.
For those not using Drupal, similar functionality can be achieved with MariaDB's INSERT ... ON DUPLICATE KEY UPDATE command, which allows for the same conditional insert or update logic based on whether the specified unique key already exists in the table.
Here are example queries, extracted from MariaDB's General Query Log to help you get started:
[code sql]INSERT INTO sensor_data (device, sensor, date, min_value, min_timestamp, max_value, max_timestamp, readings) VALUES ('0xdb123', 'temperature', '2024-01-01', 21, '2024-01-01 00:00:00', 21, '2024-01-01 00:00:00', 1); UPDATE sensor_data SET min_value = LEAST(min_value, 21), min_timestamp = IF(LEAST(min_value, 21) = 21, '2024-01-01 00:00:00', min_timestamp), max_value = GREATEST(max_value, 21), max_timestamp = IF(GREATEST(max_value, 21) = 21, '2024-01-01 00:00:00', max_timestamp), readings = readings + 1 WHERE device = '0xdb123' AND sensor = 'temperature' AND date = '2024-01-01';[/code] Generating graphsWith the data securely stored in the database, the next step involved generating the graphs. To accomplish this, I wrote some custom PHP code that generates Scalable Vector Graphics (SVGs).
Given that is blog post is already quite long, I'll spare you the details. For now, those curious can use the 'View source' feature in their web browser to examine the SVGs on the thermometer page.
ConclusionIt's fun how a visit to the Champagne cellars in France sparked an unexpected project. Choosing to build a thermometer rather than buying one allowed me to dive back into an old passion for hardware and low-level software.
I also like taking control of my own data and software. It gives me a sense of control and creativity.
As Drupal's project lead, using Drupal for an Internet-of-Things (IoT) backend brought me unexpected joy. I just love the power and flexibility of open-source platforms like Drupal.
As a next step, I hope to design and 3D print a case for my thermometer, something I've never done before. And as mentioned, I'm also considering integrating additional sensors. Stay tuned for updates!