Feeds
Seth Michael Larson: New era of slop security reports for open source
Published 2024-12-03 by Seth Larson
Reading time: minutes
I'm on the security report triage team for CPython, pip, urllib3, Requests, and a handful of other open source projects. I'm also in a trusted position such that I get "tagged in" to other open source projects to help others when they need help with security.
Recently I've noticed an uptick in extremely low-quality, spammy, and LLM-hallucinated security reports to open source projects. The issue is in the age of LLMs, these reports appear at first-glance to be potentially legitimate and thus require time to refute. Other projects such as curl have reported similar findings.
Some reporters will run a variety of security scanning tools and open vulnerability reports based on the results seemingly without a moment of critical thinking. For example, urllib3 recently received a report because a tool was detecting our usage of SSLv2 as insecure even though our usage is to explicitly disable SSLv2.
This issue is tough to tackle because it's distributed across thousands of open source projects and due to the security-sensitive nature of reports open source maintainers are discouraged from sharing their experiences or asking for help. Sharing experiences takes time and effort, something that is in short supply amongst maintainers.
Responding to security reports is expensiveIf this is happening to a handful of projects that I have visibility for, then I suspect that this is happening on a large scale to open source projects. This is a very concerning trend.
Security is already a topic that is not aligned with why many maintainers contribute their time to open source software, instead seeing security as important to help protect their users. It's critical as reporters to respect this often volunteered time.
Security reports that waste maintainers' time result in confusion, stress, frustration, and to top it off a sense of isolation due to the secretive nature of security reports. All of these feelings can add to burn-out of likely highly-trusted contributors to open source projects.
In many ways, these low-quality reports should be treated as if they are malicious. Even if this is not their intent, the outcome is maintainers that are burnt out and more averse to legitimate security work.
What platforms can doIf you're a platform accepting vulnerability reports on behalf of open source projects, here are things you can do:
- Add systems to prevent automated or abusive creation of security reports. Require reporters to solve CAPTCHAs or heavily rate-limit security report creation using automation.
- Allow a security report to be made public without publishing a vulnerability record. This would allow maintainers to "name-and-shame" offenders and better collaborate as a community how to fight back against low-quality reports. Today many of these reports aren't seen due to being private by default or when closed.
- Remove the public attribution of reporters that abuse the system, even removing previously credited reports in the case of abuse.
- Take away any positive incentive to reporting security issues, for example GitHub showing the number of GitHub Security Advisory "credits" a user appears on.
- Prevent or hamper newly registered users from reporting security issues.
If you're starting a new campaign of scanning open source projects and reporting potential vulnerabilities upstream:
- DO NOT use AI / LLM systems for "detecting" vulnerabilities. These systems today cannot understand code, finding security vulnerabilities requires understanding code AND understanding human-level concepts like intent, common usage, and context.
- DO NOT run experiments on open source volunteers. My alma-mater the University of Minnesota rightfully had its reputation thrown in the trash in 2021 over their experiment to knowingly socially deceive Linux maintainers.
- DO NOT submit reports that haven't been reviewed BY A HUMAN. This reviewing time should be paid first by you, not open source volunteers.
- DO NOT spam projects, open a handful of reports and then WAIT. You could run the script and open tons of reports all-at-once, but likely you have faults in your process that will cause mass-frustration at scale. Learn from early mistakes and feedback.
- Have someone with experience in open source maintenance for the size of projects you are scanning review your plan before you begin. If that person is not on your team, then pay them for their time and expertise.
- Show up with patches, not just reports. By providing patches this makes the work of maintainers much easier.
Doing all of the above will likely lead to better outcomes for everyone.
What maintainers can doPut the same amount of effort into responding as the reporter put into submitting a sloppy report: ie, near zero. If you receive a report that you suspect is AI or LLM generated, reply with a short response and close the report:
"I suspect this report is (AI-generated|incorrect|spam). Please respond with more justification for this report. See: https://sethmlarson.dev/slop-security-reports"
If you hear back at all then admit your mistake and you move on with the security report. Maybe the reporter will fix their process and you'll have helped other open source maintainers along the way to helping yourself.
If you don't hear back: great, you saved time and can get back to actually useful work.
Here are some questions to ask of a security report and reporter:
If you aren't sure: ask for help! Is there someone I trust in my community that I can ask for another look. You are not alone, there are many people around that are willing to help. For Python open source projects you can ask for help from me if needed.
Does the reporter have a new account, no public identity, or multiple "credited" security reports of low quality? There are sometimes legitimate reasons to want anonymity, but I've seen this commonly on very low-stakes vulnerability reports.
Is the vulnerability in the proof-of-concept code or the project itself? Oftentimes the proof-of-concept code will be using the project insecurely and thus the vulnerability is in the proof-of-concept code, not your code.
I wanted to end this article with a note that many vulnerability reporters are acting in good faith and are submitting high quality reports. Please keep in mind that vulnerability reporters are humans: not perfect and trying their best to make the world a better place.
Unfortunately, an increasing majority of reports are of low quality and are ruining the experience for others. I hope we're able to fix this issue before it gets out of hand.
Have thoughts or questions? Let's chat over email or social:
sethmichaellarson@gmail.com
@sethmlarson@fosstodon.org
Want more articles like this one? Get notified of new posts by subscribing to the RSS feed or the email newsletter. I won't share your email or send spam, only whatever this is!
Want more content now? This blog's archive has ready-to-read articles. I also curate a list of cool URLs I find on the internet.
Find a typo? This blog is open source, pull requests are appreciated.
Thanks for reading! ♡ This work is licensed under CC BY-SA 4.0
︎Bits from Debian: Bits from the DPL
This is bits from DPL for November.
MiniDebConf ToulouseI had the pleasure of attending the MiniDebConf in Toulouse, which featured a range of engaging talks, complementing those from the recent MiniDebConf in Cambridge. Both events were preceded by a DebCamp, which provided a valuable opportunity for focused work and collaboration.
DebCampDuring these events, I participated in numerous technical discussions on topics such as maintaining long-neglected packages, team-based maintenance, FTP master policies, Debusine, and strategies for separating maintainer script dependencies from runtime dependencies, among others. I was also fortunate that members of the Publicity Team attended the MiniDebCamp, giving us the opportunity to meet in person and collaborate face-to-face.
Independent of the ongoing lengthy discussion on the Debian Devel mailing list, I encountered the perspective that unifying Git workflows might be more critical than ensuring all packages are managed in Git. While I'm uncertain whether these two questions--adopting Git as a universal development tool and agreeing on a common workflow for its use--can be fully separated, I believe it's worth raising this topic for further consideration.
Attracting newcomersIn my own talk, I regret not leaving enough time for questions--my apologies for this. However, I want to revisit the sole question raised, which essentially asked: Is the documentation for newcomers sufficient to attract new contributors? My immediate response was that this question is best directed to new contributors themselves, as they are in the best position to identify gaps and suggest improvements that could make the documentation more helpful.
That said, I'm personally convinced that our challenges extend beyond just documentation. I don't get the impression that newcomers are lining up to join Debian only to be deterred by inadequate documentation. The issue might be more about fostering interest and engagement in the first place.
My personal impression is that we sometimes fail to convey that Debian is not just a product to download for free but also a technical challenge that warmly invites participation. Everyone who respects our Code of Conduct will find that Debian is a highly diverse community, where joining the project offers not only opportunities for technical contributions but also meaningful social interactions that can make the effort and time truly rewarding.
In several of my previous talks (you can find them on my talks page –just search for "team," and don't be deterred if you see "Debian Med" in the title; it's simply an example), I emphasized that the interaction between a mentor and a mentee often plays a far more significant role than the documentation the mentee has to read. The key to success has always been finding a way to spark the mentee's interest in a specific topic that resonates with their own passions.
Bug of the DayIn my presentation, I provided a brief overview of the Bug of the Day initiative, which was launched with the aim of demonstrating how to fix bugs as an entry point for learning about packaging. While the current level of interest from newcomers seems limited, the initiative has brought several additional benefits.
I must admit that I'm learning quite a bit about Debian myself. I often compare it to exploring a house's cellar with a flashlight –you uncover everything from hidden marvels to things you might prefer to discard. I've also come across traces of incredibly diligent people who have invested their spare time polishing these hidden treasures (what we call NMUs). The janitor, a service in Salsa that automatically updates packages, fits perfectly into this cellar metaphor, symbolizing the ongoing care and maintenance that keep everything in order. I hadn't realized the immense amount of silent work being done behind the scenes--thank you all so much for your invaluable QA efforts.
Reproducible buildsIt might be unfair to single out a specific talk from Toulouse, but I'd like to highlight the one on reproducible builds. Beyond its technical focus, the talk also addressed the recent loss of Lunar, whom we mourn deeply. It served as a tribute to Lunar's contributions and legacy. Personally, I've encountered packages maintained by Lunar and bugs he had filed. I believe that taking over his packages and addressing the bugs he reported is a meaningful way to honor his memory and acknowledge the value of his work.
Advent calendar bug squashingI’d like to promote an idea originally introduced by Thorsten Alteholz, who in 2011 proposed a Bug Squashing Advent Calendar for the Debian Med team. (For those unfamiliar with the concept of an Advent Calendar, you can find an explanation on Wikipedia.) While the original version included a fun graphical element —which we’ve had to set aside due to time constraints (volunteers, anyone?)— we’ve kept the tradition alive by tackling one bug per day from December 1st to 24th each year. This initiative helps clean up issues that have accumulated over the year.
Regardless of whether you celebrate the concept of Advent, I warmly recommend this approach as a form of continuous bug-squashing party for every team. Not only does it contribute to the release readiness of your team’s packages, but it’s also an enjoyable and bonding activity for team members.
Best wishes for a cheerful and productive December
Andreas.Dirk Eddelbuettel: anytime 0.3.10 on CRAN: Multiple Enhancements
A new release of the anytime package arrived on CRAN today—the first is well over four years. The package is fairly feature-complete, and code and functionality remain mature and stable, of course.
anytime is a very focused package aiming to do just one thing really well: to convert anything in integer, numeric, character, factor, ordered, … input format to either POSIXct (when called as anytime) or Date objects (when called as anydate) – and to do so without requiring a format string as well as accomodating different formats in one input vector. See the anytime page, or the GitHub repo for a few examples, and the beautiful documentation site for all documentation.
This release slowly matured over four years. It combines a number of strictly internal repository maintenance such as changes to continuous integration with small enhancements (adding for example some new formats, responding better to an error condition, dealing with logical input as an error) with a relaxation of the C++ compilation standard. While we once needed C++11, it is now a constraint as as R itself is quite proactive (the last two releases defaulted already to C++17, suitable compiler permitting) we can now relax this constraint. The documentation site is new, as some other small changes. See the full list of changes which follows.
Changes in anytime version 0.3.10 (2024-12-02)A new documentation site was added.
Continuous Integration now uses run.sh from r-ci with bspm
Logical input vectors are now recognised as an error (#121)
Additional dot-separated format '%Y.%m.%d' is supported
Other small updates were made throughout the package
No longer set a C++ compilation standard as the default choices by R are sufficient for the package
Switch Rcpp include file to Rcpp/Lightest
We recommend ~/.R/Makevars compiler flag options -Wno-ignored-attributes -Wno-nonnull -Wno-parentheses
The tinytest runner was simplified
NA values from conversion now trigger a warning
Courtesy of my CRANberries, there is also a diffstat report of changes relative to the previous release. The issue tracker tracker off the GitHub repo can be use for questions and comments. More information about the package is at the package page, the GitHub repo and the documentation site. If you like this or other open-source work I do, you can now sponsor me at GitHub.
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.
Jonathan Dowland: jungle/acid/etc
I thought it had been a full year since I last shared a playlist, but it's been two! I had a plan to produce more, but it seems I haven't. Instead here's a few tracks I've discovered recently which share a common theme.
In August I stumbled across a Sound on Sound video interviewing Pete Cannon, who creates authentically old-school Jungle music using tools and techniques from the time, including AKAI samplers and the Commodore Amiga computer.
Here's three tracks that I found since then. Some 8-bit Amiga-jungle,
bysome slower-paced acid house from someone ostensibly based on Whitley Bay,
byand a darker piece I heard on the radio.
byFSF Events: Free Software Directory meeting on IRC: Friday, December 6, starting at 12:00 EST (17:00 UTC)
Kate - 1500 accepted merge requests!
I just looked at our GitLab page today and thought: Amazing!
I thank you all for the great contributions of the last years.
Let's hope we see even more contributions in the future.
If you are unsure how to contribute, just take a look at the existing merged stuff as reference.
The upcoming 24.12 release will be a good one, we did polish Kate a lot.
I know not all is well on the world, but I still hope you have a good end of the year and an even better start in the new one!
Talking Drupal: Talking Drupal #478 - WEBAssembly
Today we are talking about WEBAssembly, How it’s used, and cool things you can use it for with Drupal with guest Matt Glaman. We’ll also cover Darkmode JS as our module of the week.
For show notes visit: https://www.talkingDrupal.com/478
Topics- What is WebAssembly
- Progressive Web Aoos
- Open source
- Does it have a community
- Browser support
- How does it work
- Common use cases
- How can you use this with Drupal
- This was an early concept for Drupal trial
- Challenges
- Wordpress playground
- Pieces that do not work for PHP
- Are there risks
- Are there resources for people that want to use WebAssembly
- Do you see it being used with Drupal
- WEBAssembly
- WEBAssembly history
- Browser support
- 2038
- WordPress Playground
- Slides from Barcelona: The Web APIs powering the Starshot trial:
Matt Glaman - mglaman.dev mglaman
HostsNic Laflin - nLighteneddevelopment.com nicxvan John Picozzi - epam.com johnpicozzi Suzanne Dergacheva - evolvingweb.com pixelite
MOTW CorrespondentMartin Anderson-Clutz - mandclu.com mandclu
- Brief description:
- Have you ever wanted your Drupal site to provide a widget that allows visitors to go over to the dark side of your theme? There’s a module for that.
- Module name/project name:
- Brief history
- How old: created in May 2022 by Arthur Baghdasaryan (arthur.baghdasar) of Last Call Media
- Versions available: 1.0.7 which works with Drupal 9, 10, and 11
- Maintainership
- Actively maintained
- Security coverage
- Number of open issues: 1 open issues which is a bug against the current branch, but is postponed, waiting for more info
- Usage stats:
- 89 sites
- Module features and usage
- The module is a wrapper for the DarkmodeJS library which gets 1,000 weekly downloads, according to NPM. That library does have its own demo / tutorial site, so if you want to understand the options it exposes, we will add a link in the show notes
- The module provides options to control where on the page you want the widget to appear, what colors it should use, whether or not to store a user’s choices in cookies, and whether or not to automatically match a visitor’s OS theme setting of light/dark
- Installing the module currently requires making some changes to your site’s composer.json file, then configuring how you want the widget to appear, and then placing the block in your site theme
- The module also doesn’t currently include a schema file for its configuration, which can cause challenges particularly for sites that run automated tests
Debug Academy: Watch as I fix a bug in a contrib Drupal module!
Follow along as I (Ashraf Abed of DebugAcademy.com) fix a bug in the contributed Drupal module, Responsive Menus. This embedded video follows as I diagnose the problem, research the issues, create a branch on gitlab, fix the issue locally, test the fix, and submit my fix to Drupal.org for the community to benefit from.
This was done as part of Debug Academy's Drupal Training course.
ashrafabed Mon, 12/02/2024Python Engineering at Microsoft: Announcing: Azure Developers – Python Day
We’re thrilled to announce Azure Developers – Python Day! Join us on December 5th for a full day of online training and discover the latest services and features in Azure designed specifically for Python developers. You’ll learn cutting-edge cloud development techniques that can save you time and money while providing your customers with the best experience possible.
December 5, 2024 from 9:30 am – 4:00 pm (Pacific Time) / 17:30 – 00:00 (UTC)
Select “Notify Me” on the YouTube Video to ensure you don’t miss the event!
During the event, you’ll hear directly from the experts behind the latest features in Azure designed for Python developers, techniques to save time and money, and a special session on our recently announced AI Toolkit for VS Code.
Whether you’re a beginner or an experienced Python developer, this event is for you. We’ll cover six main topic areas: Application Development, Artificial Intelligence, Cloud Native, Data Services, Security, Serverless, and Developer Productivity.
Agenda Session Title Theme Speaker Time( PT/ UTC) Welcome to Azure Developers – Python Day Dawn Wages, Senior Program ManagerJay Gordon, Principal Product Manager
9:50 AM / 16:50 Dev Containers and Codespaces for quick skilling and deployments Developer Productivity Sarah Kaiser, Senior Cloud Developer Advocate 10:00 AM / 17:00 Cloudy with a Chance of Jupyter – Install JupyterHub on Azure in 30 mins Data Services Dharhas Pothina, CTO, Quansight 10:30 AM / 17:30 Langchain on Azure SQL to enlighten AI with your own data AI Davide Mauri, Principal Program Manager 11:00 AM / 18:00 Integrating AI into your Python apps with App Service Sidecars AI, App Development Byron Tardif, Principal Mgr, Product ManagerTulika Chaudharie, Principal Product Manager
Jeff Martinez, Product Manager
11:30 AM / 18:30 Getting started with Python on Azure Cosmos DB App Development Theo Van Kraay, Principal Program Manager 12:00 PM / 19:00 Transforming AI development in VS Code AI, App Development, Developer Productivity Rong Lu, Principal Mgr, Product ManagerZhidi Shang, Principal Program Manager Lead
12:30 PM / 19:30 Building Scalable GenAI Apps with Azure Cosmos DB & LangChain AI, App Development James Codella, Principal Product Mgr 1:00 PM / 20:00 Python + Azure for Absolute Beginners App Development Rohit Ganguly, Product Manager II 1:30 PM / 20:30 Deploying Python apps with GitHub Copilot for @azure AI, Developer Productivity Pamela Fox, Principal Cloud Developer Advocate 2:00 PM / 21:00 Securing Python Applications Security, Cloud Native Joylynn Kirui, Senior Security Cloud Advocate 2:30 PM / 21:30 Your First Full Stack Python Web Application App Development Renee Noble, Senior Cloud Developer Advocate 3:00 PM / 22:00 Deploying a scalable Django app with Microsoft Azure App Development Velda Kiara, Senior Software Engineer, Python MVPAbigail Gbadago, Senior Software Engineer, Python MVP
3:30 PM / 22:30 Closing Remarks Dawn Wages, Senior Program ManagerJay Gordon, Principal Product Manager
4:00 PM / 23:00Don’t miss this opportunity to build the best applications with Python. Join us on December 5th on the Azure Developers YouTube and Twitch channels. See you there!
The post Announcing: Azure Developers – Python Day appeared first on Python.
The Drop Times: A Pat on the Back
Dear Readers,
Ever put your heart and soul into a project, be it an art project back in school or a work thing that sustains your corporate existence? Then you all will be able to relate that more than the work itself, it was the happiness and pride in the eyes of those who saw you, that made all the difference. For all of us, it's that 'pat on the back' that pushes us to do better each day — the recognition, a token of appreciation. The fuel for motivation is not any different for Drupal, its agencies, and the community.
Did you know the Splash Award made its debut a decade ago? This time it is widening its ambit with the first-ever Splash Awards Drupal Asia at DrupalCon Singapore 2024. 10 years of an exemplary institution for recognizing and inspiring innovation through acknowledging the outstanding websites and digital experiences built with Drupal. The much relevant, nod of approval for the Drupal agencies to tread on. To put organizations and users who are doing extraordinary things in the field of Drupal in the spotlight and add a feather to their hat.
Esmeralda Tijhoff had an opportunity to interview Bert Boerland, one of the pioneers of the award about the genesis and growth of the Splash Awards. The prestigious accord stemmed from the need to promote Drupal better.
“Our dream is to grow into a kind of Eurosplash Awards with the best European entries!"
Drupal Splash Awards Asia will take place on Monday, December 9, 2024, at 5:15 PM inside the Garden Ballroom at PARKROYAL COLLECTION Marina Bay. The evening promises to be a night of glamour and inspiration, as Drupal developers and agencies gather to celebrate the exceptional work being done in the community.
20 projects by various organizations have been shortlisted across five different categories along with Drupal’s Founder and Project Lead, Dries Buytaert, announcing the "Best in Show" award. This week The DropTimes will bring you a comprehensive overview of all the finalists in the series 'Splash Award Finalists'.
Along with these, other important stories from the last week include;
DrupalCon Singapore 2024 is less than a week away. If you are a first-time attendee, here are a few tips for you to smoothly navigate your first DrupalCon Experience: Countdown to DrupalCon Singapore 2024: Tips for First-Time Attendees
Adding to our happiness in efficiently collaborating with DrupalCon Singapore 2024, The DropTimes is now the official media partner for DrupalCon Vienna 2025. We will act as the prime location for all content surrounding the biggest Drupal event in Europe. This is the third time TDT has been named the official media partner for the European DrupalCon.
Pantheon has introduced the Content Publisher, bridging Google Docs with WordPress, Drupal, and Next.js for seamless content publishing. The tool streamlines workflows with live previews, robust editorial features, and AI-assisted enhancements. Sign up for the beta to explore this CMS integration solution.
Developed by Anand Toshniwal and recognized by Dries Buytaert himself, a Python script now automates the creation of .component.yml files for Single Directory Components in Drupal. Simplifying workflows and improving accuracy, this tool supports projects like the Starshot Demo Design System, enhancing efficiency for developers.
MidCamp 2025 has opened its call for session proposals, inviting speakers to share their expertise at the annual event. Scheduled for May 20-22, 2025, the proposals are being accepted until January 12, 2025.
Drupal Developer Days 2025 is inviting sponsors to help make the event a success. Scheduled to attract over 200 attendees from across Europe, this four-day gathering is a prime opportunity for organizations to showcase their support for one of the fastest-growing open-source communities. The event is in Leuven, Belgium, from 15 - 18 April 2025.
Read this week’s edition of Events of the Week by The DropTimes, where we highlight notable Drupal gatherings happening around the globe. Whether you're a seasoned developer, site builder, or just starting your Drupal journey, there's something for everyone in this vibrant community.
Carlos Ospina shared an update on the progress of the IXP Initiative, an effort to support company-IXP engagements within the Drupal community. Carlos replaced the proposed Google Forms tracking system with a proof-of-concept site. Utilizing the ECA and Group modules for the first time, he developed a working solution within two days to handle the entire engagement process.
Artisan, a new Drupal base theme created by Alejandro Cabarcos introduces a robust framework for building customizable and reusable Drupal themes. Developed by Metadrop, Artisan is built on Bootstrap 5 and Sass, offering extensive use of CSS variables to streamline customization and ensure consistent design across projects.
Nigel Kersten has been appointed Chief Product Officer at Platform.sh to lead Product Strategy and Upsun Development. An influential figure in the DevOps community, Nigel co-founded the State of DevOps Report, introducing DORA metrics that have elevated software delivery practices across the industry. Serving as the primary co-author for 11 years, he pioneered best practices for modernizing complex IT environments through DevOps and platform engineering.
We acknowledge that there are more stories to share. However, due to selection constraints, we must pause further exploration for now.
To get timely updates, follow us on LinkedIn, Twitter and Facebook. You can also join us on Drupal Slack at #thedroptimes.
Thank you,
Sincerely
Alka Elizabeth
Sub-editor, The DropTimes.
Drupal Association blog: See Your Design in Print - Enter the DrupalCon Atlanta T-Shirt Contest
After the huge success of the DrupalCon Portland competition, the Drupal Association is excited to announce that the t-shirt design contest will be returning for DrupalCon Atlanta!
We want to see the Drupal community's design ideas for the official t-shirt, available for all attendees to wear and enjoy. Do you have a fantastic idea in mind? Let’s see your creativity!
The winner will get THEIR design on the front of the official t-shirt for DrupalCon Atlanta 2025!
What the judges are looking forJudges are looking for a combination of creativity, impact, and relevance to the Drupal community. A design that tells a story and aligns with the values and aspirations of DrupalCon attendees is likely to capture attention.
While exploring bold ideas, consider how your design will resonate with a diverse audience. Think of classic elements that make a T-shirt memorable while pushing creative boundaries. Avoid overcomplicating things; sometimes less is more, especially if every element adds value to the message.
Now, for the finer details…Your design must include the DrupalCon Atlanta Logo and will only be featured on the front of the t-shirt. Sponsor logos will be added to the t-shirt sleeves after the design is finalized.
Specs:
- PNG or PDF preferred
- 16 inches tall, 13 inches wide
- graphics need to be 300 dpi
All designs must be submitted by 31 December 2024 at 23:59 UTC, after which the submission form will close.
The Drupal Association will then select four designs to go forward to a public vote.
The top four designs, as chosen by the Drupal Association, will then be voted upon by the public, with voting closing on 18 January at 23:59 UTC.
The winning design will be printed on the front of the official DrupalCon Atlanta 2025 t-shirt and the winner will receive a complimentary ticket to their choice of either DrupalCon Atlanta 2025 or DrupalCon North America 2026.
How to enterSimply create your design, then fill out our submission form by 18 December to submit your final design. We also ask that you include a sentence or two describing why you chose your design and how it represents the Drupal community.
So, what are you waiting for? Submit your design now, and please help us spread the word throughout the Drupal community!
Good luck!
** Drupal Association staff and members of the DrupalCon Atlanta Steering Committee will not be permitted to enter this contest.**
Qt Creator 15 released
We are happy to announce the release of Qt Creator 15!
Qt Creator 15 is here, bringing native support for Windows on ARM, refreshed visuals, and improvements to enhance your productivity. Dive in and explore the enhancements!
The Drop Times: A Weekend With the Drupal Community in Berlin
Real Python: Basic Input and Output in Python
For a program to be useful, it often needs to communicate with the outside world. In Python, the input() function allows you to capture user input from the keyboard, while you can use the print() function to display output to the console.
These built-in functions allow for basic user interaction in Python scripts, enabling you to gather data and provide feedback. If you want to go beyond the basics, then you can even use them to develop applications that are not only functional but also user-friendly and responsive.
By the end of this tutorial, you’ll know how to:
- Take user input from the keyboard with input()
- Display output to the console with print()
- Use readline to improve the user experience when collecting input on UNIX-like systems
- Format output using the sep and end keyword arguments of print()
To get the most out of this tutorial, you should have a basic understanding of Python syntax and familiarity with using the Python interpreter and running Python scripts.
Get Your Code: Click here to download the free sample code that you’ll use to learn about basic input and output in Python.
Take the Quiz: Test your knowledge with our interactive “Basic Input and Output in Python” quiz. You’ll receive a score upon completion to help you track your learning progress:
Interactive Quiz
Basic Input and Output in PythonIn this quiz, you'll test your understanding of Python's built-in functions for user interaction, namely input() and print(). These functions allow you to capture user input from the keyboard and display output to the console, respectively.
Reading Input From the KeyboardPrograms often need to obtain data from users, typically through keyboard input. In Python, one way to collect user input from the keyboard is by calling the input() function:
The input() function pauses program execution to allow you to type in a line of input from the keyboard. Once you press the Enter key, all characters typed are read and returned as a string, excluding the newline character generated by pressing Enter.
If you add text in between the parentheses, effectively passing a value to the optional prompt argument, then input() displays the text you entered as a prompt:
Python >>> name = input("Please enter your name: ") Please enter your name: John Doe >>> name 'John Doe' Copied!Adding a meaningful prompt will assist your user in understanding what they’re supposed to input, which makes for a better user experience.
The input() function always reads the user’s input as a string. Even if you type characters that resemble numbers, Python will still treat them as a string:
Python 1>>> number = input("Enter a number: ") 2Enter a number: 50 3 4>>> type(number) 5<class 'str'> 6 7>>> number + 100 8Traceback (most recent call last): 9 File "<python-input-1>", line 1, in <module> 10 number + 100 11 ~~~~~~~^~~~~ 12TypeError: can only concatenate str (not "int") to str Copied!In the example above, you wanted to add 100 to the number entered by the user. However, the expression number + 100 on line 7 doesn’t work because number is a string ("50") and 100 is an integer. In Python, you can’t combine a string and an integer using the plus (+) operator.
You wanted to perform a mathematical operation using two integers, but because input() always returns a string, you need a way to read user input as a numeric type. So, you’ll need to convert the string to the appropriate type:
Python >>> number = int(input("Enter a number: ")) Enter a number: 50 >>> type(number) <class 'int'> >>> number + 100 150 Copied!In this updated code snippet, you use int() to convert the user input to an integer right after collecting it. Then, you assign the converted value to the name number. That way, the calculation number + 100 has two integers to add. The calculation succeeds and Python returns the correct sum.
Note: When you convert user input to a numeric type using functions like int() in a real-world scenario, it’s crucial to handle potential exceptions to prevent your program from crashing due to invalid input.
The input() function lets you collect information from your users. But once your program has calculated a result, how do you display it back to them? Up to this point, you’ve seen results displayed automatically as output in the interactive Python interpreter session.
However, if you ran the same code from a file instead, then Python would still calculate the values, but you wouldn’t see the results. To display output in the console, you can use Python’s print() function, which lets you show text and data to your users.
Writing Output to the ConsoleIn addition to obtaining data from the user, a program will often need to present data back to the user. In Python, you can display data to the console with the print() function.
Read the full article at https://realpython.com/python-input-output/ »[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
Web Wash: First Look at Drupal CMS (Drupal Starshot)
In the above video, we’ll take our first look at Drupal CMS Beta, part of the Drupal Starshot initiative. This initiative aims to provide a downloadable packaged version of Drupal with pre-installed and configured contributed modules.
In the show notes below, you’ll learn about the Drupal Starshot Initiative, Drupal CMS, and how to install it using DDEV.
We’ll then explore Drupal CMS’s functionality and examine some modules included in the packaged solution.
PyCharm: The State of Data Science 2024: 6 Key Data Science Trends
Generative AI and LLMs have been hot topics this year, but are they affecting trends in data science and machine learning? What new trends in data science are worth following? Every year, JetBrains collaborates with the Python Software Foundation to carry out the Python Developer Survey, which can offer some useful insight into these questions.
The results from the latest iteration of the survey, collected between November 2023 and February 2024, included a new Data Science section. This allowed us to get a more complete picture of data science trends over the past year and highlighted how important Python remains in this domain.
While 48% of Python developers are involved in data exploration and processing, the percentage of respondents using Python for data analysis dropped from 51% in 2022 to 44% in 2023. The percentage of respondents using Python for machine learning dropped from 36% in 2022 to 34% in 2023. At the same time, 27% of respondents use Python for data engineering, and 8% use it for MLOps – two new categories that were added to the survey in 2023.
Let’s take a closer look at the trends in the survey results to put these numbers into context and get a better sense of what they mean. Read on to learn about the latest developments in the fields of data science and machine learning to prepare yourself for 2025.
Data processing: pandas remains the top choice, but Polars is gaining groundData processing is an essential part of data science. pandas, a project that is 15 years old, is still at the top of the list of the most commonly used data processing tools. It is used by 77% of respondents who do data exploration and processing. As a mature project, its API is stable, and many working examples can be found on the internet. It’s no surprise that pandas is still the obvious choice. As a NumFOCUS sponsored project, pandas has proven to the community that it is sustainable and its governance model has gained user trust. It is a great choice for beginners who may still be learning the ropes of data processing, as it’s a stable project that does not undergo rapid changes.
On the other hand, Polars, which pitches itself as DataFrames for the new era, has been in the spotlight quite a bit both last year and this year, thanks to the advantages it provides in terms of speed and parallel processing. In 2023, a company led by the creator of Polars, Ritchie Vink, was formed to support the development of the project. This ensures Polars will be able to maintain its rapid pace of development. In July of 2024, version 1.0 of Polars was released. Later, Polars expanded its compatibility with other popular data science tools like Hugging Face and NVIDIA RAPIDS. It also provides a lightweight plotting backend, just like pandas.
So, for working professionals in data science, there is an advantage to switching to Polars. As the project matures, it can become a load-bearing tool in your data science workflow and can be used to process more data faster. In the 2023 survey, 10% of respondents said that they are using Polars as their data processing tool. It is not hard to imagine this figure being higher in this year’s survey.
Whether you are a working professional or just starting to process your first dataset, it is important to have an efficient tool at hand that can make your work more enjoyable. With PyCharm, you can inspect your data as interactive tables, which you can scroll, sort, filter, convert to plots, or use to generate heat maps. Moreover, you can get analytics for each column and use AI assistance to explain DataFrames or create visualizations. Apart from pandas and Polars, PyCharm provides this functionality for Hugging Face datasets, NumPy, PyTorch, and TensorFlow.
Try PyCharm for free An interactive table in PyCharm 2024.2.2 Pro provides tools for inspecting pandas and Polars DataFramesThe popularity of Polars has led to the creation of a new project called Narwhals. Independent from pandas and Polars, Narwhals aims to unite the APIs of both tools (and many others). Since it is a very young project (started in February 2024), it hasn’t yet shown up on our list of the most popular data processing tools, but we suspect it may get there in the next few years.
Also worth mentioning are Spark (16%) and Dask (7%), which are useful for processing large quantities of data thanks to their parallel processes. These tools require a bit more engineering capability to set up. However, as the amount of data that projects depend on increasingly exceeds what a traditional Python program can handle, these tools will become more important and we may see these figures go up.
Data visualization: Will HoloViz Panel surpass Plotly Dash and Streamlit within the next year?Data scientists have to be able to create reports and explain their findings to businesses. Various interactive visualization dashboard tools have been developed for working with Python. According to the survey results, the most popular of them is Plotly Dash.
Plotly is most known in the data science community for the ggplot2 library, which is a highly popular visualization library for users of the R language. Ever since Python became popular for data science, Plotly has also provided a Python library, which gives you a similar experience to ggplot2 in Python. In recent years, Dash, a Python framework for building reactive web apps developed by Plotly, has become an obvious choice for those who are used to Plotly and need to build an interactive dashboard. However, Dash’s API requires some basic understanding of the elements used in HTML when designing the layout of an app. For users who have little to no frontend experience, this could be a hurdle they need to overcome before making effective use of Dash.
Second place for “best visualization dashboard” goes to Streamlit, which has now joined forces with Snowflake. It doesn’t have as long of a history as Plotly, but it has been gaining a lot of momentum over the past few years because it’s easy to use and comes packaged with a command line tool. Although Streamlit is not as customizable as Plotly, building the layout of the dashboard is quite straightforward, and it supports multipage apps, making it possible to build more complex applications.
However, in the 2024 results these numbers may change a little. There are up-and-coming tools that could catch up to – or even surpass – these apps in popularity. One of them is HoloViz Panel. As one of the libraries in the HoloViz ecosystem, it is sponsored by NumFocus and is gaining traction among the PyData community. Panel lets users generate reports in the HTML format and also works very well with Jupyter Notebook. It offers templates to help new users get started, as well as a great deal of customization options for expert users who want to fine-tune their dashboards.
ML models: scikit-learn is still prominent, while PyTorch is the most popular for deep learningBecause generative AI and LLMs have been such hot topics in recent years, you might expect deep learning frameworks and libraries to have completely taken over. However, this isn’t entirely true. There is still a lot of insight that can be extracted from data using traditional statistics-based methods offered by scikit-learn, a well-known machine learning library mostly maintained by researchers. Sponsored by NumFocus since 2020, it remains the most important library in machine learning and data science. SciPy, another Python library that provides support for scientific calculations, is also one of the most used libraries in data science.
Having said that, we cannot ignore the impact of deep learning and the increase in popularity of deep learning frameworks. PyTorch, a machine learning library created by Meta, is now under the governance of the Linux Foundation. In light of this change, we can expect PyTorch to continue being a load-bearing library in the open-source ecosystem and to maintain its level of active community involvement. As the most used deep learning framework, it is loved by Python users – especially those who are familiar with numpy, since “tensors”, the basic data structures in PyTorch, are very similar to numpy arrays.
You can inspect Pytorch tensors in PyCharm 2024.2.2 Pro just like you inspect Numpy arraysUnlike TensorFlow, which uses a static computational graph, PyTorch uses a dynamic one – and this makes profiling in Python a blast. To top it all off, PyTorch also provides a profiling API, making it a good choice for research and experimentation. However, if your deep learning project needs to be scalable in deployment and needs to support multiple programming languages, TensorFlow may be a better choice, as it is compatible with many languages, including C++, JavaScript, Python, C#, Ruby, and Swift. Keras is a tool that makes TensorFlow more accessible and is also popular for deep learning frameworks.
Another framework we cannot ignore for deep learning is Hugging Face Transformers. Hugging Face is a hub that provides many state-of-the-art pre-trained deep learning models that are popular in the data science and machine learning community, which you can download and train further yourself. Transformers is a library maintained by Hugging Face and the community for state-of-the-art machine learning with PyTorch, TensorFlow, and JAX. We can expect Hugging Face Transformers will gain more users in 2024 due to the popularity of LLMs.
With PyCharm you can identify and manage Hugging Face models in a dedicated tool window. PyCharm can also help you to choose the right model for your use case from the large variety of Hugging Face models directly in the IDE.
One new library that is worth paying attention to in 2024 is Scikit-LLM, which allows you to tap into Open AI models like ChatGPT and integrate them with scikit-learn. This is very handy when text analysis is needed, and you can perform analysis using models from scikit-learn with the power of modern LLM models.
MLOps: The future of data science projectsOne aspect of data science projects that is essential but frequently overlooked is MLOps (machine learning operations). In the workflow of a data science project, data scientists need to manage data, retrain the model, and have version control for all the data and models used. Sometimes, when a machine learning application is deployed in production, performance and usage also need to be observed and monitored.
In recent years, MLOps tools designed for data science projects have emerged. One of the issues that has been bothering data scientists and data engineers is versioning the data, which is crucial when your pipeline constantly has data flowing in.
Data scientists and engineers also need to track their experiments. Since the machine learning model will be retrained with new data and hyperparameters will be fine-tuned, it’s important to keep track of model training and experiment results. Right now, the most popular tool is TensorBoard. However, this may be changing soon. TensorBoard.dev has been deprecated, which means users are now forced to deploy their own TensorBoard installations locally or share results using the TensorBoard integration with Google Colab. As a result, we may see a drop in the usage of TensorBoard and an uptick in that of other tools like MLflow and PyTorch.
Another MLOps step that is necessary for ensuring that data projects run smoothly is shipping the development environment for production. The use of Docker containers, a common development practice among software engineers, seems to have been adopted by the data science community. This ensures that the development environment and the production environment remain consistent, which is important for data science projects involving machine learning models that need to be deployed as applications. We can see that Docker is a popular tool among Python users who need to deploy services to the cloud.
This year, Docker containers is slightly ahead of Anaconda in the “Python installation and upgrade” category.
2023 survey results 2022 survey results Big data: How much is enough?One common misconception is that we will need more data to train better, more complex models in order to improve prediction. However, this is not the case. Since models can be overfitted, more is not always better in machine learning. Different tools and approaches will be required depending on the use case, the model, and how much data is being handled at the same time.
The challenge of handling a huge amount of data in Python is that most Python libraries rely on the data being stored in the memory. We could just deploy cloud computing resources with huge amounts of memory, but even this approach has its limitations and would sometimes be slow and costly.
When handling huge amounts of data that are hard to fit in memory, a common solution is to use distributed computing resources. Computation tasks and data are distributed over a cluster to be performed and handled in parallel. This approach makes data science and machine learning operations scalable, and the most popular engine for this is Apache Spark. Spark can be used with PySpark, the Python API library for it.
As of Spark 2.0, anyone using Spark RDD API is encouraged to switch to Spark SQL, which provides better performance. Spark SQL also makes it easier for data scientists to handle data because it enables SQL queries to be executed. We can expect PySpark to remain the most popular choice in 2024.
Another popular tool for managing data in clusters is Databricks. If you are using Databricks to work with your data in clusters, now you can benefit from the powerful integration of Databricks and PyCharm. You can write code for your pipelines and jobs in PyCharm, then deploy, test, and run it in real time on your Databricks cluster without any additional configuration.
Communities: Events shifting focus toward data scienceMany newcomers to Python are using it for data science, and thus more Python libraries have been catering to data science use cases. In that same vein, Python events like PyCon and EuroPython are beginning to include more tracks, talks, and workshops that focus on data science, while events that are specific to data science, like PyData and SciPy, remain popular, as well.
Final thoughtsData science and machine learning are becoming increasingly active, and together with the popularity of AI and LLMs, more and more new open source tools have become available for use in data science. The landscape of data science continues to change rapidly, and we are excited to see what becomes most popular in the 2024 survey results.
Enhance your data science experience with PyCharmModern data science demands skills for a wide range of tasks, including data processing and visualization, coding, model deployment, and managing large datasets. As an integrated development environment (IDE), PyCharm helps you efficiently build this skill set. It provides intelligent coding assistance, top-tier debugging, version control, integrated database management, and seamless Docker integration. For data science, PyCharm supports Jupyter notebooks, as well as key scientific and machine learning libraries, and it integrates with tools like the Hugging Face models library, Anaconda, and Databricks.
Start using PyCharm for your data science projects today and enjoy its latest improvements, including features for inspecting pandas and Polars DataFrames, and for the layer by layer inspection of PyTorch tensors, which is handy when exploring data and building deep learning models.
Try PyCharm for freeQt for MCUs 2.9 Released
We are excited to announce Qt for MCUs 2.9, which comes with many key features to enable Qt for MCUs to support more use cases in the IoT, Consumer and Automotive segments. Here are few of the major highlights from the 2.9 release.
LostCarPark Drupal Blog: Drupal Advent Calendar day 2 - Starshot Installer
It’s day 2 of the Drupal Advent calendar and today we’re taking a look at the first step to any new website built with Drupal CMS, the site installer.
The previous Drupal installer wasn’t terrible, but it required a lot of steps, and typically needed a lot more work, finding and installing modules, when the initial install was complete.
The new installer has tried to simplify the process as much as possible, and offers a friendlier interface.
The primary question it asks is what are the main goals of your site:
At present, there are six options, but these are expected to be expanded in the future…
TagsPython Bytes: #412 Closing the loop
I think the donation notification works
A few months ago, I blogged about a change for Plasma 6.2 to show a once-a-year system notification asking for a donation, starting on December 1st. Various reasons and justifications were given in that post, so I won’t repeat them here. Instead, since December 1st was yesterday in most of the world, it’s time to check in on the day 1 experience! So let’s get right into it:
Did it work?Well, I woke up to an email inbox that looked like this:
And by the end of the day, the graph on https://kde.org/community/donations/previousdonations (which by the way only counts direct Paypal donations and still doesn’t include those made using Donorbox or direct bank transfer) wound up looking like this:
Yes that’s right, KDE e.V. received double the prior two months’ Paypal donations in a single day!!!
Do people hate us now?So far, indications point to no! I scoured https://www.reddit.com/r/kde and https://discuss.kde.org all day yesterday and literally only found one non-positive comment about it, dwarfed by a large volume of mildly to highly positive ones. I wasn’t looking at Mastodon or other social media, but a colleague reported something similar.
In addition, a large number of the donations themselves were accompanied by positive messages from the donors. Here are some of my favorites:
KDE is more than just software, it’s a family. Least I can donate, but it’s coming from someone that pirates every other thing or uses the free alternative.
Thanks for all your incredible work over the years.
KDE Plasma is a big part of why I have grown to love Linux as my daily driver
Thanks for all you have done for the linux desktop community
Thanks for Plasma! Couldn’t work without it! (Visually impaired user).
Thanks for your efforts to make the world a little more independent from Big Tech
Love the work, KDE is my daily driver and I’m glad I can help
Just got the Notification to donate in KDE and after thinking about it for a bit decided to donate for the first time, since I’ve been using Linux and specifically KDE for almost a year now. Thanks for your hard work!
Thanks for all of the work and effort put into making KDE the best DE ever!
So, yeah. On the contrary, it feels like our users really, really love us!
Is this repeatable?It’s too early to say at this point, but I hope so. It will be interesting to see how fast the donations drop off. Will it be relatively fast because everyone who was going to donate after seeing to the notification already saw it yesterday? Or will the drop-off take a while because there are more notification-based potential donors who didn’t turn on their Plasma 6.2-using computer yet, or opened the donations page in a browser tab to action later? We don’t know; we’ll have to wait and see.
However it’s also worth mentioning that these donations are coming entirely from people using distros that include Plasma 6.2. Right now that’s pretty much limited to fast-paced distros like Arch, Fedora KDE, KDE Neon, OpenSUSE Tumbleweed, and their derivatives. Notably, it excludes traditional heavy hitters like Kubuntu and Debian. So there are reasons to expect the donation notification to reach even more eyeballs in 2025 than it has this year.
Now that you’re rich are you going to buy a bunch of leopard-print Porsche steering wheel covers and other KDE e.V. board junkets?No board junkets. It’s too early to make a projection based on the performance of single day, and especially if the donations drop off quickly, this isn’t “Thunderbird money” yet. But it does look quite possible that all these donations may push KDE e.V. into ending up with a balanced budget for the 2024 financial year. That would be pretty fantastic, as we weren’t predicting a balanced budget until 2025 or 2026, instead originally expecting a deficit of over €50k in 2024. And that was already an improvement over the 110k deficit in 2023.
Balancing the budget early is huge, and opens up opportunities. As you may know, German nonprofits like KDE e.V. are required to avoid stockpiling money (hence the intentional deficits), so moving into the realm of positive cashflow means we’ll need to increase our expenditures. Thankfully, KDE e.V. has become very good at spending money over the past few years, largely by expanding our hiring on personnel in technical roles: basically sponsoring community members to improve our products directly.
The easiest way to spend more money is to simply lean into that harder: hire another person, sponsor another project, stuff like that — pretty much what I mentioned in the original post. More money means more tech work financed by KDE itself, directly increasing our institutional ability to control our own destiny. It’s pretty great stuff if you ask me. But again, this is a collective board decision, not up to me alone. And if you disagree with me that this is the right use for KDE’s money, that’s fine too, and I’ll mention that I’m up for re-election on the board next year, so please do feel free to run or vote against me if you’re a KDE e.V. member! The organization works best with a board that reflects its membership’s preferences. I have zero desire to occupy that seat if I’m not representing people properly.
Anyway, it works. It appears to really work. My conclusion is that KDE has built up enough goodwill that our user community loves and trusts us, which made this outpouring of financial support possible. It’s humbling and kind of overwhelming. But it all strengthens my conviction that KDE is pointing in the right direction and amounts to a strong positive force for humanity!
Want to help out? In addition to donating your money which is what we’ve been talking about, an arguably more impactful approach is to donate your time directly, bypassing any institutional middleman that buys time with money! It’s not hard to get started, and there are loads of resources and mentorship opportunities. So help make the world a better place through KDE today!