Feeds
Specbee: User-Centric Design: Why Your Website Needs It
Seth Michael Larson: OSS Security RFI, Guide to become a CNA, and PEP 639
Published 2023-11-07 by Seth Larson
Reading time: minutes
The past week has been almost exclusively writing for me! Here's a rundown on what I've been writing about:
Request for Information (RFI) on Open Source Software SecurityAs many folks in the Open Source security space are aware of, the deadline for the US Government RFI submissions (November 9th, 2023) is fast-approaching! I've been working with my colleagues at the Python Software Foundation to draft a response to the RFI over the past months now. The past few weeks have had a lot of time spent on collaborating and refining our response to the point where I am quite proud of it now.
If this is your first time hearing of the RFI, the Linux Foundation and Tidelift have both covered the RFI, what it is and why it's an exciting development for open source.
The Python Software Foundation's response to the RFI is about capturing what we believe is important regarding the US governments approach to securing open source software. Whatever gets done by the US government is likely to have huge implications for everyone maintaining and consuming open source software, so it's critical that policy and decisions are made with sustainability in mind.
I'm honored to be a part of this and to represent so many Pythonistas in my work both for this RFI and every day as Security Developer-in-Residence. 💜
Becoming a CVE Numbering Authority as an Open Source projectThroughout the process of joining the CVE Numbering Authority program for the Python Software Foundation I noted down all the steps and requirements to become a CNA. I transformed these notes into a digestible document that's specifically written for Open Source projects and organizations. This document has had extensive review from both the OpenSSF Vulnerability Disclosures Working Group and multiple CVE Working Groups.
This guide has recently been published under the OpenSSF Vulnerability Disclosures WG GitHub repository. I'm now in the process of drafting an announcement blog post for the OpenSSF blog.
PEP 639 - Licensing clarity in packaging metadataI've raised my hand to help PEP 639 make its way to acceptance as this PEP was one that I noted as being important for Software Bill-of-Materials being adoptable for the Python packaging ecosystem. I wanted to also thank Karolina Surma who works on Python packaging at Red Hat for joining as a coauthor of PEP 639 as well and is already making use of the PEP. Thanks so much!
The gist of this PEP is to move package tooling and maintainers to adopt SPDX License IDs and expressions in order to more accurately represent the licenses of Python packages. Previous standards would use an open-ended string License field along with License :: * trove classifiers. This approach isn't able to capture all licensing situations (such as 'MIT OR GPL-2.0-only') and especially struggles with license revisions.
Due to the inability to capture these more complication situations, it often meant that tooling consuming Python packages would need to look at LICENSE, NOTICE, or COPYING files and do their own text detection in order to have an accurate view of the licensing situation. Choosing a license is one of the more important decisions before releasing software into the wild, so ensuring that that choice is unambiguous is very important!
Other Items- Discussed "affectedness" based on modules and functions for the PyPA Advisory database. Having this information would allow vulnerability scanning tools like pip-audit to only associate a vulnerability with a project if the affected module or function is used by the project. In theory this information will reduce the amount of false-positives when a vulnerability only affects a single feature rather than the entire project.
- OSV announced broad support for C/C++ projects and vulnerabilities. Will need to test this out against the Python ecosystem to provide feedback on their detection tooling and how it applies to Python.
- Published the engagement report for October 2023 to Alpha-Omega.
That's all for this week! 👋 If you're interested in more you can read last week's report.
Wow, you made it to the end!
If you're like me, you don't believe social media should be the way to get updates on the cool stuff your friends are up to. Instead, you should either follow my blog via the RSS feed or the email newsletter for guaranteed article publication notifications.
If you really enjoyed a piece I would be grateful if you shared with a friend. If you have follow-up thoughts you can send them via email.
Thanks for reading!
— Seth
Matthew Palmer: PostgreSQL Encryption: The Available Options
On an episode of Postgres FM, the hosts had a (very brief) discussion of data encryption in PostgreSQL. While Postgres FM is a podcast well worth a subscribe, the hosts aren’t data security experts, and so as someone who builds a queryable database encryption system, I found the coverage to be somewhat… lacking. I figured I’d provide a more complete survey of the available options for PostgreSQL-related data encryption.
The Status QuoBy default, when you install PostgreSQL, there is no data encryption at all. That means that anyone who gets access to any part of the system can read all the data they have access to.
This is, of course, not peculiar to PostgreSQL: basically everything works much the same way.
What’s stopping an attacker from nicking off with all your data is the fact that they can’t access the database at all. The things that are acting as protection are “perimeter” defences, like putting the physical equipment running the server in a secure datacenter, firewalls to prevent internet randos connecting to the database, and strong passwords.
This is referred to as “tortoise” security – it’s tough on the outside, but soft on the inside. Once that outer shell is cracked, the delicious, delicious data is ripe for the picking, and there’s absolutely nothing to stop a miscreant from going to town and making off with everything.
It’s a good idea to plan your defenses on the assumption you’re going to get breached sooner or later. Having good defence-in-depth includes denying the attacker to your data even if they compromise the database. This is where encryption comes in.
Storage-Layer Defences: Disk / Volume EncryptionTo protect against the compromise of the storage that your database uses (physical disks, EBS volumes, and the like), it’s common to employ encryption-at-rest, such as full-disk encryption, or volume encryption. These mechanisms protect against “offline” attacks, but provide no protection while the system is actually running. And therein lies the rub: your database is always running, so encryption at rest typically doesn’t provide much value.
If you’re running physical systems, disk encryption is essential, but more to prevent accidental data loss, due to things like failing to wipe drives before disposing of them, rather than physical theft. In systems where volume encryption is only a tickbox away, it’s also worth enabling, if only to prevent inane questions from your security auditors. Relying solely on storage-layer defences, though, is very unlikely to provide any appreciable value in preventing data loss.
Database-Layer Defences: Transparent Database EncryptionIf you’ve used proprietary database systems in high-security environments, you might have come across Transparent Database Encryption (TDE). There are also a couple of proprietary extensions for PostgreSQL that provide this functionality.
TDE is essentially encryption-at-rest implemented inside the database server. As such, it has much the same drawbacks as disk encryption: few real-world attacks are thwarted by it. There is a very small amount of additional protection, in that “physical” level backups (as produced by pg_basebackup) are protected, but the vast majority of attacks aren’t stopped by TDE. Any attacker who can access the database while it’s running can just ask for an SQL-level dump of the stored data, and they’ll get the unencrypted data quick as you like.
Application-Layer Defences: Field EncryptionIf you want to take the database out of the threat landscape, you really need to encrypt sensitive data before it even gets near the database. This is the realm of field encryption, more commonly known as application-level encryption.
This technique involves encrypting each field of data before it is sent to be stored in the database, and then decrypting it again after it’s retrieved from the database. Anyone who gets the data from the database directly, whether via a backup or a direct connection, is out of luck: they can’t decrypt the data, and therefore it’s worthless.
There are, of course, some limitations of this technique.
For starters, every ORM and data mapper out there has rolled their own encryption format, meaning that there’s basically zero interoperability. This isn’t a problem if you build everything that accesses the database using a single framework, but if you ever feel the need to migrate, or use the database from multiple codebases, you’re likely in for a rough time.
The other big problem of traditional application-level encryption is that, when the database can’t understand what data its storing, it can’t run queries against that data. So if you want to encrypt, say, your users’ dates of birth, but you also need to be able to query on that field, you need to choose between one or the other: you can’t have both at the same time.
You may think to yourself, “but this isn’t any good, an attacker that breaks into my application can still steal all my data!”. That is true, but security is never binary. The name of the game is reducing the attack surface, making it harder for an attacker to succeed. If you leave all the data unencrypted in the database, an attacker can steal all your data by breaking into the database or by breaking into the application. Encrypting the data reduces the attacker’s options, and allows you to focus your resources on hardening the application against attack, safe in the knowledge that an attacker who gets into the database directly isn’t going to get anything valuable.
Sidenote: The Curious Case of pg_cryptoPostgreSQL ships a “contrib” module called pg_crypto, which provides encryption and decryption functions. This sounds ideal to use for encrypting data within our applications, as it’s available no matter what we’re using to write our application. It avoids the problem of framework-specific cryptography, because you call the same PostgreSQL functions no matter what language you’re using, which produces the same output.
However, I don’t recommend ever using pg_crypto’s data encryption functions, and I doubt you will find many other cryptographic engineers who will, either.
First up, and most horrifyingly, it requires you to pass the long-term keys to the database server. If there’s an attacker actively in the database server, they can capture the keys as they come in, which means all the data encrypted using that key is exposed. Sending the keys can also result in the keys ending up in query logs, both on the client and server, which is obviously a terrible result.
Less scary, but still very concerning, is that pg_crypto’s available cryptography is, to put it mildly, antiquated. We have a lot of newer, safer, and faster techniques for data encryption, that aren’t available in pg_crypto. This means that if you do use it, you’re leaving a lot on the table, and need to have skilled cryptographic engineers on hand to avoid the potential pitfalls.
In short: friends don’t let friends use pg_crypto.
The Future: EnquoAll this brings us to the project I run: Enquo. It takes application-layer encryption to a new level, by providing a language- and framework-agnostic cryptosystem that also enables encrypted data to be efficiently queried by the database.
So, you can encrypt your users’ dates of birth, in such a way that anyone with the appropriate keys can query the database to return, say, all users over the age of 18, but an attacker just sees unintelligible gibberish. This should greatly increase the amount of data that can be encrypted, and as the Enquo project expands its available data types and supported languages, the coverage of encrypted data will grow and grow. My eventual goal is to encrypt all data, all the time.
If this appeals to you, visit enquo.org to use or contribute to the open source project, or EnquoDB.com for commercial support and hosted database options.
Paolo Melchiorre: Database generated columns ⁽¹⁾: Django & SQLite
An introduction to database generated columns, using SQLite and the new GeneratedField added in Django 5.0.
PreviousNext: The Pitchburgh Diaries - decoupled Layout Builder Sprint 5 & 6
Welcome to the third edition of the Pitchburgh Diaries, a regular update on progress as we work on our plan for a decoupled Layout Builder with React.
by lee.rowlands / 7 November 2023Sprints 5 and 6 were our final sprints in the project. Keep an eye out for a final wrap-up and summary of the next steps, which we’ll publish in the coming weeks.
Bundling and extendingIn this sprint, we focused on the nuts and bolts of how contrib and custom code will extend decoupled Layout Builder functionality.
We began by creating a new Drupal 10-compatible version of the React module. Thanks to @corbacho for adding us as co-maintainers for the project!
When we bundle the decoupled Layout Builder code for use in the browser, we don't include React. Instead, we rely on the React module to load it. This allows other modules that need React (e.g. Gutenberg) to use a shared version of React. React doesn't work if two versions are loaded on the same page.
The new version of the React module makes use of a relatively new browser API called import maps. Import maps allow you to write ECMAScript modules (ESM) with naked imports and have the browser resolve the dependency for you.
So, for example, our bundled code still has import React from 'react' in it. With an import map, the browser can resolve that to a Javascript file and load it for you.
To support this functionality, we wrote and released an import maps module which both the Decoupled Layout Builder API and React module make use of. We believe this functionality belongs in core because you can only have one import map on a page. So we opened a core feature request for that too.
With this module in place, bundling for contrib and custom code that wants to add additional components to the decoupled Layout Builder becomes much simpler. Essentially the build configuration needs to mark imports of React, React DOM, the JSX runtime and the decoupled layout builder components as 'external'. This ensures the bundled code retains the original import statements. Both Vite and Webpack support this feature.
For those who recall how we built ES6 code in Drupal 9, you would know we had scripts in core's package.json and webpack configuration to transpile it into code that worked in older browsers like Internet Explorer. With Drupal 10, we removed that step as all of our supported browsers have native support for ES6 code. Similarly, if you've built a CKEditor 5 plugin, you would know it also uses Webpack for bundling.
As a result, Webpack felt like the natural choice for bundling here too. WordPress uses it to bundle block components for their Gutenberg editor. However, the web landscape moves quickly. The tool we chose N years ago may no longer be the best choice. With all modern browsers supporting ESM, the bundling landscape has changed.
Those who follow front-end web development would know that many projects are actively moving away from Webpack towards Vite. Storybook added support for Vite in v7, and just last week, Remix had a major announcement about Vite support. CKEditor5 has also added Vite support. For this reason, we evaluated both Vite and Webpack for use in our utility Drupal scripts package. Thispackage is designed to make writing and bundling code for use with the decoupled Layout Builder simpler. Based on our evaluation and the broader front-end landscape moving towards Vite, we chose it for our bundling.
As a result, we have an npm package @drupal/scripts that we will release in the coming weeks with the following features:
- A simple build step. No need to manage your own bundling configuration - just add a dependency on @drupal/scripts (more on that below) and add this to your package.json for building - drupal-scripts build -o js/dist js/src
- A simple setup process - if you install the package globally, you can run drupal-scripts init to have it automatically update your package.json with the required changes
- Support for scaffolding components for use with decoupled Layout Builder - just run drupal-scripts generate and follow the steps.
In our first four sprints, we focused on building the Layout Editor in a decoupled application. We were mocking APIs so development could occur without a Drupal site.
In these two sprints, we switched to instantiating the Layout Editor in an actual Drupal site.
The Layout Editor uses React components that mirror Blocks, Formatters, Widgets and Layout plugins from Drupal. We have always intended for these to be the extension points for the application. If you need to change how any of those work in a project, you should be able to swap in your own custom React component.
To facilitate this, the entry point for the decoupled Layout Builder is the Layout Editor component. It takes a map of components for each of the Blocks, Formatters, Widgets and Layout plugins. This map is keyed by the plugin ID (same IDs as in Drupal). The values of the map are a function that return a promise, that will resolve the components. What each component comprises depends on the type.
For example, a Block component needs an Edit and Preview component but might also need a Settings component. You can read more about what each component comprises in the storybook documentation.
In order to boot the Layout Editor, Drupal needs to construct these maps. To do this, we make use of existing plugin definitions and extend them to add an entry for the decoupled Layout Builder.
Here's an example of nominating the path to a React component for a layout plugin:
/** * Implements hook_layout_alter(). */ function mymodule_layout_alter(&$definitions) { $path = '/' . rtrim(\Drupal::service('extension.list.module')->getPath('mymodule''), '/') . '/js/dist/'; /** @var \Drupal\Core\Layout\LayoutDefinition[] $definitions */ if (isset($definitions['my_layout''])) { $definitions['mylayout'']->set('decoupled_lb', [ 'library' => 'mymodule/mylayout', 'componentPath' => $path . 'MyLayout.js', ]); } }In this example, the file MyLayout.js would be scaffolded with the drush-scripts generate command and updated according to the documentation for a Layout component.
In the Decoupled Layout Builder API module, we replace the default LayoutBuilder render element with a decoupled one. When this component is rendered, it loops over all of the block, formatter, widget and layout plugin definitions and builds up a mapping from ID to component path. This is then stored in drupalSettings. The element also attaches some Javascript to boot the React application that reads this value back and turns the file paths into promises using the browser's native import operator.
With all this in place, we were able to boot the new Layout Builder in Drupal 🎉. Here's a screenshot of that for the Recipe content-type in the Umami demo profile:
Other key highlightsSo, while we've focussed mainly on the big ticket items, we were also able to complete a fair few of our other wish list items in these final sprints, including:
- Fallback rendering when no React component exists - as seen above, we're able to fallback to Drupal rendering where no React component exists yet
- Support for layout builder restrictions so that you can only drag and drop components into supported regions
- Support for saving changes in Drupal - including autosaving
- General improvements to Drag and drop so that it was easier to drag new components into existing regions
- General normalisation improvements so that section and region UUIDs are generated Drupal side and stored in third-party settings.
- GitLab CI integration so that Storybook builds on every push.
- Additional documentation and tests.
- Layout settings pane
We've reached the end of our sprints for Pitchburgh. But that doesn't mean the work stops. We plan to continue working on the project and have quite a backlog of new features we'd like to add.
In our next post, we'll recap each of the completed stories for our Pitchburgh grant statement of work, go into more detail about our future plans and let you know where you can help.
Tagged Layout Builder, ReactJSSubtitle Composer 0.8.0 released
I’m happy to announce the 0.8.0 release of Subtitle Composer.
This release contains lots of bugfixes and new features including:
- Automatic translations using DeepL or Google Cloud
- WebVTT format support
- Subtitle positioning UI and support
- Subtitle class/comments/voice UI and support
- Improved rich text editing/preview
- Rich text rendering on waveform/video/editors
- Qt6 support
- FFmpeg 5 support
- Subtitle lines are always time sorted; remove sort lines menu action
- Replaced Kross with QJSEngine, removed ruby and python support
- Improved script manager and tools menu
As usual all binaries are available from download page.
Source tarball can be downloaded from download.kde.org.
Talking Drupal: Talking Drupal #423 - Conflict Resolution Team
Today we are talking about The Conflict Resolution Team, What they do, and Why they do it with guest Mark Casias. We’ll also cover Smart Trim as our module of the week.
For show notes visit: www.talkingDrupal.com/423
Topics- What is the Conflict Resolution Team
- Why is the Conflict Resolution Team needed
- When / Why was the Conflict Resolution Team created
- What kinds of issues does the Conflict Resolution Team deal with
- Do you meet separately from discussing issues
- How do you handle burnout
- How does one become a member
- Why are you a member
- How do you submit an issue to the Conflict Resolution Team
- How many submissions do you get
- Do you have to include the Drupal Association
- Does the DA provide tools
- Does the Conflict Resolution Team need help
- Is there any specific training you look for
- Code of conduct training
- Anything you would like to add
Nic Laflin - nLighteneddevelopment.com nicxvan John Picozzi - epam.com johnpicozzi Mark Casias - kanopi.com - markie
MOTW CorrespondentMartin Anderson-Clutz - @mandclu Smart Trim
- Brief description:
- Have you ever wanted to truncate provided user text in a more intelligent way than the “summary or trimmed” formatter in Drupal core? There’s a module for that.
- Brief history
- How old: Originally created in Apr 2012
- Versions available: 7.x-1.6 and 2.1.0 versions available, the latter compatible with Drupal 8, 9, and 10
- Maintainership
- Actively maintained
- Number of open issues
- 74, 8 of which are active bugs against the 2.x branch
- Test coverage, using the new GitLab CI
- User guide for documentation
- Usage stats:
- 66,919
- Maintainer(s):
- Friends of the podcast, markie, ultimike, volkswagenchick
- Module features and usage
- Adds a variety of configuration options, in addition to the trim length:
- Whether the length is characters or words
- An optional suffix at the trim point e.g. ellipsis
- Whether or not to add a more link after the trimmed text
- Stripping HTML tags from the trimmed output
- The ability to customize the output even more via twig template override
- Because it works by providing a field formatter it works with entity display but also views, layout builder, and more
- A module I’ve used many times myself and found extremely useful. I haven’t had a chance to try the latest release, but given the recent focus on UI improvements, documentation, and flexibility in the twig template it should be even better than I remember
The Drop Times: Break the Cage, Spread the Wings, Soar High!
Dear Readers,
"Birds born in a cage think flying is an illness."
—Alejandro Jodorowsky, Film maker.Literally? I doubt that. This quote by Alejandro Jodorowsky cannot be interpreted literally but metaphorically. It symbolizes the constraints and learned limitations prevalent in the human experience. The notion suggests that individuals accustomed to restrictive environments or societal norms may perceive natural behaviors as abnormal, not unlike the metaphorical birds in Alejandro Jodorowsky's insight.
Consider the story of a canary long confined within a cage. Upon release into the open, the canary might initially be intimidated by the freedom of the vast skies that other birds effortlessly navigate. Yet, in time, the canary adapts, spreads its wings, and takes flight. However, for us—humans like you and me—breaking free from our own mental and societal confines might not be as simple. This story illustrates how conditioning and familiarity with constraints can distort our perception of what's natural or possible. Some of us are like sharks; we only grow as big as our environment, and if it's a cage, we will forever remain caged. But once you break that shackle and find your niche, the growth can be monumental.
In a digital platform ecosystem, Drupal stands as a symbol of liberation from constraints, like the canary breaking free from its cage. In an ecosystem where popular software programs followed rigid and closed systems, Drupal was born to offer an open space for innovation and creativity. In website development, individuals constrained by traditional systems often face challenges when presented with the freedom and flexibility that Drupal offers.
Similar to the canary initially intimidated by the vast open sky, newcomers to Drupal may feel overwhelmed by its seemingly boundless possibilities. However, just as the canary adapts and embraces the open skies, individuals within the Drupal community, although initially daunted, flourish as they navigate and harness the platform's versatility.
Remember, the growth can be tremendous, but only if one breaks the shackles of constraints that hold them down.
On that note, we delve into last week's vibrant stories from The Drop Times (TDT):
DrupalCon Lille's echoes continue to resound within the Drupal Community, and TDT extends its gratitude for the overwhelming support received during our coverage. Despite numerous hurdles, particularly conducting our reporting remotely, The Drop Times embraced the challenge with ambition and soared high. Here's a recount of our successful execution of DrupalCon Lille coverage, "Way to Lille: How TDT Remotely Covered DrupalCon Lille 2023."
Throughout the week, attendees and organizations shared their invaluable feedback and experiences from DrupalCon Lille 2023. Tigin Öztürk, Irina Povjakel, SearchStax, SystemSeed, and ZenSource generously shared personal insights and event summaries. Furthermore, Scott Massey from Morpht discussed key strategies and challenges for Drupal migration in the light of Driesnote. Acquia conducted a webinar on DrupalCon Lille recap on November 02, 2023.
A new platform, "Innovation Ideas," emerged within the Drupal community, drawing inspiration from the successful Pitch-Burgh Innovation Contest. Our sub-editor, Elma John, conversed with Ricardo Marcelino to explore this initiative further.
TDT compiled a list of books aimed at decoding Drupal. Whether you're a developer or a marketer, these books have you covered.
The conclusion of Drupal GovCon Bethesda 2023 was marked with success. The Splash Awards and Drupal Business Day in Mannheim, Germany, set for November 10, will feature 28 selected Drupal projects competing across eight categories.
Notably, Drupal's debut appearance at Web Summit Lisbon and the forthcoming LocalGov Drupal Week 2023 are anticipated milestones for the Drupal community. TDT has also published a list of events within the Drupal Community for the week.
With the transition of Drupal 9 reaching its end of life on November 01, the focus has now shifted to Drupal 10 compatibility. Meanwhile, as Drupal 7 perseveres, Kristen Pol's recent blog post, shared by QuantCDN, navigates through the history of Drupal 7 and explores the available end-of-life (EOL) options. Tag1 Consultancy commenced a series on Scary Drupal migration stories featuring Janez Urevc in the past week.
On the security front, the German Federal Office for Information Security (BSI) issued an advisory for Drupal on November 02, 2023, regarding a vulnerability affecting various operating systems and the Drupal content management system. Simultaneously, a security update for the Paragraphs admin module in Drupal addressed a moderate risk vulnerability. Vallic has shared a blog post on simplifying Drupal updates using GitHub Actions.
A growing momentum within the Drupal community highlights DDEV as the official local development environment for Drupal, as outlined by Kevin Quillen in a recent blog post. Furthermore, The Argyle Report spotlighted the thriving global Drupal Development Service industry, presenting insights from a new report by Global Market Vision titled 'Global Drupal Development Service Market: Size, Share, Price, Trends, Report, and Forecast 2023-2030'.
Several articles addressed the optimization of Drupal sites, focusing on fundamental SEO practices by Otatech and security checklists by Firas Gunhaim. Reuben Walker, in Dev, recently compared Symfony Flex's Recipes and Drupal's emerging Recipes Initiative, examining their respective functionalities and operational frameworks. The latest article by Golems gives an outlook into the future of Drupal themes.
This encapsulates the highlights from The Drop Times. We invite you to share any suggestions, contributions, or feedback, as we sincerely value your engagement within our community. Thank you for being part of this journey. Until our next edition, keep exploring, sharing, and embracing the power of knowledge!
Follow us on LinkedIn, Twitter, and Facebook for regular updates.
Warm Regards,
Alka Elizabeth
Sub-editor, The Drop Times
FSF Events: Free Software Directory meeting on IRC: Friday, November 10, starting at 12:00 EST (17:00 UTC)
TechBeamers Python: How to Write Multiline Comments in Python
In Python, there is no specific syntax for creating multiline comments like some other programming languages (e.g., C, C++, Java). Instead, Python relies on other methods to achieve similar goals. Multiline comments are typically used to provide explanations, documentation, or notes within your code. In this tutorial, we will explore various methods for adding multi-line [...]
The post How to Write Multiline Comments in Python appeared first on TechBeamers.
Chromatic: Drupal 7 End-of-Life Ep 15: What Does Drupal 9 EOL Mean for Drupal 7 EOL?
Closing the 2023 rounds of Deep Dive AI with first draft piece of the Definition of Open Source AI
We embarked on a process, promising at the beginning of the year that we’d make a first announcement at All Things Open, kickstarting a public conversation. We’ve delivered, thanks to contributions of many experts and sponsors. But it’s only the starting point. There is a lot more to do.
After two community reviews in person and a first pass at online comments, we released a new draft version 0.0.3.
The base of the conversation is a preamble to explain “why Open Source AI,” followed by the beginning of a formal definition: the document will get longer. Open Source experts will recognize the heavy borrowing from the free software definition and the structure of the GNU Manifesto: it’s not a mistake. We believe that the consensus on a Definition of Open Source AI will emerge after the stakeholders will have made a similar journey that led to the Open Source Definition. The OSD is basically a checklist that appeared after decades of free software development, when developers, users, business leaders, lawyers and policymakers had time to learn what freedom meant in the context of software. We don’t have decades to wait for AI but we can accelerate by building on top of what many of us already know and reach out to diverse communities to join the conversation.
That’s what the OSI is doing with these Deep Dive: AI cycles: inviting multiple stakeholders to learn and share their knowledge as we all make progress together towards a common understanding of AI systems.
What’s in draft v.0.0.3The four freedoms have received a bit of wordsmithing for consistency and clarity, making them shorter compared to previous drafts. I removed the words “without any limitation” from the Use and Share principles as recommended by Chestek, and because a question about copyleft also came up at the workshop in Monterey.
The current version reflects the consensus of the suggestions emerged from the workshops in Raleigh and Monterey, and the online comments to v.0.0.2.
In addition to those changes, I did some cleanup of the word soup, removing all instances of the most loaded concepts like trustworthy, reliable, fair, etc. from the preamble: they only appear in the “Out of scope” section.
Enjoy and comment on draft 0.0.3.
Known issues and next stepsThere is no consensus on what definition of AI system to use. The draft 0.0.3 still uses the definition introduced by OECD in 2019 for lack of a better option. We’ll continue the conversation.
We have two more in-person workshops scheduled before the end of the year: Nov 15 at the DPGA annual summit in Addis Ababa; and Dec 12-13 at the Linux Foundation AI.Dev conference in San Jose. These were not planned at the beginning of the year when we announced the 2023 series but they’re extremely important to reach African tech leaders and policy makers and AI developers.
At this point we want to close DDAI 2023 thanking the sponsors Google, Amazon, GitHub, OSS Capital, GitLab, Weaviate and Sourcegraph; the Linux Foundation for their travel grants; and individual donors, because we couldn’t have hosted the webinar series and run three in-person meetings without them.
We’re working on a plan for 2024 that includes expanding our reach to other communities with an eye on reaching consensus on a 1.0 release of the Open Source AI Definition in the quickest amount of time.
The post <span class='p-name'>Closing the 2023 rounds of Deep Dive AI with first draft piece of the Definition of Open Source AI</span> appeared first on Voices of Open Source.
Real Python: Python News: What's New From October 2023
October 2023 saw the release of the new Python 3.12. At the same time, focus has shifted to new developments in the language. The acceptance of the long-discussed PEP 703 means that developers can ramp up their work on a free-threading version of Python.
The steering council does an important job governing Python and its development. Nominations for the next steering council are now open. Currently, groups are being established that’ll support the steering council by specifically focusing on typing and documentation.
Dive in to learn more about the most important Python news from the last month.
Join Now: Click here to join the Real Python Newsletter and you'll never miss another Python tutorial, course update, or post.
Python 3.12 ReleasedThe latest version of Python, Python 3.12, came out on October 2. You can read more about the release in last month’s newsletter.
If you haven’t tried Python 3.12 for yourself yet, then you should give it a quick spin! Especially if you’re working on a library or an application, it’s good to check that it works on the latest version of Python. You can install several versions of Python on your computer to try them side by side.
The biggest challenge with upgrading a project to a brand-new version of Python is that some of your dependencies may not be ready yet. In general, pure Python libraries should continue to work well, while you may experience issues with extension modules that need to be compiled especially for Python 3.12. However, most of the popular libraries are on the ball and are providing compatible wheels already.
Python Enjoys New DevelopmentsEven though Python 3.12 has just been released, the core developers have been working on Python 3.13 for several months already. You can expect the next version of Python to be released in October 2024.
The first alpha version of Python 3.13 is now available. As it’s still early in development, you won’t find many new features yet.
Instead, the main differences between Python 3.12 and 3.13 so far are deprecations and removals of old functionality. In particular, the dead batteries identified in Python 3.11 have now been removed, and many private functions in Python’s C API have been removed.
There are several places where you can follow the discussions and work that goes into developing and maintaining Python, including GitHub, discussion forums, and PEP documents. Last month, core developers Pablo Galindo Salgado and Łukasz Langa unveiled a new platform for Python news: a podcast named core.py.
In the first episode, Pablo and Łukasz discuss some of the features that are in development for Python 3.13. These include an improved editing experience in the REPL and the Faster CPython project’s just-in-time (JIT) compiler prototype.
In the second episode, they cover PEP 703 and the road toward a version of Python without the GIL. The GIL—or global interpreter lock—is a mutex that ensures that only one thread accesses the Python interpreter at a time. A GIL has several advantages in single-threaded programs. However, it also makes parallel processing harder.
If you’re interested in everything that happens under the hood in Python, then you won’t find better guides than Pablo and Łukasz. They’ve both been instrumental in several of the recent new features of the language. Additionally, Pablo was the release manager for Python 3.10 and 3.11, while Łukasz had the same role for Python 3.8 and 3.9.
Steering Council Accepting Nominations Read the full article at https://realpython.com/python-news-october-2023/ »[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
Mike Driscoll: PyDev of the Week: Gláucia Esppenchutz
This week we welcome Gláucia Esppenchutz (@glauesppen) as our PyDev of the Week! Gláucia is the author of Data Ingestion with Python Cookbook.
Let’s spend some time getting to know Gláucia better!
Can you tell us a little about yourself (hobbies, education, etc):Hi, my name is Glaucia; 31 years old, Brazilian, and living in Portugal.
Married and “mother” of dog beautiful dogs! Last year, I bought a 3D printer and got utterly addicted to it. So, my hobbies include printing random stuff, playing video games, and reading.
I have worked as a Data Engineer for the past eight years and love what I do. I enjoy reading about data, how to optimize ingestion and transformation pipelines, and how to better monitor them.
I’ve been recently allocated to a team focused on Data Operations, which thrills me! Monitoring data and ensuring data quality is challenging.
A fun fact about me is that I have yet to graduate in Science Computing or any engineering grad school. Actually, I graduated in the biomedical field. I changed my career when I met my husband, who is a software engineer.
I am a late diagnosed autistic, and the diagnosis saved my life.
Why did you start using Python?Python is my mother language! I started using it when I shifted my career path. The language’s simplicity helped me learn it quickly and start working in a small startup.
What other programming languages do you know, and which is your favorite?I learned how to program in JavaScript and PHP, but it was so long ago that I had no idea how to do it anymore, haha.
I had to learn Scala because of a project in a previous work. It’s not my favorite language, but it helps me a lot when I need to debug something in Spark.
Python will always be the language of my heart <3
What projects are you working on now?Currently, I am working on two personal projects. One is called Apache Wayang, and it is in the incubator phase at Apache Org. I work with them as a release manager, improving the docs and website.
The other project I am working on is the DE&I initiative in the Apache Org. The idea is to increase the diversity in the open-source community and remove biases we find in the tech area.
Both are long-term projects but very exciting!
Which Python libraries are your favorite (core or 3rd party)?Hum… that’s a tricky question.. Based on what I work, I will say Pandas. I can’t make a count of how many times this lib saved me when analyzing data. Even when using PySpark, I sometimes invoke the inner compatibility with Pandas (.toPandas()) to analyze something.
On the core side, datetime lib is on my top list. Who didn’t have any problems with date formats when working with data? This core lib always saves me.
How did your book, Data Ingestion with Python Cookbook, come about?I got an invitation from Packt publisher. They wanted to make a book about Python and Data in a cookbook format. Then, I proposed something for beginners to start in the data world, but with some intermediate topics for the ones who already work with data pipelines.
The book covers the beginning of the data journey, like understanding the data we will work on and how to plan and monitor the pipelines.
What are the top three things you learned writing a book?The first thing I learned was how to structure and plan a chapter. It seems simple, but creating a content flow and connecting the topics can take a lot of work. Now, I feel more confident to create writing content for my Medium blog, which I started to write posts after the book was released.
Second, my English improved a lot! I had to search for synonyms and different ways to write some things constantly, which made me read a lot of new things.
Third was how to do proper research. All the explanations in the book were made using pieces of code or documentation present in the source codes. Of course, there are citations of other writers and blog posts. Still, I double-checked all the information I needed to make correct assumptions and content.
Is there anything else you’d like to say?Thank you for the invitation! I am pleased to be part of this! And, of course, you can follow me on LinkedIn or Twitter by the username @glauesppen.
Thanks for doing the interview, Gláucia!
The post PyDev of the Week: Gláucia Esppenchutz appeared first on Mouse Vs Python.
Jacob Rockowitz: Love thy CMS!
Adding Love thy CMS to my DrupalCon session proposal
I've been continually defining and redefining the goals of the Schema.org Blueprints module for Drupal. Initially, I started with the simple goal of taking a Schema.org-first approach to building standardized content models in Drupal with JSON:API and JSON-LD to provide great APIs and fantastic SEO. The reality is that for myself and my organization, I am also working on building a best-in-class content management and authoring experience. Ultimately, this secondary goal has led me to reimagine how to build and maintain an instance of Drupal.
In short, there is so much going on with the Schema.org Blueprints modules that my presentation proposal for DrupalCon Portland is a live demo showing how to go from 0-60 with the Schema.org Blueprints module. I am confident that attendees will learn something valuable from my demo of leveraging Schema.org to build an ideal content authoring experience in Drupal. Still, I needed something direct and straightforward to define the overarching goal of the module and presentation, and I decided to prefix my session's title with "Love thy CMS!"
Learning to love your CMS
"Loving your CMS" has been floating around for a few years. For example, Greg Dunlap's presentation at DrupalCon Pittsburg and related blog post about Designing Content Authoring Experiences opens with the statement that "Most people don't love their content management system." A List Apart moderated a conversation between Eileen Webb, Karen McGrane, Jeff Eaton, and Ryan Irelan titled "Love Your CMS."
Drupal - the software and the community - makes it possible for people to...Read More
PyCharm: PyCharm 2023.2.4 Is Out!
We’ve just released another update for v2023.2.
You can update to this version from inside the IDE, using the Toolbox App, or by using snaps if you are a Ubuntu user. You can also download it from our website.
This version brings the following refinements:
- Absolute directory paths in the Commit tool window no longer take up too much space as a result of redundantly replicating parent paths. [IDEA-326271]
- We’ve addressed the issue that made it impossible to copy text from the local machine’s editor and paste it into the editor on Windows Remote Desktop. [IDEA-313776]. If you’re using older versions of Windows Server, you can go to File | Settings | Advanced Settings and select the Paste at the caret position option instead of When pasting a line copied with no selection. This workaround will fix the clipboard sync issue. Alternatively, you can use the latest Windows Remote Desktop client as a solution.
To see the full list of issues addressed in this version, please refer to the release notes.
If you encounter any bugs, please report them using our issue tracker.
LN Webworks: Migration from Drupal 7 to 10 : Answering Your Top 8 Questions
Migrating from Drupal 7 to Drupal 10 is a transformative journey for your website. Even for seasoned Drupal experts, the process can present challenges. Drupal migration encompasses a wide range of considerations, and making the right decisions is crucial to your business's digital success.
In this blog, we will shed light on some of the “what” and “how” of Drupal 7 to 10 Migration doubts and provide solutions from our experts. So, let’s get down to the most common Drupal FAQs.
ListenData: NumPy argmax() Function : Learn with Examples
In this tutorial, we will see how to use the NumPy argmax() function in Python along with examples.
The numpy.argmax() function in Python is used to find the indices of the maximum element in an array.
Syntax of NumPy argmax() FunctionBelow is the syntax of the NumPy argmax() function:
import numpy as np np.argmax(array, axis, out) To read this article in full, please click hereThis post appeared first on ListenDataRead the Docs: Read the Docs newsletter - November 2023
Work continues on hardening Addons, our new in-documentation JavaScript client that supports all documentation tools. We’re looking for people in the community to test out this new functionality, and will be expanding access in the near future.
Python 3.12 is now supported on builds, and is the default version used when you specify build.tools.python: 3 in your configuration file.
Language codes are now normalized in URLs: https://blog.readthedocs.com/language-codes-are-now-normalized/.
Our search infrastructure was upgraded to Elastic Search 8, which should provide faster and better search results.
We submitted a fix to the API Key library we use to make API calls to our servers much faster, which should result in faster build times for all users.
We are working to expand the functionality of our redirects feature to support more use cases. More will be announced here in the coming month.
We are planning an upgrade to our dashboard notification system, so that users have more control and better context for on-site notifications.
Our beta dashboard continues to be tested in public beta, and new functionality for Addons configuration will only be available in that new interface.
Want to follow along with our development progress? View our full roadmap 📍️
Possible issuesWe don’t have any possible issues to report this month.
Questions? Comments? Ideas for the next newsletter? Contact us!