Feeds

AI Chatbot demo with Drupal and RAG Peter Pónya Thu, 11/21/2024 - 12:30 AI Chatbot demo with Drupal and RAG

At DrupalCon Barcelona, we were amazed to see all the powerful and functional Drupal AI integrations in action. I have been following the development closely, and after the recent release announcement of the AI (Artificial Intelligence) module, we decided to recreate one of its use cases: a RAG chatbot enabling semantic search. 

Learn how the Event Platform module simplified building the DrupalCamp Berlin 2024 website, its key features, benefits, and potential for improvements.

In this quiz, you’ll test your understanding of Expression vs Statement in Python: What’s the Difference?

By working through this quiz, you’ll revisit the key differences between expressions and statements in Python, and how to use them effectively in your code.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

In this quiz, you’ll test your understanding of the different ways you can interact with Python.

By working through this quiz, you’ll revisit key concepts related to Python interaction in interactive mode using the Read-Eval-Print Loop (REPL), through Python script files, and within Integrated Development Environments (IDEs) and code editors.

You’ll also test your knowledge of some other options that may be useful, such as Jupyter Notebooks.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Indus Action seeks volunteers with Drupal expertise to enhance RTE-MIS, its open-source platform supporting the implementation of the Right to Education Act. Volunteer 4-8 hours weekly for this social impact project aimed at improving educational access across India.

Today, we’re announcing early elections for the Django Software Foundation Steering Council over the 6.x Django release cycle. Elected members will be on the Steering Council for two years, from the end of those elections in December, until April 2027 with the scheduled start of the Django 7.x release cycle.

Why we have early elections

The DSF Board of Directors previously shared Django’s technical governance challenges, and opportunities. Now that the Board elections are completed, we’re ready to proceed with this other, separate election, following existing processes. We will want a Steering Council who strives  to meet the group’s intended goals:

1. To safeguard big decisions that affect Django projects at a fundamental level.

2. To help shepherd the project’s future direction.

We expect the new Steering Council will take on those known challenges, resolve those questions of technical leadership, and update Django’s technical governance. They will have the full support of the Board of Directors to address this threat to Django’s future. And the Board will also be more decisive in intervening, should similar issues keep arising.

Elections timeline

Here are the important dates of the Steering Council elections, subject to change:

• 2024-11-21: announcement & opening of voter registration
• 2024-11-26 23:59 AoE (Anywhere on Earth): voter registration closes
• 2024-11-27: opening of Steering Council candidates registration
• 2024-12-04 23:59 AoE: candidates registration closes
• (one week gap per defined processes)
• 2024-12-10: voting starts
• 2024–12-17 23:59 AoE: voting ends
• 2024-12-18: results ratification by DSF Board of Directors
• 2024-12-19: results announcement

Voter registration

If you’re an Individual Member of the Django Software Foundation, you’re already registered to vote. There’s nothing further for you to do. If you aren’t, consider nominating yourself for individual membership. Once approved, you will be registered to vote for this election.

Alternatively, for members of our community who want to vote in this election but don’t want to become Individual Members, you can register to vote from now until 2024-11-26 23:59 Anywhere on Earth, use our form: Django 6.x Steering Council Voter Registration.

Candidate registration

If you’re interested, don’t wait until formal candidate registration. You can already fill in our 6.x Steering Council expression of interest form. At the end of the form, select “I would like what my submissions to this form to be used as part of my candidate registration for the elections”.

Django 6.x Steering Council elections - Expression of interest

Voting

Once voting opens, those eligible to vote in this election will receive information on how to vote via email. Please check for an email with the subject line “6.x Steering Council elections voting”. Voting will be open until 23:59 on December 17, 2024 Anywhere on Earth.

—

Any questions? Ask on our dedicated forum discussion thread, or reach out via email to foundation\@djangoproject.com.

A preview of our chat with Anna Makarudze. Watch the full episode on November 20, 2024A preview of our chat with Anna Makarudze. Watch the full episode on November 20, 2024

Sneak Peek of our chat with Anna Makarudze, hosted by Mariatta Wijaya and Cheuk Ting Ho.

Since discovering Python and Django in 2015, Anna has been actively involved in the Django community. She helped organize PyCon Zimbabwe, and she has coached at Django Girls in Harare and Windhoek.

She served on the Board of Directors at Django Software Foundation for five years, and she is currently a Django Girls Foundation Trustee & Fundraising Coordinator.

Anna became aware of the lack of representation of women in tech industry, something that became more evident as she attended Django Under the Hood in 2016 where most of the attendees were white men, and only a few are women. That’s when she realized the importance of communities like Django Girls in supporting more women in the Django Community.

In this chat, Anna shared ways on how you can contribute and help support Django Girls+ Foundation.

Full episode is coming on November 27, 2024! Subscribe to our podcast now!

Hi, All:

Please join me in welcoming our new member:

User Details:
-------------
Name:    Haoran Du
Login:   bingchuanjuzi
Email:   dududu233@outlook.com

I wish bingchuanjuzi a wonderful journey in GNU CTT.

Happy Hacking
wxie

Well, it's official. My Drupal Lenient Composer Plugin has allowed the lenient Composer repository endpoint on Drupal.org to be sunset and removed. I created the mglaman/composer-drupal-lenient repository two years ago at DrupalCon Portland. It is pretty wild how much it has been adopted in just two years. Not only has it allowed the Drupal Association to dismantle some infrastructure, but it is also baked into the Drupal.org GitLab CI. The package is pushing over 3,000,000 downloads from Packagist!

Libtoolers!

The Libtool Team is pleased to announce the release of libtool 2.5.4.

GNU Libtool hides the complexity of using shared libraries behind a
consistent, portable interface. GNU Libtool ships with GNU libltdl, which
hides the complexity of loading dynamic runtime libraries (modules)
behind a consistent, portable interface.

There have been 49 commits by 16 people in the 8 weeks since 2.5.3.

See the NEWS below for a brief summary.

Thanks to everyone who has contributed!
The following people contributed changes to this release:

  Adrien Destugues (1)
  Alastair McKinstry (6)
  Bruno Haible (1)
  Ileana Dumitrescu (27)
  Jerome Duval (1)
  Jonathan Nieder (2)
  Joshua Root (1)
  Khalid Masum (1)
  Markus Mützel (1)
  Martin Storsjö (1)
  Richard Purdie (1)
  Sergey Poznyakoff (1)
  Tim Schumacher (1)
  Vincent Lefevre (2)
  mintsuki (1)
  streaksu (1)

Ileana
 [on behalf of the libtool maintainers]
==================================================================

Here is the GNU libtool home page:
    https://gnu.org/s/libtool/

For a summary of changes and contributors, see:
  https://git.sv.gnu.org/gitweb/?p=libtool.git;a=shortlog;h=v2.5.4
or run this command from a git-cloned libtool directory:
  git shortlog v2.5.3..v2.5.4

Here are the compressed sources:
  https://ftpmirror.gnu.org/libtool/libtool-2.5.4.tar.gz   (2.0MB)
  https://ftpmirror.gnu.org/libtool/libtool-2.5.4.tar.xz   (1.1MB)

Here are the GPG detached signatures:
  https://ftpmirror.gnu.org/libtool/libtool-2.5.4.tar.gz.sig
  https://ftpmirror.gnu.org/libtool/libtool-2.5.4.tar.xz.sig

Use a mirror for higher download bandwidth:
  https://www.gnu.org/order/ftp.html

Here are the SHA1 and SHA256 checksums:

  77227188ead223ed8ba447301eda3761cb68ef57  libtool-2.5.4.tar.gz
  2o67LOTc9GuQCY2vliz/po9LT2LqYPeY0O8Skp7eat8=  libtool-2.5.4.tar.gz
  9781a113fe6af1b150571410b29d3eee2e792516  libtool-2.5.4.tar.xz
  +B9YYGZrC8fYS63e+mDRy5+m/OsjmMw7rKavqmAmZnU=  libtool-2.5.4.tar.xz

Verify the base64 SHA256 checksum with cksum -a sha256 --check
from coreutils-9.2 or OpenBSD's cksum since 2007.

Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify libtool-2.5.4.tar.gz.sig

The signature should match the fingerprint of the following key:

  pub   rsa4096 2021-09-23 [SC]
        FA26 CA78 4BE1 8892 7F22  B99F 6570 EA01 146F 7354
  uid   Ileana Dumitrescu <ileanadumi95@protonmail.com>
  uid   Ileana Dumitrescu <ileanadumitrescu95@gmail.com>

If that command fails because you don't have the required public key,
or that public key has expired, try the following commands to retrieve
or refresh it, and then rerun the 'gpg --verify' command.

  gpg --locate-external-key ileanadumi95@protonmail.com

  gpg --recv-keys 6570EA01146F7354

  wget -q -O- 'https://savannah.gnu.org/project/release-gpgkeys.php?group=libtool&download=1' | gpg --import -

As a last resort to find the key, you can try the official GNU
keyring:

  wget -q https://ftp.gnu.org/gnu/gnu-keyring.gpg
  gpg --keyring gnu-keyring.gpg --verify libtool-2.5.4.tar.gz.sig

This release was bootstrapped with the following tools:
  Autoconf 2.72e
  Automake 1.17
  Gnulib v1.0-1108-gea58a72d4d

NEWS

• Noteworthy changes in release 2.5.4 (2024-11-20) [stable]

** New features:

  - New libtool command line flag, --no-finish, to skip executing
    finish_cmds that would alter the shared library cache during testing.

  - New libtool command line flag, --reorder-cache=DIRS, to reorder the
    shared library cache, only on OpenBSD.

** Bug fixes:

  - Fix incorrect use of workarounds designed for Darwin versions that
    don't have -single_module support.

  - Fix errors when executing 'make distclean' and 'make maintainer-clean'.

  - Fix bug where the constructed rpath omit directories, instead of
    appending them to the end.

  - Fix configure error for when variable 'multlib' is unset.

  - Fix searching for -L in link paths being over-greedy and incorrectly
    handling paths with -L in them.

  - Avoid using AC_TRY_EVAL macro, "dangerous and undocumented".

  - Fix linking libraries at runtime with tcc by adding run path.

  - Fix path comparison by removing trailing slashes on install commands.

  - Fix linking for mingw with lld by prefering response files over the
    linker script.

  - Fix '-Fe' usage with linking in MSVC.

  - Fix '--no-warnings' flag.

  - Fix handling xlc(1)-specific options.

  - Fix Haiku support.

** Changes in supported systems or compilers:

  - Support additional flang-based compilers, 'f18' and 'f95'.

  - Support for 'netbsdelf*-gnu'.

  - Support for '*-mlibc', and subsequently Ironclad and Managarm.

  - Support for SerenityOS.

  - Support for wasm32-emscripten.

Enjoy!

As we announced back in September, the OSI has partnered again with OpenLogic by Perforce to produce a comprehensive report on global, industry-wide Open Source software adoption trends. The 2025 State of Open Source Report will be based on responses to a survey of those working with Open Source software in their organizations, from developers to CTOs and everyone in between. 

“This is our fourth year being involved in the State of Open Source Report, and there is never any shortage of surprises in the data,” says Stefano Maffulli, Executive Director, Open Source Initiative. “Now, however, the aim of the survey is not to determine whether or not organizations are using Open Source — we know they are — but to find out how they are handling complexities related to AI, licensing, and of course, security.”

This year, the survey includes new sections on Big Data, the impact of CentOS EOL, and security/compliance. As always, there are questions about technology usage in various categories such as infrastructure, cloud-native, frameworks, CI/CD, automation, and programming languages. Finally, a few questions toward the end look at Open Source maturity and stewardship, including sponsoring or being involved with open source foundations and organizations like OSI. 

Of course, any report like this is only as valuable as its data and the more robust and high-quality the dataset, the stronger the report will be. As stewards of the Open Source community, OSI members are encouraged to take the survey so that the 2025 State of Open Source Report accurately reflects the interests, concerns, and preferences of Open Source software users around the world. 

You can access the State of Open Source Survey here: https://www.research.net/r/SLQWZGF

Ready for some Python skill-building sales?

This is my seventh annual compilation of Python learning deals.

I’m publishing this post extra early this year, so bookmark this page and set a calendar event for yourself to check back on Friday November 29.

Currently live sales

Here are Python-related sales that are live right now:

• Python Jumpstart with Python Morsels: 50% off my brand new Python course, an introduction to Python that’s very hands-on ($99 instead of $199)
• Rodrigo 50% off Rodrigo’s all books bundle with code BF24
• The Python Coding Place: 40% off The Python Coding Book and 40% off a lifetime membership to The Python Coding Place with code black2024
• Sundeep Agarwal: ~50% off Sundeep’s all book and Python bundles with code FestiveOffer
• O'Reilly Media: 40% off the first year with code CYBERWEEK24 ($299 instead of $499)

Anticipated sales

Here are sales that will be live soon:

• Data School 40% off all Kevin’s courses or get a bundle with all 5 of his courses
• Mike Driscoll: 35% off Mike’s Python books and courses with code BF24

Here are some sales I expect to see, but which haven’t been announced yet:

• Talk Python: usually holds a sale on a variety of courses
• Brian Okken: often holds a sale on his pytest course
• Reuven Lerner: usually holds a sale
• Pragmatic Bookshelf: I’m guessing they’ll hold a 40% off sale with code turkeycode2024

Even more sales

Also see Adam Johnson’s Django-related Deals for Black Friday 2024 for sales on Adam’s books, courses from the folks at Test Driven, Django templates, and various other Django-related deals.

And for non-Python/Django Python deals, see the Awesome Black Friday / Cyber Monday deals GitHub repository and the BlackFridayDeals.dev website.

If you know of another sale (or a likely sale) please comment below or email me.

Project: Drupal coreDate: 2024-November-20Security risk: Moderately critical 14 ∕ 25 AC:Complex/A:Admin/CI:All/II:All/E:Theoretical/TD:UncommonVulnerability: Gadget chainAffected versions: >= 8.0.0 < 10.2.11 || >= 10.3.0 < 10.3.9Description: 

Drupal core contains a potential PHP Object Injection vulnerability that (if combined with another exploit) could lead to Remote Code Execution. It is not directly exploitable.

This issue is mitigated by the fact that in order for it to be exploitable, a separate vulnerability must be present to allow an attacker to pass unsafe input to unserialize(). There are no such known exploits in Drupal core.

To help protect against this potential vulnerability, some additional checks have been added to Drupal core's database code. If you use a third-party database driver, check the release notes for additional configuration steps that may be required in certain cases.

Solution: 

Install the latest version:

• If you are using Drupal 7, update to Drupal 7.102.
• If you are using Drupal 10.2, update to Drupal 10.2.11.
• If you are using Drupal 10.3, update to Drupal 10.3.9.

All versions of Drupal 10 prior to 10.2 are end-of-life and do not receive security coverage. (Drupal 8 and Drupal 9 have both reached end-of-life.)

Reported By: 

• Drew Webber of the Drupal Security Team

Fixed By: 

• Drew Webber of the Drupal Security Team
• Fabian Franz
• Juraj Nemec of the Drupal Security Team
• Lee Rowlands of the Drupal Security Team
• Dave Long of the Drupal Security Team
• Alex Pott of the Drupal Security Team

Coordinated By: 

• Juraj Nemec of the Drupal Security Team
• Benji Fisher of the Drupal Security Team
• xjm of the Drupal Security Team

Project: Drupal coreDate: 2024-November-20Security risk: Moderately critical 14 ∕ 25 AC:Complex/A:Admin/CI:All/II:All/E:Theoretical/TD:UncommonVulnerability: Gadget chainAffected versions: >= 8.0.0 < 10.2.11 || >= 10.3.0 < 10.3.9 || >= 11.0.0 < 11.0.8Description: 

Drupal core contains a potential PHP Object Injection vulnerability that (if combined with another exploit) could lead to Remote Code Execution. It is not directly exploitable.

This issue is mitigated by the fact that in order for it to be exploitable, a separate vulnerability must be present to allow an attacker to pass unsafe input to unserialize(). There are no such known exploits in Drupal core.

To help protect against this potential vulnerability, types have been added to properties in some of Drupal core's classes. If an application extends those classes, the same types may need to be specified on the subclass to avoid a TypeError.

Solution: 

Install the latest version:

• If you are using Drupal 10.2, update to Drupal 10.2.11.
• If you are using Drupal 10.3, update to Drupal 10.3.9.
• If you are using Drupal 11.0, update to Drupal 11.0.8.
• Drupal 7 is not affected.

All versions of Drupal 10 prior to 10.2 are end-of-life and do not receive security coverage. (Drupal 8 and Drupal 9 have both reached end-of-life.)

Reported By: 

• Drew Webber of the Drupal Security Team

Fixed By: 

• Drew Webber of the Drupal Security Team
• Lee Rowlands of the Drupal Security Team

Coordinated By: 

• Juraj Nemec of the Drupal Security Team
• Greg Knaddison of the Drupal Security Team
• Benji Fisher of the Drupal Security Team
• xjm of the Drupal Security Team

Project: Drupal coreDate: 2024-November-20Security risk: Less critical 8 ∕ 25 AC:Complex/A:User/CI:None/II:Some/E:Theoretical/TD:UncommonVulnerability: Gadget chainAffected versions: >= 8.0.0 < 10.2.11 || >= 10.3.0 < 10.3.9 || >= 11.0.0 < 11.0.8Description: 

Drupal core contains a potential PHP Object Injection vulnerability that (if combined with another exploit) could lead to Artbitrary File Deletion. It is not directly exploitable.

This issue is mitigated by the fact that in order to be exploitable, a separate vulnerability must be present that allows an attacker to pass unsafe input to unserialize(). There are no such known exploits in Drupal core.

To help protect against this vulnerability, types have been added to properties in some of Drupal core's classes. If an application extends those classes, the same types may need to be specified on the subclass to avoid a TypeError.

Solution: 

Install the latest version:

• If you are using Drupal 10.2, update to Drupal 10.2.11.
• If you are using Drupal 10.3, update to Drupal 10.3.9.
• If you are using Drupal 11.0, update to Drupal 11.0.8.
• Drupal 7 is not affected.

All versions of Drupal 10 prior to 10.2 are end-of-life and do not receive security coverage. (Drupal 8 and Drupal 9 have both reached end-of-life.)

Reported By: 

• Drew Webber of the Drupal Security Team

Fixed By: 

• Drew Webber of the Drupal Security Team
• Lee Rowlands of the Drupal Security Team

Coordinated By: 

• Juraj Nemec of the Drupal Security Team
• Benji Fisher of the Drupal Security Team
• xjm of the Drupal Security Team

Project: Drupal coreDate: 2024-November-20Security risk: Critical 17 ∕ 25 AC:None/A:None/CI:Some/II:Some/E:Theoretical/TD:DefaultVulnerability: Cross Site ScriptingDescription: 

Drupal 7 core's Overlay module doesn't safely handle user input, leading to reflected cross-site scripting under certain circumstances.

Only sites with the Overlay module enabled are affected by this vulnerability.

Solution: 

Install the latest version:

• If you are using Drupal 7, update to Drupal 7.102
• Sites may also disable the Overlay module to avoid the issue.

Drupal 10 and Drupal 11 are not affected, as the Overlay module was removed from Drupal core in Drupal 8.

Reported By: 

• Cesar

Fixed By: 

• Cesar
• Greg Knaddison of the Drupal Security Team
• Matthew Grill
• Wim Leers
• Drew Webber of the Drupal Security Team
• Ra Mänd
• Fabian Franz
• Juraj Nemec of the Drupal Security Team

Coordinated By: 

• Juraj Nemec of the Drupal Security Team
• Greg Knaddison of the Drupal Security Team
• xjm of the Drupal Security Team

Project: Drupal coreDate: 2024-November-20Security risk: Moderately critical 10 ∕ 25 AC:Basic/A:User/CI:None/II:Some/E:Theoretical/TD:DefaultVulnerability: Access bypassAffected versions: >= 8.0.0 < 10.2.11 || >= 10.3.0 < 10.3.9 || >= 11.0.0 < 11.0.8Description: 

Drupal's uniqueness checking for certain user fields is inconsistent depending on the database engine and its collation.

As a result, a user may be able to register with the same email address as another user.

This may lead to data integrity issues.

Solution: 

Install the latest version:

• If you are using Drupal 10.2, update to Drupal 10.2.11.
• If you are using Drupal 10.3, update to Drupal 10.3.9.
• If you are using Drupal 11.0, update to Drupal 11.0.8.
• Drupal 7 is not affected.

All versions of Drupal 10 prior to 10.2 are end-of-life and do not receive security coverage. (Drupal 8 and Drupal 9 have both reached end-of-life.)

Updating Drupal will not solve potential issues with existing accounts affected by this bug. See Fixing emails that vary only by case for additional guidance.

Reported By: 

• Wayne Eaker

Fixed By: 

• Wayne Eaker
• cilefen of the Drupal Security Team
• Kristiaan Van den Eynde
• Drew Webber of the Drupal Security Team
• Lee Rowlands of the Drupal Security Team

Coordinated By: 

• Juraj Nemec of the Drupal Security Team
• Benji Fisher of the Drupal Security Team
• xjm of the Drupal Security Team

Project: Drupal coreDate: 2024-November-20Security risk: Moderately critical 13 ∕ 25 AC:Basic/A:User/CI:Some/II:Some/E:Theoretical/TD:DefaultVulnerability: Cross Site ScriptingAffected versions: >= 8.8.0 < 10.2.11 || >= 10.3.0 < 10.3.9 || >= 11.0.0 < 11.0.8Description: 

Drupal uses JavaScript to render status messages in some cases and configurations. In certain situations, the status messages are not adequately sanitized.

Solution: 

Install the latest version:

• If you are using Drupal 10.2, update to Drupal 10.2.11.
• If you are using Drupal 10.3, update to Drupal 10.3.9.
• If you are using Drupal 11.0, update to Drupal 11.0.8.

All versions of Drupal 10 prior to 10.2 are end-of-life and do not receive security coverage. (Drupal 8 and Drupal 9 have both reached end-of-life.)

Reported By: 

• Jay Beaton

Fixed By: 

• Lee Rowlands of the Drupal Security Team
• catch of the Drupal Security Team
• Mingsong
• Juraj Nemec of the Drupal Security Team
• Dave Long of the Drupal Security Team
• Benji Fisher of the Drupal Security Team

Coordinated By: 

• Juraj Nemec of the Drupal Security Team
• Greg Knaddison of the Drupal Security Team

The NumPy library is a Python library used for scientific computing. It provides you with a multidimensional array object for storing and analyzing data in a wide variety of ways. In this tutorial, you’ll see examples of some features NumPy provides that aren’t always highlighted in other tutorials. You’ll also get the chance to practice your new skills with various exercises.

In this tutorial, you’ll learn how to:

• Create multidimensional arrays from data stored in files
• Identify and remove duplicate data from a NumPy array
• Use structured NumPy arrays to reconcile the differences between datasets
• Analyze and chart specific parts of hierarchical data
• Create vectorized versions of your own functions

If you’re new to NumPy, it’s a good idea to familiarize yourself with the basics of data science in Python before you start. Also, you’ll be using Matplotlib in this tutorial to create charts. While it’s not essential, getting acquainted with Matplotlib beforehand might be beneficial.

Get Your Code: Click here to download the free sample code that you’ll use to work through NumPy practical examples.

Take the Quiz: Test your knowledge with our interactive “NumPy Practical Examples: Useful Techniques” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

NumPy Practical Examples: Useful Techniques

This quiz will challenge your knowledge of working with NumPy arrays. You won't find all the answers in the tutorial, so you'll need to do some extra investigating. By finding all the answers, you're sure to learn some interesting things along the way.

Setting Up Your Working Environment

Before you can get started with this tutorial, you’ll need to do some initial setup. In addition to NumPy, you’ll need to install the Matplotlib library, which you’ll use to chart your data. You’ll also be using Python’s pathlib library to access your computer’s file system, but there’s no need to install pathlib because it’s part of Python’s standard library.

You might consider using a virtual environment to make sure your tutorial’s setup doesn’t interfere with anything in your existing Python environment.

Using a Jupyter Notebook within JupyterLab to run your code instead of a Python REPL is another useful option. It allows you to experiment and document your findings, as well as quickly view and edit files. The downloadable version of the code and exercise solutions are presented in Jupyter Notebook format.

The commands for setting things up on the common platforms are shown below:

• Windows
• Linux + macOS

Fire up a Windows PowerShell(Admin) or Terminal(Admin) prompt, depending on the version of Windows that you’re using. Now type in the following commands:

Windows PowerShell PS> python -m venv venv\ PS> venv\Scripts\activate (venv) PS> python -m pip install numpy matplotlib jupyterlab (venv) PS> jupyter lab Copied!

Here you create a virtual environment named venv\, which you then activate. If the activation is successful, then the virtual environment’s name will precede your Powershell prompt. Next, you install numpy and matplotlib into this virtual environment, followed by the optional jupyterlab. Finally, you start JupyterLab.

Note: When you activate your virtual environment, you may receive an error stating that your system can’t run the script. Modern versions of Windows don’t allow you to run scripts downloaded from the Internet as a security feature.

To fix this, you need to type the command Set-ExecutionPolicy RemoteSigned, then answer Y to the question. Your computer will now run scripts that Microsoft has verified. Once you’ve done this, the venv\Scripts\activate command should work.

Fire up a terminal and type in the following commands:

Shell $ python -m venv venv/ $ source venv/bin/activate (venv) $ python -m pip install numpy matplotlib jupyterlab (venv) $ jupyter lab Copied!

Here you create a virtual environment named venv/, which you then activate. If the activation is successful, then the virtual environment’s name will precede your command prompt. Next, you install numpy and matplotlib into this virtual environment, followed by the optional jupyterlab. Finally, you start JupyterLab.

You’ll notice that your prompt is preceded by (venv). This means that anything you do from this point forward will stay in this environment and remain separate from other Python work you have elsewhere.

Now that you have everything set up, it’s time to begin the main part of your learning journey.

NumPy Example 1: Creating Multidimensional Arrays From Files

When you create a NumPy array, you create a highly-optimized data structure. One of the reasons for this is that a NumPy array stores all of its elements in a contiguous area of memory. This memory management technique means that the data is stored in the same memory region, making access times fast. This is, of course, highly desirable, but an issue occurs when you need to expand your array.

Suppose you need to import multiple files into a multidimensional array. You could read them into separate arrays and then combine them using np.concatenate(). However, this would create a copy of your original array before expanding the copy with the additional data. The copying is necessary to ensure the updated array will still exist contiguously in memory since the original array may have had non-related content adjacent to it.

Constantly copying arrays each time you add new data from a file can make processing slow and is wasteful of your system’s memory. The problem becomes worse the more data you add to your array. Although this copying process is built into NumPy, you can minimize its effects with these two steps:

1. When setting up your initial array, determine how large it needs to be before populating it. You may even consider over-estimating its size to support any future data additions. Once you know these sizes, you can create your array upfront.

2. The second step is to populate it with the source data. This data will be slotted into your existing array without any need for it to be expanded.

Next, you’ll explore how to populate a three-dimensional NumPy array.

Populating Arrays With File Data

In this first example, you’ll use the data from three files to populate a three-dimensional array. The content of each file is shown below, and you’ll also find these files in the downloadable materials:

The first file has two rows and three columns with the following content:

CSV file1.csv 1.1, 1.2, 1.3 1.4, 1.5, 1.6 Copied! Read the full article at https://realpython.com/numpy-example/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

tl;dr: The Rust Foundation’s new trademark policy still forbids unapproved modifications: this would forbid both the Rust Community’s own development work(!) and normal Free Software distribution practices.

Background

In April 2023 I wrote about the Rust Foundation’s ham-fisted and misguided attempts to update the Rust trademark policy. This turned into drama.

The new draft

Recently, the Foundation published a new draft. It’s considerably less bad, but the most serious problem, which I identified last year, remains.

It prevents redistribution of modified versions of Rust, without pre-approval from the Rust Foundation. (Subject to some limited exceptions.) The people who wrote this evidently haven’t realised that distributing modified versions is how free software development works. Ie, the draft Rust trademark policy even forbids making a github branch for an MR to contribute to Rust!

It’s also very likely unacceptable to Debian. Rust is still on track to repeat the Firefox/Iceweasel debacle.

Below is a copy of my formal response to the consultation. The consultation closes at 07:59:00 UTC tomorrow (21st November), ie, at the end of today (Wednesday) US Pacific time, so if you want to reply, do so quickly.

My consultation response

Hi. My name is Ian Jackson. I write as a Rust contributor and as a Debian Developer with first-hand experience of Debian’s approach to trademarks. (But I am not a member of the Debian Rust Packaging Team.)

Your form invites me to state any blocking concerns. I’m afraid I have one:

PROBLEM

The policy on distributing modified versions of Rust (page 4, 8th bullet) is far too restrictive.

PROBLEM - ASPECT 1

On its face the policy forbids making a clone of the Rust repositories on a git forge, and pushing a modified branch there. That is publicly distributing a modified version of Rust.

I.e., the current policy forbids the Rust’s community’s own development workflow!

PROBLEM - ASPECT 2

The policy also does not meet the needs of Software-Freedom-respecting downstreams, including community Linux distributions such as Debian.

There are two scenarios (fuzzy, and overlapping) which provide a convenient framing to discuss this:

Firstly, in practical terms, Debian may need to backport bugfixes, or sometimes other changes. Sometimes Debian will want to pre-apply bugfixes or changes that have been contributed by users, and are intended eventually to go upstream, but are not included upstream in official Rust yet. This is a routine activity for a distribution. The policy, however, forbids it.

Secondly, Debian, as a point of principle, requires the ability to diverge from upstream if and when Debian decides that this is the right choice for Debian’s users. The freedom to modify is a key principle of Free Software. This includes making changes that the upstream project disapproves of. Some examples of this, where Debian has made changes, that upstream do not approve of, have included things like: removing user-tracking code, or disabling obsolescence “timebombs” that stop a particular version working after a certain date.

Overall, while alignment in values between Debian and Rust seems to be very good right now, modifiability it is a matter of non-negotiable principle for Debian. The 8th bullet point on page 4 of the PDF does not give Debian (and Debian’s users) these freedoms.

POSSIBLE SOLUTIONS

Other formulations, or an additional permission, seem like they would be able to meet the needs of both Debian and Rust.

The first thing to recognise is that forbidding modified versions is probably not necessary to prevent language ecosystem fragmentation. Many other programming languages are distributed under fully Free Software licences without such restrictive trademark policies. (For example, Python; I’m sure a thorough survey would find many others.)

The scenario that would be most worrying for Rust would be “embrace - extend - extinguish”. In projects with a copyleft licence, this is not a concern, but Rust is permissively licenced. However, one way to address this would be to add an additional permission for modification that permits distribution of modified versions without permission, but if the modified source code is also provided, under the original Rust licence.

I suggest therefore adding the following 2nd sub-bullet point to the 8th bullet on page 4:

• changes which are shared, in source code form, with all recipients of the modified software, and publicly licenced under the same licence as the official materials.

This means that downstreams who fear copyleft have the option of taking Rust’s permissive copyright licence at face value, but are limited in the modifications they may make, unless they rename. Conversely downstreams such as Debian who wish to operate as part of the Free Software ecosystem can freely make modifications.

It also, obviously, covers the Rust Community’s own development work.

NON-SOLUTIONS

Some upstreams, faced with this problem, have offered Debian a special permission: ie, said that it would be OK for Debian to make modifications that Debian wants to. But Debian will not accept any Debian-specific permissions.

Debian could of course rename their Rust compiler. Debian has chosen to rename in the past: infamously, a similar policy by Mozilla resulted in Debian distributing Firefox under the name Iceweasel for many years. This is a PR problem for everyone involved, and results in a good deal of technical inconvenience and makework.

“Debian could seek approval for changes, and the Rust Foundation would grant that approval quickly”. This is unworkable on a practical level - requests for permission do not fit into Debian’s workflow, and the resulting delays would be unacceptable. But, more fundamentally, Debian rightly insists that it must have the freedom to make changes that the Foundation do not approve of. (For example, if a future Rust shipped with telemetry features Debian objected to.)

“Debian and Rust could compromise”. However, Debian is an ideological as well as technological project. The principles I have set out are part of Debian’s Foundation Documents - they are core values for Debian. When Debian makes compromises, it does so very slowly and with great deliberation, using its slowest and most heavyweight constitutional governance processes. Debian is not likely to want to engage in such a process for the benefit of one programming language.

“Users will get Rust from upstream”. This is currently often the case. Right now, Rust is moving very quickly, and by Debian standards is very new. As Rust becomes more widely used, more stable, and more part of the infrastructure of the software world, it will need to become part of standard, stable, reliable, software distributions. That means Debian.

(The consultation was a Google Forms page with a single text field, so the formatting isn’t great. I have edited the formatting very lightly to avoid rendering bugs here on my blog.)

comments