FLOSS Project Planets

The Drop Times: Drupal Page Builders—Part 1: Paragraph-Based Solutions

Planet Drupal - Mon, 2024-03-18 12:44
Dive into the changing dynamics of Drupal's page-building features with André Angelantoni's new series on The DropTimes. Discover the progression of Drupal's page layout options as the author begins exploring Paragraph-Based Solutions. Gain historical insights, learn about the shift towards Layout Builder, and examine the Paragraphs module and related tools like Layout Paragraphs and Mercury Editor. Anticipate detailed discussions on contributed modules, alternative solutions, and potent distributions in the forthcoming parts of the series. Ideal for developers and content managers seeking to upgrade their Drupal 10+ projects.
Categories: FLOSS Project Planets

Simon Josefsson: Apt archive mirrors in Git-LFS

Planet Debian - Mon, 2024-03-18 12:15

My effort to improve transparency and confidence of public apt archives continues. I started to work on this in “Apt Archive Transparency” in which I mention the debdistget project in passing. Debdistget is responsible for mirroring index files for some public apt archives. I’ve realized that having a publicly auditable and preserved mirror of the apt repositories is central to being able to do apt transparency work, so the debdistget project has become more central to my project than I thought. Currently I track Trisquel, PureOS, Gnuinos and their upstreams Ubuntu, Debian and Devuan.

Debdistget download Release/Package/Sources files and store them in a git repository published on GitLab. Due to size constraints, it uses two repositories: one for the Release/InRelease files (which are small) and one that also include the Package/Sources files (which are large). See for example the repository for Trisquel release files and the Trisquel package/sources files. Repositories for all distributions can be found in debdistutils’ archives GitLab sub-group.

The reason for splitting into two repositories was that the git repository for the combined files become large, and that some of my use-cases only needed the release files. Currently the repositories with packages (which contain a couple of months worth of data now) are 9GB for Ubuntu, 2.5GB for Trisquel/Debian/PureOS, 970MB for Devuan and 450MB for Gnuinos. The repository size is correlated to the size of the archive (for the initial import) plus the frequency and size of updates. Ubuntu’s use of Apt Phased Updates (which triggers a higher churn of Packages file modifications) appears to be the primary reason for its larger size.

Working with large Git repositories is inefficient and the GitLab CI/CD jobs generate quite some network traffic downloading the git repository over and over again. The most heavy user is the debdistdiff project that download all distribution package repositories to do diff operations on the package lists between distributions. The daily job takes around 80 minutes to run, with the majority of time is spent on downloading the archives. Yes I know I could look into runner-side caching but I dislike complexity caused by caching.

Fortunately not all use-cases requires the package files. The debdistcanary project only needs the Release/InRelease files, in order to commit signatures to the Sigstore and Sigsum transparency logs. These jobs still run fairly quickly, but watching the repository size growth worries me. Currently these repositories are at Debian 440MB, PureOS 130MB, Ubuntu/Devuan 90MB, Trisquel 12MB, Gnuinos 2MB. Here I believe the main size correlation is update frequency, and Debian is large because I track the volatile unstable.

So I hit a scalability end with my first approach. A couple of months ago I “solved” this by discarding and resetting these archival repositories. The GitLab CI/CD jobs were fast again and all was well. However this meant discarding precious historic information. A couple of days ago I was reaching the limits of practicality again, and started to explore ways to fix this. I like having data stored in git (it allows easy integration with software integrity tools such as GnuPG and Sigstore, and the git log provides a kind of temporal ordering of data), so it felt like giving up on nice properties to use a traditional database with on-disk approach. So I started to learn about Git-LFS and understanding that it was able to handle multi-GB worth of data that looked promising.

Fairly quickly I scripted up a GitLab CI/CD job that incrementally update the Release/Package/Sources files in a git repository that uses Git-LFS to store all the files. The repository size is now at Ubuntu 650kb, Debian 300kb, Trisquel 50kb, Devuan 250kb, PureOS 172kb and Gnuinos 17kb. As can be expected, jobs are quick to clone the git archives: debdistdiff pipelines went from a run-time of 80 minutes down to 10 minutes which more reasonable correlate with the archive size and CPU run-time.

The LFS storage size for those repositories are at Ubuntu 15GB, Debian 8GB, Trisquel 1.7GB, Devuan 1.1GB, PureOS/Gnuinos 420MB. This is for a couple of days worth of data. It seems native Git is better at compressing/deduplicating data than Git-LFS is: the combined size for Ubuntu is already 15GB for a couple of days data compared to 8GB for a couple of months worth of data with pure Git. This may be a sub-optimal implementation of Git-LFS in GitLab but it does worry me that this new approach will be difficult to scale too. At some level the difference is understandable, Git-LFS probably store two different Packages files — around 90MB each for Trisquel — as two 90MB files, but native Git would store it as one compressed version of the 90MB file and one relatively small patch to turn the old files into the next file. So the Git-LFS approach surprisingly scale less well for overall storage-size. Still, the original repository is much smaller, and you usually don’t have to pull all LFS files anyway. So it is net win.

Throughout this work, I kept thinking about how my approach relates to Debian’s snapshot service. Ultimately what I would want is a combination of these two services. To have a good foundation to do transparency work I would want to have a collection of all Release/Packages/Sources files ever published, and ultimately also the source code and binaries. While it makes sense to start on the latest stable releases of distributions, this effort should scale backwards in time as well. For reproducing binaries from source code, I need to be able to securely find earlier versions of binary packages used for rebuilds. So I need to import all the Release/Packages/Sources packages from snapshot into my repositories. The latency to retrieve files from that server is slow so I haven’t been able to find an efficient/parallelized way to download the files. If I’m able to finish this, I would have confidence that my new Git-LFS based approach to store these files will scale over many years to come. This remains to be seen. Perhaps the repository has to be split up per release or per architecture or similar.

Another factor is storage costs. While the git repository size for a Git-LFS based repository with files from several years may be possible to sustain, the Git-LFS storage size surely won’t be. It seems GitLab charges the same for files in repositories and in Git-LFS, and it is around $500 per 100GB per year. It may be possible to setup a separate Git-LFS backend not hosted at GitLab to serve the LFS files. Does anyone know of a suitable server implementation for this? I had a quick look at the Git-LFS implementation list and it seems the closest reasonable approach would be to setup the Gitea-clone Forgejo as a self-hosted server. Perhaps a cloud storage approach a’la S3 is the way to go? The cost to host this on GitLab will be manageable for up to ~1TB ($5000/year) but scaling it to storing say 500TB of data would mean an yearly fee of $2.5M which seems like poor value for the money.

I realized that ultimately I would want a git repository locally with the entire content of all apt archives, including their binary and source packages, ever published. The storage requirements for a service like snapshot (~300TB of data?) is today not prohibitly expensive: 20TB disks are $500 a piece, so a storage enclosure with 36 disks would be around $18.000 for 720TB and using RAID1 means 360TB which is a good start. While I have heard about ~TB-sized Git-LFS repositories, would Git-LFS scale to 1PB? Perhaps the size of a git repository with multi-millions number of Git-LFS pointer files will become unmanageable? To get started on this approach, I decided to import a mirror of Debian’s bookworm for amd64 into a Git-LFS repository. That is around 175GB so reasonable cheap to host even on GitLab ($1000/year for 200GB). Having this repository publicly available will make it possible to write software that uses this approach (e.g., porting debdistreproduce), to find out if this is useful and if it could scale. Distributing the apt repository via Git-LFS would also enable other interesting ideas to protecting the data. Consider configuring apt to use a local file:// URL to this git repository, and verifying the git checkout using some method similar to Guix’s approach to trusting git content or Sigstore’s gitsign.

A naive push of the 175GB archive in a single git commit ran into pack
size limitations:

remote: fatal: pack exceeds maximum allowed size (4.88 GiB)

however breaking up the commit into smaller commits for parts of the
archive made it possible to push the entire archive. Here are the
commands to create this repository:

git init
git lfs install
git lfs track 'dists/**' 'pool/**'
git add .gitattributes
git commit -m"Add Git-LFS track attributes." .gitattributes
time debmirror --method=rsync --host ftp.se.debian.org --root :debian --arch=amd64 --source --dist=bookworm,bookworm-updates --section=main --verbose --diff=none --keyring /usr/share/keyrings/debian-archive-keyring.gpg --ignore .git .
git add dists project
git commit -m"Add." -a
git remote add origin git@gitlab.com:debdistutils/archives/debian/mirror.git
git push --set-upstream origin --all
for d in pool//; do
echo $d;
time git add $d;
git commit -m"Add $d." -a
git push
done

The resulting repository size is around 27MB with Git LFS object storage around 174GB. I think this approach would scale to handle all architectures for one release, but working with a single git repository for all releases for all architectures may lead to a too large git repository (>1GB). So maybe one repository per release? These repositories could also be split up on a subset of pool/ files, or there could be one repository per release per architecture or sources.

Finally, I have concerns about using SHA1 for identifying objects. It seems both Git and Debian’s snapshot service is currently using SHA1. For Git there is SHA-256 transition and it seems GitLab is working on support for SHA256-based repositories. For serious long-term deployment of these concepts, it would be nice to go for SHA256 identifiers directly. Git-LFS already uses SHA256 but Git internally uses SHA1 as does the Debian snapshot service.

What do you think? Happy Hacking!

Categories: FLOSS Project Planets

Drupal Mountain Camp: Thanks for being a part of Drupal Mountain Camp 2024

Planet Drupal - Mon, 2024-03-18 10:10
Thanks for being a part of Drupal Mountain Camp 2024 admin Mon, 03/18/2024 - 15:10

Wow, what an event!

We would like to thank you for contributing to the magic of Drupal Mountain Camp 2024 and making it tremendous success!

From the snow-capped peaks of learning to the comfy fireside chats of networking, the event was a one-of-a-kind experience that left a lasting imprint on all of us.

As I look back on the laughter-filled evenings and the insightful discussions that echoed through the venue to the fondue evening, I'm reminded of the incredible sense of community that defines us.

We are immensely grateful for the insightful presentations by our honoured speakers, the generous support from our valuable sponsors, and the dedication and hard work of our volunteers to ensure the event's success.

Together, we explored new ideas, shared knowledge, and forged meaningful connections within the Drupal community.

As we reflect on the success of this year's conference, we're already looking forward to what lies ahead for the next Drupal Mountain Camp.

 

 

Feedback

We implore everyone to submit their feedback to each session they attended by going to the schedule and clicking on each session, then giving a quick anonymous feedback.

The schedule should still be available via the custom domain:
https://event.drupalmountaincamp.ch/drupal-mountain-camp-2024/schedule/

Furthermore, we would greatly appreciate all feedback regarding the conference as a whole to be submitted on the "Closing Session".

 

Highlights

Here's some of the statistics that we calculated from the event.

  • 77 participants
  • 350+ coffees consumed
  • 63 Fondues eaten at Schatzalp
  • 39 People took the sled down Schatzalp
  • 0 Injuries
  • 11 Sponsors
  • 12 Team members
  • 6 Volunteers
  • 17.65% Diverse Speakers
  • 58 Posts in LinkedIn, Instagram Twitter and Mastodon
  • 236 Unique Visitors on LinkedIn since Nov 11
  • 25 issues worked on #MountainCamp2024

Participants per Country

  • 46    Switzerland (CH)
  • 6    Belgium (BE)
  • 5    United Kingdom (GB)
  • 4    Germany (DE)
  • 3    Finland (FI)
  • 3    France (FR)
  • 2    Slovenia (SI)
  • 2    United States (US)
  • 1    Australia (AU)
  • 1    Austria (AT)
  • 1    Bulgaria (BG)
  • 1    Portugal (PT)
  • 1    Spain (ES)
  • 1    Suriname (SR)

 

Media

We're very grateful to have such an awesome media team to capture every single moment.

Official photos by Patrick Itten
https://foto.patrickitten.ch/drupal-mountain-camp-2024

Photos by Josef Kruckenberg
https://www.flickr.com/photos/185226958@N05/albums/72177720315270848/

Relive the recap here by Dan Lemon
https://youtu.be/Z6AX_gexOg0

Extended fondue and sledding with the little happy birthday surprise
https://youtu.be/BgBzHbveSdE

 

Session Videos

We were able to record all the keynote sessions along with all sessions that took place in the Pischa room.

You can find the session videos on our YouTube Channel, under the playlist title "Sessions - Drupal Mountain Camp 2024"

https://www.youtube.com/playlist?list=PL6C9y4dEueZhksow0hSYFsPKGIo2c735C

 

Aaron Winborn Award

This is a reminder that the Aaron Winborn award nominations are open until Friday, March 29, 2024

This annual award recognizes an individual who demonstrates personal integrity, kindness, and above-and-beyond commitment to the Drupal community.
It includes a scholarship and travel stipend for the winner to attend DrupalCon North America and recognition in a plenary session at the event.

https://www.drupal.org/community/cwg/blog/nominations-are-now-open-for-the-2024-aaron-winborn-award

 

What's Next

Drupal Switzerland Association plans to host further in-person events throughout the year, such as gatherings in Zurich, Bern, and hopefully somewhere in the Swiss Romandie area.

We will post new events to our meetup.com account, as well as cross-posting to our Drupal Switzerland Slack and also to Drupal Groups:
https://www.meetup.com/zurich-drupal-meetup/
https://groups.drupal.org/switzerland

Furthermore, we'd like to highlight these two upcoming events for the Drupal community.

 

Drupal Dev Days Burgas

Drupal Developer Days is an annual gathering of people loving, learning, and discussing all things relating to Drupal. This year it'll be hosted by Drupal Bulgaria in Burgas, by the seaside.

The event is planned to take place between June 26-28, 2024 at Burgas Free University in Burgas, Bulgaria.

Drupal Dev Days 2024 is going to be a 3-day event full of amazing sessions, workshops and discussions as well as various social events and other surprises. 

Find out more on their website
https://ddd2024.drupalcamp.bg/

 

DrupalCon Barcelona

You may have seen Mercè in Davos, the mascot for DrupalCon Barcelona. Kuoni-Tumlare will calculate out who won the contest for a free ticket to DrupalCon Barcelona 2024.

This year's DrupalCon takes place from September 24-27, 2024 at the stunning Centre de convencions internacional de Barcelona (CCIB).

Barcelona's accessibility ensures that this event is within reach for all Drupal enthusiasts.

https://events.drupal.org/barcelona2024

 

A Final Thank You

We'd like to extend our deepest gratitude to our valuable sponsors, whose generous support made Drupal Mountain Camp 2024 possible.

To our esteemed speakers, thank you for sharing your expertise and insights, enriching the experience for all attendees.

A heartfelt thank you to our dedicated volunteers, whose hard work and enthusiasm ensured the smooth execution of every detail.

And last but certainly not least, to all our participants, thank you for bringing your passion and energy, making Drupal Mountain Camp a vibrant and unforgettable gathering.

Your contributions have truly made a difference and have left a lasting impact on our community.

Until we meet again, let's continue to collaborate, learn, and grow together in the spirit of Drupal.

 

On behalf of the Drupal Mountain Camp organisation team,
Dan Lemon

 

Categories: FLOSS Project Planets

Real Python: Model-View-Controller (MVC) in Python Web Apps: Explained With Lego

Planet Python - Mon, 2024-03-18 10:00

If you’re curious about web development, then you’ve likely encountered the abbreviation MVC, which stands for Model-View-Controller. You may know that it’s a common design pattern that’s fundamental to many Python web frameworks and even desktop applications.

But what exactly does it mean? If you’ve had a hard time wrapping your head around the concept, then keep on reading.

In this tutorial, you’ll:

  • Approach understanding the MVC pattern through a Lego-based analogy
  • Learn what models, views, and controllers are conceptually
  • Tie your conceptual understanding back to concrete web development examples
  • Investigate Flask code snippets to drive the point home

Maybe you built things with Lego as a kid, or maybe you’re still a Lego-aficionado today. But even if you’ve never pieced two Lego blocks together, keep on reading because the analogy might still be a good building block for your understanding.

Get Your Code: Click here to download an example Flask app that will help you understand MVC in Python web apps.

Take the Quiz: Test your knowledge with our interactive “Model-View-Controller (MVC) in Python Web Apps: Explained With Lego” quiz. Upon completion you will receive a score so you can track your learning progress over time:

Take the Quiz »

Explaining the Model-View-Controller Pattern With Lego

Imagine that you’re ten years old and sitting on your family room floor. In front of you is a big bucket of Lego, or similar modular building blocks. There are blocks of all different shapes and sizes:

  • 🟦🟦🟦 Some are blue, tall, and long.
  • 🟥 Some are red and cube-shaped.
  • 🟨🟨 Some are yellow, big, and wide.

With all of these different Lego pieces, there’s no telling what you could build!

Just as your mind is filling with the endless possibilities, you hear something coming from the direction of the couch. It’s your older brother, voicing a specific request. He’s saying, “Hey! Build me a spaceship!”

“Alright,” you think, “that could actually be pretty cool.” A spaceship it is!

So you get to work. You start pulling out the Lego blocks that you think you’re going to need. Some big, some small. Different colors for the outside of the spaceship, different colors for the engines.

Now that you have all of your building blocks in place, it’s time to assemble the spaceship. And after a few hours of hard work, you now have in front of you—a spaceship:

🟦 🟦🟥🟦 🟦🟥🟥🟥🟦 🟦🟥🟥🟥🟥🟥🟦 🟦🟥🟥🟥🟥🟥🟦 🟦🟥🟩🟩🟩🟥🟦 🟦🟥🟩🟦🟩🟥🟦 🟦🟥🟩🟩🟩🟥🟦 🟦🟥🟥🟥🟥🟥🟦 🟦🟥🟥🟥🟥🟥🟦 🟦🟥🟥🟥🟥🟥🟦 🟦🟥🟥🟥🟥🟥🟥🟥🟥🟥🟦 🟦🟥🟥🟥🟥🟥🟥🟥🟥🟥🟦 🟦🟥🟨🟨🟥🟥🟥🟨🟨🟥🟦 🟨🟨 🟨🟨

You run to find your brother and show him the finished product. “Wow, nice work!”, he says. Then he quietly adds:

Huh, I just asked for that a few hours ago, I didn’t have to do a thing, and here it is. I wish everything was that easy.

Your Brother

What if I told you that building a web application using the MVC pattern is exactly like building something with Lego blocks?

User Sends a Request Read the full article at https://realpython.com/lego-model-view-controller-python/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Categories: FLOSS Project Planets

Christoph Berg: vcswatch and git --filter

Planet Debian - Mon, 2024-03-18 08:45

Debian is running a "vcswatch" service that keeps track of the status of all packaging repositories that have a Vcs-Git (and other VCSes) header set and shows which repos might need a package upload to push pending changes out.

Naturally, this is a lot of data and the scratch partition on qa.debian.org had to be expanded several times, up to 300 GB in the last iteration. Attempts to reduce that size using shallow clones (git clone --depth=50) did not result more than a few percent of space saved. Running git gc on all repos helps a bit, but is tedious and as Debian is growing, the repos are still growing both in size and number. I ended up blocking all repos with checkouts larger than a gigabyte, and still the only cure was expanding the disk, or to lower the blocking threshold.

Since we only need a tiny bit of info from the repositories, namely the content of debian/changelog and a few other files from debian/, plus the number of commits since the last tag on the packaging branch, it made sense to try to get the info without fetching a full repo clone. The question if we could grab that solely using the GitLab API at salsa.debian.org was never really answered. But then, in #1032623, Gábor Németh suggested the use of git clone --filter blob:none. As things go, this sat unattended in the bug report for almost a year until the next "disk full" event made me give it a try.

The blob:none filter makes git clone omit all files, fetching only commit and tree information. Any blob (file content) needed at git run time is transparently fetched from the upstream repository, and stored locally. It turned out to be a game-changer. The (largish) repositories I tried it on shrank to 1/100 of the original size.

Poking around I figured we could even do better by using tree:0 as filter. This additionally omits all trees from the git clone, again only fetching the information at run time when needed. Some of the larger repos I tried it on shrank to 1/1000 of their original size.

I deployed the new option on qa.debian.org and scheduled all repositories to fetch a new clone on the next scan:

The initial dip from 100% to 95% is my first "what happens if we block repos > 500 MB" attempt. Over the week after that, the git filter clones reduce the overall disk consumption from almost 300 GB to 15 GB, a 1/20. Some repos shrank from GBs to below a MB.

Perhaps I should make all my git clones use one of the filters.

Categories: FLOSS Project Planets

ListenData: Python for Data Science: Beginner's Guide

Planet Python - Mon, 2024-03-18 08:32

This tutorial would help you to learn Data Science with Python by examples. It is designed for beginners who want to get started with Data Science in Python. Python is an open source language and it is widely used as a high-level programming language for general-purpose programming. It has gained high popularity in data science world. In the PyPL Popularity of Programming language index, Python leads with a 29 percent share. In advanced analytics and predictive analytics market, it is ranked among top 3 programming languages for advanced analytics.

Data Science with Python Tutorial Table of Contents Introduction Python is widely used and very popular for a variety of software engineering tasks such as website development, cloud-architecture, back-end etc. It is equally popular in data science world. In advanced analytics world, there has been several debates on R vs. Python. There are some areas such as number of libraries for statistical analysis, where R wins over Python but Python is catching up very fast.With popularity of big data and data science, Python has become first programming language of data scientists.

There are several reasons to learn Python. Some of them are as follows -

  1. Python runs well in automating various steps of a predictive model.
  2. Python has awesome robust libraries for machine learning, natural language processing, deep learning, big data and artificial Intelligence.
  3. Python wins over R when it comes to deploying machine learning models in production.
  4. It can be easily integrated with big data frameworks such as Spark and Hadoop.
  5. Python has a great online community support.
Do you know these sites are developed in Python?
  1. YouTube
  2. Instagram
  3. Reddit
  4. Dropbox
  5. Disqus
Python 2 vs. 3 Google yields thousands of articles on this topic. Some bloggers opposed and some in favor of 2.7. If you filter your search criteria and look for only recent articles, you would find Python 2 is no longer supported by the Python Software Foundation. Hence it does not make any sense to learn 2.7 if you start learning it today. Python 3 supports all the packages. Python 3 is cleaner and faster. It is a language for the future. It fixed major issues with versions of Python 2 series. Python 3 was first released in year 2008. It has been 12 years releasing robust versions of Python 3 series. You should go for latest version of Python 3. How to install Python? There are two ways to download and install Python
  1. Download Anaconda. It comes with Python software along with preinstalled popular libraries.
  2. Download Pythonfrom its official website. You have to manually install libraries.
Recommended : Go for first option and download anaconda. It saves a lot of time in learning and coding Python Coding Environments Anaconda comes with two popular IDE :
  1. Jupyter (Ipython) Notebook
  2. Spyder
Spyder. It is like RStudio for Python. It gives an environment wherein writing python code is user-friendly. If you are a SAS User, you can think of it as SAS Enterprise Guide / SAS Studio. It comes with a syntax editor where you can write programs. It has a console to check each and every line of code. Under the 'Variable explorer', you can access your created data files and function. I highly recommend Spyder! Spyder - Python Coding Environment Jupyter (Ipython) Notebook Jupyter is equivalent to markdown in R. It is useful when you need to present your work to others or when you need to create step by step project report as it can combine code, output, words, and graphics.
To read this article in full, please click hereThis post appeared first on ListenData
Categories: FLOSS Project Planets

Drupal.org blog: What's next for (Drupal) innovation

Planet Drupal - Mon, 2024-03-18 06:39

Last year the Drupal Association brought me onto the team using a special board vision fund to help kick off our first steps in increasing our focus on innovation.

This phase is coming to an end in May with the end of my contract, and right in time for Drupalcon Portland. I am incredible grateful for this past months, for all the people and friends I've met inside and outside of the Drupal Association, for all the work we've done together, and for all the work that is still to come.

But before I wrap up my work with the team I want to report on the progress we've made so far, what we're still working on until May, and what happens next.

Alex took on a herculean task when he joined us for this special engagement to help determine our strategy for accelerating innovation in Drupal. I'm extremely grateful for what he's accomplished and proud to have had him join our team.

Thanks to his work we've fostered 6 fantastic projects through Pitchburgh grant funding, completed a study of friction points in our new user registration, onboarding and contribution process, identified an action plan to better onboard new contributors, improve open source contribution and set our sights on even higher goals.

I'd love to see Drupal step forward to help us take the next step!

Tim Lehnen CTO
Drupal Association

Lots of work to be proud of

Innovation Working Group

The Innovation working group is a gathering happening every other week, in  which we report progress on ongoing initiatives as well as discuss everything related to innovation, from contributor guidance, to documentation. As part of my role I took the leadership of the working group, and steered the direction of the discussions as well as the topics to be discussed.

Some of the initiatives that started as a seed from that group are the research on the documentation improvements (spearheaded by Mike Herschel), the onboarding analysis (executed by myself), the Drupal Acceleration team proposal (led by Laurii and Cristina Chumillas), a conversation about the relative value of a credit to the contributor wage invested, and many more.

As a personal note, I have to thank the whole group for their passion and commitment to improving Drupal, and especially for their patience with me while I was trying to find my place in this new world that I joined in 2023.

Pitchburgh

Pitchburgh was kick-started by Dries during Drupalcon Pittsburgh 2023, and consists of an innovation contest in which participants would propose their innovative ideas for Drupal, and a mix between judges and people at Drupalcon itself would vote the winners.

Acting as the program manager / coordinator of the 6 projects, my role helped on a diverse range of tasks, sometimes outside of my own zone of comfort, like getting contracts ready for sign up, syncing with core committers when needed, communications and relations with the community and a broad range of other tasks.

We're proud to be sharing the success stories of each of these projects at DrupalCon Portland in May.

Contribution Health Dashboards

As part of the feedback from the Innovation Working Group we took care of creating a series of dashboards and Drush scripts with the goal of getting a better understanding of the contribution ecosystem in Drupal, finding what needs improvement and how any changes introduced in the ecosystem would impact the platform.

From these dashboards we can track the impact of our work to improve contributor onboarding.

Onboarding new users/contributors

The first contact a big portion of our users have with Drupal is our drupal.org registration. Improving that process not only ensures that we are delivering the image and branding that Drupal deserves, but also we maximize the opportunities to convert those users from takers to makers. The start of 2024 was full of research and investigation on the outdated registration flow, a set of recommendations have been put in place, and changes have already started, including a more prominent presence of guidance and help for new want-to-be contributors.

We'll be implementing a new 'persona' field to help us categorize new users and set them on the appropriate journey for their role.

Contribution friction analysis

When a hindrance obstructs innovation, it often persists for months or even years. The extended duration of the contribution pipeline presents an opportunity to reduce this time to contribution, enhance the velocity of Open Source projects, in this case Drupal, and accelerate Drupal's development process.

During different times we went through the contribution process in Drupal, aiming to identify friction points and other blockers, while I also kept conversations with different members of the community, contributors and core members. I expect to publish this research and my findings and recommendations prior to Drupalcon Portland, and I hope that this will provide valuable insights into the factors impeding contribution and suggesting strategies to optimize efforts.

For more information, join my session, co-presented with Irina Zaks (who has also been researching on this topic) at DrupalCon portland, titled: Innovation and contribution challenges

Bounty program

The Bounty program was a pilot program with the idea to research possible solutions to a few common problems, the most important one, ensure contributions happen where Drupal and the Drupal Association needs them more, which is, for example (but not only), strategic initiatives and innovation projects.

The program was successful as it resuscitated some old, sleeping issues, and even solved one of them (which was open for more than 4 years) in just a few weeks. The program demonstrated the need of soft steering in the right direction, and a phase 2 has already been started with the aim to make an ever wider impact.

Read about the Bounty program here, and the Bounty program extension here.

Contributor (priorities) guide

The Bounty program, together with conversations we’ve been having with companies and partners, also inspired us to create a contributor guide which helps companies and individuals to steer everyone on the right path towards contributions that are more meaningful and have higher impact in the community.

You can read the priority guide contributor announcement here: Maximizing contributions

Innovation landing

One of the realizations during this time collaborating with the Innovation Working Group is that innovation will not come from specific individuals or even necessarily from a group. 

Drupal is a highly decentralized community, and innovation will come from that community. We have a huge amount of modules, which is one of our biggest strengths. But that as well means that we need to ensure that we keep our conversations and meetings diverse enough. The community needs to surface those projects that are innovative.

The next big leap is probably hidden, deep in a sea of ideas and projects. Hence, the only way to ensure that we don’t miss the next wave is to ensure that everyone has a voice.

That’s why we agreed early to create an innovation landing page, which could capture the essence of Drupal Innovation, serve as a center of information, and inspire others to do more of the great things that we are used to seeing in our community.
 

More for us to do!

Drupal has always been a do-ocracy- where passionate, motivated contributors with a great idea can roll up their sleeves and get things done. The work above represents some great first steps (and it's hard to believe how fast time flew by!) but as a community we have to come together with a collective commitment to do more.

Innovation does not stop here, my work was just the beginning, setting up the base of what’s next. The Drupal association needs your help, so moving forward we could:

  • Creating a dedicated innovation fund that partner organizations and individuals can contribute to, to make specific, targeted investments in innovation
  • Grant Fundraising and Sponsorship for dedicated roles within the Drupal Association
    • For example - partners could come together to 'endow' a position for a role like mine of Innovation Program manager to be made permanent.
    • And/Or for roles like a Contribution Success Manager
    • And/Or for direct core contribution engineers

Only together will we be able to scale the pace of innovation in Drupal - and we have so many capable people in our community that our biggest barrier is only finding the resources to make our vision a reality. That is something we can solve together.

On a personal note, while my engagement is coming to a close, this is not a goodbye. I'm more committed than ever to the Drupal community and excited about its future. If you want to support our continued efforts to work with the Drupal Association on these goals, please let us know: alex@association.drupal.org

Categories: FLOSS Project Planets

Golems GABB: Advanced Content Management with the Paragraphs Module in Drupal

Planet Drupal - Mon, 2024-03-18 06:13
Advanced Content Management with the Paragraphs Module in Drupal Editor Mon, 03/18/2024 - 12:13

Creating user-friendly and stylish web pages can sometimes make you nervous, and maybe you want to turn off your computer - does that sound familiar? Even using popular CMSs, including Drupal, sometimes it is difficult to understand this issue and create a functional, modern website. Drupal can offer several effective solutions, and today, we will look at one of them - the Drupal Paragraphs module.
Creating a home page, or even a regular page on a website, requires much effort and time unless you're a professional web developer. Many people with initial design skills face similar problems and spend much time searching for solutions. Often, everything comes with experience. Through trial and error, reading various articles, and watching videos on YouTube, we are looking for an opportunity to achieve the goal.

Categories: FLOSS Project Planets

LN Webworks: Drupal Community Module Evaluation: A Go-to Guide

Planet Drupal - Mon, 2024-03-18 05:47

It's not only about what a module from the Drupal community can achieve when you consider using it. Along with how long you want your website to remain, you need also to consider who will maintain it after it goes up. These factors influence the level of danger you are willing to accept. 

For instance, you might desire modules with an excellent track record and robust community support if you're not too familiar with Drupal.

Every module available on Drupal.org is licensed under the GPLv2 and is open source. This means that while they are free to use, any dangers are your responsibility but every module has a community of users who assist with problems and updates, as well as a team of maintainers for each module. Additionally, partnering with a reputable Drupal web development company can provide further assurance in terms of module selection, ongoing support, and maintenance.

Let's discuss some factors to consider when evaluating a community module.

Categories: FLOSS Project Planets

How to write a QML effect for KWin

Planet KDE - Mon, 2024-03-18 02:00

Since the dawn of the times, the only way to implement any effect that has fancy user interface used to be in C++. It would be rather an understatement to say that the C++ API is difficult to use or get it right so there are no glitches. On the other hand, it would be really nice to be able to implement dashboard-like effects while not compromise on code simplicity and maintainability, especially given the rise of popularity of overview effects a few years ago, which indicated that there is demand for such effects.

In order solve that problem, we started looking for some options and the most obvious one was QtQuick. It’s a quite powerful framework and it’s already used extensively in Plasma. So, in Plasma 5.24, we introduced basic support for implementing kwin effects written in QML and even added a new overview effect. Unfortunately, if you wanted to implement a QtQuick-based effect yourself, you would still have to write a bit of C++ glue yourself. This is not great because effects that use C++ are a distribution nightmare. They can’t be just uploaded to the KDE Store and then installed by clicking “Get New Effects…”. Furthermore, libkwin is a fast moving target with lots of ABI and API incompatible changes in every release. That’s not good if you’re an effect developer because it means you will need to invest a bit of time after every plasma release to port the effects to the new APIs or at least rebuild the effects to resolve ABI incompatibilities.

This has been changed in Plasma 6.0.

In Plasma 6, we’ve had the opportunity to address that pesky problem of requiring some C++ code and also improve the declarative effect API after learning some lessons while working on the overview and other effects. So, enough of history and let’s jump to the good stuff, shall we?

Project Structure

Declarative effects require some particular project structure that we need to learn first before writing any code

└── package
├── contents
│   └── ui
│   └── main.qml
└── metadata.json

The package directory is a toplevel directory, it should contain two things: a metadata.json file and a contents directory. The metadata.json file contains information about the name of the effect, what API it uses, the author, etc.

{
"KPackageStructure": "KWin/Effect",
"KPlugin": {
"Authors": [
{
"Email": "user@example.com",
"Name": "Name"
}
],
"Category": "Appearance",
"Description": "Yo",
"EnabledByDefault": false,
"Id": "hello-world",
"License": "MIT",
"Name": "Hello World"
},
"X-KDE-Ordering": 60,
"X-Plasma-API": "declarativescript"
}

The contents directory contains the rest of QML code, config files, assets, etc. Keep in mind that ui/main.qml is a “magical” file, it acts as an entry point, every effect must have it.

In order to install the effect and make it visible in Desktop Effects settings, you will need to run the following command

kpackagetool6 --type KWin/Effect --install package/

This is quite a lot to memorize. That’s why kwin provides an example qtquick effect that you can grab, tweak some metadata and you’re good to go. You can find the example project at https://invent.kde.org/plasma/kwin/-/tree/master/examples/quick-effect?ref_type=heads. Note that the example project also contains a CMakeLists.txt file, which provides an alternative way to install the effect by the means of cmake, i.e. make install or cmake --install builddir.

Hello World

Let’s start with an effect that simply shows a hello world message on the screen:

import QtQuick
import org.kde.kwin

SceneEffect {
id: effect

delegate: Rectangle {
color: "blue"

Text {
anchors.centerIn: parent
color: "white"
text: "Hello world!"
}
}

ScreenEdgeHandler {
enabled: true
edge: ScreenEdgeHandler.TopEdge
onActivated: effect.visible = !effect.visible
}

ShortcutHandler {
name: "Toggle Hello World Effect"
text: "Toggle Hello World Effect"
sequence: "Meta+H"
onActivated: effect.visible = !effect.visible
}

PinchGestureHandler {
direction: PinchGestureHandler.Direction.Contracting
fingerCount: 3
onActivated: effect.visible = !effect.visible
}
}

import QtQuick is needed to use basic QtQuick components such as Rectangle. import org.kde.kwin imports kwin specific components.

The SceneEffect is a special type that every declarative effect must use. Its delegate property specifies the content for every screen. In this case, it’s a blue rectangle with a “Hello World!” label in the center.

The ShortcutHandler is a helper that’s used to register global shortcuts. ShortcutHandler.name is the key of the global shortcut, it’s going to be used to store the shortcut in the config and other similar purposes. ShortcutHandler.text is a human readable description of the global shortcut, it’s going to be visible in the Shortcuts settings.

The ScreenEdgeHandler allows to reserve a screen edge. When the pointer hits that screen edge, some code can be executed by listening to the activated signal.

The PinchGestureHandler and SwipeGestureHandler allow to execute some code when the user makes a pinch or a swipe gesture, respectively.

effect.visible = !effect.visible toggles the visibility of the effect. When effect.visible is true, the effect is active and visible on the screen; otherwise it’s hidden. You need to set effect.visible to true in order to show the effect.

If you press Meta+H or make a pinch gesture or move the pointer to the top screen edge, you’re going to see something like this

Note that there are no windows visible anymore, it is the responsibility of the effect to decide what should be displayed on the screen now.

Displaying Windows

Being able to display text is great, but it’s not useful. Usually, effects need to display some windows, so let’s display the active window

delegate: Rectangle {
color: "blue"

WindowThumbnail {
anchors.centerIn: parent
client: Workspace.activeWindow
}
}

The change is quite simple. Instead of displaying a Text component, there’s a WindowThumbnail component now. The WindowThumbnail type is provided by the org.kde.kwin module. WindowThumbnail.client indicates what window the thumbnail item should display.

Input

Input processing contains no kwin specific APIs. TapHandler, MouseArea, Keys and other stuff available in QtQuick should just work. For example, let’s implement an effect that arranges windows in a grid and if somebody middle clicks a window, it will be closed

delegate: Rectangle {
color: "pink"

GridView {
id: grid
anchors.fill: parent
cellWidth: 300
cellHeight: 300

model: WindowModel {}
delegate: WindowThumbnail {
client: model.window
width: grid.cellWidth
height: grid.cellHeight

TapHandler {
acceptedButtons: Qt.MiddleButton
onTapped: client.closeWindow()
}
}
}
}

The code looks pretty straightforward except maybe the model of the GridView. WindowModel is a helper provided by org.kde.kwin module that lists all the windows. It can be passed to various views, Repeater, and so on.

The result can be seen here

Delegates are Per Screen

One thing to keep in mind is that the delegates are instantiated per screen. For example,

delegate: Rectangle {
color: "yellow"

Text {
anchors.centerIn: parent
color: "black"
text: SceneView.screen.name
}
}

When you activate the effect on a setup with several outputs, each output will be filled with yellow color and the output name in the center

Usually, the output is irrelevant, but if you need to know what output particular delegate is displayed on, you could use the SceneView.screen attached property.

Configuration

As your effect grows, you will probably face the need to provide an option or two. Let’s say that we want the background color in our hello world effect to be configurable. How do we achieve that? The first step, is to add a main.xml file in package/contents/config directory, i.e.

package/
├── contents
│   ├── config
│   │   └── main.xml
│   └── ui
│   └── main.qml
└── metadata.json

The main.xml file lists all options

<?xml version="1.0" encoding="UTF-8"?>
<kcfg xmlns="http://www.kde.org/standards/kcfg/1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.kde.org/standards/kcfg/1.0
http://www.kde.org/standards/kcfg/1.0/kcfg.xsd" >
<kcfgfile name=""/>
<group name="">
<entry name="BackgroundColor" type="Color">
<default>#ff00ff</default>
</entry>
</group>
</kcfg>

In our case, only one option is needed: BackgroundColor, which has Color type and #ff00ff default value. You can refer to the KConfigXT documentation to learn more what other entry types are supported.

The next step is to actually use the BackgroundColor option

delegate: Rectangle {
color: effect.configuration.BackgroundColor

Text {
anchors.centerIn: parent
color: "white"
text: "Hello world!"
}
}

effect.configuration is a map object that contains all the options listed in the main.xml.

Now, if you toggle the hello world effect, you’re going to see

There are a few more thing left to do though. If you navigate to Desktop Effects settings, you’re not going a configure button next to the hello world effect

Besides providing a main.xml file, the effect also needs to provide a config.ui file containing a configuration ui

package/
├── contents
│   ├── config
│   │   └── main.xml
│   └── ui
│   ├── config.ui
│   └── main.qml
└── metadata.json

The config.ui file is a regular Qt Designer UI file. The only special thing about it is that the controls that represent options should have special name format: kcfg_ + OptionName. For example, kcfg_BackgroundColor

<?xml version="1.0" encoding="UTF-8"?>
<ui version="4.0">
<class>QuickEffectConfig</class>
<widget class="QWidget" name="QuickEffectConfig">
<property name="geometry">
<rect>
<x>0</x>
<y>0</y>
<width>455</width>
<height>177</height>
</rect>
</property>
<layout class="QFormLayout" name="formLayout">
<item row="0" column="0">
<widget class="QLabel" name="label">
<property name="text">
<string>Background color:</string>
</property>
</widget>
</item>
<item row="0" column="1">
<widget class="KColorButton" name="kcfg_BackgroundColor">
<property name="flat">
<bool>false</bool>
</property>
</widget>
</item>
</layout>
</widget>
<customwidgets>
<customwidget>
<class>KColorButton</class>
<extends>QPushButton</extends>
<header>kcolorbutton.h</header>
</customwidget>
</customwidgets>
<resources/>
<connections/>
</ui>

The last final piece in order to expose the configuration ui is to add the following line in the metadata.json file

"X-KDE-ConfigModule": "kcm_kwin4_genericscripted"

With all of that, the effect is finally displayed as configurable in the system settings and the background color can be changed

Sharing Your Effect With Other People

The preferred method to distribute third party extensions is via the KDE Store. Both JS and QML effects can be uploaded to the same “KWin Effects” category.

Documentation and Other Useful Resources

Documentation is still something that needs a lot of work (and writing it is no fun ). KWin scripting API documentation can be found here https://develop.kde.org/docs/plasma/kwin/api/.

Besides the link above, it’s worth having a look at the examples https://invent.kde.org/plasma/kwin/-/tree/master/examples?ref_type=heads in kwin git repository.

Window Heap

If you need to pack or arrange windows like how the overview effect does, you could use the WindowHeap component from org.kde.kwin.private.effects module. BUT you need to keep in mind that that helper is private and has no stable API yet, so use it on your own risk (or copy paste the relevant code in kwin). Eventually, the WindowHeap will be stabilized once we are confident about its API.

The WindowHeap source code can be found at https://invent.kde.org/plasma/kwin/-/tree/2a13a330404c8d8a95f6264512aa06b0a560f55b/src/plugins/private.

More Examples

If you need more examples, I suggest to have a look at the desktop cube effect in kdeplasma-addons. It’s implemented using the same QML effect API in kwin + QtQuick3D. The source code can be found at https://invent.kde.org/plasma/kdeplasma-addons/-/tree/master/kwin/effects/cube?ref_type=heads

Conclusion

I hope that some people find this quick introduction to QML-based effects in KWin useful. Despite being a couple years old, declarative effects can be still considered being in the infancy stage and there’s a lot of untapped potential in them yet. The QML effect API is not perfect, and that’s why we are interested in feedback from third party extension developers. If you have some questions or feedback, feel free to reach out us at https://webchat.kde.org/#/room/#kwin:kde.org. Happy hacking!

Categories: FLOSS Project Planets

Gunnar Wolf: After miniDebConf Santa Fe

Planet Debian - Mon, 2024-03-18 00:00

Last week we held our promised miniDebConf in Santa Fe City, Santa Fe province, Argentina — just across the river from Paraná, where I have spent almost six beautiful months I will never forget.

Around 500 Kilometers North from Buenos Aires, Santa Fe and Paraná are separated by the beautiful and majestic Paraná river, which flows from Brazil, marks the Eastern border of Paraguay, and continues within Argentina as the heart of the litoral region of the country, until it merges with the Uruguay river (you guessed right — the river marking the Eastern border of Argentina, first with Brazil and then with Uruguay), and they become the Río de la Plata.

This was a short miniDebConf: we were lent the APUL union’s building for the weekend (thank you very much!); during Saturday, we had a cycle of talks, and on sunday we had more of a hacklab logic, having some unstructured time to work each on their own projects, and to talk and have a good time together.

We were five Debian people attending: {santiago|debacle|eamanu|dererk|gwolf}@debian.org. My main contact to kickstart organization was Martín Bayo. Martín was for many years the leader of the Technical Degree on Free Software at Universidad Nacional del Litoral, where I was also a teacher for several years. Together with Leo Martínez, also a teacher at the tecnicatura, they contacted us with Guillermo and Gabriela, from the APUL non-teaching-staff union of said university.

We had the following set of talks (for which there is a promise to get electronic record, as APUL was kind enough to record them! of course, I will push them to our usual conference video archiving service as soon as I get them)

Hour Title (Spanish) Title (English) Presented by 10:00-10:25 Introducción al Software Libre Introduction to Free Software Martín Bayo 10:30-10:55 Debian y su comunidad Debian and its community Emanuel Arias 11:00-11:25 ¿Por qué sigo contribuyendo a Debian después de 20 años? Why am I still contributing to Debian after 20 years? Santiago Ruano 11:30-11:55 Mi identidad y el proyecto Debian: ¿Qué es el llavero OpenPGP y por qué? My identity and the Debian project: What is the OpenPGP keyring and why? Gunnar Wolf 12:00-13:00 Explorando las masculinidades en el contexto del Software Libre Exploring masculinities in the context of Free Software Gora Ortiz Fuentes - José Francisco Ferro 13:00-14:30 Lunch     14:30-14:55 Debian para el día a día Debian for our every day Leonardo Martínez 15:00-15:25 Debian en las Raspberry Pi Debian in the Raspberry Pi Gunnar Wolf 15:30-15:55 Device Trees Device Trees Lisandro Damián Nicanor Perez Meyer (videoconferencia) 16:00-16:25 Python en Debian Python in Debian Emmanuel Arias 16:30-16:55 Debian y XMPP en la medición de viento para la energía eólica Debian and XMPP for wind measuring for eolic energy Martin Borgert

As it always happens… DebConf, miniDebConf and other Debian-related activities are always fun, always productive, always a great opportunity to meet again our decades-long friends. Lets see what comes next!

Categories: FLOSS Project Planets

ListenData: 15 Free Open Source ChatGPT Alternatives (with Code)

Planet Python - Sun, 2024-03-17 20:50

In this article we will explain how Open Source ChatGPT alternatives work and how you can use them to build your own ChatGPT clone for free. By the end of this article you will have a good understanding of these models and will be able to compare and use them.

Benefits of Open Source ChatGPT Alternatives

There are various benefits of using open source large language models which are alternatives to ChatGPT. Some of them are listed below.

  1. Data Privacy: Many companies want to have control over data. It is important for them as they don't want any third-party to have access to their data.
  2. Customization: It allows developers to train large language models with their own data and some filtering on some topics if they want to apply
  3. Affordability: Open source GPT models let you to train sophisticated large language models without worrying about expensive hardware.
  4. Democratizing AI: It opens room for further research which can be used for solving real-world problems.
Table of Contents Llama Introduction : Llama

Llama stands for Large Language Model Meta AI. It includes a range of model sizes from 7 billion to 65 billion parameters. Meta AI researchers focused on scaling the model's performance by increasing the volume of training data, rather than the number of parameters. They claimed the 13 billion parameter model outperformed 175 billion parameters of GPT-3 model. It uses the transformer architecture and was trained on 1.4 trillion tokens extracted by web scraping Wikipedia, GitHub, Stack Exchange, Books from Project Gutenberg, scientific papers on ArXiv.

Python Code : Llama # Install Package pip install llama-cpp-python from llama_cpp import Llama llm = Llama(model_path="./models/7B/ggml-model.bin") output = llm("Q: Name the planets in the solar system? A: ", max_tokens=128, stop=["Q:", "\n"], echo=True) print(output) In the model path, you need to have weights for Llama in GGML format and then store them into the models folder. You can search it on Hugging Face website. See one of them here Llama 2 What's New in Llama 2

Here are some of the key differences between Llama 2 and Llama:

  • Training data: Llama 2 is trained on 40% more tokens than Llama, a total of 2 trillion tokens. This gives it a larger knowledge base and allows it to generate more accurate responses.
  • Model size: Llama 2 is available in three sizes: 7 billion parameters, 13 billion parameters, and 70 billion parameters. Whereas, the maximum size of Llama is 65 billion parameters.
  • Chat optimization: Llama 2-Chat is a specialized version of Llama 2 that is optimized for engaging in two-way conversations. It has been trained on a dataset of human conversations, which allows it to generate more natural and engaging responses.
  • Safety and bias mitigation: Llama 2 has been trained with a focus on safety and bias mitigation. This means that it is less likely to generate toxic or harmful content.
  • Open source: Llama 2 is open source, which means that anyone can use it for research or commercial purposes. Whereas, Llama can't be used for commercial purposes.
Python Code : Llama 2 Llam2: 7 Billion Parameters

To run Llama2 7B model, refer the code below. The following code uses a 4-bit quantization technique that reduces the size of the LLM, which can make it easier to deploy and use on sytems with limited memory.

  Colab: Llama2 7B Model %cd /content !apt-get -y install -qq aria2 !git clone -b v1.3 https://github.com/camenduru/text-generation-webui %cd /content/text-generation-webui !pip install -r requirements.txt !pip install -U gradio==3.28.3 !mkdir /content/text-generation-webui/repositories %cd /content/text-generation-webui/repositories !git clone -b v1.2 https://github.com/camenduru/GPTQ-for-LLaMa.git %cd GPTQ-for-LLaMa !python setup_cuda.py install !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/4bit/Llama-2-7b-Chat-GPTQ/raw/main/config.json -d /content/text-generation-webui/models/Llama-2-7b-Chat-GPTQ -o config.json !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/4bit/Llama-2-7b-Chat-GPTQ/raw/main/generation_config.json -d /content/text-generation-webui/models/Llama-2-7b-Chat-GPTQ -o generation_config.json !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/4bit/Llama-2-7b-Chat-GPTQ/raw/main/special_tokens_map.json -d /content/text-generation-webui/models/Llama-2-7b-Chat-GPTQ -o special_tokens_map.json !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/4bit/Llama-2-7b-Chat-GPTQ/resolve/main/tokenizer.model -d /content/text-generation-webui/models/Llama-2-7b-Chat-GPTQ -o tokenizer.model !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/4bit/Llama-2-7b-Chat-GPTQ/raw/main/tokenizer_config.json -d /content/text-generation-webui/models/Llama-2-7b-Chat-GPTQ -o tokenizer_config.json !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/4bit/Llama-2-7b-Chat-GPTQ/resolve/main/gptq_model-4bit-128g.safetensors -d /content/text-generation-webui/models/Llama-2-7b-Chat-GPTQ -o gptq_model-4bit-128g.safetensors %cd /content/text-generation-webui !python server.py --share --chat --wbits 4 --groupsize 128 --model_type llama Llam2: 13 Billion Parameters

To run Llama2 13B model, refer the code below.

  Colab: Llama2 13B Model %cd /content !apt-get -y install -qq aria2 !git clone -b v1.8 https://github.com/camenduru/text-generation-webui %cd /content/text-generation-webui !pip install -r requirements.txt !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/4bit/Llama-2-13b-chat-hf/resolve/main/model-00001-of-00003.safetensors -d /content/text-generation-webui/models/Llama-2-13b-chat-hf -o model-00001-of-00003.safetensors !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/4bit/Llama-2-13b-chat-hf/resolve/main/model-00002-of-00003.safetensors -d /content/text-generation-webui/models/Llama-2-13b-chat-hf -o model-00002-of-00003.safetensors !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/4bit/Llama-2-13b-chat-hf/resolve/main/model-00003-of-00003.safetensors -d /content/text-generation-webui/models/Llama-2-13b-chat-hf -o model-00003-of-00003.safetensors !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/4bit/Llama-2-13b-chat-hf/raw/main/model.safetensors.index.json -d /content/text-generation-webui/models/Llama-2-13b-chat-hf -o model.safetensors.index.json !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/4bit/Llama-2-13b-chat-hf/raw/main/special_tokens_map.json -d /content/text-generation-webui/models/Llama-2-13b-chat-hf -o special_tokens_map.json !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/4bit/Llama-2-13b-chat-hf/resolve/main/tokenizer.model -d /content/text-generation-webui/models/Llama-2-13b-chat-hf -o tokenizer.model !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/4bit/Llama-2-13b-chat-hf/raw/main/tokenizer_config.json -d /content/text-generation-webui/models/Llama-2-13b-chat-hf -o tokenizer_config.json !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/4bit/Llama-2-13b-chat-hf/raw/main/config.json -d /content/text-generation-webui/models/Llama-2-13b-chat-hf -o config.json !aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/4bit/Llama-2-13b-chat-hf/raw/main/generation_config.json -d /content/text-generation-webui/models/Llama-2-13b-chat-hf -o generation_config.json %cd /content/text-generation-webui !python server.py --share --chat --load-in-8bit --model /content/text-generation-webui/models/Llama-2-13b-chat-hf Alpaca Introduction : Alpaca

A team of researchers from Stanford University developed an open-source language model called Alpaca. It is based on Meta's large-scale language model Llama. The team used OpenAI's GPT API (text-davinci-003) to fine tune the Llama 7 billion (7B) parameters sized model. The goal of the team is to make AI available for everyone for free so that academicians can do further research without worrying about expensive hardwares to execute these memory-intensive algorithms. Although these open source models are not available for commercial use, small businesses can still utilize it for building their own chatbots.

How does Alpaca work

The Stanford team began their research with the smallest language model among Llama models, which was the Llama 7B model, and pre-trained it with 1 trillion tokens. They started with the 175 human-written instruction-output pairs from the self-instruct seed set. They then used OpenAI API to ask ChatGPT to generate more instructions using the seed set. It is to obtain roughly 52,000 sample conversations, which the team used to further fine-tune the Llama models using Hugging Face's training framework.

To read this article in full, please click hereThis post appeared first on ListenData
Categories: FLOSS Project Planets

Setting up a Toggleable Side Panel in KDE Plasma 6

Planet KDE - Sun, 2024-03-17 20:00

Since a brief tryst with Ubuntu Budgie Edition, I’ve dearly missed its Raven side-panel, a special panel on the side of the screen that can be opened and closed with a click. As someone who needs a clean, minimal desktop, the workflow is just too perfect — when you have two or three widgets that you use frequently, but not frequently enough that they warrant permanent homes on a main panel, just stuff them into a disappearing side-panel that can be called with a quick key-combination or by clicking on an icon; It’s a great way to keep things out of the way, but within reach, without having a permanently cluttered system tray that you might want to keep clear for things like email notifications.

My side panel, and the widget that launches it.

There are some drawbacks; this workflow isn’t well supported on KDE Plasma, so it’s a bit of a faff to set up, and only a few widgets will display nicely on a wide side-panel. For instance, it would be a dream to have the KDE weather widget automatically take advantage of the horizontal space and display the information that would usually be in its dropdown, but what you get instead is a giant icon, for now at least. I use my side-panel for my clipboard history and the media player widget, both of which play nicely with a side-panel. Another niggle I have with it is that, as far as I know, there’s no way to disable activation of the panel when your mouse pointer makes contact with the screen edge. This is a mild to moderate inconvenience when you’re working with applications that have toolbars on the sides of the window, like design applications often do.

For me, personally, the drawbacks aren’t so severe as to put me off of the workflow.

Creating and configuring the panel #

First, you’ll need to create a panel. To do this, right click on an empty section of your desktop, and select “Add Panel > Empty Panel.” When the panel appears, right click it and select “Enter Edit Mode.” Set up your panel however you like, but you will need to set “Visibility” to “Auto Hide” and may want to give it a width of at least 400px or so.

The panel settings configuration window. Setting up the script #

Now, if you wanted to show and hide your panel with a keyboard shortcut, you can set up a focus shortcut in the panel settings window and stop here. If, like me, you want to toggle your panel by clicking on an icon somewhere, we’re going to have to use a wee script, but don’t worry, it’s not as hard as it sounds and I’ll take you through it step by step.

Before we can put our script together, we’re going to need to know what the ID of our panel is. Open up KRunner with Alt+F2 or Alt+Space and run plasma-interactiveconsole. This will launch KDE’s Desktop Shell Scripting Console. In the console, type print(panelIds); and click “Execute.” Assuming you entered that in correctly, what you should see now in the output console beneath the text editor is a series of numbers — the ID numbers of our panels. Keep a note of these numbers.

Look at those IDs.

Clear the text editor and enter the following:

let panel = panelById(401); panel.hiding === "autohide" ? panel.hiding = "windowsgobelow" : panel.hiding = "autohide";

This will check if our panel is set to auto-hide; if it is, the script will set the panel to “windows go below” mode, otherwise it will set the panel to auto-hide.

Now to make use of those panel ID numbers. Which number corresponds to your new side-panel? While I can’t be sure, chances are it’s the last number on the list as we’ve just made the new panel a moment ago. So in the script above, where I have entered 401, enter the last number in your ID list and click “Execute.” At this point, if the ID number is correct, your panel should appear; click “Execute” once more to hide it.

Setting up the Scriptinator widget #

Alright, we’ve got our script ready, so we just need one more thing in place: a button or icon that we can click on to show and hide the panel. Fortunately, we can use a widget called “Scriptinator” to provide just this. Right click on an empty area of your desktop or a panel, click “Add Widgets,” and “Get New Widgets.”

Let’s get that widget.

From here, find and install Scriptinator. Once installed, simply drag it where you’d like it to live, either on your desktop, or on a panel. Once you’ve done that, right click on the widget and choose “Configure Scriptinator.” Here, enter the path of the icon you’d like to use in “Custom icon full path;” I used /usr/share/icons/breeze-dark/actions/22/sidebar-expand-right-symbolic.svg. In “OnClick Script,” enter the following:

qdbus org.kde.plasmashell /PlasmaShell evaluateScript ''

and between those single-quote marks, paste in the full script we put together in the Desktop Shell Scripting Console, like this:

qdbus org.kde.plasmashell /PlasmaShell evaluateScript 'let panel = panelById(401); panel.hiding === "autohide" ? panel.hiding = "windowsgobelow" : panel.hiding = "autohide";' The Scriptinator configuration window.

Set up a tooltip if you like, hit apply, and test out your toggle button.

Success! #

If you’ve done everything correctly, you should see your side-panel appear when you click the widget and disappear when you click a second time. You may need to restart to see your icon applied to the widget; if you don’t want to wait, you can drop the file path into “OnClick icon full path” in your Scriptinator configuration.

Categories: FLOSS Project Planets

Brett Cannon: State of WASI support for CPython: March 2024

Planet Python - Sun, 2024-03-17 18:43

The biggest update since June 2023 is WASI is now a tier 2 platform for CPython! This means that the main branch of CPython should never be broken more than 24 hours for WASI and that a release will be blocked if WASI support is broken. This only applies to Python 3.13 and later, although I have been trying to keep Python 3.11 and 3.12 working with WASI as well.

To help make this support easier, the devguide has build instructions for WASI. There is also now a WASI step in CI to help make things easier for core developers.

Starting in wasmtime 14, a new command line interface was introduced. All the relevant bits of code that call wasmtime have been updated to use the new CLI in Python 3.11, 3.12, and 3.13/main.

Lastly, 3.13/main and 3.12 now support WASI SDK 21 – which is the official name of the project – and 3.11 is one bug fix away in the test suite from also having support.

At this point I think CPython has caught up to what&aposs available in WASI 0.2 and wasi-libc via WASI SDK. The open issues are mostly feature requests or checking if assumptions related to what&aposs supported still hold.

I&aposm on parental leave at this point, so future WASI work from me is on hold until I return to work in June. Another side effect of me becoming a parent soon is I stepped down as the sponsor of Emscripten support in CPython. That means CPython 3.13 does not officially support Emscripten and probably starting in 3.14, I will be removing any code that complicates supporting WASI. The Pyodide project already knows about this and they don&apost expect it to be a major hindrance for them since they are already used to patching CPython source code.

Categories: FLOSS Project Planets

Kile 2.9.95 / 3.0 beta 4 released

Planet KDE - Sun, 2024-03-17 16:25

We have a release of Kile 2.9.95, also known as 3.0 beta 4! Earlier today, Michel Ludwig tagged the current Git master. This is the first beta release since October 2019. Beside the port to KDE Frameworks 6 and Qt 6, it provides a couple of new features and bug fixes.

New features
  • Port to KDE Frameworks 6 & Qt 6 (Port by Carl Schwan)
  • Enable high-dpi support
  • Provide option to hide menu bar (Patch by Daniel Fichtner, #372295)
  • Configurable global default setting for the LivePreview engines (Patch by Florian Zumkeller-Quast, #450332)
  • Remove offline copy of "LaTeX2e: An unofficial reference manual", use online version instead (Patch by myself, Christoph Grüninger, Issue #7)
Fixed bugs
  • Kile crashes on selecting "Browse" or "Zoom" for document preview (Patch by Carl Schwan, #465547, #476207, #467435, #452618, #429452)
  • Kile crashes when generating new document (Patch by Carl Schwan, #436837)
  • Ensure \end{env} is inserted in the right place even when the user uses tabs for indentation (Patch by Kishore Gopalakrishnan, #322654)
  • Avoid saving console commands in bash history (Patch by Alessio Bonfiglio, #391537, #453935)
  • Don't crash when deleting templates (#413506)
  • Avoid crashing when closing a document that is being parsed (#404164)

Thanks to all the contributors. They fixed bugs, wrote documentation, modernized the code, and in general took care of Kile.

Enjoy the latest Kile release!

Categories: FLOSS Project Planets

Thomas Koch: Minimal overhead VMs with Nix and MicroVM

Planet Debian - Sun, 2024-03-17 06:13
Posted on March 17, 2024 Tags: debian, free software, nix

Joachim Breitner wrote about a Convenient sandboxed development environment and thus reminded me to blog about MicroVM. I’ve toyed around with it a little but not yet seriously used it as I’m currently not coding.

MicroVM is a nix based project to configure and run minimal VMs. It can mount and thus reuse the hosts nix store inside the VM and thus has a very small disk footprint. I use MicroVM on a debian system using the nix package manager.

The MicroVM author uses the project to host production services. Otherwise I consider it also a nice way to learn about NixOS after having started with the nix package manager and before making the big step to NixOS as my main system.

The guests root filesystem is a tmpdir, so one must explicitly define folders that should be mounted from the host and thus be persistent across VM reboots.

I defined the VM as a nix flake since this is how I started from the MicroVM projects example:

{ description = "Haskell dev MicroVM"; inputs.impermanence.url = "github:nix-community/impermanence"; inputs.microvm.url = "github:astro/microvm.nix"; inputs.microvm.inputs.nixpkgs.follows = "nixpkgs"; outputs = { self, impermanence, microvm, nixpkgs }: let persistencePath = "/persistent"; system = "x86_64-linux"; user = "thk"; vmname = "haskell"; nixosConfiguration = nixpkgs.lib.nixosSystem { inherit system; modules = [ microvm.nixosModules.microvm impermanence.nixosModules.impermanence ({pkgs, ... }: { environment.persistence.${persistencePath} = { hideMounts = true; users.${user} = { directories = [ "git" ".stack" ]; }; }; environment.sessionVariables = { TERM = "screen-256color"; }; environment.systemPackages = with pkgs; [ ghc git (haskell-language-server.override { supportedGhcVersions = [ "94" ]; }) htop stack tmux tree vcsh zsh ]; fileSystems.${persistencePath}.neededForBoot = nixpkgs.lib.mkForce true; microvm = { forwardPorts = [ { from = "host"; host.port = 2222; guest.port = 22; } { from = "guest"; host.port = 5432; guest.port = 5432; } # postgresql ]; hypervisor = "qemu"; interfaces = [ { type = "user"; id = "usernet"; mac = "00:00:00:00:00:02"; } ]; mem = 4096; shares = [ { # use "virtiofs" for MicroVMs that are started by systemd proto = "9p"; tag = "ro-store"; # a host's /nix/store will be picked up so that no # squashfs/erofs will be built for it. source = "/nix/store"; mountPoint = "/nix/.ro-store"; } { proto = "virtiofs"; tag = "persistent"; source = "~/.local/share/microvm/vms/${vmname}/persistent"; mountPoint = persistencePath; socket = "/run/user/1000/microvm-${vmname}-persistent"; } ]; socket = "/run/user/1000/microvm-control.socket"; vcpu = 3; volumes = []; writableStoreOverlay = "/nix/.rwstore"; }; networking.hostName = vmname; nix.enable = true; nix.nixPath = ["nixpkgs=${builtins.storePath <nixpkgs>}"]; nix.settings = { extra-experimental-features = ["nix-command" "flakes"]; trusted-users = [user]; }; security.sudo = { enable = true; wheelNeedsPassword = false; }; services.getty.autologinUser = user; services.openssh = { enable = true; }; system.stateVersion = "24.11"; systemd.services.loadnixdb = { description = "import hosts nix database"; path = [pkgs.nix]; wantedBy = ["multi-user.target"]; requires = ["nix-daemon.service"]; script = "cat ${persistencePath}/nix-store-db-dump|nix-store --load-db"; }; time.timeZone = nixpkgs.lib.mkDefault "Europe/Berlin"; users.users.${user} = { extraGroups = [ "wheel" "video" ]; group = "user"; isNormalUser = true; openssh.authorizedKeys.keys = [ "ssh-rsa REDACTED" ]; password = ""; }; users.users.root.password = ""; users.groups.user = {}; }) ]; }; in { packages.${system}.default = nixosConfiguration.config.microvm.declaredRunner; }; }

I start the microVM with a templated systemd user service:

[Unit] Description=MicroVM for Haskell development Requires=microvm-virtiofsd-persistent@.service After=microvm-virtiofsd-persistent@.service AssertFileNotEmpty=%h/.local/share/microvm/vms/%i/flake/flake.nix [Service] Type=forking ExecStartPre=/usr/bin/sh -c "[ /nix/var/nix/db/db.sqlite -ot %h/.local/share/microvm/nix-store-db-dump ] || nix-store --dump-db >%h/.local/share/microvm/nix-store-db-dump" ExecStartPre=ln -f -t %h/.local/share/microvm/vms/%i/persistent/ %h/.local/share/microvm/nix-store-db-dump ExecStartPre=-%h/.local/state/nix/profile/bin/tmux new -s microvm -d ExecStart=%h/.local/state/nix/profile/bin/tmux new-window -t microvm: -n "%i" "exec %h/.local/state/nix/profile/bin/nix run --impure %h/.local/share/microvm/vms/%i/flake"

The above service definition creates a dump of the hosts nix store db so that it can be imported in the guest. This is necessary so that the guest can actually use what is available in /nix/store. There is an effort for an overlayed nix store that would be preferable to this hack.

Finally the microvm is started inside a tmux session named “microvm”. This way I can use the VM with SSH or through the console and also access the qemu console.

And for completeness the virtiofsd service:

[Unit] Description=serve host persistent folder for dev VM AssertPathIsDirectory=%h/.local/share/microvm/vms/%i/persistent [Service] ExecStart=%h/.local/state/nix/profile/bin/virtiofsd \ --socket-path=${XDG_RUNTIME_DIR}/microvm-%i-persistent \ --shared-dir=%h/.local/share/microvm/vms/%i/persistent \ --gid-map :995:%G:1: \ --uid-map :1000:%U:1:
Categories: FLOSS Project Planets

digiKam 8.3.0 is released

Planet KDE - Sat, 2024-03-16 20:00
Dear digiKam fans and users, After four months of active maintenance and long bugs triage, the digiKam team is proud to present version 8.3.0 of its open source digital photo manager. This version arrives with the internal RAW decoder Libraw updated to the rolling-release snapshot 2024-02-02. Long time bugs present in older versions have been fixed and we spare a lot of time to contact users to validate changes in pre-release to confirm fixes before deploying the program in production.
Categories: FLOSS Project Planets

digiKam 8.3.0 is released

Planet KDE - Sat, 2024-03-16 20:00
Dear digiKam fans and users, After four months of active maintenance and long bugs triage, the digiKam team is proud to present version 8.3.0 of its open source digital photo manager. This version arrives with the internal RAW decoder Libraw updated to the rolling-release snapshot 2024-02-02. Long time bugs present in older versions have been fixed and we spare a lot of time to contact users to validate changes in pre-release to confirm fixes before deploying the program in production.
Categories: FLOSS Project Planets

Breeze Icon Update March 16, 2024

Planet KDE - Sat, 2024-03-16 11:11

Hi all,

I am back with another update for adapting icons to the 24px grid and doing some larger edits. This week, I worked on 3 rows, which is great, and was able to hit just past the 50% mark, it seems

Here is a video with the changes:

Categories: FLOSS Project Planets

Pages