FLOSS Project Planets

It is a really impressive Season of KDE this year -- we have selected a total of 18 projects!

To all of our new contributors, thank you very much for your drive to gain experience and improve FOSS at the same time. The mentors and mentorship team, as well as the broader KDE community, are sure you'll create something fantastic over the next 10 weeks.

Figure :

Group photo of several members of the KDE community at Akademy 2022 in Barcelona, Spain. The KDE community enthusiastically welcomes all new contributors! (Image: CC BY 4.0)

Translation Projects

We have 2 projects to translate multiple apps into Hindi:

• Asish Kumar will work on translating KTorrent, KDE connect, Cantor, and Merkuro
• Akash Kumar will work on translating Tellico and Merkuro

They will be mentored by Benson Muite and Raghavendra Kamath.

Kdenlive

We have 2 projects:

• Ajay Chauhan will work on adding multi-format rendering to Kdenlive
• Iceneko will enable editing of keyframe curves and advanced keyframe types in Kdenlive

They will be mentored by Julius Künzel and Jean-Baptiste Mardelle.

KDE Eco / Accessibility

We have 5 projects:

• Athul Raj Kollareth will work on improving KdeEcoTest tool using kwin scripting API and uinput
• Amartya Chakraborty will work on adding support for Windows in KdeEcoTest
• Sarthak Negi will work on Testing and development of KEcoLab
• Aakarsh MJ will work on Testing and development of KEcoLab
• Pradyot Ranjan will work on Selenium automation of energy measurements / accessibility tests for KDE applications

They will be mentored by Karanjot Singh, Emmanuel Charruau, Nitin Tejuja, Rishi Kumar, and Joseph P. De Veaugh-Geiss.

Cantor / LabPlot

8 projects have been accepted for these two applications:

• Dhairya Majmudar and Saksham Gupta will work together on extending the embedded documentation for supported Computer Algebra Systems Project
• Mahfuza Humayra Mohona will work on enhancing Cantor's and LabPlot's documentation by adding QtHelp Conversion and User Accessibility
• Anshul Dalal will work on integrating KTextEditor into Cantor
• Krish Jain will work on enhancing Cantor with package management capabilities
• Israel Galadima will work on downloading/importing of data sets from kaggle.com
• Mihir Kumar Singh will work on integrating TSDB support to LabPlot
• Raphael Wirth will work on MCAP Support for LabPlot

They will be mentored by Alexander Semke.

KWin

1 project has been selected:

• Anurag Thakur will work on moving brightness control to KWin and adding per-screen brightness control

They will be mentored by Xaver Hugl.

The 18 contributors will start developing on 17 January under the guidance of their mentors. SOK will end 31 March. Check out the full SOK 2024 timeline.

Let's warmly welcome all of them and make sure the beginning of their journey with KDE is successful!

You will be able to follow the progress via blog posts published on KDE's planet.

We are excited to announce the creation of an EU-funded consortium with the central objective to launch GNU Taler as a privacy-preserving payment system across Europe. You can find more information on the consortium page.

Introduction

It’s been a while since I’ve posted anything on the blog. One of the primary reasons for the hiatus is that I have been using python and pandas but not to do anything very new or different.

In order to shake things up and hopefully get back into the blog a bit, I’m going to write about polars. This article assumes you know how to use pandas and are interested in determining if polars can fit into your workflow. I will cover some basic polars concepts that should get you started on your journey.

Along the way I will point out some of the things I liked and some of the differences that that might limit your usage of polars if you’re coming from pandas.

Ultimately, I do like polars and what it is trying to do. I’m not ready to throw out all my pandas code and move over to polars. However, I can see where polars could fit into my toolkit and provide some performance and capability that is missing from pandas.

As you evaluate the choice for yourself, it is important to try other frameworks and tools and evaluate them on their merits as they apply to your needs. Even if you decide polars doesn’t meet your needs it is good to evaluate options and learn along the way. Hopefully this article will get you started down that path.

Polars

As mentioned above, pandas has been the data analysis tool for python for the past few years. Wes McKinney started the initial work on pandas in 2008 and the 1.0 release was in January 2020. Pandas has been around a long time and will continue to be.

While pandas is great, it has it’s warts. Wes McKinney wrote about several of these challenges. There are many other criticisms online but most will boil down to two items: performance and awkward/complex API.

Polars was initially developed by Richie Vink to solve these issues. His 2021 blog post does a thorough job of laying out metrics to back up his claims on the performance improvements and underlying design that leads to these benefit with polars.

The user guide concisely lays out the polars philosophy:

The goal of Polars is to provide a lightning fast DataFrame library that:

• Utilizes all available cores on your machine.
• Optimizes queries to reduce unneeded work/memory allocations.
• Handles datasets much larger than your available RAM.
• Has an API that is consistent and predictable.
• Has a strict schema (data-types should be known before running the query).

Polars is written in Rust which gives it C/C++ performance and allows it to fully control performance critical parts in a query engine.

As such Polars goes to great lengths to:

• Reduce redundant copies.
• Traverse memory cache efficiently.
• Minimize contention in parallelism.
• Process data in chunks.
• Reuse memory allocations.

Clearly performance is an important goal in the development of polars and key reason why you might consider using polars.

This article won’t discuss performance but will focus on the polars API. The main reason is that for the type of work I do, the data easily fits in RAM on a business-class laptop. The data will fit in Excel but it is slow and inefficient on a standard computer. I rarely find myself waiting on pandas once I have read in the data and have done basic data pre-processing.

Of course performance matters but it’s not everything. If you’re trying to make a choice between pandas, polars or other tools don’t make a choice based on general notions of “performance improvement” but based on what works for your specific needs.

Getting started

For this article, I’ll be using data from an earlier post which you can find on github.

I would recommend following the latest polar installation instructions in the user guide .

I chose to install polars with all of the dependencies:

python -m pip install polars[all]

Once installed, reading the downloaded Excel file is straightforward:

import polars as pl df = pl.read_excel( source="2018_Sales_Total_v2.xlsx", schema_overrides={"date": pl.Datetime} )

When I read this specific file, I found that the date column did not come through as a DateTime type so I used the scheme_override argument to make sure the data was properly typed.

Since data typing is so important, here’s one quick way to check on it:

df.schema OrderedDict([('account number', Int64), ('name', Utf8), ('sku', Utf8), ('quantity', Int64), ('unit price', Float64), ('ext price', Float64), ('date', Datetime(time_unit='us', time_zone=None))])

A lot of the standard pandas commands such as head , tail , describe work as expected with a little extra output sprinkled in:

df.head() df.describe()

The polars output has a couple of notable features:

• The shape is included which is useful to make sure you’re not dropping rows or columns inadvertently
• Underneath each column name is a data type which is another useful reminder
• There are no index numbers
• The string columns include ” ” around the values

Overall, I like this output and do find it useful for analyzing the data and making sure the data is stored in the way I expect.

Basic concepts - selecting and filtering rows and columns

Polars introduces the concept of Expressions to help you work with your data. There are four main expressions you need to understand when working with data in polars:

• select to choose the subset of columns you want to work with
• filter to choose the subset of rows you want to work with
• with_columns to create new columns
• group_by to group data together

Choosing or reordering columns is straightforward with select()

df.select(pl.col("name", "quantity", "sku"))

The pl.col() code is used to create column expressions. You will want to use this any time you want to specify one or more columns for an action. There are shortcuts where you can use data without specifying pl.col() but I’m choosing to show the recommended way.

Filtering is a similar process (note the use of pl.col() again):

df.filter(pl.col("quantity") > 50)

Coming from pandas, I found selecting columns and filtering rows to be intuitive.

Basic concepts - adding columns

The next expression, with_columns , takes a little more getting used to. The easiest way to think about it is that any time you want to add a new column to your data, you need to use with_columns .

To illustrate, I will add a month name column which will also show how to work with date and strings.

df.with_columns((pl.col("date").dt.strftime("%b").alias("month_name")))

This command does a couple of things to create a new column:

• Select the date column
• Access the underlying date with dt and convert it to the 3 character month name using strftime
• Name the newly created column month_name using the alias function

As a brief aside, I like using alias to rename columns. As I played with polars, this made a lot of sense to me.

Here’s another example to drive the point home.

Let’s say we want to understand how much any one product order contributes to the total percentage unit volume for the year:

df.with_columns( (pl.col("quantity") / pl.col("quantity").sum()).alias("pct_total") )

In this example we divide the line item quantity by the total quantity pl.col("quantity").sum() and label it as pct_total .

You may have noticed that the previous month_name column is not there. That’s because none of the operations we have done are in-place. If we want to persist a new column, we need to assign it to a new variable. I will do so in a moment.

I briefly mentioned working with strings but here’s another example.

Let’s say that any of the sku data with an “S” at the front is a special product and we want to indicate that for each item. We use str in a way very similar to the pandas str accessor.

df.with_columns(pl.col("sku").str.starts_with("S").alias("special"))

Polars has a useful function when then otherwise which can replace pandas mask or np.where

Let’s say we want to create a column that indicates a special or includes the original sku if it’s not a special product.

df.with_columns( pl.when(pl.col("sku").str.starts_with("S")) .then(pl.lit("Special")) .otherwise(pl.col("sku")) .alias("sales_status") )

Which yields:

This is somewhat analogous to an if-then-else statement in python. I personally like this syntax because I alway struggle to use pandas equivalents.

This example also introduces pl.lit() which we use to assign a literal value to the columns.

Basic concepts - grouping data

The pandas groupby and polars group_by functional similarly but the key difference is that polars does not have the concept of an index or multi-index.

There are pros and cons to this approach which I will briefly touch on later in this article.

Here’s a simple polars group_by example to total the unit amount by sku by customer.

df.group_by("name", "sku").agg(pl.col("quantity").sum().alias("qty-total"))

The syntax is similar to pandas groupby with agg dictionary approach I have mentioned before. You will notice that we continue to use pl.col() to reference our column of data and then alias() to assign a custom name.

The other big change here is that the data does not have a multi-index, the result is roughly the same as using as_index=False with a pandas groupby. The benefit of this approach is that it is easy to work with this data without flattening or resetting your data.

The downside is that you can not use unstack and stack to make the data wider or narrower as needed.

When working with date/time data, you can group data similar to the pandas grouper function by using group_by_dynamic :

df.sort(by="date").group_by_dynamic("date", every="1mo").agg( pl.col("quantity").sum().alias("qty-total-month") )

There are a couple items to note:

• Polars asks that you sort the data by column before doing the group_by_dynamic
• The every argument allows you to specify what date/time level to aggregate to

To expand on this example, what if we wanted to show the month name and year, instead of the date time? We can chain together the group_by_dynamic and add a new column by using with_columns

df.sort(by="date").group_by_dynamic("date", every="1mo").agg( pl.col("quantity").sum().alias("qty-total-month") ).with_columns(pl.col("date").dt.strftime("%b-%Y").alias("month_name")).select( pl.col("month_name", "qty-total-month") )

This example starts to show the API expressiveness of polars. Once you understand the basic concepts, you can chain them together in a way that is generally more straightforward than doing so with pandas.

To summarize this example:

• Grouped the data by month
• Totaled the quantity and assigned the column name to qty-total-month
• Change the date label to be more readable and assigned the name month_name
• Then down-selected to show the two columns I wanted to focus on

Chaining expressions

We have touched on chaining expressions but I wanted to give one full example below to act as a reference.

Combining multiple expressions is available in pandas but it’s not required. This post from Tom Augspurger shows a nice example of how to use different pandas functions to chain operations together. This is also a common topic that Matt Harrison (@__mharrison__) discusses.

Chaining expressions together is a first class citizen in polars so it is intuitive and an essential part of working with polars.

Here is an example combining several concepts we showed earlier in the article:

df_month = df.with_columns( (pl.col("date").dt.month().alias("month")), (pl.col("date").dt.strftime("%b").alias("month_name")), (pl.col("quantity") / pl.col("quantity").sum()).alias("pct_total"), ( pl.when(pl.col("sku").str.starts_with("S")) .then(pl.lit("Special")) .otherwise(pl.col("sku")) .alias("sales_status") ), ).select( pl.col( "name", "quantity", "sku", "month", "month_name", "sales_status", "pct_total" ) ) df_month

I made this graphic to show how the pieces of code interact with each other:

The image is small on the blog but if you open it in a new window, it should be more legible.

It may take a little time to wrap your head around this approach to programming. But the results should pay off in more maintainable and performant code.

Additional notes

As you work with pandas and polars there are convenience functions for moving back and forth between the two. Here’s an example of creating a pandas dataframe from polars:

df.with_columns( pl.when(pl.col("sku").str.starts_with("S")) .then(pl.lit("Special")) .otherwise(pl.lit("Standard")) .alias("sales_status") ).to_pandas()

Having this capability means you can gradually start to use polars and go back to pandas if there are activities you need in polars that don’t quite work as expected.

If you need to work the other way, you can convert a pandas dataframe to a polars one using from_pandas()

Finally, one other item I noticed when working with polars is that there are some nice convenience features when saving your polars dataframe to Excel. By default the dataframe is stored in a table and you can make a lot of changes to the output by tweaking the parameters to the write_excel() . I recommend reviewing the official API docs for the details.

To give you a quick flavor, here is an example of some simple configuration:

df.group_by("name", "sku").agg(pl.col("quantity").sum().alias("qty-total")).write_excel( "sample.xlsx", table_style={ "style": "Table Style Medium 2", }, autofit=True, sheet_zoom=150, )

There are a lot of configuration options available but I generally find this default output easier to work with thank pandas.

Additional resources

I have only touched on the bare minimum of capabilities in polars. If there is interest, I’ll write some more. In the meantime, I recommend you check out the following resources:

• Official user guide
• Modern Polars from Kevin Heavey

The Modern Polars resource goes into a much more detailed look at how to work with pandas and polars with code examples side by side. It’s a top notch resource. You should definitely check it out.

Conclusion

Pandas has been the go-to data analysis tool in the python ecosystem for over a decade. Over that time it has grown and evolved and the surrounding ecosystem has changed. As a result some of the core parts of pandas might be showing their age.

Polars brings a new approach to working with data. It is still in the early phases of its development but I am impressed with how far it has come in the first few years. As of this writing, polars is moving to a 1.0 release. This milestone means that the there will be fewer breaking changes going forward and the API will stabilize. It’s a good time to jump in and learn more for yourself.

I’ve only spent a few hours with polars so I’m still developing my long-term view on where it fits. Here are a few of my initial observations:

Polars pros:

• Performant design from the ground up which maximizes modern hardware and minimizes memory usage
• Clean, consistent and expressive API for chaining methods
• Not having indices simplifies many cases
• Useful improvement in displaying output, saving excel files, etc.
• Good API and user documentation
• No built in plotting library.

Regarding the plotting functionality, I think it’s better to use the available ones than try to include in polars. There is a plot namespace in polars but it defers to other libraries to do the plotting.

Polars cons:

• Still newer code base with breaking API changes
• Not as much third party documentation
• Not as seamlessly integrated with other libraries (although it is improving)
• Some pandas functions like stacking and unstacking are not as mature in polars

Pandas pros:

• Tried and tested code base that has been improved significantly over the years
• The multi-index support provides helpful shortcuts for re-shaping data
• Strong integrations with the rest of the python data ecosystem
• Good official documentation as well as lots of 3rd party sources for tips and tricks

Pandas cons:

• Some cruft in the API design. There’s more than one way to do things in many cases.
• Performance for large data sets can get bogged down

This is not necessarily exhaustive but I think hits the highlights. At the end of the day, diversity in tools and approaches is helpful. I intend to continue evaluating the integration of polars into my analysis - especially in cases where performance becomes an issue or the pandas code gets too be too messy. However, I don’t think pandas is going away any time soon and I continue to be excited about pandas evolution.

I hope this article helps you get started. As always, if you have experiences, thoughts or comments on the article, let me know below.

I needed to run random subsets of my test suite to narrow down the cause of some mysterious behavior. I didn’t find an existing tool that worked the way I wanted to, so I cobbled something together.

I wanted to run 10 random tests (out of 1368), and keep choosing randomly until I saw the bad behavior. Once I had a selection of 10, I wanted to be able to whittle it down to try to reduce it further.

I tried a few different approaches, and here’s what I came up with, two tools in the coverage.py repo that combine to do what I want:

• A pytest plugin (select_plugin.py) that lets me run a command to output the names of the exact tests I want to run,
• A command-line tool (pick.py) to select random lines of text from a file. For convenience, blank or commented-out lines are ignored.

More details are in the comment at the top of pick.py, but here’s a quick example:

1. Get all the test names in tests.txt. These are pytest “node” specifications: pytest --collect-only | grep :: > tests.txt
   
2. Now tests.txt has a line per test node. Some are straightforward: tests/test_cmdline.py::CmdLineStdoutTest::test_version
   tests/test_html.py::HtmlDeltaTest::test_file_becomes_100
   tests/test_report_common.py::ReportMapsPathsTest::test_map_paths_during_html_report
   but with parameterization they can be complicated: tests/test_files.py::test_invalid_globs[bar/***/foo.py-***]
   tests/test_files.py::FilesTest::test_source_exists[a/b/c/foo.py-a/b/c/bar.py-False]
   tests/test_config.py::ConfigTest::test_toml_parse_errors[[tool.coverage.run]\nconcurrency="foo"-not a list]
   
3. Run a random bunch of 10 tests: pytest --select-cmd="python pick.py sample 10 < tests.txt"
   We’re using --select-cmd to specify the shell command that will output the names of tests. Our command uses pick.py to select 10 random lines from tests.txt.
4. Run many random bunches of 10, announcing the seed each time: for seed in $(seq 1 100); do
       echo seed=$seed
       pytest --select-cmd="python pick.py sample 10 $seed < tests.txt"
   done
   
5. Once you find a seed that produces the small batch you want, save that batch: python pick.py sample 10 17 < tests.txt > bad.txt
   
6. Now you can run that bad batch repeatedly: pytest --select-cmd="cat bad.txt"
   
7. To reduce the bad batch, comment out lines in bad.txt with a hash character, and the tests will be excluded. Keep editing until you find the small set of tests you want.

I like that this works and I understand it. I like that it’s based on the bedrock of text files and shell commands. I like that there’s room for different behavior in the future by adding to how pick.py works. For example, it doesn’t do any bisecting now, but it could be adapted to it.

As usual, there might be a better way to do this, but this works for me.

LangChain is a toolkit for building apps powered by large language models like GPT-3. Think of it as Legos for AI apps – it simplifies connecting these powerful models to build things like text generators, chatbots, and question answerers. It was created by an open-source community and lets developers quickly prototype and deploy AI-powered apps. […]

The post Understanding LangChain: A Guide for Beginners appeared first on TechBeamers.

KDE e.V. is looking for you as an employee for a new project to promote environmentally-sustainable software and long-term hardware use. You will be the person who makes sure the administration of the project and the organization of the campaigns and workshops are on track and successful. Please see the full job listing for more details about this opportunity.

Applications will be accepted until mid-february, and we look forward to your application.

GNU cpio version 2.15 is available for download. This is a bug-fixing release.  Short summary of changes:

• Fix the operation of --no-absolute-filenames --make-directories.
• Restore access and modification times of symlinks in copy-in and copy-pass modes.

Starting with version 2.33, the GNU C library (glibc) grew the capability to search for shared libraries using additional paths, based on the hardware capabilities of the machine running the code. This was a great boon for x86_64, which was first released in 2003, and has seen many changes in the capabilities of the hardware since then. While it is extremely common for Linux distributions to compile for a baseline which encompasses all of an architecture, there is performance being left on the table by targeting such an old specification and not one of the newer revisions.

One option used internally in glibc and in some other performance-critical libraries is indirect functions, or IFUNCs (see also here) The loader, ld.so uses them to pick function implementations optimized for the available CPU at load time. GCC's (functional multi-versioning (FMV))[https://gcc.gnu.org/wiki/FunctionMultiVersioning] generates several optimized versions of functions, using the IFUNC mechanism so the approprate one is selected at load time. These are strategies which most performance-sensitive libraries do, but not all of them.

With the --tune using package transformation option, Guix implements so-called package multi-versioning, which creates package variants using compiler flags set to use optimizations targeted for a specific CPU.

Finally - and we're getting to the central topic of this post! - glibc since version 2.33 supports another approach: ld.so would search not just the /lib folder, but also the glibc-hwcaps folders, which for x86_64 included /lib/glibc-hwcaps/x86-64-v2, /lib/glibc-hwcaps/x86-64-v3 and /lib/glibc-hwcaps/x86-64-v4, corresponding to the psABI micro-architectures of the x86_64 architecture. This means that if a library was compiled against the baseline of the architecture then it should be installed in /lib, but if it were compiled a second time, this time using (depending on the build instructions) -march=x86-64-v2, then the libraries could be installed in /lib/glibc-hwcaps/x86-64-v2 and then glibc, using ld.so, would choose the correct library at runtime.

These micro-architectures aren't a perfect match for the different hardware available, it is often the case that a particular CPU would satisfy the requirements of one tier and part of the next but would therefore only be able to use the optimizations provided by the first tier and not by the added features that the CPU also supports.

This of course shouldn't be a problem in Guix; it's possible, and even encouraged, to adjust packages to be more useful for one's needs. The problem comes from the search paths: ld.so will only search for the glibc-hwcaps directory if it has already found the base library in the preceding /lib directory. This isn't a problem for distributions following the File System Hierarchy (FHS), but for Guix we will need to ensure that all the different versions of the library will be in the same output.

With a little bit of planning this turns out to not be as hard as it sounds. Lets take for example, the GNU Scientific Library, gsl, a math library which helps with all sorts of numerical analysis. First we create a procedure to generate our 3 additional packages, corresponding to the psABIs that are searched for in the glibc-hwcaps directory.

(define (gsl-hwabi psabi) (package/inherit gsl (name (string-append "gsl-" psabi)) (arguments (substitute-keyword-arguments (package-arguments gsl) ((#:make-flags flags #~'()) #~(append (list (string-append "CFLAGS=-march=" #$psabi) (string-append "CXXFLAGS=-march=" #$psabi)) #$flags)) ((#:configure-flags flags #~'()) #~(append (list (string-append "--libdir=" #$output "/lib/glibc-hwcaps/" #$psabi)) #$flags)) ;; The building machine can't necessarily run the code produced. ((#:tests? _ #t) #f) ((#:phases phases #~%standard-phases) #~(modify-phases #$phases (add-after 'install 'remove-extra-files (lambda _ (for-each (lambda (dir) (delete-file-recursively (string-append #$output dir))) (list (string-append "/lib/glibc-hwcaps/" #$psabi "/pkgconfig") "/bin" "/include" "/share")))))))) (supported-systems '("x86_64-linux" "powerpc64le-linux")) (properties `((hidden? . #t) (tunable? . #f)))))

We remove some directories and any binaries since we only want the libraries produced from the package; we want to use the headers and any other bits from the main package. We then combine all of the pieces together to produce a package which can take advantage of the hardware on which it is run:

(define-public gsl-hwcaps (package/inherit gsl (name "gsl-hwcaps") (arguments (substitute-keyword-arguments (package-arguments gsl) ((#:phases phases #~%standard-phases) #~(modify-phases #$phases (add-after 'install 'install-optimized-libraries (lambda* (#:key inputs outputs #:allow-other-keys) (let ((hwcaps "/lib/glibc-hwcaps/")) (for-each (lambda (psabi) (copy-recursively (string-append (assoc-ref inputs (string-append "gsl-" psabi)) hwcaps psabi) (string-append #$output hwcaps psabi)) '("x86-64-v2" "x86-64-v3" "x86-64-v4")))))))) (native-inputs (modify-inputs (package-native-inputs gsl) (append (gsl-hwabi "x86-64-v2") (gsl-hwabi "x86-64-v3") (gsl-hwabi "x86-64-v4")))) (supported-systems '("x86_64-linux")) (properties `((tunable? . #f)))))

In this case the size of the final package is increased by about 13 MiB, from 5.5 MiB to 18 MiB. It is up to you if the speed-up from providing an optimized library is worth the size trade-off.

To use this package as a replacement build input in a package package-input-rewriting/spec is a handy tool:

(define use-glibc-hwcaps (package-input-rewriting/spec ;; Replace some packages with ones built targeting custom packages build ;; with glibc-hwcaps support. `(("gsl" . ,(const gsl-hwcaps))))) (define-public inkscape-with-hwcaps (package (inherit (use-glibc-hwcaps inkscape)) (name "inkscape-with-hwcaps")))

Of the Guix supported architectures, x86_64-linux and powerpc64le-linux can both benefit from this new capability.

Through the magic of newer versions of GCC and LLVM it is safe to use these libraries in place of the standard libraries while compiling packages; these compilers know about the glibc-hwcap directories and will purposefully link against the base library during build time, with glibc's ld.so choosing the optimized library at runtime.

One possible use case for these libraries is crating guix packs of packages to run on other systems. By substituting these libraries it becomes possible to crate a guix pack which will have better performance than a standard package used in a guix pack. This works even when the included libraries don't make use of the IFUNCs from glibc or functional multi-versioning from GCC. Providing optimized yet portable pre-compiled binaries is a great way to take advantage of this feature.

About GNU Guix

GNU Guix is a transactional package manager and an advanced distribution of the GNU system that respects user freedom. Guix can be used on top of any system running the Hurd or the Linux kernel, or it can be used as a standalone operating system distribution for i686, x86_64, ARMv7, AArch64 and POWER9 machines.

In addition to standard package management features, Guix supports transactional upgrades and roll-backs, unprivileged package management, per-user profiles, and garbage collection. When used as a standalone GNU/Linux distribution, Guix offers a declarative, stateless approach to operating system configuration management. Guix is highly customizable and hackable through Guile programming interfaces and extensions to the Scheme language.

Está aberta a inscrição de participantes e a chamada de atividades para a MiniDebConf Belo Horizonte 2024 e para o FLISOL - Festival Latino-americano de Instalação de Software Livre.

Veja abaixo algumas informações importantes:

Data e local da MiniDebConf e do FLISOL

A MiniDebConf acontecerá de 27 a 30 de abril no Campus Pampulha da UFMG - Universidade Federal de Minas Gerais.

No dia 27 (sábado) também realizaremos uma edição do FLISOL - Festival Latino-americano de Instalação de Software Livre, evento que acontece no mesmo dia em várias cidades da América Latina.

Enquanto a MiniDebConf terá atividades focados no Debian, o FLISOL terá atividades gerais sobre Software Livre e temas relacionados como linguagem de programação, CMS, administração de redes e sistemas, filosofia, liberdade, licenças, etc.

Inscrição gratuita e oferta de bolsas

Você já pode realizar a sua inscrição gratuita para a MiniDebConf Belo Horizonte 2024.

A MiniDebConf é um evento aberto a todas as pessoas, independente do seu nível de conhecimento sobre Debian. O mais importante será reunir a comunidade para celebrar um dos maiores projeto de Software Livre no mundo, por isso queremos receber desde usuários(as) inexperientes que estão iniciando o seu contato com o Debian até Desenvolvedores(as) oficiais do projeto. Ou seja, estão todos(as) convidados(as)!

Este ano estamos ofertando bolsas de hospedagem e passagens para viabilizar a vinda de pessoas de outras cidades que contribuem para o Projeto Debian. Contribuidores(as) não oficiais, DMs e DDs podem solicitar as bolsas usando o formulário de inscrição.

Também estamos ofertando bolsas de alimentação para todos(as) os(as) participantes, mesmo não contribuidores(as), e pessoas que moram na região de BH.

Os recursos financeiros são bastante limitados, mas tentaremos atender o máximo de pedidos.

Se você pretende pedir alguma dessas bolsas, acesse este link e veja mais informações antes de realizar a sua inscrição:

A inscrição (sem bolsas) poderá ser feita até a data do evento, mas temos uma data limite para o pedido de bolsas de hospedagem e passagens, por isso fique atento(a) ao prazo final: até 18 de fevereiro.

Como estamos usando mesmo formulário para os dois eventos, a inscrição será válida tanto para a MiniDebConf quanto para o FLISOL.

Para se inscrever, acesse o site, vá em Criar conta. Criei a sua conta (preferencialmente usando o Salsa) e acesse o seu perfil. Lá você verá o botão de Se inscrever.

https://bh.mini.debconf.org

Chamada de atividades

Também está aberta a chamada de atividades tanto para MiniDebConf quanto para o FLISOL.

Para mais informações, acesse este link.

Fique atento ao prazo final para enviar sua proposta de atividade: até 18 de fevereiro.

Contato

Qualquer dúvida, mande um email para contato@debianbrasil.org.br

Organização

In this tutorial, we will learn about Python lists (creating lists, changing list items, removing items and other list operations) with the help of examples.

Lithuania and Latvia

Caused by a small discussion about how it is difficult to get from Berlin to Riga by train, and in direct consequence a quick look at how the official app for trains in Latvia finds its connections, I added support for it in KDE Itinerary. KDE Itinerary is KDE’s travel planning app.

After I understood how it works, adding support for new data sources seemed pretty doable, so I directly moved on to do the same for trains in Lithuania as well.

As a result of this, it is now possible to travel from Berlin to Riga with Itinerary and continue further with the local trains there:

The connection is still far from good, but fear I can’t fix that in software.

What still does not work, is directly searching from Berlin to Riga, as that depends on having a single data source that has data on the entire route to find it. So it is necessary to split the route and search for the parts yourself.

Why you can’t always find a route even though there is one

The main data source for Itinerary in Europe is the API of the “Deutsche Bahn”, the main railway operator in Germany. Its API also has data for neighbouring countries, and even beyond that. According to Jon Worth their data comes from UIC Merits, which is a common system that railway operators can submit their routes to. However that probably comes with high costs, so many smaller operators like the ones in Latvia and Lithuania don’t do that. For that reason there is no such single data source that can route for example from Berlin to Riga.

What most of the operators in Europe do however, is publish schedule data in a common format (GTFS). What is missing so far, is a single service that can route on all of the available data and has an API that we can use. Setting something like this up would require a bit of community and hosting resources, but I am hopeful that we can have something like this in the future.

In the meantime, it already helps to fill in the missing countries one by one, so at least local users can already find their routes in Itinerary, and for Interrail and other cross border travel, people can at least patch routes together.

More countries

The next country I worked on was Montenegro. The reason for that is that it is close to the area that the DB API can still give results for, and also still has useful train services. Getting their API to work well was a bit more difficult though, as it doesn’t provide some of the information that Itinerary usually depends on. For example coordinates for stations. Those are needed to select where to search for trains going from a station. Luckily, exporting the list of stations and their coordinates from OpenStreetMap was relatively easy and provided me with all the data I needed.

Thanks to that Itinerary can now even show the route on a map properly.

Now only the API for Serbia is missing to actually connect to the part of the network DB knows about.

The new backends are not yet included in any release, but you can already find them in the nightly builds. Be aware that the nightly builds have switched to Qt6 and KF6 faily recently, which means there are still a few rough edges and small bugs in the UI.

On Linux, you can use the nightly flatpak:

flatpak install https://cdn.kde.org/flatpak/itinerary-nightly/org.kde.itinerary.flatpakref

On Android, the usual KDE Nightly F-Droid repository has up to date builds.

Julian Sequeira is a cofounder of Pybites. 
He's a Python coach, a podcaster, a career mindset advocate, and is learning guitar.

Topics include:

• Learning guitar
• Vacationing in Canada
• Pybites
• Splitting finances with Bob
• Building a community and a team
• Coaching
• Conscious positivity
• Australia is full of animals that want to kill you. Except kangaroos.
• Teaching Python to non-technical people

The Complete pytest Course

• Level up your testing skills and save time during coding and maintenance.
• Check out courses.pythontest.com

★ Support this podcast on Patreon ★ <p>Julian Sequeira is a cofounder of Pybites. <br>He's a Python coach, a podcaster, a career mindset advocate, and is learning guitar.</p><p>Topics include:</p><ul><li>Learning guitar</li><li>Vacationing in Canada</li><li>Pybites</li><li>Splitting finances with Bob</li><li>Building a community and a team</li><li>Coaching</li><li>Conscious positivity</li><li>Australia is full of animals that want to kill you. Except kangaroos.</li><li>Teaching Python to non-technical people</li></ul> <br><p><strong>The Complete pytest Course</strong></p><ul><li>Level up your testing skills and save time during coding and maintenance.</li><li>Check out <a href="https://courses.pythontest.com/p/complete-pytest-course">courses.pythontest.com</a></li></ul> <strong> <a href="https://www.patreon.com/PythonPeople" rel="payment" title="★ Support this podcast on Patreon ★">★ Support this podcast on Patreon ★</a> </strong>

After a few days of work the Fedora KDE SIG is proud to announce the availability of KDE 6th Megarelease Release Candidate 1 on Fedora Rawhide!

For those who like bleeding edge, feel free to try it!

We are very excited and looking forward to Fedora 40 + KDE 6 + Wayland only

Note: right now the update is sitting on testing. If you don’t want to wait a few hours until it reaches stable…

You can access it via a dnf repository like:

[main] cachedir=/var/cache/yum debuglevel=1 logfile=/var/log/yum.log reposdir=/dev/null retries=20 obsoletes=1 gpgcheck=0 assumeyes=1 keepcache=1 install_weak_deps=0 strict=1 # repos [build] name=build baseurl=https://kojipkgs.fedoraproject.org/repos/f40-build-side-81132/5738561/x86_64

In 2002, Kent Beck released a book called  "Test Driven Development by Example".
In December of 2023, Kent wrote an article called "Canon TDD".
With Kent's permission, this episode contains the full content of the article.

Brian's commentary is saved for a followup episode.

Links:

• Canon TDD
• Test Driven Development by Example

The Complete pytest Course

• Level up your testing skills and save time during coding and maintenance.
• Check out courses.pythontest.com

<p>In 2002, Kent Beck released a book called  "Test Driven Development by Example".<br>In December of 2023, Kent wrote an article called "Canon TDD".<br>With Kent's permission, this episode contains the full content of the article.</p><p>Brian's commentary is saved for a followup episode.</p><p>Links:</p><ul><li><a href="https://tidyfirst.substack.com/p/canon-tdd">Canon TDD</a></li><li><a href="https://bookshop.org/p/books/test-driven-development-by-example-kent-beck/115205">Test Driven Development by Example</a></li></ul> <br><p><strong>The Complete pytest Course</strong></p><ul><li>Level up your testing skills and save time during coding and maintenance.</li><li>Check out <a href="https://courses.pythontest.com/p/complete-pytest-course">courses.pythontest.com</a></li></ul>

Here are 50 Python Data Structure exercises covering List, Set, Dictionary, and Tuple operations. These are excellent exercises for any beginner learning Python. In the following exercises, you’ll engage in a series of hands-on challenges that traverse the landscape of Lists, Sets, Dictionaries, and Tuples, as well as more advanced concepts like list comprehension. Each […]

The post Top 50 Python Data Structure Exercises (List, Set, Dictionary, and Tuple) appeared first on TechBeamers.

Here are 45 Python exercises on loops (for, while), if-else statements, and the range() function, along with their solutions. Each exercise comes with a brief description of the problem and a solution that utilizes the mentioned constructs. 45 Python Exercises on Loops, Conditions, and Range() Function If you already have learned the basics of Python […]

The post Top 45 Python Exercises on Loops, Conditions, and Range() Function appeared first on TechBeamers.

Wow, it feels like it’s been a while! And while many of KDE’s contributors have been enjoying some holiday and vacation time, quite a lot happened too! We’re getting pretty close to the projected February 28th release day for the KDE 6 megarelease, so all hands have been on the bug-fixing deck.

Overall we’re in good shape. Despite the large number of open bug reports, most are not serious, and I have confidence that we’ll get the remaining major ones done before the final release. Of course, the best way to make sure that happens is to help out!

KDE 6 Mega-Release

(Includes all software to be released on the February 28th mega-release: Plasma 6, Frameworks 6, and apps from Gear 24.02)

General info – Open issues: 238

UI improvements

It’s now possible to quickly apply a wallpaper to all screens at once (Prajna Sariputra, link)

You can now set a custom mouse pointer speed, just like you can for the touchpad (Denis Zhdanov, link)

Discover Notifier no longer appears in your System Tray (even just the inactive area) unless it actually has something to notify you about–which means you can remove it permanently by simply telling Discover to never check for updates automatically on System Settings’ Updates page (Yifan Zhu, link)

Breeze-themed buttons and text field now always have the same height so they’ll never look slightly different when adjacent to one another (Akseli Lahtinen, link)

Throughout KDE software, the way symbolic icons (i.e. those whose names end with -symbolic) are found has been improved: when a symbolic icon is not found and the system has to fall back to a more generic icon, you’ll now get a symbolic version of that icon if it exists (Joshua Goins, link)

You can now configure Plasma to turn off the screen at the same moment when it locks (Jakob Petsovits, link 1 and link 2)

Various utility and dialog windows that don’t show up in the Task Manager also no longer show up in the Overview effect either (Akseli Lahtinen, link)

Bug fixes

Important note: I don’t mention fixes for bugs that were never released to users; it’s just too much for me (it would probably be too much for you to read as well), and most people never encountered them in the first place. Because we’re in the middle of a big Plasma dev cycle, there are a lot of these bugs! So big thanks to everyone who’s made it a priority to fix them!

Ark now preserved extended attributes of files in archives when editing and saving their contents (Kristen McWilliam, link)

In Dolphin’s Details view, expandable folders now expand to the correct places again when sorting by size (Akseli Lahtinen, link)

Fixed a file descriptor leak in Plasma that could cause it to run out of file descriptors and crash under certain circumstances (Moody Liu, link)

Fixed an issue that could prevent the GlobalProtect SAML authentication method in OpenConnect VPNs from working (Rahul Rameshbabu, link)

Night color can now co-exist with color correction using ICC profiles–at least in the Wayland session (Xaver Hugl, link)

When using the weather widget’s DWD weather provider, you’ll no longer sometimes see nonsensically high temperature values shown in the forecast (Ismael Asensio, link)

Tabs in the weather widget no longer sometimes overlap in certain circumstances (Ismael Asensio, link)

Plasma’s “Alternatives” popup now follows the opacity level of its parent panel (Niccolò Venerandi, link)

When showing a preview of the selected Task Switcher style, clicking away from the preview now closes it as expected in the Plasma Wayland session (Ismael Asensio, link)

Other bug information of note:

• 4 Very high priority Plasma bugs (up from 3 three weeks ago). Current list of bugs
• 37 15-minute Plasma bugs (up from 35 three weeks ago). Current list of bugs
• 262 KDE bugs of all kinds fixed over the past 3 weeks. Full list of bugs

Performance & Technical

The colord-kde repo has been ported to Qt6, so that color management on X11 works too (Nicolas Fella, link)

Reduced the background CPU usage when moving the pointer (Xaver Hugl, link)

A wide variety of pieces of transient state in QtWidgets-based apps are now stored in the state config file, not the settings file used for persistent user-set settings. This is a part of a very vocal crowd’s favorite bug! (Alexander Lohnau, link)

Automation & Systematization

Added GUI tests to make sure several Plasma bugs don’t recur (Fushan Wen, link 1, link 2, link 3, and link 4)

Added an autotest to make sure symbolic icon fallback works properly (Joshua Goins, link)

…And Everything Else

This blog only covers the tip of the iceberg! If you’re hungry for more, check out https://planet.kde.org, where you can find more news from other KDE contributors.

How You Can Help

Thanks to you, our Plasma 6 fundraiser has been a crazy success! I originally thought the goal of 500 new KDE e.V. supporting members was over-optimistic, but you’ve all proven me happily wrong. We’re now up to an incredible 627 members and unlocked multiple stretch goals! The first one has been met, but the second one is still on offer. So if you like the work we’re doing, spreading the wealth via this fundraiser is a great way to share the love.

If you’re a developer, work on Qt6/KF6/Plasma 6 issues! Which issues? These issues. Plasma 6 is very usable for daily driving now, but still in need of bug-fixing and polishing to get it into a releasable state by February.

Otherwise, visit https://community.kde.org/Get_Involved to discover other ways to be part of a project that really matters. Each contributor makes a huge difference in KDE; you are not a number or a cog in a machine! You don’t have to already be a programmer, either. I wasn’t when I got started. Try it, you’ll like it! We don’t bite!

We are thrilled to announce Python Data Science Day will be taking place March 14th, 2024; a “PyDay” on Pi Day: 3.14 . If you’re a Python developer, entrepreneur, data scientist, student, or researcher working on projects from hobbyist and start up to enterprise level, you’ll find solutions to modernize your data pipelines and answer complex queries with data.

We’re looking forward to your content submission!

Submit a session or a lightning talk proposal to join the list of amazing speakers for Data Science Day. The CFP opened January 11th, 2024 and closes January 25th, 2024 AOE (Anywhere on Earth). Full schedule will be announced early mid February.

There are two types of submissions:

• Session (25 minutes, pre-recorded or live) Sessions are delivered by no more than two people and can cover high level programming topics. Many times these talks are accompanied by slides, demonstrations or blog posts. This could cover a programming story, how it was approached and the solution; or it could dive deeper into a particular topic or using a suite of features.
• Lightning Talk (5 to 7 minutes, pre-recorded) A Lightning talk is a very short presentation. It is a great introduction to public speaking or a great way to present something short and sweet. These talks typically prioritize a singular idea in a digestible way that inspires further learning.

Good submissions will…

• discuss a hot or cool tool, product, or skill.
• include accessible content with slides, Notebook, repository, tutorial, or blog post.
• create a thoughtful, cohesive story for a Data Science audience.

Special Guest Speakers on Data Science Day

You could be among the speakers on Python Data Science Day.

Sarah Kaiser, PhD Soojin Choi You! Python Cloud Developer Advocate Jupyter Notebook Product Manager Speaker at Python Data Science Day Submit your CFP

 

More ways to engage with all the Data Science fun:

• Join the Microsoft Fabric Global AI Hack Together on February 15th to March 4th, 2024. Fabric is an end-to-end AI-powered analytics platform that unites your data and services, including data science and data lakes. Register for the event to participate in live streams every week and solve real-world problems with guidance and a community.
• Read our 14 Days of Python Data Science series where for fourteen days leading up to Python Data Science Day, we will drop cool articles and recipes for using Data Science on Microsoft tools. Link coming soon!
• Check out the Data Science Cloud Skills Challenge if you want to go through some self-paced learning! This challenge is active until April 15th, 2024.
• Join us on Discord at https://aka.ms/python-discord

More Data Science at Microsoft…

• Data Scientist Certifications
• Data Scientist Training Path
• Data Science for Beginners – GitHub Repo

The post Data Science Day Announcement and Call for Speaker Proposals appeared first on Python.

Contributing to Debian is part of Freexian’s mission. This article covers the latest achievements of Freexian and their collaborators. All of this is made possible by organizations subscribing to our Long Term Support contracts and consulting services.

LXD/Incus backend bug in autopkgtest by Stefano Rivera

While working on the Python 3.12 transition, Stefano repeatedly ran into a bug in autopkgtest when using LXD (or in the future Incus), that caused it to hang when running cython’s multi-hour autopkgtests. After some head-banging, the bug turned out to be fairly straightforward: LXD didn’t shut down on receiving a SIGTERM, so when a testsuite timed out, it would hang forever. A simple fix has been applied.

/usr-merge, by Helmut Grohne

Thanks to Christian Hofstaedtler and others, the effort is moving into a community effort and the work funded by Freexian becomes more difficult to separate from non-funded work. In particular, since the community fully handled all issues around lost udev rules, dh_installudev now installs rules to /usr.

The story around diversions took another detour. We learned that conflicts do not reliably prevent concurrent unpack and the reiterated mitigation for molly-guard triggered this. After a bit of back and forth and consultation with the developer mailing list, we concluded that avoiding the problematic behavior when using apt or an apt-based upgrader combined with a loss mitigation would be good enough. The involved packages bfh-container, molly-guard, progress-linux-container and systemd have since been uploaded to unstable and the matter seems finally solved except that it doesn’t quite work with sysvinit yet. The same approach is now being proposed for the diversions of zutils for gzip. We thank involved maintainers for their timely cooperation.

gcc-for-host, by Helmut Grohne

Since forever, it has been difficult to correctly express a toolchain build dependency. This can be seen in the Build-Depends of the linux source package for instance. While this has been solved for binutils a while back, the patches for gcc have been unfinished. With lots of constructive feedback from gcc package maintainer Matthias Klose, Helmut worked on finalizing and testing these patches. Patch stacks are now available for gcc-13 and gcc-14 and Matthias already included parts of them in test builds for Ubuntu noble. Finishing this work would enable us to resolve around 1000 cross build dependency satisfiability issues in unstable.

Miscellaneous contributions

• Stefano continued work on the Python 3.12 transition, including uploads of cython, pycxx, numpy, python-greenlet, twisted, foolscap and dh-python.
• Stefano reviewed and selected from a new round of DebConf 24 bids, as part of the DebConf Committee. Busan, South Korea was selected.
• For debian-printing Thorsten uploaded hplip to unstable to fix a /usr-merge bug and cups to Bookworm to fix bugs related to printing in color.
• Utkarsh helped newcomers in mentoring and reviewing their packaging; eg: golang-github-prometheus-community-pgbouncer-exporter.
• Helmut sent patches for 42 cross build failures unrelated to the gcc-for-host work.
• Helmut continues to maintain rebootstrap. In December, blt started depending on libjpeg and this poses a dependency loop. Ideally, Python would stop depending on blt. Also linux-libc-dev having become Multi-Arch: foreign poses non-trivial issues that are not fully resolved yet.
• Enrico participated in /usr-merge discussions with Helmut.

Saturday, 13 January 2024

KDE today announces the release of KDE Frameworks 5.114.0.

KDE Frameworks are 83 addon libraries to Qt which provide a wide variety of commonly needed functionality in mature, peer reviewed and well tested libraries with friendly licensing terms. For an introduction see the KDE Frameworks release announcement.

This release is part of a series of planned monthly releases making improvements available to developers in a quick and predictable manner.

New in this version Baloo

• [IndexCleaner] Remove no-op recursion over includedFolders
• [ExtractorProcess] Remove unused members
• [TimeEstimator] Cleanup, remove QObject depency
• Use forward declaration for Baloo::Database
• Remove inacurate mimetype check in xattrindexer and indexcleaner

Extra CMake Modules

• Fixes to FindLibGit2.cmake (bug 478669)

KActivities

• Drop unused KF5WindowSystem from cli

KCodecs

• KEmailAddress: Only trim surrounding whitespace between E-Mail addresses instead of also replacing all whitespace within E-Mail address names with a single ASCII space

KCoreAddons

• Fix license text loading on Android

KHolidays

• Introduce holidays observed in Kenya

KImageFormats

• avif: new quality settings
• Update CI template
• HEIF plug-in extended to support HEJ2 format

KIO

• kpropertiesdialog: don't trip over malformed Exec (bug 465290)
• WidgetsAskUserActionHandler: fix backport (bug 448532)
• WidgetsAskUserActionHandler: Use QPointer to check the validity of parent widgets (bug 448532)

Kirigami

• Make drawer actions accessible

KJobWidgets

• KUiServerV2JobTracker: prevent potenial use-after-free (bug 471531)

KRunner

• DBusRunner: Use /runner as default for X-Plasma-DBusRunner-Path property

Plasma Framework

• [CI] Fix pipeline include

Purpose

• Adapt to KAccounts API change

Security information

The released code has been GPG-signed using the following key: pub rsa2048/58D0EE648A48B3BB 2016-09-05 David Faure faure@kde.org Primary key fingerprint: 53E6 B47B 45CE A3E0 D5B7 4577 58D0 EE64 8A48 B3BB