Feeds

Python for Beginners: Drop Elements From a Series in Python

Planet Python - Wed, 2022-11-30 09:00

Pandas series is very useful for handling data having ordered key-value pairs. In this article, we will discuss different ways to drop elements from a pandas series.

Table of Contents
  1. Drop Elements From a Pandas Series Using the drop() Method
  2. Drop a Single Element From a Pandas Series
  3. Delete Multiple Elements From a Pandas Series
  4. Drop Elements Inplace From a Pandas Series
  5. Delete an Element From a Series if the Index Exists
  6. Drop NaN Values From a Pandas Series
  7. Drop NaN Values Inplace From a Pandas Series
  8. Drop Duplicates From a Pandas Series
  9. Drop Duplicates Inplace in a Pandas Series
  10. Drop All Duplicate Values From a Pandas Series
  11. Conclusion
Drop Elements From a Pandas Series Using the drop() Method

We can drop elements from a pandas series using the drop() method. It has the following syntax.

Series.drop(labels=None, *, axis=0, index=None, columns=None, level=None, inplace=False, errors='raise')

Here, 

  • The labels parameter takes the index of the elements that we need to delete from the series. You can pass a single index label or a list of indices to the labels parameter. 
  • The axis parameter is used to decide if we want to delete a row or column. For a pandas series, the axis parameter isn’t used. It is defined in the function just to ensure the compatibility of the drop() method with pandas dataframes.
  • The index parameter is used to select the index of the elements to delete for given labels in the dataframe. The index parameter is redundant for series objects. However, you can use the index parameter instead of the labels parameter. 
  • The columns parameter is used to select the columns to delete in a dataframe. The “columns” parameter is also redundant here. You can use labels or index parameters to drop elements from a series. 
  • The levels parameter is used to delete elements from a series when the series contains a multi-index. The levels parameter takes the level or list of levels from which the elements need to be deleted for specified labels. 
  • By default, the drop() method returns a new series object after deleting elements from the original series. In this process, the original series isn’t modified. If you want to modify the original series instead of creating a new series, you can set the inplace parameter to True.
  • The drop() method raises an exception whenever it runs into an error while dropping the elements from the series. For example, if an index or label that we want to delete doesn’t exist in the series, the drop() method raises a python KeyError exception. To suppress such errors while deleting an element from the series, you can set the errors parameter to “ignore”.

After execution, the drop() method returns a new series if the inplace parameter is set to False. Otherwise, it returns None. 

Drop a Single Element From a Pandas Series

To drop a single element from a series, you can pass the index of the element to the labels parameter in the drop() method as shown below.

import pandas as pd import numpy as np letters=["a","b","c","ab","abc","abcd","bc","d"] numbers=[3,23,11,14,16,2,45,65] series=pd.Series(letters) series.index=numbers print("The original series is:") print(series) series=series.drop(labels=11) print("The modified series is:") print(series)

Output:

The original series is: 3 a 23 b 11 c 14 ab 16 abc 2 abcd 45 bc 65 d dtype: object The modified series is: 3 a 23 b 14 ab 16 abc 2 abcd 45 bc 65 d dtype: object

In the above example, we first created a Series object using the Series() constructor. Then we dropped the element having index 11 using the drop() method. For this, we have passed the value 11 to the drop() method. After execution of the drop() method, you can observe that the element with index 11 has been removed from the output series.

Instead of the labels parameter, you can also use the index parameter in the drop() method as shown below.

import pandas as pd import numpy as np letters=["a","b","c","ab","abc","abcd","bc","d"] numbers=[3,23,11,14,16,2,45,65] series=pd.Series(letters) series.index=numbers print("The original series is:") print(series) series=series.drop(index=11) print("The modified series is:") print(series)

Output:

The original series is: 3 a 23 b 11 c 14 ab 16 abc 2 abcd 45 bc 65 d dtype: object The modified series is: 3 a 23 b 14 ab 16 abc 2 abcd 45 bc 65 d dtype: object

In this example, we have used the index parameter instead of the labels parameter. However, the resultant series after execution of the drop() method is the same in both cases.

Delete Multiple Elements From a Pandas Series

To drop multiple elements from a series, you can pass a python list of indices of the elements to be deleted to the labels parameter. For instance, if you want to delete elements at indices 11, 16, and 2 of the given Series, you can pass the list [11,16,2] to the labels parameter in the drop() method as shown below.

import pandas as pd import numpy as np letters=["a","b","c","ab","abc","abcd","bc","d"] numbers=[3,23,11,14,16,2,45,65] series=pd.Series(letters) series.index=numbers print("The original series is:") print(series) series=series.drop(labels=[11,16,2]) print("The modified series is:") print(series)

Output:

The original series is: 3 a 23 b 11 c 14 ab 16 abc 2 abcd 45 bc 65 d dtype: object The modified series is: 3 a 23 b 14 ab 45 bc 65 d dtype: object

In this example, we have passed the list [11, 16, 2] as input to the labels parameter. Hence, after execution of the drop() method, the elements at index 11, 16, and 2 are deleted from the original series object.

Instead of the labels parameter, you can pass the list of indices to the index parameter as shown below.

import pandas as pd import numpy as np letters=["a","b","c","ab","abc","abcd","bc","d"] numbers=[3,23,11,14,16,2,45,65] series=pd.Series(letters) series.index=numbers print("The original series is:") print(series) series=series.drop(index=[11,16,2]) print("The modified series is:") print(series)

Output:

The original series is: 3 a 23 b 11 c 14 ab 16 abc 2 abcd 45 bc 65 d dtype: object The modified series is: 3 a 23 b 14 ab 45 bc 65 d dtype: object Drop Elements Inplace From a Pandas Series

By default, the drop() method returns a new series and doesn’t delete specified elements from the original series. To drop elements inplace from a pandas series, you can set the inplace parameter to True as shown below.

import pandas as pd import numpy as np letters=["a","b","c","ab","abc","abcd","bc","d"] numbers=[3,23,11,14,16,2,45,65] series=pd.Series(letters) series.index=numbers print("The original series is:") print(series) series.drop(index=[11,16,2],inplace=True) print("The modified series is:") print(series)

Output:

The original series is: 3 a 23 b 11 c 14 ab 16 abc 2 abcd 45 bc 65 d dtype: object The modified series is: 3 a 23 b 14 ab 45 bc 65 d dtype: object

In all the previous examples, the drop() method returned a new Series object. In this example, we have set the inplace parameter to True in the drop() method. Hence, the elements are deleted from the original series and it is modified. In this case, the drop() method returns None.

Delete an Element From a Series if the Index Exists

While deleting elements from a series using the drop() method, it is possible that we might pass an index to the labels or index parameter that is not present in the Series object. If the value passed to the labels or index parameter isn’t present in the Series, the drop() method runs into a KeyError exception as shown below.

import pandas as pd import numpy as np letters=["a","b","c","ab","abc","abcd","bc","d"] numbers=[3,23,11,14,16,2,45,65] series=pd.Series(letters) series.index=numbers print("The original series is:") print(series) series.drop(index=1117,inplace=True) print("The modified series is:") print(series)

Output:

KeyError: '[1117] not found in axis'

In the above example, we have passed the value 1117 to the index parameter. As the value 1117 is not present in the Series, we get a KeyError exception.

To avoid errors and drop elements from a series if the index exists, you can use the errors parameter. By default, the errors parameter is set to "raise". Due to this, the drop() method raises an exception every time it runs into an error. To suppress the exception, you can set the errors parameter to “ignore” as shown in the following example.

import pandas as pd import numpy as np letters=["a","b","c","ab","abc","abcd","bc","d"] numbers=[3,23,11,14,16,2,45,65] series=pd.Series(letters) series.index=numbers print("The original series is:") print(series) series.drop(index=1117,inplace=True,errors="ignore") print("The modified series is:") print(series)

Output:

The original series is: 3 a 23 b 11 c 14 ab 16 abc 2 abcd 45 bc 65 d dtype: object The modified series is: 3 a 23 b 11 c 14 ab 16 abc 2 abcd 45 bc 65 d dtype: object

In the above example, we have passed the value 1117 to the index parameter. As 1117 is not present in the series index, the drop() method would have run into a KeyError exception. However, we have set the errors parameter to "ignore" in the drop() method. Hence, it suppresses the error. You can also observe that the series returned by the drop() method is the same as the original series.

Suggested Reading: If you are into machine learning, you can read this article on regression in machine learning. You might also like this article on clustering mixed data types in Python.

Drop NaN Values From a Pandas Series

NaN values are special numbers having floating-point data type in Python. NaN values are used to represent the absence of a value. Most of the times, NaN values have no importance in a given dataset and we need to remove these values.

You can drop NaN values from a pandas series using the dropna() method. It has the following syntax.

Series.dropna(*, axis=0, inplace=False, how=None)

Here,  

  • The axis parameter is used to decide if we want to delete nan values from a row or column from the series. For a pandas series, the axis parameter isn’t used. It is defined just to ensure the compatibility of the dropna() method with pandas dataframes.
  • By default, the dropna() method returns a new series object after deleting nan values from the original series. In this process, the original series isn’t modified. If you want to delete the nan values from the original series instead of creating a new series, you can set the inplace parameter to True.
  • The “how” parameter is not used for a series. 

After execution, the dropna() method returns a new series if the inplace parameter is set to False. Otherwise, it returns None. 

You can drop nan values from a pandas series as shown in the following example.

import pandas as pd import numpy as np letters=["a","b","c",np.nan,"ab","abc",np.nan,"abcd","bc","d"] series=pd.Series(letters) print("The original series is:") print(series) series=series.dropna() print("The modified series is:") print(series)

Output:

The original series is: 0 a 1 b 2 c 3 NaN 4 ab 5 abc 6 NaN 7 abcd 8 bc 9 d dtype: object The modified series is: 0 a 1 b 2 c 4 ab 5 abc 7 abcd 8 bc 9 d

In the above example, you can observe that the original series has two NaN values. After execution, the dropna() method deletes both the NaN values with their indices and returns a new series.

Drop NaN Values Inplace From a Pandas Series

If you want to drop NaN values from the original series instead of creating a new series, you can set the inplace parameter to True in the dropna() method as shown below.

import pandas as pd import numpy as np letters=["a","b","c",np.nan,"ab","abc",np.nan,"abcd","bc","d"] series=pd.Series(letters) print("The original series is:") print(series) series.dropna(inplace=True) print("The modified series is:") print(series)

Output:

import pandas as pd import numpy as np letters=["a","b","c",np.nan,"ab","abc",np.nan,"abcd","bc","d"] series=pd.Series(letters) print("The original series is:") print(series) series.dropna(inplace=True) print("The modified series is:") print(series)

Here, we have set the inplace parameter to True. Hence, the dropna() method modified the original series instead of creating a new one. In this case, the dropna() method returns None after execution.

Drop Duplicates From a Pandas Series

We data preprocessing, we often need to remove duplicate values from the given data. To drop duplicate values from a pandas series, you can use the drop_duplicates() method. It has the following syntax.

Series.drop_duplicates(*, keep='first', inplace=False)

Here,

  • The keep parameter is used to decide what values we need to keep after removing the duplicates. To drop all the duplicate values except for the first occurrence, you can set the keep parameter to “first” which is its default value. To drop all the duplicate values except for the last occurrence, you can set the keep parameter to “last”. If you want to drop all the duplicate values, you can set the keep parameter to False.
  • By default, the drop_duplicates() method returns a new series object after deleting duplicate values from the original series. In this process, the original series isn’t modified. If you want to delete the duplicate values from the original series instead of creating a new series, you can set the inplace parameter to True.

After execution, the drop_duplicates() method returns a new series if the inplace parameter is set to False. Otherwise, it returns None. You can observe this in the following example.

import pandas as pd import numpy as np letters=["a","b","a","a","ab","abc","ab","abcd","bc","abc","ab"] series=pd.Series(letters) print("The original series is:") print(series) series=series.drop_duplicates() print("The modified series is:") print(series)

Output:

The original series is: 0 a 1 b 2 a 3 a 4 ab 5 abc 6 ab 7 abcd 8 bc 9 abc 10 ab dtype: object The modified series is: 0 a 1 b 4 ab 5 abc 7 abcd 8 bc dtype: object

In the above example, you can observe that strings “a”, “ab”, and “abc” are present multiple times in the series. Hence, when we invoke the drop_duplicates() method on the series objects, all the duplicates except the one occurrence of the strings are removed from the series.

Looking at the indices, you can observe that first occurrence of the elements have been retained if the elements are present multiple times in the series. If you want to preserve the last occurrence of the elements having duplicate values, you can set the keep parameter to "last" as shown below.

import pandas as pd import numpy as np letters=["a","b","a","a","ab","abc","ab","abcd","bc","abc","ab"] series=pd.Series(letters) print("The original series is:") print(series) series=series.drop_duplicates(keep="last") print("The modified series is:") print(series)

Output:

The original series is: 0 a 1 b 2 a 3 a 4 ab 5 abc 6 ab 7 abcd 8 bc 9 abc 10 ab dtype: object The modified series is: 1 b 3 a 7 abcd 8 bc 9 abc 10 ab dtype: object

In the above example, we have set the keep parameter to "last". Hence, you can observe that the drop_duplicates() method preserves the last occurrence of the elements that have duplicate values.

Drop Duplicates Inplace in a Pandas Series

By default, the drop_duplicates() method doesn’t modify the original series object. It returns a new series. If you want to modify the original series by deleting the duplicates, you can set the inplace parameter to True in the drop_duplicates() method as shown below.

import pandas as pd import numpy as np letters=["a","b","a","a","ab","abc","ab","abcd","bc","abc","ab"] series=pd.Series(letters) print("The original series is:") print(series) series.drop_duplicates(inplace=True) print("The modified series is:") print(series)

Output:

The original series is: 0 a 1 b 2 a 3 a 4 ab 5 abc 6 ab 7 abcd 8 bc 9 abc 10 ab dtype: object The modified series is: 0 a 1 b 4 ab 5 abc 7 abcd 8 bc dtype: object

In this example, we have set the inplace parameter to True. Hence, the drop_duplicates() method modified the original series instead of creating a new one. In this case, the drop_duplicates() method returns None after execution.

Drop All Duplicate Values From a Pandas Series

To drop all the duplicates from a pandas series, you can set the keep parameter to False as shown below.

import pandas as pd import numpy as np letters=["a","b","a","a","ab","abc","ab","abcd","bc","abc","ab"] series=pd.Series(letters) print("The original series is:") print(series) series=series.drop_duplicates(keep=False) print("The modified series is:") print(series)

Output:

The original series is: 0 a 1 b 2 a 3 a 4 ab 5 abc 6 ab 7 abcd 8 bc 9 abc 10 ab dtype: object The modified series is: 1 b 7 abcd 8 bc dtype: object

In this example, we have set the keep parameter to False in the drop_duplicates() method. Hence, you can observe that all the elements having duplicate values are removed from the series.

Conclusion

In this article, we have discussed different ways to drop elements from a pandas series. To know more about pandas module, you can read this article on how to sort a pandas dataframe. You might also like this article on how to drop columns from a pandas dataframe.

The post Drop Elements From a Series in Python appeared first on PythonForBeginners.com.

Categories: FLOSS Project Planets

OpenSense Labs: An overview of Automatic Updates in Drupal 10

Planet Drupal - Wed, 2022-11-30 07:46
An overview of Automatic Updates in Drupal 10 Maitreayee Bora Wed, 11/30/2022 - 18:16

Between November 2020 and October 2021, 5212 organizations worldwide experienced data breaches. (source: statista).

And the number is steadily increasing. 

While every business that operates online faces some cyber threats, there are many ways to prevent data breaches or at least minimize their impact.

Delays before security updates are applied on site can result in compromised sites as seen in Drupalgeddon.

Manually updating a Drupal site can be an expensive, difficult, & time-consuming. 

The goal of the Automatic Updates Initiative is to provide safe and secure automatic updates for Drupal sites. It aims to solve the problem of any security concerns while over ridding the troublesome manual update process of a Drupal site.

Explained: Drupal Automatic Updates

Drupal’s Automatic Updates focus on resolving some of the most difficult usability concerns in maintaining Drupal websites. It is listed as one of the Drupal Core Strategic Initiatives for Drupal 9. 

It comprises of updates on production, development, and staging environments, with some integrations required in existing CI/CD processes. 

Automatic Updates in Drupal offers some major benefits to its users such as a reduction in the total cost of ownership (TCO) for Drupal websites and also a decrease in the maintenance cost.

Presently, we get to see a stable release that comprises features such as public safety alerts and readiness checks which will be discussed below. 

Importance of updating website

Here is the importance of updating a website. Take a look below:

  • Helps in increasing brand exposure

If we update a website by changing the outdated information with newly updated content then it will lead to an increase in brand exposure. But if we do not take this responsibility of updating content then it can be an obstacle in increasing the brand exposure which is essentially important.

  • Increases security

One of the major reasons for updating a website can be security concerns. For example, if a website is hacked then it can bring trouble for both the business and clients. But if we frequently update our website with the latest security features then such troubles of website hacking can be avoided. 

  • Provides mobile-friendly facility

By updating our website to a mobile-friendly website we enable our users to go through our website across various devices and platforms with ease and comfort. This leads to an increase in website traffic also resulting in a better company reputation.

Key Features of the Automatic Updates Module 

Here’s a list of features in the Automatic Updates module.  

  • Update readiness checks

We might not be always capable of updating all websites. Therefore, in instances like such, the readiness checks, one of the key features of Automatic Updates helps in identifying if a website is ready for updating automatically after a new release is offered to the Drupal community. 

For instance, websites that have un-run database updates, not having sufficient disk space for updating, or working on read-only file systems, won’t be able to get automatic updates. And if our website fails readiness checks and a Public service announcement (PSA) happens to be released, then it is essentially important to solve the readiness issue so that the website can be updated instantly.

  • In-place updates
  1. After the PSA service provides a notification to a Drupal site owner of an available update, and also the readiness checks happen to confirm that the website is ready to be updated, the website administrator is then able to update through the Update form.
     
  2. Tarball-based installations are well supported by this particular module and it doesn’t happen to choose some of the requirements in order to secure updating, rollback, etc which will come under the core solution.
     
  3. This module doesn’t support contrib updates or composer-based site installations. And also, the work on composer integration has begun already and is in progress.
  • Public service announcements (PSAs)

We get to see that infrequent announcements are done especially for critical security releases in regard to core and contrib modules. After a PSA is released, site owners need to review their websites so that they are updated well with the latest releases. Also, the website needs to be in a good position in order to quickly update if any fixes are given to the community.

Here is a quick video on the above-discussed features of automatic updates.


Conclusion

The Drupal community never fails to make an honest effort in building a community where its users can be benefited by making their software and websites safer and more user-friendly. The Automatic Updates initiative is a great example of it and by far it has made tremendous progress that cannot be unseen. 
 

Articles Off
Categories: FLOSS Project Planets

Russell Coker: Links November 2022

Planet Debian - Wed, 2022-11-30 05:26

Here’s the US Senate Statement of Frances Haugen who used to work for Facebook countering misinformation and espionage [1]. She believes that Facebook is capable of dealing with the online radicalisation and promotion of bad things on it’s platform but is unwilling to do so for financial reasons. We need strong regulation of Facebook and it probably needs to be broken up.

Interesting article from The Atlantic about filtered cigarettes being more unhealthy than unfiltered [2]. Every time I think I know how evil tobacco companies are I get surprised by some new evidence.

Cory Doctorow wrote an insightful article about resistance to “rubber hose cryptanalysis” [3].

Cory Doctorow wrote an interesting article “When Automation Becomes Enforcement” with a new way of thinking about Snapchat etc [4].

Cory Doctorow wrote an insightful and informative article Big Tech Isn’t Stealing News Publishers’ Content, It’s Stealing Their Money [5] which should be read by politicians from all countries that are trying to restrict quoting news on the Internet.

Interesting articl;e on Santiago Genoves who could be considered as a pioneer of reality TV for deliberately creating disputes between a group of young men and women on a raft in the Atlantic for 3 months [6].

Matthew Garrett wrote an interesting review of the Freedom Phone, seems that it’s not good for privacy and linked to some companies doing weird stuff [7]. Definitely worth reading.

Cory Doctorow wrote an interesting and amusing article about backdoors for machine learning [8]

Petter Reinholdtsen wrote an informative post on how to make a bootable USB stick image from an ISO file [9]. Apparently Lenovo provides ISO images to update laptops that don’t have DVD drives. :(

Barry Gander wrote an interesting article about the fall of Rome and the decline of the US [10]. It’s a great concern that the US might fail in the same way as Rome.

Ethan Siegel wrote an interesting article about Iapetus, a moon of Saturn that is one of the strangest objects in the solar system [11].

Cory Doctorow’s article Revenge of the Chickenized Reverse-Centaurs has some good insights into the horrible things that companies like Amazon are doing to their employees and how we can correct that [12].

Charles Stross wrote an insightful blog post about Billionaires [13]. They can’t do much for themselves with the extra money beyond about $10m or $100m (EG Steve Jobs was unable to extend his own life much when he had cancer) and their money is trivial when compared to the global economy. They are however effective parasites capable of performing great damage to the country that hosts them.

Cory Doctorow has an interesting article about how John Deere is being evil again [14]. This time with potentially catastrophic results.

Related posts:

  1. Links September 2022 Tony Kern wrote an insightful document about the crash of...
  2. Links November 2021 The Guardian has an amusing article by Sophie Elmhirst about...
  3. Links September 2020 MD5 cracker, find plain text that matches MD5 hash [1]....
Categories: FLOSS Project Planets

Qt for MCUs 2.3 released

Planet KDE - Wed, 2022-11-30 04:19

Since the very first release of Qt for MCUs, your feedback and requests have been driving the development of Qt for MCUs. Today, we are happy to announce the release of version 2.3, which includes several of the most requested features and improvements. This new version adds the Loader QML type to Qt Quick Ultralite, support for partial framebuffers to substantially reduce the overall memory requirements of your applications, support for building applications using MinGW on Windows, and much more!

Categories: FLOSS Project Planets

Morpht: Data migration for the Book module

Planet Drupal - Wed, 2022-11-30 04:14
A tutorial on how to preserve the hieracrhial structure of book pages during a migration from Drupal 7 to Drupal 9.
Categories: FLOSS Project Planets

The Drop Times: Behind the New Logo for NEDCamp: A Short Conversation with John Picozzi

Planet Drupal - Wed, 2022-11-30 03:59
TDT asked John Picozzi, one of the organizers for NEDCamp, about the process and idea behind creating a new logo for the camp. Read through to know, what we learned from him.
Categories: FLOSS Project Planets

John Ludhi/nbshare.io: Pandas Read and Write Excel File

Planet Python - Wed, 2022-11-30 02:38
Pandas Read and Write Excel File

Make sure you have openpyxl package installed. Otherwise you will get following error
...

ModuleNotFoundError: No module named 'openpyxl'

Install the package with following command
... pip install openpyxl

Pandas print excel sheet names In [1]: import pandas as pd

Pandas has ExcelFile method which returns Pandas excel object.

In [2]: excel = pd.ExcelFile("stocks.xlsx") excel.sheet_names Out[2]: ['Sheet12']

Note you might run in to following error

ValueError: Worksheet index 0 is invalid, 0 worksheets found

which usually means the Excel file is corrupt. To fix this error, copy the data in to another excel file and save it.

ExcelFile has many methods. For example excel.dict will print the data of spreadsheet in dictionary format.

In [3]: excel.__dict__ Out[3]: {'io': 'stocks.xlsx', '_io': 'stocks.xlsx', 'engine': 'openpyxl', 'storage_options': None, '_reader': <pandas.io.excel._openpyxl.OpenpyxlReader at 0x7f4cb232c8e0>}

To convert the data in to Pandas Dataframe. We will use ExcelFile.parse() method.

Pandas Read Excel Files In [4]: excel = pd.ExcelFile("stocks.xlsx") df = excel.parse() In [5]: df.head() Out[5]: Unnamed: 0 Unnamed: 1 Unnamed: 2 Unnamed: 3 0 NaN Stock Price Date 1 NaN INTC 28.9 2022-11-29 00:00:00 2 NaN AAPL 141.17 2022-11-29 00:00:00

Since our excel sheet has first column and row empty that is why we see headers and ist column as Unnamed and NaN respectively.

Let us fix it by specifying that header starts at row1.

In [6]: excel.parse(header=1) Out[6]: Unnamed: 0 Stock Price Date 0 NaN INTC 28.90 2022-11-29 1 NaN AAPL 141.17 2022-11-29

To fix the column indexing, we can use "usecols" option as shown below.

In [7]: excel.parse(usecols=[1,2,3],header=1) Out[7]: Stock Price Date 0 INTC 28.90 2022-11-29 1 AAPL 141.17 2022-11-29

To specify stock symbol as our index column, we can ues "index_col" option.

In [8]: excel.parse(index_col="Stock",usecols=[1,2,3],header=1) Out[8]: Price Date Stock INTC 28.90 2022-11-29 AAPL 141.17 2022-11-29

We can also use pd.read_excel() method to achieve the same

In [9]: pd.read_excel("stocks.xlsx",index_col="Stock",usecols=[1,2,3],header=1) Out[9]: Price Date Stock INTC 28.90 2022-11-29 AAPL 141.17 2022-11-29

Instead of specifying each column number, we can use range function to specify the columns which contain the data.

In [10]: excel.parse(usecols=range(1,4),header=1) Out[10]: Stock Price Date 0 INTC 28.90 2022-11-29 1 AAPL 141.17 2022-11-29

let us save the dataframe in to a variable.

In [11]: dfef = pd.read_excel("stocks.xlsx",usecols=range(1,4),header=1) In [12]: dfef.head() Out[12]: Stock Price Date 0 INTC 28.90 2022-11-29 1 AAPL 141.17 2022-11-29 Pandas write Dataframe to Excel File

We can write the dataframe in to Excel file using pd.to_excel() method.

In [13]: dfef.to_excel("stocktmp.xlsx") In [14]: !ls -lrt stocktmp.xlsx -rw-r--r-- 1 root root 5078 Nov 30 05:21 stocktmp.xlsx
Categories: FLOSS Project Planets

Russ Allbery: Review: The Fed Unbound

Planet Debian - Wed, 2022-11-30 00:08

Review: The Fed Unbound, by Lev Menand

Publisher: Columbia Global Reports Copyright: 2022 ISBN: 1-7359137-1-5 Format: Kindle Pages: 156

The Fed Unbound is a short non-fiction exploration of US Federal Reserve actions to reducing systemic risk caused by shadow banking. Its particular focus is the role of the Fed from the 2008 financial crisis to the present, including the COVID shock, but it includes a history of what Menand calls the "American Monetary Settlement," the political compromise that gave rise to the Federal Reserve.

In Menand's view, a central cause of instability in the US financial system (and, given the influence of the dollar system, the world financial system as well) is shadow banking: institutions that act as banks without being insured like banks or subject to bank regulations. A bank, in this definition, is an organization that takes deposits. I'm simplifying somewhat, but what distinguishes a deposit from a security or an investment is that deposits can be withdrawn at any time, or at least on very short notice. When you want to spend the money in your checking account, you don't have to wait for a three-month maturity period or pay an early withdrawal penalty. You simply withdraw the money, with immediate effect. This property is what makes deposits "money," rather than something that you can (but possibly cannot) sell for money, such as stocks or bonds.

Most people are familiar with the basic story of how banks work. Essentially no bank simply takes people's money and puts it in a vault until the person wants it again. If that were the case, you would need to pay the bank to store your money. Instead, a bank takes in deposits and then lend some portion of that money out to others. Those loans, for things like cars or houses or credit card spending, come due over time, with interest. The interest rate the bank charges on the loans is much higher than the rate it has to pay on its deposits, and it pockets the difference.

The problem with this model, of course, is that the bank doesn't have your money, so if all the depositors go to the bank at the same time and ask for their money, the bank won't be able to repay them and will collapse. (See, for example, the movie It's a Wonderful Life, or Mary Poppins, or any number of other movies or books.) Retail banks are therefore subject to stringent regulations designed to promote public trust and to ensure that traditional banking is a boring (if still lucrative) business. Banks are also normally insured, which in the US means that if they do experience a run, federal regulators will step in, shut down the bank in an orderly fashion, and ensure every depositor gets their money bank (at least up to the insurance limit).

Alas, if you thought people would settle for boring work that makes a comfortable profit, you don't know the financial industry. Highly-regulated insured deposits are less lucrative than acting like a bank without all of those restrictions and rules and deposit insurance payments. As Menand relates in his brief history of US banking, financial institutions constantly invent new forms of deposits with similar properties but without all the pesky rules: eurodollars (which have nothing to do with the European currency), commercial paper, repo, and many others. These forms of deposits are primarily used by large institutions like corporations. The details vary, but they tend to be prone to the same fundamental instability as bank deposits: if there's a run on the market, there may not be enough liquidity for everyone to withdraw their money at once. Unlike bank deposits, though, there is no insurance, no federal regulator to step in and make depositors whole, and much less regulation to ensure that runs are unlikely.

Instead, there's the Federal Reserve, which has increasingly become the bulwark against liquidity crises among shadow banks. This happened in 2008 during the financial crisis (which Menand argues can be seen as a shadow bank run sparked by losses on mortgage securities), and again at a larger scale in 2020 during the initial COVID crisis.

Menand is clear that these interventions from the Federal Reserve were necessary. The alternative would have been an uncontrolled collapse of large sections of the financial system, with unknown consequences. But the Fed was not intended to perform those types of interventions. It has no regulatory authority to reform the underlying financial systems to make them safer, remove executives who failed to maintain sufficient liquidity for a crisis, or (as is standard for all traditional US banks) prohibit combining banking and more speculative investment on the same balance sheet. What the Federal Reserve can do, and did, is function as a buyer of last resort, bailing out shadow banks by purchasing assets with newly-created money. This works, in the sense that it averts the immediate crisis, but it creates other distortions. Most importantly, constant Fed intervention doesn't create an incentive to avoid situations that require that intervention; if anything, it encourages more dangerous risk-taking.

The above, plus an all-too-brief history of the politics of US banking, is the meat of this book. It's a useful summary, as far as it goes, and I learned a few new things. But I think The Fed Unbound is confused about its audience.

This type of high-level summary and capsule history seems most useful for people without an economics background and who haven't been following macroeconomics closely. But Menand doesn't write for that audience. He assumes definitions of words like "deposits" and "money" that are going to be confusing or even incomprehensible to the lay reader.

For example, Menand describes ordinary retail banks as creating money, even saying that a bank loans money by simply incrementing the numbers in a customer's deposit account. This is correct in the technical economic definition of money (fractional reserve banking effectively creates new money), but it's going to sound to someone not well-versed in the terminology as if retail banks can create new dollars out of the ether. That power is, of course, reserved for the Federal Reserve, and indeed is largely the point of its existence. Much of this book that relies on a very specific definition of money and money supply that will only be familiar to those with economics training.

Similarly, the history of the Federal Reserve is interesting but slight, and at no point does Menand explain clearly how the record-keeping between it and retail banks works, or what the Fed's "balance sheet" means in practice. I realize this book isn't trying to be detailed description or history of the Federal Reserve system, but the most obvious audience is likely to flounder at the level of detail Menand provides.

Perhaps, therefore, this book is aimed at an audience already familiar with macroeconomics? But, if so, I'm not sure it says anything new. I follow macroeconomic policy moderately closely and found most of Menand's observations obvious and not very novel. There were tidbits here and there that I hadn't understood, but my time would probably have been better invested in another book. Menand proposes some reforms, but they mostly consist of "Congress should do its job and not create situations where the Federal Reserve has to act without the right infrastructure and political oversight," and, well, yes. It's hard to disagree with that, and it's also hard to see how it will ever happen. It's far too convenient to outsource such problems to central banking, where they are hidden behind financial mechanics that are incomprehensible to the average voter.

This is an important topic, but I don't think this is the book to read about it. If you want a clearer and easier-to-understand role of the Federal Reserve in shadow banking crises, read Crashed instead. If you want to learn more about how retail bank regulation works, and hear a strong case for why the same principles should be applied to shadow banks, see Sheila Bair's Bull by the Horns. I'm still looking for a great history and explainer of the Federal Reserve system as a whole, but that is not this book.

Rating: 5 out of 10

Categories: FLOSS Project Planets

scikit-learn: Interview with Meekail Zain, scikit-learn Team Member

Planet Python - Tue, 2022-11-29 19:00
Author: Reshama Shaikh , Meekail zain

Posted by Sangam SwadiK

Meekail Zain is a computer science PhD student at University of Georgia (USA), a member of Quinn Research Group and a software engineer at Quansight. Meekail officially joined the scikit-learn team as a maintainer in October 2022.

  1. Tell us about yourself.

    I’m currently attending the University of Georgia, pursuing a PhD in computer science. My area of research predominantly focuses on deep learning, generative modeling, and statistical approaches to clustering. I’m in my third year, and at the time of writing about to begin my comprehensive exams.

  2. How did you first become involved in open source and scikit-learn?

    I got my first computer when I was writing my master thesis, and back then a friend installed Linux on it for me. Since then I am a near-exclusive Linux user and learned to love the advantages of open source.

  3. We would love to learn of your open source journey.

    My journey really kicked off when I went to work at Quansight and received funding through the NASA Roses grant to be able to dedicate time to contributing to scikit-learn. It was a huge jump from what I had known up until that point. I learned Python very informally in order to be able to use PyTorch to develop/deploy models for my research, and had little-to-no experience with things like continuous integration or strong API. At first I felt incredibly intimidated and unqualified, but at the same time absolutely thrilled that I was in a position to learn so many new things! I started working on really simple changes to get used to the contribution workflow — things like removing excess whitespace and fixing typos — and then graduated to slightly more complex tasks. Eventually I got to the point where I started to “understand” small corners of the codebase and could actually offer help on new issues because of that familiarity. After that, I started reviewing others’ pull requests (PRs) and offering feedback in an unofficial capacity, as well as taking on more challenging tasks across the codebase. That process of growth and escalation is still ongoing, and truly I hope it never ends.

  4. To which OSS projects and communities do you contribute?

    NumPy, scikit-learn, and scipy. Right now it is heavily skewed towards scikit-learn with numpy being second most, but I’m hoping to take some more time to work on scipy in the near future!

  5. What advice or tips you have for people starting out in your field of work?

    Find a way to enjoy the feeling of being surrounded by things that you haven’t yet mastered. If you aim for growth — and indeed I think we all should — then you’ll find that you spend the majority of your time surrounded by things that you don’t quite understand, and the natural reaction to that is frustration and intimidation. If you can somehow convince yourself to also be excited by such an environment, you’ll find yourself growing every single day. Nobody starts off knowing everything :)

  6. What do you find alluring about OSS?

    This is a tough one, there are many amazing points. If I had to select just a few, it would be (in no particular order):

    • The growth potential
    • The community
    • The impact

    I’ve already discussed the growth potential so I’ll leave it at that.

    The community is fantastic as well! On every project the community base has its own unique personality of sorts, and they are all wonderful! It’s amazing being able to see recurring users that post interesting issues, or take a stab at opening more complex PRs (pull requests). There’s a strong sense of companionship with the people that are also trying to improve the same project as you! It’s akin to a very niche club in high school. It’s a wonderful experience finding people obsessed with the same cool project as you are.

    Finally, the impact. At the end of the day, the work we do has some serious consequences. Each project is essential to so many different workflows and enables brilliant researchers and software engineers to build complex systems and solutions to cutting edge problems. It’s sometimes surreal to think about how essential some of these projects really are.

  7. What pain points do you observe in community-led OSS?

    Consensus is difficult. This is a double-edged sword, since it carries some benefits too. With community-lead OSS, changes at every scale need to meet some kind of consensus. This ensures that the changes are well thought out and provides a layer of safety since the chance of uncaught mistakes propagating goes down with the number of people carefully reviewing changes (for the most part).

    For example, in scikit-learn a PR with changes to code needs to meet a lazy consensus where two official reviewers (currently just core developers) explicitly approve, and no other official reviewer officially disapproves. Going a bit further up, a new feature request in a project could require the consensus of several core developers that are well-versed in the topic area. Large systemic changes manifest in the form of SLEPs (scikit-learn enhancement proposals) which require a ⅔ consensus across all core developers. Above even that, there are cross-community discussions where the idea of a “consensus” itself isn’t always really clear.

    This system is a critical one, but there are important issues intrinsic to it that need to be addressed. For example, who gets to contribute to a consensus at each scale? What qualifications does one need, and how do we codify that? There’s also the intrinsic tradeoff where the stronger the consensus required, the less likely it is that changes will be adopted. This is by design since wide-reaching changes need to be held to high standards, but it does also mean that occasionally even for narrow-scoped problems no solution will be reached despite options being raised that are better than the status quo.

  8. If we discuss how far OS has evolved in 10 years, what would you like to see happen?

    I can’t speak to its evolution in the past 10 years, since I am still fairly new to OSS overall, but I would like to see systematic data-driven analysis on contributor’s needs. Different OSS projects have issued contributor surveys in the past, but in general I think a lot of emphasis is placed on the feedback given from users in meta issues or over community calls. While that is definitely helpful, there’s a lot of extrapolation that takes place when projects try to determine the needs of their contributor base like this.

    Some questions I would love to see studied include:

    • What distribution does the expertise of the contributor base follow?
    • What are the greatest bottlenecks at each level of expertise?
    • Aside from expertise, are there other socio-economic or general demographics that exhibit consistent bottlenecks? (e.g. access to hardware)
    • How do we create informed and effective DEI policies from this information? OSS projects thrive and prosper based on their community, so I would love to see more systematic research on community needs and pain points.
  9. What are your favorite resources, books, courses, conferences, etc?

    I absolutely adore “Probability and Statistics” by Evans and Rosenthal. It does a fantastic job of constructing a lot of otherwise daunting statistical concepts from very elementary ideas. It is my favorite book to recommend to eager students that do not have a rigorous foundation in probability and statistics, since this book does a great job of building up the reader’s intuition and making everything feel natural and derived, rather than arbitrarily defined.

    Regarding conferences, I have to go with SciPy! I was definitely scared going into the conference thinking that I would be the least-qualified person in every room and that I’d have nothing to talk about. I realized very quickly that there is always something to talk about, and qualifications don’t matter. It’s a gathering of super passionate people that are each eager to talk about the things that interest them, so regardless of whether you’re an expert or a beginner, they will happily explain things to you. Every single attendee has some area, no matter how specific, that they can talk about for hours. That genuine interest and excitement felt rejuvenating and reminded me why I love OSS so much.

  10. What are your hobbies, outside of work and open source?

    I really enjoy hiking, camping and playing DnD (Dungeons & Dragons)! Camping especially is an important hobby for me since whenever I have a computer in reach I feel inclined to check my GitHub notifications, so the occasional total disconnect for a weekend is a fantastic tool for me to give myself a break with no pressure of “I could work on that new feature right now…”

    If you have ever had difficulty with relaxing because of that little voice in your head that says “How dare you relax? You could be doing this and that right now!” then I highly recommend going camping, even just for one night! When that voice strikes during camping, I retort “Ah but you see, I don’t have my laptop, so I can’t work on that right now. All I can do right now is relax.” and suddenly the anxiety washes away :)

Categories: FLOSS Project Planets

Plasma Mobile Gear ⚙ 22.11 is Out

Planet KDE - Tue, 2022-11-29 19:00
Updates in Plasma Mobile for September to November 2022

The Plasma Mobile team is happy to announce the result of all the project's development work carried out in September and November of 2022.

Plasma Mobile Gear

We have decided to migrate the releases of Plasma Mobile applications to KDE Gear, starting with KDE Gear 23.04. This means that Plasma Mobile Gear will be discontinued in the future, and Plasma Mobile applications will follow the release schedule of most other KDE applications, simplifying packaging. To prepare for this, an ongoing effort was made to ensure all applications have proper Bugzilla categories created.

Akademy

Akademy 2022 was held in Barcelona, and Devin and Bhushan presented some of the work in the project in the following talk:

Several Plasma Mobile BoF (birds-of-feather) meetings were also held. More details about them can be read over at Devin's blog.

Shell

Plasma 5.27 will be released on February 9th, 2023. This will be the last Plasma 5 release, with work after that being focused on Plasma 6!

Action Drawer

Devin added a feature that lets you tap the media player so that the audio source app window opens. He also fixed the quicksettings so that it now always opens the mobile settings application. Several issues with the mobile data quicksetting not accounting for certain scenarios were also fixed; and he also worked on fixing the marquee label in the WiFi quicksetting, stopping them from overflowing when multiple network devices are attached.

Navigation Panel

Devin fixed the close button not being usable while an application is launching.

Halcyon (Homescreen)

Devin fixed some performance issues when scrolling in the grid app list. This should improve performance a lot on slower devices.

Yari fixed support for the Meta key, and it should now properly bring up the homescreen when pressed.

Lockscreen

Devin did some performance refactoring, and also set the clock text to bold to improve contrast.

KScreen

Aleix fixed wakeups while the screen is off and the device is rotated. Rotations are now only tracked when the display is on.

KWin

Xaver added support for device panel orientations. This means that devices like the OnePlus 5 (which has an upside-down mounted display) will now have the orientation correct by default, and not inverted for touch input.

Other

The bug that led to shell configurations sometimes being wiped at start has been fixed in the upcoming Plasma 5.26.4 release.

Seshan worked on an updated design for the power menu, and it now includes a logout button.

Weather

Devin spent time addressing feedback through the KDEReview process, in preparation for moving the application to the KDE Gear release cycle. These changes include:

  • The settings dialog being switched to use a window in desktop mode
  • The scrollbars being added to views
  • Re-implementing the location list reordering to be much nicer to use
  • Many bugfixes
Recorder

Devin also spent time on the Recorder app, addressing feedback through the KDEReview process in preparation for moving the application to the KDE Gear release cycle. These changes include:

  • The Recorder page now uses a fullscreen layout
  • The recording player layout has been reworked to be easier to use
  • The settings dialog is a window in desktop mode
  • Recordings now start immediately when the record button is pressed
  • You can now export recordings to a different location
  • A bug that added suffixes to recorded file names for no reason was corrected
  • Many bugfixes and UX improvements
Clock

Devin fixed an issue where looping timers could have multiple ongoing notifications and the user was not able to dismiss them.

Terminal

Devin did some bug fixing work on the terminal application. He fixed command deletion not saving in certain cases, and also fixed the bug which made the whole window to close when Ctrl-D was pressed.

Dialer

According to the feedback obtained after the previous incoming call screen updates, Alexey introduced support for changing the answer controls. He provided buttons, and a selection of asymmetric and symmetric answer swipes. He also implemented call duration and caller id support for the incoming call screen with updates both for the daemon and GUI logic.

Marco, along with Alexey, fixed an issue when there was no ringtone when the phone was in lock screen mode without an additional notification. Marco also helped Alexey improve KWin's logic when parsing application privileges, like the lock screen overlay.

Volker introduced initial support for the Qt6 CI builds.

Devin ported Dialer settings to the new form components.

Spacebar

Michael added attachment previews to notifications. He also made it so that image attachment previews are shown in the chat's list. Another thing he implemented is support for tapback reactions. There is now a confirmation dialog before deleting a chat conversation to prevent accidental deletion of a conversation. Michael also made it so that MMS messages can be downloaded even when wifi is also connected.

Discover

Aleix worked on a more helpful homepage that better displays featured applications.

Tokodon

Carl ported Tokodon's settings to the new form components. He also updated the timeline by automatically cropping and rounding the images, improving the typography and fixing some sizing issues.

Volker fixed multiple bugs in the timeline and reduced the transfer volume on a "warm" start-up by 80%. For the technical details, you might want to read his blog post: Secure and efficient QNetworkAccessManager use

NeoChat

Tobias has made a lot of progress on end-to-end encryption. You can read more about it in his blog post: NeoChat, encryption, and thanks for all the olms

But that's not all, aside from the end-to-end encryption implementation, there was also a lot of changes to NeoChat's configuration settings. James and Carl ported many settings to the new form components. James additionally created a new settings component for managing your notifications settings directly from NeoChat. Gary Wang made it possible to configure a proxy for NeoChat. Tobias improved the settings on Android (hiding the irrelevant settings).

Tobias implemented a basic developer tool that allows inspecting raw matrix events.

Carl added a confirmation dialog when signing out and Tobias added another confirmation dialog when enabling end-to-end encryption.

Tobias rewrote the account switcher to make it easier to switch between accounts.

Kasts

Bart added support for streaming to Kasts and episodes can now be listened to without the need to download them first. For people that don't care about downloading episodes, there is a new setting allowing you to select streaming over downloading. If this setting is activated, it will show streaming buttons on the UI instead of download and play buttons.

Settings

Devin did some major fixes to the cellular network settings module, ensuring that the toggle state always matches the one used in the shell. He also improved behavior for when there is no SIM, as well as added more helpful messages if an APN is not configured. Some UI issues on the APN page were also fixed.

Devin also fixed accent colors being set from the wallpaper not working in the colors settings module.

Raven

Devin fixed some issues with the new account setup. At Akademy we discussed sharing code between Kalendar and Raven.

Łukasz fixed the edit button showing on the time page even when no entries are listed.

Audiotube

Jonah implemented a lyrics view in the player, and made it possible to filter recent search queries. He also added real album cover images instead of monochrome icons in all song lists.

Mathis made a few UI improvements, including rounded images and new list headers.

Actions for each song (like add to queue, etc.) are now in a popup menu. This allows you to favorite songs without having to play them.

Contributing

Want to help with the development of Plasma Mobile? Take Plasma Mobile for a spin! Check out the device support for each distribution and find the version which will work on your phone.

Our documentation gives information on how and where to report issues. Also, consider joining our Matrix channel, and let us know what you would like to work on!

Categories: FLOSS Project Planets

Spyder IDE: Improvements to the Spyder IDE installation experience

Planet Python - Tue, 2022-11-29 19:00

Juan Sebastian Bautista, C.A.M. Gerlach and Carlos Cordoba also contributed to this post.

Spyder 5.4.0 was released recently, featuring some major enhancements to its Windows and macOS standalone installers. You'll now get more detailed feedback when new versions are available, and you can download and start the update to them from right within Spyder, instead of having to install them manually. In this post, we'll go over how these new update features work and how you can start using them!

Before proceeding, we want to acknowledge that this work was made possible by a Small Development Grant awarded to Spyder by NumFOCUS, which has enabled us to hire a new developer (Juan Sebastian Bautista Rojas) to be in charge of all the implementation details.

Before these improvements, Spyder already had a mechanism to detect more recent versions, but that functionality was very simple. There was a pop-up dialog warning that a new version was available, but users had to follow a link to manually download the installer and then run it themselves:

Once you upgrade to Spyder 5.4.0 or above, you'll get this message on future Spyder updates:

Spyder will now be able to automatically download and install a new version for you, much like many other popular applications.

After clicking "Yes" on that dialog, Spyder will display another with the status and percent completion of the download.

If it is closed, the download will continue in the background, with its progress shown in a new status bar widget.

After the download completes, Spyder will ask if you want to update immediately, cancel the update or defer it to when you close Spyder, to avoid interrupting your current workflow.

If you chose to update immediately, or once you close Spyder if you deferred the update, our installer will be started automatically. On Windows, the installer has a series of automated prompts to close the current instance, uninstall the previous version and finally install the new one:

On macOS, Spyder will automatically mount the new version's DMG, so you can simply drag and drop it in the Applications folder

We hope these improvements will make updating to future Spyder versions smoother and more straightforward, so we can bring you new features and enhancements more easily in the future!

Categories: FLOSS Project Planets

FSF Blogs: GNU Guix: Support the growth of the distribution into 2023 and beyond

GNU Planet! - Tue, 2022-11-29 17:25
An update from GNU Guix co-maintainer Maxim Cournoyer on the impressive work they did in 2022.
Categories: FLOSS Project Planets

GNU Guix: Support the growth of the distribution into 2023 and beyond

FSF Blogs - Tue, 2022-11-29 17:25
An update from GNU Guix co-maintainer Maxim Cournoyer on the impressive work they did in 2022.
Categories: FLOSS Project Planets

Jonathan McDowell: onak 0.6.2 released

Planet Debian - Tue, 2022-11-29 16:41

Over the weekend I released a new version of onak, my OpenPGP compatible keyserver. At 2 years since the last release that means I’ve at least managed to speed up a bit, but it’s fair to say its development isn’t a high priority for me at present.

This release is largely driven by a collection of minor fixes that have built up, and the knowledge that a Debian freeze is coming in the new year. The fixes largely revolve around the signature verification that was introduced in 0.6.0, which makes it a bit safer to run a keyserver by only accepting key material that can be validated. All of the major items I wanted to work on post 0.6.0 remain outstanding.

For the next release I’d like to get some basic Stateless OpenPGP Command Line Interface support integrated. That would then allow onak to be tested with the OpenPGP interoperability test suite, which has recently added support for verification only OpenPGP implementations.

I realise most people like to dismiss OpenPGP, and the tooling has been fairly dreadful for as long as I’ve been using it, but I do think it fills a space that no competing system has bothered to try and replicate. And that’s the web of trust, which helps provide some ability to verify keys without relying on (but also without preventing) a central authority to do so.

Anyway. Available locally or via GitHub.

0.6.2 - 27th November 2022

  • Don’t take creation time from unhashed subpackets
  • Fix ECDSA/SHA1 signature check
  • Fix handling of other signature requirement
  • Fix deletion of keys with PostgreSQL backend
  • Add support for verifying v3 signature packets
Categories: FLOSS Project Planets

PyCoder’s Weekly: Issue #553 (Nov. 29, 2022)

Planet Python - Tue, 2022-11-29 14:30

#553 – NOVEMBER 29, 2022
View in Browser »

Microsoft Power BI and Python: Two Superpowers Combined

In this tutorial, you’ll learn how to install and configure Microsoft Power BI to work with Python. Using Python, you’ll import data from a SQLite database, transform and augment your dataset with pandas, and visualize it with Matplotlib.
REAL PYTHON

Parallel Nested for-Loops in Python

Nested for-loops often are an opportunity for parallel code. This article covers when it is a good idea to split them up and the variety of different parallel coding approaches you can use.
JASON BROWNLEE

Painless and Worry Free Postgres

Find out why vanilla Postgres is a more cost effective and simple way to manage your database than shiny Postgres “compatible” databases. Performance tuned, no-forks, quality Postgres with expert support. Built for developer experience. Get started today →
CRUNCHY DATA sponsor

How We Run Tests in Hundreds of Environments Really Fast

Anton describes the test setup at Sentry and how they use both tox in parallel as well as GitHub actions to run a large test suite quickly.
ANTON PIRKER

Discussions The Origins of Python

This discussion is around the excellent article by Lambert Meertens called The Origins of Python that delves into Python’s history.
HACKER NEWS

Python Jobs Software Engineer - Weissman Lab (Cambridge, MA, USA)

Whitehead Institute for Biomedical Research

Senior Software Engineer (Python, Qt, Linux) (Anywhere)

VIOLET

More Python Jobs >>>

Articles & Tutorials Everyday Project Packaging With pyproject.toml

In this Code Conversation video course, you’ll learn how to package your everyday projects with pyproject.toml. Playing on the same team as the import system means you can call your project from anywhere, ensure consistent imports, and have one file that’ll work for many build systems.
REAL PYTHON course

Always Use [closed, open) Intervals

“Intervals or ranges pop-up everywhere in the programming world. The classic example is picking a start and end date, like you would when booking an AirBnB or a flight. Have you ever wondered why they are always implemented as [closed, open) as opposed to [closed, closed]?”
FERNANDO HURTADO CARDENAS

Top 10 Vulns Impacting Open Source in 2022

You might know all about the incredibly useful and insightful OWASP Top 10 list from 2021, but what about the exact CVEs that could be lurking in your applications? Check out Snyk Top 10 Open Source Vulnerability report to get up to date on 2022’s most common vulnerabilities →
SNYK.IO sponsor

16 Reasons to Use VS Code for Developing Jupyter Notebooks

“Visual Studio Code is one of the most popular text editors with a track record of continual improvements. One area where VS Code has been recently innovating is its Jupyter Notebook support.” Read on to see how this might help you.
CHRIS MOFFITT

Python Bytecode Explained

When a Python program is run, the interpreter first parses your code and checks for syntax errors, then it translates it into bytecode instructions. This article explains some of the features of Python bytecode.
MOSER MICHAEL

plydata: Piping for Pandas

The plydata Python package enables you to use the pipe operator, ">>", to chain operations on a pandas dataframe. Read on to learn how to use it and how it compares to the equivalent operation in R.
MARCIN KOZAK • Shared by Marcin

REPL Driven Development

REPL Driven Development is about fast feedback loops during development. It is not about typing code into a terminal window. David talks about this coding workflow and how it is similar to TDD.
DAVID VUJIC • Shared by David Vujic

Python JSONPath With Example

JSONPath is an expression language that is used to parse the JSON data in Python, similar to XPath in XML. This article covers the basics of finding paths in JSON using the library.
SRINIVAS RAMAKRISHNA

Deploy Django, Celery, Redis & Postgres With Docker-Compose

Deployments can be painful. This article describes one approach to deploying Django, Celery, Redis, and Postgres with docker-compose so you can reuse it in your app!
PIOTR PŁOŃSKI • Shared by Piotr Płoński

Deepnote Is a Modern Notebook Where Data Teams Go to Explore, Collaborate, and Solve Hard Problems

Explore data with Python & SQL from your browser. Add context with data visualizations and rich text editing. Share analysis with stakeholders by simply sending a link. Plans start at $0 (free).
DEEPNOTE sponsor

Private, “Protected” Attributes in Python Demystified

A guide to private and protected attributes in Python, learn all about when to use and when not to use leading underscores and double underscores (dunder).
AMIR AFIANIAN • Shared by Amir Afianian

Investigating a Backdoored PyPI Package Targeting FastAPI

Using an open source security scanner, the authors found a backdoored package on PyPI. Read on for details about how they found it and what it contained.
TURCKHEIM & TAFANI-DEREEPER

Projects & Code quickadd: Parse Natural Language Time and Date Expressions

GITHUB.COM/ACREOM

Python Tools for the Polylith Architecture

GITHUB.COM/DAVIDVUJIC • Shared by David Vujic

django-virtual-models: Django ORM Prefetching Layer

GITHUB.COM/VINTASOFTWARE

Colossal-AI: Unified Deep Learning System for Big Model Era

GITHUB.COM/HPCAITECH

pytorch-image-models: Models, Scripts, Pre-Trained Weights

GITHUB.COM/RWIGHTMAN

Events NZPUG-Auckland Coding Challenge “Office Hours”

November 30, 2022
MEETUP.COM

Deep Learning With PyTorch

November 30, 2022
MEETUP.COM

Weekly Real Python Office Hours Q&A (Virtual)

November 30, 2022
REALPYTHON.COM

PyDelhi User Group Meetup

December 3, 2022
MEETUP.COM

Sydney Python User Group (SyPy)

December 1, 2022
SYPY.ORG

Happy Pythoning!
This was PyCoder’s Weekly Issue #553.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

Categories: FLOSS Project Planets

PyCharm: The Second Release Candidate for PyCharm 2022.3 Is Out!

Planet Python - Tue, 2022-11-29 12:21

PyCharm 2022.3 is just around the corner and the second Release Candidate is here!

You can download the new version from our website, update directly from the IDE or via the free Toolbox App, or use snaps for Ubuntu.

Download PyCharm 2022.3 RC

This build requires an active subscription to PyCharm Professional.

Here are the most important improvements in this build:

  • You can now use Conda with remote interpreters, such as interpreters on WSL and Docker. To install the packages on remote Conda environments, please use the Python Packages tool window. [PY-35978]
  • We fixed several issues that were affecting the use of Python interpreters on Docker and Docker Compose, including the issue preventing users from choosing a specific interpreter for their run/debug configurations. [PY-55360], [PY-57484], [PY-57460].
  • Debug Console now works as expected after starting the debugging process. [PY-57296]
  • Mapping now works normally for debugging files on WSL. [PY-55800
  • On Windows, Python Console now has the UTF-8 encoding by default [PY-57519]

Please share your feedback in the comments, on Twitter, or on our issue tracker. Your feedback is highly appreciated!

Categories: FLOSS Project Planets

Start the holiday season by becoming a member of the Open Source Initiative.

Open Source Initiative - Tue, 2022-11-29 11:15

Together we can make a difference – become an OSI member today. Only with a strong...

The post Start the holiday season by becoming a member of the Open Source Initiative. first appeared on Voices of Open Source.

Categories: FLOSS Research

Ixis.co.uk - Thoughts: 5 Top Tips to Securing Your Drupal Website

Planet Drupal - Tue, 2022-11-29 09:10
Keeping your website secure is vital for every website and business; in this blog, we cover 5 top tips to keep your Drupal site secure.
Categories: FLOSS Project Planets

Real Python: Using Python's pathlib Module

Planet Python - Tue, 2022-11-29 09:00

Have you struggled with file path handling in Python? With the pathlib module, the struggle is now over! You no longer need to scratch your head over code like this:

>>>>>> path.rsplit('\\', maxsplit=1)[0]

And you don’t have to cringe at the verbosity of something like this:

>>>>>> os.path.isfile(os.path.join(os.path.expanduser('~'), 'realpython.txt'))

In this video course, you’ll learn how to:

  • Work with file paths in Python
  • Read and write files in new ways
  • Manipulate paths and the underlying file system
  • List files and iterate over them

Using the pathlib module, the two examples above can be rewritten using elegant, readable, and Pythonic code:

>>>>>> path.parent >>> (pathlib.Path.home() / 'realpython.txt').is_file()

That’s what you’ll master in this video course!

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Categories: FLOSS Project Planets

Django Weblog: 2023 DSF Board Election Results

Planet Python - Tue, 2022-11-29 08:19

The 2023 Board is transitional to our new staggered 2 year term board membership model.
Here are the results of this year's election as selected by ranked choice voting:

  1. Chaim Kirby*
  2. Jacob Kaplan-Moss
  3. Katie McLaughlin*
  4. Aaron Bassett
  5. Kátia Yoshime Nakamura*
  6. Peter Baumgartner
  7. Cagil Ulusahin Sonmez*
* Elected to a two (2) year term

Congratulations to our winners and a huge thank you to our departing board members Anna Makarudze, Mfon Eti-mfon, William Vincent, and Žan Anderle. You all set the bar very high for the next board.

Also a special thank you to the 30 candidates we had this year. The DSF simply isn't possible without the help of all of our volunteers.

Categories: FLOSS Project Planets

Pages