Planet Python

This is the next video in my Python Built-ins Series that I started last week.

Learn a little about aiter() and anext() in this one!

The post The Python Built-in Functions – aiter and anext (Video) appeared first on Mouse Vs Python.

Installing packages with Python is just one pip install command away. That’s one of the many great qualities that the Python ecosystem has to offer.

However, you may have downloaded a third-party package once that didn’t work out for you in one way or another. For example, the package didn’t support the Python version that you were using in your project, or the package didn’t do what you expected it to do.

By understanding the characteristics of a high-quality Python package, you can avoid introducing incompatible or even harmful code into your project. In this tutorial, you’ll learn how the Python Package Index can give you a first impression of a package. Then you’ll dig even deeper by checking out Libraries.io, the GitHub repository, and the license of any Python package that you want to use.

In the end, you’ll know how to evaluate third-party packages that you can find online before you implement them into your Python projects.

For future reference, you can also download this handy flowchart that’ll help you decide if a third-party Python package works for your particular situation:

Bonus Download: Click here to download the free flowchart that you can use to evaluate the quality of Python packages.

How to Evaluate the Quality of Third-Party Python Packages

The Python Package Index (PyPI) provides the largest collection of external Python packages. Before you install a package with pip, you should make sure that it’s available on PyPI:

Being able to find the package on PyPI is a good indicator that it’s legit, although you still need to be careful about what you’re getting.

Note: It’s possible to pip install a package from a Git repository or directly install a Python wheel you find online.

While you can download a Python package from anywhere, you should ask yourself if the package is high-quality and safe.

On PyPI, you’ve got the option to either browse categories or search for keywords. Unless you’re looking for a very niche package, chances are that PyPI will present you with a list of thousands of packages that match your topic.

To sort the order of the list, PyPI gives you two options:

1. Relevance
2. Date last updated

The option to sort by Relevance is ambiguous because you don’t know what PyPI takes into consideration for this order. Still, when you sort by relevance, the packages appearing on top are indeed often the ones that suit your needs:

Sometimes, it can be tricky to find the right package on PyPI, even if you know the package’s name. For example, when you search for beautiful soup, then you’ll get a number of similar-looking results. That’s when Relevance can come in handy.

Out of curiosity, you may also peek into the packages that have been updated lately by using the Date last updated order. But usually the sorting makes irrelevant packages appear on top, just because they were updated recently.

In most cases, a package doesn’t need to be cutting-edge unless it addresses a security issue. Instead, it’s important that a package supports the Python version of your project. This is when the PyPI filter comes into play:

For example, when you want to use the cool new features of Python 3.11 to full capacity, then you can select Python 3.11 in the list of Programming Languages filters.

Additionally, you can combine filters on PyPI. It’s a good idea to also add Development Status into your consideration by filtering for Production/Stable, too. This way, you increase the chances of working with reliable packages that have gone through thorough testing.

When you’ve found a package that seems to fit your needs, then it’s time to put it under the microscope to make sure it’s safe and reliable. For this evaluation, you’ll study the PyPI details page. Clicking a package name brings you to a page that’s dedicated to the package:

Read the full article at https://realpython.com/python-package-quality/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Pandas dataframes are used to manipulate tabular data in python. While data manipulation, we sometimes need to convert data from other python objects such as lists, strings, and tuples into a dataframe. During conversion, you might get into an exception with the message ValueError: DataFrame constructor not properly called! This article discusses the ValueError: DataFrame constructor not properly called! error, its causes, and the solutions.

Table of Contents

1. What Is ValueError: Dataframe Constructor Not Properly Called! Error in Python?
2. When Does the ValueError: DataFrame Constructor Not Properly Called Error Occur?
   1. When We Pass a String as the Input to the DataFrame() Function
   2. We Pass a String to the DataFrame() Function
   3. When We Pass a Scalar Value to the DataFrame() Function
3. How to Solve ValueError: DataFrame Constructor Not Properly Called Exception in Python?
4. Use the columns Parameter to Assign Column Names to the Dataframe
   1. Pass a List of Strings as Input to the DataFrame() Function
   2. Convert Strings Into Python Objects Before Passing Them to the DataFrame() Function
5. Conclusion

What Is ValueError: Dataframe Constructor Not Properly Called! Error in Python?

As the message suggests the error “ValueError: DataFrame constructor not properly called! ” is a python ValueError exception. It means that the error occurs when we pass an incompatible value as input to the DataFrame() function. This may happen in the following cases.

1. We pass a string as the input to the DataFrame() function.
2. When we pass a string representation of a list instead of a list to the DataFrame() function.
3. We pass a JSON string directly to the DataFrame() function. 
4. When we pass a string representation of a python dictionary instead of a dictionary to the DataFrame() function.
5. We pass other scalar values such as integers or floating point numbers to the  DataFrame() function. 

When Does the ValueError: DataFrame Constructor Not Properly Called Error Occur?

As introduced above the exception occurs in five cases. Let us discuss each of them one by one.

When We Pass a String as the Input to the DataFrame() Function

In most instances,  ValueError: DataFrame constructor not properly called error occurs when we try to create an empty dataframe with a given column name. When we pass the column name directly to the DataFrame() function, the program runs into the ValueError exception. You can observe this in the following example.

import pandas as pd columnName="Column1" print("The column name is:") print(columnName) df=pd.DataFrame(columnName) print("The dataframe is:") print(df)

Output:

The column name is: Column1 ValueError: DataFrame constructor not properly called!

In this example, we tried to create an empty dataframe with the column name Column1. As we passed the column name directly to the DataFrame() function, the program runs into ValueError exception.

We Pass a String to the DataFrame() Function

We often create a dataframe from a list in python. You might think that you can also create a dataframe of characters in the string by passing the string to the DataFrame() function as an input argument. However, the program runs into the ValueError: DataFrame constructor not properly called!  Exception.

You can observe this in the following example.

import pandas as pd myStr="PFB" print("The string is:") print(myStr) df=pd.DataFrame(myStr) print("The dataframe is:") print(df)

Output:

The string is: PFB ValueError: DataFrame constructor not properly called!

In this example, we passed the string "PFB" to the DataFrame() function to create a dataframe. Due to this, the program runs into ValueError Exception.

When we pass a string representation of a list to the DataFrame() function, the program runs into ValueError: DataFrame constructor not properly called exception as shown below.

import pandas as pd listStr='[1,22,333,4444,55555]' print("The list string is:") print(listStr) df=pd.DataFrame(listStr) print("The dataframe is:") print(df)

Output:

The list string is: [1,22,333,4444,55555] ValueError: DataFrame constructor not properly called!

In the above example, we passed a string "[1,22,333,4444,55555]" to the DataFrame() function. Due to this, the program runs into ValueError exception.

In a similar manner, when we pass a string representation of a dictionary to the DataFrame() function, the program runs into a ValueError exception as shown in the following example.

import pandas as pd dictStr='{"Roll":1,"Maths":100, "Physics":80, "Chemistry": 90}' print("The dictionary string is:") print(dictStr) df=pd.DataFrame(dictStr) print("The dataframe is:") print(df)

Output:

The dictionary string is: {"Roll":1,"Maths":100, "Physics":80, "Chemistry": 90} ValueError: DataFrame constructor not properly called!

In some cases, we might also directly try to convert a JSON string into a pandas dataframe using the DataFrame() function. In these cases, the program will run into errors as shown below.

import pandas as pd jsonStr='{"Roll":1,"Maths":100, "Physics":80, "Chemistry": 90}' print("The json string is:") print(jsonStr) df=pd.DataFrame(jsonStr) print("The dataframe is:") print(df)

Output:

The json string is: {"Roll":1,"Maths":100, "Physics":80, "Chemistry": 90} ValueError: DataFrame constructor not properly called! When We Pass a Scalar Value to the DataFrame() Function

We can create a dataframe from an iterable object such as a list, tuple, set, or dictionary. However, when we pass an object of primitive data types such as integer or floating point number as input to the DataFrame() function, the program runs into the ValueError exception with the message ValueError: DataFrame constructor not properly called!.

You can observe this in the following example.

import pandas as pd myInt=1117 print("The integer is:") print(myInt) df=pd.DataFrame(myInt) print("The dataframe is:") print(df)

Output:

The integer is: 1117 ValueError: DataFrame constructor not properly called!

In this example, we passed the integer 1117 to the DataFrame() function. Due to this, the program runs into ValueError exception.

How to Solve ValueError: DataFrame Constructor Not Properly Called Exception in Python?

In accordance with the reasons of the error, we can solve the ValueError: DataFrame constructor not properly called exception using various ways.

Use the columns Parameter to Assign Column Names to the Dataframe

The first way to solve the  ValueError: DataFrame constructor not properly called exception in Python is to not pass a string directly to the DataFrame() constructor. If you are trying to create a dataframe with a given column name as a string, use the columns parameter in the constructor as shown below.

import pandas as pd columnName="Column1" print("The column name is:") print(columnName) df=pd.DataFrame(columns=[columnName]) print("The dataframe is:") print(df)

Output:

The column name is: Column1 The dataframe is: Empty DataFrame Columns: [Column1] Index: []

In this example, we passed the string "Column1" to the columns parameter after putting it in a list. Due to this, the program executes successfully and we get an empty dataframe with the given column name.

Pass a List of Strings as Input to the DataFrame() Function

If you want to create a dataframe from the characters of the list. You can first convert the string to a list of characters. Then, you can pass the list as input to the DataFrame() constructor to avoid the  ValueError: DataFrame constructor not properly called exception in Python. You can observe this in the following example.

import pandas as pd myStr="PFB" print("The string is:") print(myStr) df=pd.DataFrame(list(myStr)) print("The dataframe is:") print(df)

Output:

The string is: PFB The dataframe is: 0 0 P 1 F 2 B

In this example, we first created a list of characters using the string "PFB" and the list() function. Then, we passed the list of characters to the the DataFrame() function to create the output dataframe.

If you want to put the string as an element of the data frame, you can put the string in a list and then pass the list to the DataFrame() function as shown below.

import pandas as pd myStr="PFB" print("The string is:") print(myStr) df=pd.DataFrame([myStr]) print("The dataframe is:") print(df)

Output:

The string is: PFB The dataframe is: 0 0 PFB Convert Strings Into Python Objects Before Passing Them to the DataFrame() Function

If you want to convert a JSON string to a dataframe, first convert the json string to a python dictionary. Then, you can pass the dictionary to the DataFrame() function as shown below.

import pandas as pd import json jsonStr='{"Roll":1,"Maths":100, "Physics":80, "Chemistry": 90}' print("The json string is:") print(jsonStr) myDict=json.loads(jsonStr) df=pd.DataFrame([myDict]) print("The dataframe is:") print(df)

Output:

The json string is: {"Roll":1,"Maths":100, "Physics":80, "Chemistry": 90} The dataframe is: Roll Maths Physics Chemistry 0 1 100 80 90

If you have a string representation of a list or dictionary and you want to convert it into a dataframe, first convert the string into a list or dictionary. For this, you can use the eval() method. The eval() method takes the string representation of the list or dictionary and converts them into a python list or dictionary respectively. After this, you can use the list to create a dataframe as shown below.

import pandas as pd dictStr='{"Roll":1,"Maths":100, "Physics":80, "Chemistry": 90}' print("The dictionary string is:") print(dictStr) myDict=eval(dictStr) df=pd.DataFrame([myDict]) print("The dataframe is:") print(df)

Output:

The dictionary string is: {"Roll":1,"Maths":100, "Physics":80, "Chemistry": 90} The dataframe is: Roll Maths Physics Chemistry 0 1 100 80 90 Conclusion

In this article, we discussed the  ValueError: DataFrame constructor not properly called exception in Python. We also discussed the possible cause and solutions for this error. To learn more about python programming, you can read this article on how to overwrite a file in python. You might also like this article on CPython vs Python.

I hope you enjoyed reading this article. Stay tuned for more informative articles!

Happy Learning!

The post Solved: Dataframe Constructor Not Properly Called Error in Pandas appeared first on PythonForBeginners.com.

This content first appeared on our friends list, you can subscribe here.

I remember a long time ago I applied for a job and … I never heard back.

I applied for another job… rejected.

And for yet another job one of my former colleagues got it. 

As I had to deal with disappointment, I found it tough to not give up, to not to question myself. Luckily, I did get myself up after multiple “failures” and kept putting in the reps letting data trump emotion (by the way, I don’t believe in the word “failure”. We learn more from our failures than our successes!)

And after a while I did get a different job that, while I could not see it at the time, was strategically better and ended up changing my career.

Setbacks, disappointment, not meeting expectations. It can be very hard to deal with. We get rejected, ghosted, maybe even ridiculed, and we have to keep believing in ourselves, in spite of what seems the “evidence” that “this might not be for me”.

But here’s the thing. People that do keep at it, no matter what, end at the top. I know this is a cliche, but we’ve seen success story after success story where this has been the case. People were not handed position Y, they had to fight for it. These are people that often came from very humble beginnings.

So, this week we just want you to think about any disappointment you’ve had or might be going through right now and reflect on how it’s affecting your next steps.

And we’d like to ask you: What are you going to do next? Push, learn and grow? Or throw in the towel and give up?

The former will propel you forward, it’s what you can control. The latter is destructive and a downward spiral, getting you nowhere.

A few tips to better cope with this:

• Take a short break. We have to process a lot and sometimes just letting things go for a bit makes all the difference. Even a good night of sleep, a weekend “offline” can totally change your perspective. Related podcast.
  
• Find an accountability buddy to vent, share ideas and hold you to higher standards. Talk with other people, loneliness exacerbates the problem. Join our Slack community and engage with the great Pytonistas that have joined over the years. Related podcast.
  
• Reflect on the situation and think about what you’ll do differently next time (because there will be a next time). A great tool is to keep a journal. Some people do it daily and say it greatly helps them reducing overall anxiety, but that daily habit can be hard, so at the very least do it when you’re going through tough times. On the flip side we encourage you to also note down your weekly wins (even “small” ones), this can be tremendously empowering. Related article.
  
• Give back. Helping others is a great way to be less self centered while still building your portfolio with work you can show for / talk about. Doubts about your work? Externalizing the goal can be very motivating and again you’ll find accountability and other people to collaborate with.
  
• Drop perfectionism. One setback does not ruin everything. Again, from failure you learn, and you often can still get back on track and finish strong. Often not everything is lost, it’s all about the averages and coming out stronger. Related podcast.

We hope these tips help you better handle setbacks.

– Bob and Julian

As a Python developer it can be tough to go at it alone, especially when disappointment and obstacles inevitably happen (be it with building apps, writing quality code, and finding a dev job or seeking a promotion).

Our PDM community is an incredibly supportive group of people that are often in a similar situation as you. The results people achieve both technically and mindset wise, boosts their confidence and resilience, and lifetime relations form, we’re really thrilled what our program has grown into. 

So, if you want to grow your career, better deal with the imposter syndrome we all face as developers, our PDM program is the place to make that life altering change.

Watch here:

Listen here:

Welcome back to the podcast. Today we share 8 tips in response to a question that we were tagged on @ Twitter:

1. Communication is everything.
2. Deliberate practice.
3. Adopt a growth mindset.
4. Be a generalist.
5. Focus on the “compound movements”.
6. Know the business domain you are in.
7. Share your work / teach others.
8. Network every single week.

As always we also discuss wins and books.

Links:

• Pybites Flashcards
• PDM program

Mentioned books:

• Mindset
• Peak
• Grit
• 177 mental toughness secrets
• The obstacle is the way
• Range
• Domain-driven design
• All books mentioned on the podcast

And last but not least thanks for all your feedback

You can reach out to us through our Slack or send an email to info at pybit dot es.

Sometimes you want to be able to pass command line arguments to your GUI applications. For example, you may want to be able to pass files which the application should open, or change the initial startup state.

In that case it can be helpful to be able to pass custom command-line arguments to your application. Qt supports this using the QCommandLineOption and QCommandLineParser classes, the first defining optional parameters to be identified from the command line, and the latter to actually perform the parsing.

In this short tutorial we'll create a small demo application which accepts arguments on the command line.

Building the Parser

We'll start by creating an outline of our application. As usual we create a QApplication instance, however, we won't initially create any windows. Instead, we construct our command-line parser function, which accepts our app instance and uses QCommandLineParser to parse the command line arguments.

• PyQt6
• PySide6

python import sys from PyQt6.QtWidgets import QApplication from PyQt6.QtCore import QCommandLineOption, QCommandLineParser def parse(app): """Parse the arguments and options of the given app object.""" parser = QCommandLineParser() parser.addHelpOption() parser.addVersionOption() parser.process(app) app = QApplication(sys.argv) app.setApplicationName("My Application") app.setApplicationVersion("1.0") parse(app) python import sys from PySide6.QtWidgets import QApplication from PySide6.QtCore import QCommandLineOption, QCommandLineParser def parse(app): """Parse the arguments and options of the given app object.""" parser = QCommandLineParser() parser.addHelpOption() parser.addVersionOption() parser.process(app) app = QApplication(sys.argv) app.setApplicationName("My Application") app.setApplicationVersion("1.0") parse(app)

We don't call app.exec() yet, to start the event loop. That's because we don't have any windows, so there would be no way to exit the app once it is started! We'll create the windows in a later step.

It is important to note that you must pass sys.argv to QApplication for the command line parser to work -- the parser processes the arguments from app. If you don't pass the arguments, there won't be anything to process.

With this outline structure in place we can move on to creating our command line options.

Adding Optional Parameters

We'll add two optional parameters -- the first to allow users to toggle the window start up as maximized, and the second to pass a Qt style to use for drawing the window.

The available styles are 'windows', 'windowsvista', 'fusion' and 'macos' -- although the platform specific styles are only available on their own platform. If you use macos on Windows, or vice versa, it will have no effect. However, 'fusion' can be used on all platforms.

• PyQt6
• PySide6

python import sys from PyQt6.QtWidgets import QApplication from PyQt6.QtCore import QCommandLineOption, QCommandLineParser QT_STYLES = ["windows", "windowsvista", "fusion", "macos"] def parse(app): """Parse the arguments and options of the given app object.""" parser = QCommandLineParser() parser.addHelpOption() parser.addVersionOption() maximize_option = QCommandLineOption( ["m", "maximize"], "Maximize the window on startup." ) parser.addOption(maximize_option) style_option = QCommandLineOption( "s", "Use the specified Qt style, one of: " + ', '.join(QT_STYLES), "style" ) parser.addOption(style_option) parser.process(app) app = QApplication(sys.argv) app.setApplicationName("My Application") app.setApplicationVersion("1.0") parse(app) app.exec() python import sys from PySide6.QtWidgets import QApplication from PySide6.QtCore import QCommandLineOption, QCommandLineParser QT_STYLES = ["windows", "windowsvista", "fusion", "macos"] def parse(app): """Parse the arguments and options of the given app object.""" parser = QCommandLineParser() parser.addHelpOption() parser.addVersionOption() maximize_option = QCommandLineOption( ["m", "maximize"], "Maximize the window on startup." ) parser.addOption(maximize_option) style_option = QCommandLineOption( "s", "Use the specified Qt style, one of: " + ', '.join(QT_STYLES), "style" ) parser.addOption(style_option) parser.process(app) app = QApplication(sys.argv) app.setApplicationName("My Application") app.setApplicationVersion("1.0") parse(app) app.exec()

The optional arguments are now in place. We'll now add positional arguments, which will be used to open specific files.

Adding Positional Arguments

Strictly speaking, positional arguments are any arguments not interpreted as optional arguments. You can define multiple positional arguments if you like but this is only used for help text display. You will still need to handle them yourself internally, and limit the number if necessary by throwing an error.

In our example we are specifying a single position argument file, but noting in the help text that you can provide more than one. There is no limit in our example -- if you pass more files, more windows will open.

• PyQt6
• PySide6

python import sys from PyQt6.QtWidgets import QApplication from PyQt6.QtCore import QCommandLineOption, QCommandLineParser QT_STYLES = ["windows", "windowsvista", "fusion", "macos"] def parse(app): """Parse the arguments and options of the given app object.""" parser = QCommandLineParser() parser.addHelpOption() parser.addVersionOption() parser.addPositionalArgument("file", "Files to open.", "[file file file...]") maximize_option = QCommandLineOption( ["m", "maximize"], "Maximize the window on startup." ) parser.addOption(maximize_option) style_option = QCommandLineOption( "s", "Use the specified Qt style, one of: " + ', '.join(QT_STYLES), "style" ) parser.addOption(style_option) parser.process(app) has_maximize_option = parser.isSet(maximize_option) app_style = parser.value(style_option) # Check for positional arguments (files to open). arguments = parser.positionalArguments() print("Has maximize option?", has_maximize_option) print("App style:", app_style) print("Arguments (files): ", arguments) app = QApplication(sys.argv) app.setApplicationName("My Application") app.setApplicationVersion("1.0") parse(app) app.exec() python import sys from PySide6.QtWidgets import QApplication from PySide6.QtCore import QCommandLineOption, QCommandLineParser QT_STYLES = ["windows", "windowsvista", "fusion", "macos"] def parse(app): """Parse the arguments and options of the given app object.""" parser = QCommandLineParser() parser.addHelpOption() parser.addVersionOption() parser.addPositionalArgument("file", "Files to open.", "[file file file...]") maximize_option = QCommandLineOption( ["m", "maximize"], "Maximize the window on startup." ) parser.addOption(maximize_option) style_option = QCommandLineOption( "s", "Use the specified Qt style, one of: " + ', '.join(QT_STYLES), "style" ) parser.addOption(style_option) parser.process(app) has_maximize_option = parser.isSet(maximize_option) app_style = parser.value(style_option) # Check for positional arguments (files to open). arguments = parser.positionalArguments() print("Has maximize option?", has_maximize_option) print("App style:", app_style) print("Arguments (files): ", arguments) app = QApplication(sys.argv) app.setApplicationName("My Application") app.setApplicationVersion("1.0") parse(app) app.exec()

We've added a series of print calls to display the arguments and options extracted by Qt's QCommandLineParser. If you run the application now you can experiment by passing different arguments and seeing the result on the command line.

For example --- with no arguments.

bash $ python command_line.py Has maximize option? False App style:

With -m maximize flag and a single file

bash $ python command_line.py -m example.txt Has maximize option? True App style: Arguments (files): ['example.txt']

With a single file and using Fusion style -- there is no window yet, so this will have effect yet!

bash $ python command_line.py -s fusion example.txt Has maximize option? False App style: fusion Arguments (files): ['example.txt']

With the argument handling in place, we can now write the remainder of the example.

Using the Arguments & Options

We'll be using a standard QPlainTextEdit widget as our file viewer. In Qt any widget without a parent is a window, so these editors will be floating independently on the desktop. If the -m flag is used we'll set these windows to be displayed maximized.

If windows are created, we'll need to start the Qt event loop to draw the windows and allow interaction with them. If no windows are created, we'll want to show the command-line help to help the user understand why nothing is happening! This will output the format of position and optional arguments that our app takes.

• PyQt6
• PySide6

python import sys from PySide6.QtWidgets import QApplication, QPlainTextEdit from PySide6.QtCore import QCommandLineOption, QCommandLineParser QT_STYLES = ["windows", "windowsvista", "fusion", "macos"] windows = [] # Store references to our created windows. def parse(app): """Parse the arguments and options of the given app object.""" parser = QCommandLineParser() parser.addHelpOption() parser.addVersionOption() parser.addPositionalArgument("file", "Files to open.", "[file file file...]") maximize_option = QCommandLineOption( ["m", "maximize"], "Maximize the window on startup." ) parser.addOption(maximize_option) style_option = QCommandLineOption( "s", "Use the specified Qt style, one of: " + ', '.join(QT_STYLES), "style" ) parser.addOption(style_option) parser.process(app) has_maximize_option = parser.isSet(maximize_option) app_style = parser.value(style_option) if app_style and app_style in QT_STYLES: app.setStyle(app_style) # Check for positional arguments (files to open). arguments = parser.positionalArguments() # Iterate all arguments and open the files. for tfile in arguments: try: with open(tfile, 'r') as f: text = f.read() except Exception: # Skip this file if there is an error. continue window = QPlainTextEdit(text) # Open the file in a maximized window, if set. if has_maximize_option: window.showMaximized() # Keep a reference to the window in our global list, to stop them # being deleted. We can test this list to see whether to show the # help (below) or start the event loop (at the bottom). windows.append(window) if not windows: # If we haven't created any windows, show the help. parser.showHelp() app = QApplication(sys.argv) app.setApplicationName("My Application") app.setApplicationVersion("1.0") parse(app) if windows: # We've created windows, start the event loop. app.exec() python import sys from PySide6.QtWidgets import QApplication, QPlainTextEdit from PySide6.QtCore import QCommandLineOption, QCommandLineParser QT_STYLES = ["windows", "windowsvista", "fusion", "macos"] windows = [] # Store references to our created windows. def parse(app): """Parse the arguments and options of the given app object.""" parser = QCommandLineParser() parser.addHelpOption() parser.addVersionOption() parser.addPositionalArgument("file", "Files to open.", "[file file file...]") maximize_option = QCommandLineOption( ["m", "maximize"], "Maximize the window on startup." ) parser.addOption(maximize_option) style_option = QCommandLineOption( "s", "Use the specified Qt style, one of: " + ', '.join(QT_STYLES), "style" ) parser.addOption(style_option) parser.process(app) has_maximize_option = parser.isSet(maximize_option) app_style = parser.value(style_option) if app_style and app_style in QT_STYLES: app.setStyle(app_style) # Check for positional arguments (files to open). arguments = parser.positionalArguments() # Iterate all arguments and open the files. for tfile in arguments: try: with open(tfile, 'r') as f: text = f.read() except Exception: # Skip this file if there is an error. continue window = QPlainTextEdit(text) # Open the file in a maximized window, if set. if has_maximize_option: window.showMaximized() # Keep a reference to the window in our global list, to stop them # being deleted. We can test this list to see whether to show the # help (below) or start the event loop (at the bottom). windows.append(window) if not windows: # If we haven't created any windows, show the help. parser.showHelp() app = QApplication(sys.argv) app.setApplicationName("My Application") app.setApplicationVersion("1.0") parse(app) if windows: # We've created windows, start the event loop. app.exec()

The arguments are handled and processed as before however, now they actually have an effect!

Firstly, if the user passes the -s <style> option we will apply the specified style to our app instance -- first checking to see if it is one of the known valid styles.

python if app_style and app_style in QT_STYLES: app.setStyle(app_style)

Next we take the list of position arguments and iterate, creating a QPlainTextEdit window and displaying the text in it. If has_maximize_option has been set, these windows are all set to be maximized with window.showMaximized().

References to the windows are stored in a global list windows, so they are not cleaned up (deleted) on exiting the function. After creating windows we test to see if this is empty, and if not show the help:

bash $ python command_line.py Usage: command_line.py [options] [file file file...] Options: -?, -h, --help Displays help on commandline options. --help-all Displays help including Qt specific options. -v, --version Displays version information. -m, --maximize Maximize the window on startup. -s <style> Use the specified Qt style, one of: windows, windowsvista, fusion, macos Arguments: file Files to open.

If there are windows, we finally start up the event loop to display them and allow the user to interact with the application.

python if windows: # We've created windows, start the event loop. app.exec() Conclusion

In this short tutorial we've learned how to develop an app which accepts custom arguments and options on the command line, and uses them to modify the UI behavior. You can use this same approach in your own applications to provide command-line control over the behavior of your own applications.

For an in-depth guide to building GUIs with Python see my PySide6 book.

Django is a popular web framework for Python developers, known for its robustness, flexibility, and security. One of the features that make Django powerful is its signal system. Signals allow developers to trigger certain actions when specific events occur, such as when a model is saved or deleted. However, there is often confusion about whether Django signals are asynchronous or not. In this article, we will explore this question and discuss the tradeoffs associated with using Django signals.

#568 – MARCH 14, 2023
View in Browser »

Sharing Your Python App Across Platforms With BeeWare

Are you interested in deploying your Python project everywhere? This week on the show, Russell Keith-Magee, founder and maintainer of the BeeWare project, returns. Russell shares recent updates to Briefcase, a tool that converts a Python application into native installers on macOS, Windows, Linux, and mobile devices.
REAL PYTHON podcast

Overhead of Python Asyncio Tasks

The Textual library uses a lot of asyncio tasks. In order to determine whether to spend time optimizing them, Will measured the cost of creating asyncio tasks. TLDR; optimize something else. This article also spawned a conversation on Hacker News.
WILL MCGUGAN

Retire Your Legacy CMS With ButterCMS

ButterCMS is your new content backend. We’re SaaS so we host, maintain, and scale the CMS. Enable your marketing team to update website + app content without needing you. Try the #1 rated SaaS Headless CMS for your Python app today. Free for 30 days →
BUTTERCMS sponsor

Julia and Python Better Together

Julia is a popular programming language among data scientists, but if you ever code in that space and miss some of the key Python libraries, this article is for you. Julia packages can wrap other languages, so you can have the best of both worlds.
BOGUMIŁ KAMIŃSKI

Python 3.12.0 Alpha 6 Released

CPYTHON DEV BLOG

Django Developers Survey 2022 Results Available

DJANGO SOFTWARE FOUNDATION

Discussions Nearly 40% of Software Engineers Will Only Work Remotely

HACKER NEWS

Articles & Tutorials Predicting Wine Quality Using Chemical Properties

Alfredo discovered some datasets about wine quality including chemical properties and decided it was time to do some predictive model building. This article walks you through what he did with torch, sklearn, numpy, pandas, and seaborn to predict wine quality.
ALFREDO GONZÁLEZ-ESPINOZA

Are Those Numbers Realistic or Fake? Try Using Benford’s Law

How can you tell whether a set of figures is trustworthy? It’s not always simple, but Benford’s Law gives you one way to find out. There’s even a Python Package to help you check: randalyze.
JASON ROSS

The Best Way to Structure Your NoSQL Data Using Python

Data modeling can be challenging. The question that most often comes up is, “How do I structure my data?” The short answer: it depends. That’s why the Redis folks wrote a comprehensive e-book that goes through 8 different optimal scenarios and shows how to model them in Redis →
REDIS LABS sponsor

Building a ChatGPT-based Assistant With Python

This article demonstrates a workflow for integrating multiple AI services to perform speech-to-text (STT), natural language processing (NLP), and text-to-speech (TTS) using OpenAI’s ChatGPT and Whisper API’s in Python.
FAIZANBASHIR.ME • Shared by Faizan Bashir

How to Strangle Old Code Using Python Decorators

The “strangler pattern” is a way of simultaneously running old and replacement code to determine through live behavior whether the replacement works. This article shows you how to use decorators to help.
JON JAGGER

Deploying a Django App to Azure App Service

This tutorial looks at how to deploy a Django application to Azure App Service. Includes details on configuring your Django and what Azure services to use to get going quickly.
NIK TOMAZIC

Python’s Mutable vs Immutable Types: What’s the Difference?

In this tutorial, you’ll learn how Python mutable and immutable data types work internally and how you can take advantage of mutability or immutability to power your code.
REAL PYTHON

Formatting Gone Wrong

Your code formatter may have reformatted your API key. This can cause confusing errors. Read Kojo’s tale to learn what happened and how he figured it out.
KOJO IDRISSA

De-Duplicating a List While Maintaining Order

There’s more than one way deduplicate an iterable in Python. Which approach you take will depend on whether you care about the order of your items.
TREY HUNNER • Shared by Trey Hunner

Find Your Next Tech Job Through Hired

Hired is home to thousands of companies, from startups to Fortune 500s, that are actively hiring the best engineers, designers, data scientists, and more. Create a profile to let hiring managers extend interview requests to you. Sign up for free today!
HIRED sponsor

Python Assertions, or Checking if a Cat Is a Dog

The articles explains the rules of using assertions in Python, and when not to use them. Includes details on the __debug__ object.
MARCIN KOZAK • Shared by Marcin

XKCD-style Plots in Python With Matplotlib

This short article shows how to add a twist to your matplotlib plots by styling them like the web-famous comic xkcd.
RODRIGO GIRÃO SERRÃO • Shared by Rodrigo Girão Serrão

Projects & Code Interact With ChatGPT Through a Single-File Python Script

GITHUB.COM/REORX • Shared by Reorx

meerkat: Interactive Data Frames for Unstructured Data

GITHUB.COM/HAZYRESEARCH

sketch: AI Code-Writing Assistant for Pandas

GITHUB.COM/APPROXIMATELABS

replbuilder: Tool for Building Custom REPLs

GITHUB.COM/APEROCKY

pylyzer: A Fast Static Code Analyzer for Python

GITHUB.COM/MTSHIBA

Events Python Web Conf 2023

March 13 to March 18, 2023
PYTHONWEBCONF.COM

The Long White Computing Cloud

March 15, 2023
MEETUP.COM

Weekly Real Python Office Hours Q&A (Virtual)

March 15, 2023
REALPYTHON.COM

PyData Bristol Meetup

March 16, 2023
MEETUP.COM

PyLadies Dublin

March 16, 2023
PYLADIES.COM

Chattanooga Python User Group

March 17 to March 18, 2023
MEETUP.COM

PyCascades

March 18 to March 20, 2023 in Vancouver, BC + Remote
PYCASCADES.COM

Happy Pythoning!
This was PyCoder’s Weekly Issue #568.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

Sphinx is a document generation tool that’s become the de facto standard for Python projects. It uses the reStructuredText (RST) markup language to define document structure and styling, and it can output in a wide variety of formats, including HTML, ePub, man pages, and much more. Sphinx is extendable and has plugins for incorporating pydoc comments from your code into your docs and for using MyST Markdown instead of RST.

Read the Docs is a free document hosting site where many Python projects host their documentation. It integrates with GitHub, GitLab, and Bitbucket to automatically pull new documentation sources from your repositories and build their Sphinx sources.

In this video course, you’ll learn how to:

• Write your documentation with Sphinx
• Structure and style your document with RST syntax
• Incorporate your pydoc comments into your documentation
• Host your documentation on Read the Docs

With these skills, you’ll be able to write clear, reliable documentation that’ll help your users get the most out of your project.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Today I released Eve v2.1, which comes with official Flask 2.2+ support and the ability to modify the pagination limit on a per-resource basis thanks to the new pagination_limit setting. You can find the release on PyPI, while the changelog is available here—special thanks to Pieter De Clercq and smeng9 for the help with this release. Subscribe to the newsletter, the RSS feed, or follow me on Mastodon

Below Python example will use the re module to group the telephone number without the ‘-‘ sign and the period ‘.’.

Let’s say the phone number has been written in this manner.

123-456-7810.

The below python program will group the numbers by taking out both the ‘-‘ sign and ‘.’.

find = re.compile(r'\d+(?=\-|\.)') match = find.findall('123-456-7810.')

The match array now stored the following numbers:-

['123', '456', '7810']

Next what if I just want to extract all the numbers without the ‘-‘ sign? Then use the below program.

find = re.compile(r'\d+(?!\d)') match = find.findall('123-456-7810')

If you’re interested in Django development, you might have come across the django-simple-history package. It’s a great tool that can help you keep track of changes made to your models over time. In this article, we’ll take a closer look at django-simple-history and how it can benefit your projects. What is django-simple-history? django-simple-history is a third-party Django package that provides version control for your models. It allows you to keep track of changes made to your models, including who made the change, when it was made, and what the change was.

I wrote a simple thing to watch the progress of GitHub Actions: watchgha.

I started by using gh run list, and tailoring the output, but that required running the command obsessively to see the changes. Then I tried gh run watch, but I didn’t like the way it worked. You had to pick one action to watch, but my branch has two actions that run automatically, and I need to know when both are done. Plus, it lists all of the steps, which makes my complex action matrix fill more than a screen, so I can’t even see the progress.

So I wrote my own. It buckets together the start times of the runs to present them as a coherent run triggered by a single event. It keeps showing runs as long as they have a new action to show. It presents the steps in a compact form, and collapses jobs and runs once they succeed.

Give it a try and let me know if it works for you.

Listen here:

Or watch here:

Welcome back to the Pybites podcast. This week we have a very special guest: Blaise Pabon.

We talk about his background in software development, how he started with Python and his journey with us in PDM.

We also pick his brains about why Python is such a great language, the importance of open source and his active role in it, including a myriad of developer communities he actively takes part in.

Lastly, we talk about the books we’re currently reading.

Links:
– Vitrina: a portfolio development kit for DevOps
– Boston Python hosts several online meetings a week
– The mother of all (web) demo apps
– Cucumberbdd New contributors ensemble programming sessions
– Reach out to Blaise: blaise at gmail dot com | Slack

Books:
– Antifragile
– The Tombs of Atuan (Earthsea series)
– How to write

February is the shortest month, but it brought no shortage of activity in the Python world! Exciting developments include a new company aiming to improve cloud services for developers, publication of the PyCon US 2023 schedule, and the first release candidate for pandas 2.0.0.

In the world of artifical intelligence, OpenAI has continued to make strides. But while the Big Fix has worked to reduce vulnerabily for programmers, more malicious programs showed up on PyPI.

Read on to dive into the biggest Python news from the last month.

Join Now: Click here to join the Real Python Newsletter and you'll never miss another Python tutorial, course update, or post.

Pydantic Launches Commercial Venture

With over 40 million downloads per month, pydantic is the most-used data validation library in Python. So when its founder, Samuel Colvin, announces successful seed funding, there’s plenty of reason to believe there’s a game changer in the works.

In his announcement, Colvin compares the current state of cloud services to that of a tractor fifteen years after its invention. In both cases, the technology gets the job done, but without much consideration for the person in the driver’s seat.

Colvin’s new company builds on what pydantic has learned about putting developer experience and expertise first. The exact details of this new venture are still under wraps, but it sets out to answer these questions:

What if we could build a platform with the best of all worlds? Taking the final step in reducing the boilerplate and busy-work of building web applications — allowing developers to write nothing more than the core logic which makes their application unique and valuable? (Source)

This doesn’t mean that the open-source project is going anywhere. In fact, pydantic V2 is on its way and will be around seventeen times faster than V1, thanks to being rewritten in Rust.

To stay on up to date on what’s happening with pydantic’s new venture, subscribe to the GitHub issue.

Read the full article at https://realpython.com/python-news-february-2023/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

File handling is one of the first tasks we do while working with data in python. Sometimes, we need to change the contents of the original file or completely overwrite it. This article discusses different ways to overwrite a file in python.

Table of Contents

1. Overwrite a File Using open() Function in Write Mode
2. Using Read Mode and Write Mode Subsequently
3. Overwrite a File Using seek() and truncate() Method in Python
4. Conclusion

Overwrite a File Using open() Function in Write Mode

To overwrite a file in python, you can directly open the file in write mode. For this, you can use the open() function. The open() function takes a string containing the file name as its first input argument and the python literal “w” as its second input argument.

If the file with the given name doesn’t exist in the file system, the open() function creates a new file and returns the file pointer. If the file is already present in the system, the open() function deletes all the file contents and returns the file pointer. 

Once we get the file pointer, we can write the new data to the file using the file pointer and the write() method. The write() method, when invoked on the file pointer, takes the file content as its input argument and writes it to the file. Next, we will close the file using the close() method. After this, we will get the new file with overwritten content.

For instance, suppose that we want to overwrite the following text file in python.

Sample text file

To overwrite the above file, we will use the following code.

file=open("example.txt","w") newText="I am the updated text from python program." file.write(newText) file.close()

The text file after execution of the above code looks as follows.

In this example, we have directly opened the text file in write mode. Due to this, the original content of the file gets erased. After this, we have overwritten the file using new text and closed the file using the close() method.

Using Read Mode and Write Mode Subsequently

In the above example, we cannot read the original file contents. If you want to read the original file contents, modify it and then overwrite the original file, you can use the following steps.

• First, we will open the file in read mode using the open() function. The open() function will return a file pointer after execution. 
• Then, we will read the file contents using the read() method. The read() method, when invoked on the file pointer, returns the file contents as a string. 
• After reading the file contents, we will close the file using the close() method. 
• Now, we will again open the file, but in write mode. 
• After opening the file in write mode, we will overwrite the file contents using the write() method. 
• Finally, we will close the file using the close() method.

After executing the above steps, we can overwrite a file in python. You can observe this in the following example.

file=open("example.txt","r") fileContent=file.read() print("The file contents are:") print(fileContent) file.close() file=open("example.txt","w") newText="I am the updated text from python program." file.write(newText) file.close()

Output:

The file contents are: This a sample text file created for pythonforbeginners.

In this example, we first opened the file in read mode and read the contents as shown above. Then, we overwrote the file contents by opening it again in write mode. After execution of the entire code, the text file looks as shown below.

Modified text file

Although this approach works for us, it is just a workaround. We aren’t actually overwriting the file but reading and writing to the file after opening it separately. Instead of this approach, we can use the seek() and truncate() methods to overwrite a file in python. 

Overwrite a File Using seek() and truncate() Method in Python

To overwrite a file in a single step after reading its contents, we can use the read(), seek() truncate() and write() methods.

• When we read the contents of a file using the read() method, the file pointer is moved to the end of the file. We can use the seek() method to again move to the start of the file. The seek() method, when invoked on the file pointer, takes the desired position of the file pointer as its input argument. After execution, it moves the file pointer to the desired location. 
• To remove the contents of the file after reading it, we can use the truncate() method. The truncate() method, when invoked on a file pointer, truncates the file. In other words, it removes all the file contents after the current position of the file pointer.

To overwrite a file in python using the seek() and truncate() methods, we can use the following steps.

• First, we will open the file in append mode using the open() function. 
• Next, we will read the file contents using the read() method. 
• After reading the file contents, we will move the file pointer to the start of the file using the seek() method. For this, we will invoke the seek() method on the file pointer with 0 as its input. The file pointer will be moved to the first character in the file.
• Next, we will use the truncate() method to delete all the contents of the file.
• After truncating, we will overwrite the file with new file contents using the write() method.
• Finally, we will close the file using the close() method.

After executing the above steps, we can easily overwrite a file in python. You can observe this in the following example.

file=open("example.txt","r+") fileContent=file.read() print("The file contents are:") print(fileContent) newText="I am the updated text from python program." file.seek(0) file.truncate() file.write(newText) file.close()

Output:

The file contents are: This a sample text file created for pythonforbeginners.

After execution of the above code, the text file looks as follows.

Conclusion

In this article, we discussed different ways to overwrite a file in python. To learn more about python programming, you can read this article on how to convert a pandas series into a dataframe. You might also like this article on how to read a file line by line in python.

I hope you enjoyed reading this article. Stay tuned for more informative articles.

Happy Learning!

The post Overwrite a File in Python appeared first on PythonForBeginners.com.

This week we welcome Logan Thomas as our PyDev of the Week! If you’d like to see some of the things that Logan has been up to, you can do so on Logan’s GitHub profile or by visiting his personal website. Logan works at Enthought as a teacher. If you haven’t done so, you should check out their list of courses.

Now let’s spend some time getting to know Logan better!

Can you tell us a little about yourself (hobbies, education, etc):

I currently work for Enthought as a Scientific Software Developer and Technical Trainer. Previously, I worked for a protective design firm as a machine learning engineer and I’ve also been employed as a data scientist in the digital media industry.

My educational background consists of mathematics, statistics, and mechanical engineering. During my final semester of graduate school at the University of Florida, I took a data mining class from the Department of Information Science and was hooked. I’ve worked in data science ever since.

I’m passionate about software and love to (humbly) share my knowledge with others. Working for Enthought gives me the opportunity to help students, and larger organizations, accelerate their journey toward greater digital maturity. We focus on digital transformation – transforming an organization’s business model to take full advantage of the possibilities that new technologies can provide. It’s so fulfilling being able to help scientists and engineers develop the digital skills and tools they need to be able to automate away the trivial tasks and free their minds of this cognitive load to focus on the things they love – the science and engineering behind the problems they face.

Outside of work, I love to stay active by running, brewing a good cup of coffee, and have recently started smoking brisket / pork (a must if you live in Texas like me).

Why did you start using Python?

I was originally hesitant to get into software development as both my parents are software engineers and I wanted my own career path. But, with their encouragement, I took an Introduction to Programming course during my undergraduate degree. The class was taught in Visual Basic and given it was my first true taste of any type of software development, I didn’t have the best experience. Thus, I decided to stay away from computer science and ended up going to graduate school for mechanical engineering. I’m glad the story didn’t end there.

In one of my mechanical design courses, we used a CAD (computer-aided design) program that supported Python scripting (if you knew how to use it!). I remember spending an entire weekend running simulations by hand with Microsoft Excel. When I asked my friend how he managed to get his testing done so quickly, he showed me roughly 10 lines of Python code he wrote to automate the simulation loop. After that exchange, I made it my goal to give computer science another try and take advantage of the superpower a general-purpose programming language like Python could provide.

What other programming languages do you know and which is your favorite?

Since graduate school, I’ve worked in the data science space for about 8 years – all of which my primary programming language has been Python. I love Python for its ease of use and ability to stretch across domain areas like scripting / automation, data analysis / machine learning, and even web design. For me, Python is the clear favorite.

Besides Python, I have the most experience with MATLAB, R, SQL, and Spark. I’ve dabbled in a few other languages as well: Scala, Ruby, C++, HTML / CSS.

What projects are you working on now?

Currently, I am working on building a curriculum for Enthought Academy – a scientific Python catalog of courses for R&D professionals. We have some really great courses in the queue for 2023! I’m putting final touches on our new Data Analysis course that covers Python libraries like NumPy, pandas, Xarray, and Awkward Array.

Personally, I’m working on a new project called Coffee & Code. The idea is to share small code snippets and video shorts for exploring the Python programming language — all during a morning coffee break. My first major project will be trying to teach my brother, who has 0 programming experience, the Python programming language. I plan to film it live and release videos as we walk through installation, environment setup, and language syntax together.

I’m also on the planning committee for two major Python conferences: PyTexas and SciPy Conference. I love both of these events and can’t wait for their 2023 appearance.

Which Python libraries are your favorite (core or 3rd party)?

There are so many to choose from! From the standard library, I have to give a shout-out to the collections module and itertools. The namedtuple, double-ended queue (deque), Counter, and defaultdict are gamechangers. The itertool recipes in the official Python documentation are a goldmine.

My favorite third-party libraries from the scientific Python stack are NumPy and scikit-learn. I use PyTorch and Tensorflow with Keras (in that order) for deep learning. I also have to give credit to IPython. It is my default choice for any type of interactive computing and is quite honestly my development environment of choice (when partnered with Vim). One lesser-known library that I have come to enjoy is DEAP (Distributed Evolutionary Algorithms in Python). I worked with this library extensively for a genetic programming project in the past.

I see you are an instructor at Enthought. What sorts of classes do you teach?

We teach all sorts of classes at Enthought. Given that all of our instructors started out in industry first, we can boast that our curriculum is “for scientists and engineers by scientists and engineers”. We have introduction to Python and programming courses as well as more advanced machine learning and deep learning courses. We also have a software engineering course, data analysis course, data management course, and desktop application development course. All of these live in what we call a learning path or “track”. We have an Enthought Academy Curriculum Map that displays the various tracks we offer: Data Analysis, Machine Learning, Tool Maker, and Manager.

What are some common issues that you see new students struggling with?

I love Python but can admit that there are some rough edges when first getting started. Most students struggle with understanding virtual environments and why they need one if they’ve never encountered dependency management before. Other students are surprised at how many third-party libraries exist in the ecosystem. I often get questions on which ones to use, how to know which ones are safe, and where to look for them.

Some students struggle with IDEs and which development environment to use. I’ve gotten questions like “which is the single best tool: JupyterLab, IPython, VSCode, PyCharm, Spyder, …”. For new students, I try to help them understand that these are tools and there may not be a single correct answer. Sometimes I reach for VSCode when needing to navigate a large code base. If I’m sharing an analysis with a colleague or onboarding a new analyst, I may reach for a Jupyter Notebook. If I’m prototyping a new algorithm, I’ll probably stay in IPython. But, these are my preferences and may not be the same for all students. When looking at these as tools in a toolbelt, rather than a single permanent choice, it becomes easier to pick the right one for the task at hand.

Is there anything else you’d like to say?

Thank you for the opportunity! I love to meet new people in the Python space and appreciate you reaching out!

The post PyDev of the Week: Logan Thomas appeared first on Mouse Vs Python.

In my all in with Nuitka post and my first post Python 3.11 and Nuitka and then progress post Python 3.11 and Nuitka Progress , I promised to give you more updates on Python 3.11 and in general.

So this is where 3.11 is at, and the TLDR is, experimental support has arrives with Nuitka 1.5 release, follow develop branch for best support, and 1.6 is expected to support new 3.11 features.

What is now

The 1.5 release passes the CPython3.10 test suite practically as good as with Python3.11 as with Python3.10, with only a handful of tests failing and these do not seem significant, and it is expected to be resolved later when making the CPython3.11 test suite working.

The 1.5 release now gives this kind of output.

Nuitka:WARNING: The Python version '3.11' is not officially supported by Nuitka '1.5', but an Nuitka:WARNING: upcoming release will change that. In the mean time use Python version '3.10' Nuitka:WARNING: instead or newer Nuitka.

Using develop should always be relatively good, it doesn’t often have regressions, but Python3.11 improvements will accumulate there until 1.6 release happens. Follow it there if you want. However, checking those standalone cases that can be done, as many packages are not available for 3.11 yet, I have not found a single issue.

What you can do?

Try your software with Nuitka and Python3.11 now. Very likely your code base is not using 3.11 specific features, or is it? If it is, of course you may have to wait until develop catches up with new features and changes in behavior.

In case you are wondering, how I can invest this much time into doing all of what I do, consider becoming a subscriber of Nuitka commercial, even if you do not need the IP protection features it mostly has. All commonly essential packaging and performance features are entirely free, and I have put incredible amounts of works in this, and I need to now make a living off it, while I do not plan to make Nuitka annoying or unusable for non-commercial non-subscribers at all.

What was done?

Getting all of the test suite to work, is a big thing already. Also a bunch of performance degradations have been addressed. However right now, attribute lookups and updates e.g. are not as well optimized, and that despite and of course because Python 3.11 changed the core a lot in this area.

The Process

This was largely explained in my previous posts. I will just put where we are now and skip completed steps and avoid repeating it too much.

In the next phase, during 1.6 development the 3.11 test suite is used in the same way as the 3.10 test suite. Then we will get to support new features, new behaviors, newly allowed things, and achieve super compatibility with 3.11 as we always do for every CPython release. All the while doing this, the CPython3.10 test suite will be executed with 3.11 by my internal CI, immediately reporting when things change for the worse.

This phase is starting today actually.

When

It is very hard to predict what will be encountered in the test suite. It didn’t look like many things are there, but e.g. exception groups might be an invasive feature, otherwise I am not aware of too many things at this point. It sure feels close now.

These new features will be relatively unimportant to the masses of users who didn’t immediately change their code to use 3.11 only features.

The worst things with debugging is that I just never know how much time it will be. Often things are very quick to add to Nuitka, and sometimes they hurt a lot or cause regressions for other Python versions by mistake.

Benefits for older Python too

I mentioned stuff before, that I will not repeat only new stuff.

Most likely, attribute lookups will lead to adding the same JIT approach the Python 3.11 allows for now, and maybe that will be possible to backport to old Python as well. Not sure yet. For now, they are actually worse than with 3.10, while CPython made them faster.

Expected results

Not quite good for benchmarking at this time. From the comparisons I did, the compiled code of 3.10 and 3.11 seemed equally fast, allowing CPython to catch up. When Nuitka takes advantage of the core changes to dict and attributes more closely, hope is that will change.

So in a sense, using 3.11 with Nuitka over 3.10 actually doesn’t have much of a point yet.

I need to repeat this. People tend to expect that gains from Nuitka and enhancements of CPython stack up. The truth of the matter is, no they do not. CPython is now applying some tricks that Nuitka already did, some a decade ago. Not using its bytecode will then become less of a benefit, but that’s OK, this is not what Nuitka is about.

We need to get somewhere else entirely anyway, in terms of speed up. I will be talking about PGO and C types a lot in the coming year, that is at least the hope. The boost of 1.4 and 1.5 was only be the start. Once 3.11 support is sorted out, int will be getting dedicated code too, that’s where things will become interesting.

Final Words

So, this post is kind of too late. My excuse is that due to having Corona, I did kind of close down on some of the things, and actually started to do optimization that will lead towards more scalable class code. This is also already in 1.5 and important.

But now that I feel better, I actually forced myself to post this. I am getting better at this. Now back and starting the CPython3.11 test suite, I will get back to you once I have some progress with that. Unfortunately adding the suite is probably also a couple of days work, just technically before I encounter interesting stuff.

<a href='https://www.youtube.com/watch?v=rduFjpb-fAw' style='font-weight: bold;'>Watch on YouTube</a><br> <br> <p><strong>About the show</strong></p> <p>Sponsored by <a href="https://pythonbytes.fm/compiler"><strong>Compiler Podcast from Red Hat</strong></a>.</p> <p><strong>Connect with the hosts</strong></p> <ul> <li>Michael: <a href="https://fosstodon.org/@mkennedy"><strong>@mkennedy@fosstodon.org</strong></a></li> <li>Brian: <a href="https://fosstodon.org/@brianokken"><strong>@brianokken@fosstodon.org</strong></a></li> <li>Show: <a href="https://fosstodon.org/@pythonbytes"><strong>@pythonbytes@fosstodon.org</strong></a></li> </ul> <p>Join us on YouTube at <a href="https://pythonbytes.fm/stream/live"><strong>pythonbytes.fm/live</strong></a> to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too.</p> <p><strong>Michael #1:</strong> <a href="https://pydantic-xml.readthedocs.io/en/latest/"><strong>pydantic-xml extension</strong></a></p> <ul> <li>via Ilan</li> <li>Recall untangle. How about some pydantic in the mix?</li> <li>pydantic-xml is a pydantic extension providing model fields xml binding and xml serialization / deserialization. It is closely integrated with pydantic which means it supports most of its features.</li> </ul> <p><strong>Brian #2:</strong> <a href="https://snarky.ca/how-virtual-environments-work"><strong>How virtual environments work</strong></a></p> <ul> <li>Brett Cannon</li> <li>This should be required reading for anyone learning Python. <ul> <li>Maybe right after “Hello World” and right before “My first pytest test”, approximately.</li> </ul></li> <li>Some history of environments <ul> <li>Back in the day, there was global and your directory.</li> </ul></li> <li>How environments work <ul> <li>structure: bin, include, and lib</li> <li>pyvenv.cfg configuration file </li> </ul></li> <li>How Python uses virtual environments</li> <li>What activation does, and that it’s <strong>optional.</strong> <ul> <li>Yes, activation is optional. </li> </ul></li> <li>A new project called microvenv that helps VS Code. <ul> <li>Mostly to fix the “Debian doesn’t ship python3 with venv” problem.</li> <li>It doesn’t include script activation stuff</li> <li>It’s super small, less than 100 lines of code, in one file.</li> </ul></li> </ul> <p><strong>Michael #3:</strong> <a href="https://github.com/raaidarshad/dbdeclare"><strong>DbDeclare</strong></a></p> <ul> <li>Declarative layer for your database.</li> <li>https://raaidarshad.github.io/dbdeclare/guide/controller/#example</li> <li>Sent in by creator raaid</li> <li>DbDeclare is a Python package that helps you create and manage entities in your database cluster, like databases, roles, access control, and (eventually) more. </li> <li>It aims to fill the gap between SQLAlchemy (SQLA) and infrastructure as code (IaC).</li> <li>You can: <ul> <li>Declare desired state in Python</li> <li>Avoid maintaining raw SQL</li> <li>Tightly integrate your databases, roles, access control, and more with your tables</li> </ul></li> <li>Migrations like alembic coming too.</li> </ul> <p><strong>Brian #4:</strong> <a href="https://sethmlarson.dev/nox-pyenv-all-python-versions"><strong>Testing multiple Python versions with nox and pyenv</strong></a></p> <ul> <li>Seth Michael Larson</li> <li>This is a cool “what to do first” with nox.</li> <li>Specifically, how to use it to run pytest against your project on multiple versions of Python.</li> <li><p>Example noxfile.py is super small</p> <pre><code> import nox @nox.session(python=["3.8", "3.9", "3.10", "3.11", "3.12", "pypy3"]) def test(session): session.install(".") session.install("-rdev-requirements.txt") session.run("pytest", "tests/") </code></pre></li> <li><p>How to run everything, <code>nox</code> or <code>nox -s test</code>.</p></li> <li>How to run single sessions, <code>nox -s test-311</code> for just Python 3.11</li> <li>Also how to get this to work with pyenv. <ul> <li><code>pyenv global 3.8 3.9 3.10 3.11 3.12-dev</code></li> </ul></li> <li>This reminds me that I keep meaning to write a workflow comparison post about nox and tox.</li> </ul> <p><strong>Extras</strong> </p> <p>Michael:</p> <ul> <li><a href="https://www.bleepingcomputer.com/news/security/github-makes-2fa-mandatory-next-week-for-active-developers/">GitHub makes 2FA mandatory next week for active developers</a></li> <li>New adventure bike [<a href="https://python-bytes-static.nyc3.digitaloceanspaces.com/IMG_0205.jpeg">image 1</a>, <a href="https://python-bytes-static.nyc3.digitaloceanspaces.com/IMG_1002.jpeg">image 2</a>]. <ul> <li>Who’s got good ideas for where to ride in the PNW? </li> <li>Wondering why I got it, here’s <a href="https://www.youtube.com/watch?v=U1HfsqjnEc0">a fun video</a>.</li> </ul></li> </ul> <p><strong>Joke:</strong> <a href="https://www.reddit.com/r/ProgrammerHumor/comments/10p4wo0/anybody_else_having_this_kind_of_colleague_way_to/">Case of the Mondays</a></p>

The next Tryton Unconference is scheduled to begin in just over two months. As you can see in the linked topic the community is already booking their flights and accommodation, if you’ve already done this then please share your booking details so we can all meet up and spend some time together.

As is the case for all unconferences the content is provided by its participants, so if you have a topic that you want to present then please submit details of your talk by email to tub2023@tryton.org. We accept long presentations (30 minutes) and lighting talks (of just 5-10 minutes). So please include a estimate for the duration of your talk when submitting it.

If you don’t feel confident enough to present a topic all by yourself, or there is something you don’t fully understand and want explained, then please create a topic on the forum to see if anyone else is interested in joining forces, or doing a talk to try and make things clearer for you.

I would like to announce that the Foundation board has agreed to accept sponsors for this event. Sponsorship starts from 500€ and grants free entry to the unconference and the inclusion of your logo on the event’s page. If you are interested in sponsoring the event, please send us a message.

We will do our best to live stream the talks and make them available on Youtube. If you can help us with this, then please also raise your voice.

I hope to see you all in Berlin!

1 post - 1 participant

Read full topic