Planet Python

Exactly 10 years ago I started my first job as a software tester.

That doesn’t mean I started testing 10 years ago. Back when I was in high school and at university, I did spend some time doing testing and quality-related stuff for various open source projects - Debian, KDE, Kadu, LibreOffice, Cantata, and some more. I don’t remember any longer which was the first and when exactly that happened. I imagine my first contribution was pretty uneventful - perhaps a message on users forum, a response confirming this is a bug, and an encouragement to report it on bug tracking system or devs mailing list.

Nonetheless, “first job as software tester” is a good place to start counting. First, it’s easy - I have papers to prove the exact date. Second, from that day I have spent about eight hours a day, five days a week, every week, on testing-related things. That adds up to a lot of time, but it’s the consistency that sets it apart from any open source work I have done. Last but not least, the decision to start this specific job set me on a path to treat testing much more seriously, and which eventually led me to where I am today.

I’m not much of a job hopper. In these 10 years, I have only had two employers. But I did change teams and projects quite a lot - I’ve been on 4 projects in first company, and now I’m on my 5th project in second company. The longest time I’ve ever been in a single project is 2 years and 7 months. Details are on LinkedIn.

I came into testing after getting a degree in sociology. In my time at university, I had an opportunity to get my feet wet in empirical social research. I approached testing the same way I approached empirical sociology, even if only because I didn’t really know anything else - I assumed there’s a number of things the team would like to know and my job is to learn about them and report my findings. The hard part is that we don’t have direct access to some of the things we would like to know more about, so we need to depend on a number of proxies of uncertain reliability. X can be caused by Y, and we observed X, but is this because of Y, or some other factor Z? How can we rule out Z? Today, I can confidently say this is not the worst way to approach testing.

When I started my first job, I have been using Linux as my main operating system for about 7 years. During that time I learned how to use shell, I got familiar with the idea that things change and move around, I faced various breakages after updates. Often trying to fix them was frustrating, but I did learn how to search for information, I picked up few tricks and I learned how various components can interact in complex system. That was another major source of experiences that influenced my approach to testing.

I guess I also have certain character traits that helped me to become a decent tester. I tend to be stubborn, I don’t give up easily, I self-identify as perfectionist and I strive to actually understand the thing I am dealing with.

After a year and a half I decided that I want to know more about testing, especially established testing techniques and solutions. My work was praised, but it was all based on intuition and past experiences from other fields. I felt I was missing fundamentals and I feared I might be missing some obvious and elementary testing techniques or skills. I tried to fill these gaps by attending an ISTQB preparation course, but it did not deliver what I was looking for.

My manager knew about my disappointment and at one point presented me with the opportunity to attend a testing conference in another city. One of the talks given there was called “Context-Driven Testing: A New Hope”. This is a funny title, as Context-Driven Testing was already 15 years old at that time and “schools of testing” debate has long left community conciousness. I don’t remember many details of the talk itself, but I did left the conference with a feeling that I should learn more about CDT, as they might have at least some of the answers I was looking for.

I think I started by reading “Lessons Learned in Software Testing”, and what a book it was! It not only revolutionized the way I think about testing to this day, but also gave me much-needed confidence. I found I was already doing some of the things that book recommended, but now I knew why they were worth doing. This is the book that everyone who is serious about testing should read, and probably re-read thorough their career. I think I read it at very good moment, too - I had about three years of experience at the time. I feel I wouldn’t get that much from it if I read it earlier.

Later I have read “Perfect Software” by late Jerry Weinberg. I think this is a great book for people who just start in testing. It surely helped to establish some of my knowledge, but I don’t think it was as influential for me as “Lessons Learned”. It would have been if I read it earlier.

Finally, I have read the complete archives of James Bach and Michael Bolton blogs. This is not something I can recommend to anyone, as both are very prolific writers - each authored few hundreds articles. I think it took me well over a year to get through them all. Nonetheless, this allowed me to fully immerse myself in their thinking and I can confidently say I understand where they are coming from and where they are going to. This also allowed me to stumble upon few very valuable articles and resources that I still refer to.

There’s a lot that I learned from all these resources, but I would like to point out two overarching principles that I often come back to. One, my role as a tester is to show possibilities and broaden the view of the team. My job is to go beyond simple and quick answers. Two, every single day I need to ask myself: what is the most important, most impactful thing I can do right now? And then do this exact thing, even if it means putting aside earlier plans and ideas. Change is something to embrace, not to be afraid of.

About five years into my career, I began to slowly move into more software development-heavy role. To some extent, that was out of necessity - I saw many tasks that could be rectified with a tiny bit of programming. At the same time, I was in the environment where development was considered higher on organizational totem pole than “manual testing”, and showing programming skills was a clear way for more respectable assignments and higher salary. Similar to my testing journey, that was not the moment I started to learn programming - I have written my first shell scripts and perl programs back in high school. While I did struggle, I felt confident enough in my programming prowess to do some simple things.

The event that really helped me to take off to the next level happened about a year after I joined Red Hat. We had a UI test automation framework, which was recently rewritten by a couple of contractors. They worked in a silo and as a result most of the team was not familiar with that code. My job was to learn it, contribute to it and become one of the maintainers.

I think contractors felt threatened by my presence and thought their job security depended on them being the only people capable of working with the framework. As a result, they made code review a nightmare. They threw it all - passive-aggressive comments, unhelpful comments, misleading comments, requests to change code that was already approved in earlier review cycle, demands to explain almost every single line of code, replying anytime between a day and a week. That was all on top of working with unfamiliar, complex and barely documented libraries.

I don’t look back at that time with fondness, but I have to admit it was an effective learning exercise. I was forced to understand things above my capabilities, and eventually I did understand them. This was very much the moment programming finally clicked for me. Also, I learned precisely what to avoid during code reviews and when teaching others.

Since then, my interests started to move more in direction of software design and architecture. I know I can write good enough code that works. But I also want to write code that is maintainable in the long term and allows for adjustments in response to changing environment or requirements.

In these 10 years, I have primarily been an individual contributor. This is the role I feel comfortable in and which I think suits me well. However, I did act as a kind of team lead in two separate occasions. Both times I was not formally a manager for other people and I didn’t feel I have all the tools necessary to make them do the required work. The first time I was completely unprepared for a challenge in front of me. The second time went a little bit better, as I knew more about ways to informally influence people.

These would be the rough summary and most important highlights of my 10 years in testing. There’s no narrative closure, as I am still here and intend to stay for a while longer. I’m happy to talk about testing, open source, software engineering and related topics, so feel free to get in touch with me if this is something you find interesting, or if you would like to draw from my experience.

The latest episode of The Python Podcast came out this week. In this episode, I chatted with Steve Dower.

Steve is a part of the Python team at Microsoft and is also a core developer of the Python language. If you have ever used Python on Windows, then you’ve used some of the code he’s worked on as he completely rewrote the Windows Python installer, got Python into the Windows Store and more.

In this episode, we talk about Steve’s contributions, Python in Excel, how Python has been improved on Windows and much more!

Listen to the podcast using any of the following:

• Python Show on Apple Podcast
• Python Show on Spotify
• Python Show on Amazon Music
• Python Show on Pocket Cast
• Python Show on Google Podcasts

The post Episode 20 – Python on Windows with Steve Dower appeared first on Mouse Vs Python.

Are you getting by with a few fundamental commands for Git when building your Python projects? Would you like to improve your version control techniques and become more efficient with the Git command line? This week on the show, Adam Johnson is back to talk about his new book, "Boost Your Git DX."

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

I received a question the other day about combining the notion of marking slow tests and ordering them to run the slow tests first. This post describes a bit of background and a solution to the problem. @pytest.mark.slow It’s possible to mark slow tests with @pytest.mark.slow and then either run or skip the slow tests. To run slow tests: pytest -m slow To skip slow tests: pytest -m "not slow" With the pytest-skip-slow plugin, you can:

We’re excited to announce the November 2023 release of the Python and Jupyter extensions for Visual Studio Code!

This release includes the following announcements:

• Improvements to Shift + Enter run line in terminal
• Deprecated built-in linting and formatting features
• Improvements to Python linting extensions
• Reconfiguration of test output
• Virtual environment deactivation help
• Announcing Python in VS Code release videos

If you’re interested, you can check the full list of improvements in our changelogs for the Python, Jupyter and Pylance extensions.

Improvements to Shift + Enter run line in terminal

The Python extension is working on improving the behavior of sending code to the Python REPL (Shift+Enter) when no code has been explicitly selected to run. Previously, when you placed your cursor on a line of Python code and pressed Shift+Enter, the Python extension would send the exact line contents to the REPL, even if it would fail, for example, due to being part of a multi-line command.

With our new experimental Smart Send feature, the Python extension will send the smallest runnable block of code surrounding the line where your cursor is to the REPL for execution. This ensures that only complete and executable sections of code are sent to the REPL. The cursor will also be automatically placed to the next executable line, to provide a smooth experience when executing multiple chunks iteratively.

To try it out, you can add the following User setting: "python.experiments.optInto: ["pythonREPLSmartSend"]. While this feature is currently behind an experiment, we expect it to be the default behavior in the future. If you have feedback or suggestions on how we can further improve this feature, please let us know!

Deprecated built-in linting and formatting features

With all the ongoing work and improvements we have been making to the linting and formatting extensions in VS Code for the last year, we have deprecated the built-in linting and formatting features that are shipped within the Python extension. This includes all the linting and formatting commands as well as settings (python.linting.* and python.formatting.*). We recommend you to remove these deprecated settings if you are still using them, and use the linting and formatting extensions instead, as they offer a more configurable and flexible experience.

If you were using a linter or formatter that does not yet have an extension, you can create linter or formatter VS Code extensions for your favorite Python tool! Check out our Python Tools Extension Template for a quick start.

Furthermore, we would like to highlight the community-contributed Ruff extension. Ruff is a Python linter written in Rust and it supports various linters such as pyflakes, pycodestyle, pydocstyle, and more! They have also recently enabled support for using Ruff as a formatter in VS Code ("[python]": { "editor.defaultFormatter": "charliermarsh.ruff" }). Try it out for your Python linting and formatting support in VS Code!

Improvements to Python linting extensions

We have made several improvements to the linting extensions our team supports to allow for an even more configurable and flexible experience with your favorite Python tools.

The Pylint, Mypy and Flake8 extensions now offer settings that allow you to specify glob patterns for files that you wish to exclude from linting. This can be useful if you are working with a large codebase with many sub-projects, and want to exclude certain folders from being linted.

These settings are "pylint.ignorePatterns", "mypy-type-checker.ignorePatterns" and "flake8.ignorePatterns", respectively.

These extensions also support cwd settings, which allows you to specify the working directory for the linter. This setting has been updated to support the variable ${fileDirname}, so the working directory can be dynamically set to the parent folder of the file you have open in the editor. This can be particularly useful if you are working with mono-repos, and want the linter’s working directory to be dynamically updated as you open files from different sub-projects. These settings are "pylint.cwd", "mypy-type-checker.cwd" and "flake8.cwd", respectively.

We have also changed the default value of the "mypy-type-checker.preferDaemon", only applicable to the Mypy extension. Previously, it was set to true, which meant that the Mypy daemon would be used by default. After receiving feedback, we have changed the default value to false. If you are wondering which value would be best for you, our recommendation is to use the Mypy daemon if you enabled the Mypy reporting scope to be the entire workspace ("mypy-type-checker.reportingScope": "workspace") for performance reasons. Otherwise, if the reporting scope is set to the current file, we recommend you use the Mypy executable that is shipped with the extension.

Reconfiguration of test output

We’ve made significant improvements to how you can view and interact with the Test Output in the Python extension when the pythonTestAdapter experiment is enabled, which we announced a few months ago. Previously, output from test discovery and execution was inconsistently scattered across the Python Test Log output channel and the Test Results panel, with some information being duplicated in both places. To consolidate the experience, we are now displaying output related to test execution in the Test Results panel, and test discovery in the Python output channel. To learn more see our related wiki.

This new approach also supports colorization in the output if you are using Pytest and set "python.testing.pytestArgs": ["--color=yes"] in your Workspace settings.json. Colorization only works for test execution in the Test Results panel and will not work for discovery or for output in the Python Test Log panel.

There is also a new button that allows you to open the Test Logs more easily from the Test Explorer view when test discovery errors occur.

Virtual environment deactivation help

A couple of months ago we announced a new feature behind an experiment that enables terminal activation using environment variables, which aims to enhance your development workflow by automatically activating the selected environment in the terminal without the need for explicit activation commands. However, since there are no explicit activation scripts working, the deactivate command was no longer working when this experiment was enabled.

The Python extension will now detect when you attempt to run the deactivate command and show a helper notification that will guide you on how to add scripts for your shell to enable the command to work again when the environment is activated through environment variables. It also offers a button to more conveniently open your shell profile file for you to add the necessary scripts.

You can find the full documentation on how to add the necessary scripts for your shell here.

If you are not in the experiment and would like to try out this feature, you can add the following User setting: "python.experiments.optInto: ["pythonTerminalEnvVarActivation"].

Python in VS Code release videos

Python Community Advocate, Dawn Wages walks through key release news and enhancements in the Python in VS Code release video! These short videos will come out with all subsequent releases on the Visual Studio Code YouTube channel highlighting key changes and enhancements along with video content demonstrating how to interact with these new features!

 

For more Python news and community engagement, check out the Python Pulse hosted by Dawn Wages every second Friday of the month!

Other Changes and Enhancements

We have also added small enhancements and fixed issues requested by users that should improve your experience working with Python and Jupyter Notebooks in Visual Studio Code. Some notable changes include:

• A new FastAPI tutorial has been added to our docs! Learn how to get the most out of VS Code and the Python extension when developing a FastAPI application.
• The Python extension now displays a notification to help create virtual environments when you attempt to run or debug a Python file within a workspace that contains a dependency file (such as requirements.txt or requirements/*.txt) and there is no current .venv or .conda folder. This notification provides a quick and easy way to create a new virtual environment using the Python: Create Environment command. This setting can be disabled to setting python.createEnvironment.trigger to off.
• The Python Debugger extension now has platform-specific versions, which means only the necessary files for your platform will be installed on update. This reduces the size of the extension and helps with improving startup time.
• Tensorboard functionality has now been moved out of the Python extension and into the standalone Tensorboard extension. Feedback can be provided in the Tensorboard extension GitHub repo.
• You can now enable and disable Pylance auto import completions more easily through the language status icon on the Status Bar (pylance-release#3894)
• IntelliSense support for __setattr__, __delattr__ and __getattribute__ when using Pylance (pylance-release#4786)

We would also like to extend special thanks to this month’s contributors:

• @PeterJClaw Bump Jedi to 0.19.1 for Python 3.12 support in @vscode-python#22132
• @BoltonBailey Remove unmatched parenthesis from error message in @vscode-python#22254
• @HVbajoria Fixed capitalization of “Python” in user-facing text in @vscode-pylint#390

Call for Community Feedback

As we are planning and prioritizing future work, we value your feedback! Below are a few issues we would love feedback on:

• Proposed design for Django testing on rewrite (@vscode-python#22206)
• Automatically import libraries when user does Shift+Enter on methods before actually importing (@vscode-python#22329)

Try out these new improvements by downloading the Python extension and the Jupyter extension from the Marketplace, or install them directly from the extensions view in Visual Studio Code (Ctrl + Shift + X or ⌘ + ⇧ + X). You can learn more about Python support in Visual Studio Code in the documentation. If you run into any problems or have suggestions, please file an issue on the Python VS Code GitHub page.

The post Python in Visual Studio Code – November 2023 Release appeared first on Python.

You can download the build from our website, get it from the free Toolbox App, or update to it using snaps if you’re an Ubuntu user.

Download PyCharm 2023.3 EAP

The fifth build of the Early Access Program for PyCharm 2023.3 introduces:

• A new inspection, Missing super method, to support PEP 698 (@override Decorator for Static Typing).
• A new UI for the Django Server run configuration.

You can download the build from our website, get it from the free Toolbox App, or update to it using snaps if you’re an Ubuntu user.

These are the most important updates for this build. For the full list of changes in this EAP build, read the release notes.

We’re dedicated to giving you the best possible experience, and your feedback is vital. If you find any bugs, please report them via our issue tracker. And if you have any questions or comments, feel free to share them in the comments below or get in touch with us on X (formerly Twitter).

Introduction

While some data structures are versatile and can be used in a wide range of applications, others are specialized and designed to handle specific problems. One such specialized structure, known for its simplicity yet remarkable utility, is the stack.

So, what is a stack? At its core, a stack is a linear data structure that follows the LIFO (Last In First Out) principle. Think of it as a stack of plates in a cafeteria; you only take the plate that's on top, and when placing a new plate, it goes to the top of the stack.

The last element added is the first element to be removed

But, why is understanding the stack crucial? Over the years, stacks have found their applications in a plethora of areas, from memory management in your favorite programming languages to the back-button functionality in your web browser. This intrinsic simplicity, combined with its vast applicability, makes the stack an indispensable tool in a developer's arsenal.

In this guide, we will deep dive into the concepts behind stacks, their implementation, use cases, and much more. We'll define what stacks are, how they work, and then, we'll take a look at two of the most common ways to implement stack data structure in Python.

Fundamental Concepts of a Stack Data Structure

At its essence, a stack is deceptively simple, yet it possesses nuances that grant it versatile applications in the computational domain. Before diving into its implementations and practical usages, let's ensure a rock-solid understanding of the core concepts surrounding stacks.

The LIFO (Last In First Out) Principle

LIFO is the guiding principle behind a stack. It implies that the last item to enter the stack is the first one to leave. This characteristic differentiates stacks from other linear data structures, such as queues.

Note: Another useful example to help you wrap your head around the concept of how stacks work is to imagine people getting in and out of an elevator - the last person who enters an elevator is the first to get out!

Basic Operations

Every data structure is defined by the operations it supports. For stacks, these operations are straightforward but vital:

• Push - Adds an element to the top of the stack. If the stack is full, this operation might result in a stack overflow.

• Pop - Removes and returns the topmost element of the stack. If the stack is empty, attempting a pop can cause a stack underflow.

• Peek (or Top) - Observes the topmost element without removing it. This operation is useful when you want to inspect the current top element without altering the stack's state.

By now, the significance of the stack data structure and its foundational concepts should be evident. As we move forward, we'll dive into its implementations, shedding light on how these fundamental principles translate into practical code.

How to Implement a Stack from Scratch in Python

Having grasped the foundational principles behind stacks, it's time to roll up our sleeves and delve into the practical side of things. Implementing a stack, while straightforward, can be approached in multiple ways. In this section, we'll explore two primary methods of implementing a stack - using arrays and linked lists.

Implementing a Stack Using Arrays

Arrays, being contiguous memory locations, offer an intuitive means to represent stacks. They allow O(1) time complexity for accessing elements by index, ensuring swift push, pop, and peek operations. Also, arrays can be more memory efficient because there's no overhead of pointers as in linked lists.

On the other hand, traditional arrays have a fixed size, meaning once initialized, they can't be resized. This can lead to a stack overflow if not monitored. This can be overcome by dynamic arrays (like Python's list), which can resize, but this operation is quite costly.

With all that out of the way, let's start implementing our stack class using arrays in Python. First of all, let's create a class itself, with the constructor that takes the size of the stack as a parameter:

class Stack: def __init__(self, size): self.size = size self.stack = [None] * size self.top = -1

As you can see, we stored three values in our class. The size is the desired size of the stack, the stack is the actual array used to represent the stack data structure, and the top is the index of the last element in the stack array (the top of the stack).

From now on, we'll create and explain one method for each of the basic stack operations. Each of those methods will be contained within the Stack class we've just created.

Let's start with the push() method. As previously discussed, the push operation adds an element to the top of the stack. First of all, we'll check if the stack has any space left for the element we want to add. If the stack is full, we'll raise the Stack Overflow exception. Otherwise, we'll just add the element and adjust the top and stack accordingly:

def push(self, item): if self.top == self.size - 1: raise Exception("Stack Overflow") self.top += 1 self.stack[self.top] = item

Now, we can define the method for removing an element from the top of the stack - the pop() method. Before we even try removing an element, we'd need to check if there are any elements in the stack because there's no point in trying to pop an element from an empty stack:

def pop(self): if self.top == -1: raise Exception("Stack Underflow") item = self.stack[self.top] self.top -= 1 return item

Finally, we can define the peek() method that just returns the value of the element that's currently on the top of the stack:

def peek(self): if self.top == -1: raise Exception("Stack is empty") return self.stack[self.top]

And that's it! We now have a class that implements the behavior of stacks using lists in Python.

Implementing a Stack Using Linked Lists

Linked lists, being dynamic data structures, can easily grow and shrink, which can be beneficial for implementing stacks. Since linked lists allocate memory as needed, the stack can dynamically grow and reduce without the need for explicit resizing. Another benefit of using linked lists to implement stacks is that push and pop operations only require simple pointer changes. The downside to that is that every element in the linked list has an additional pointer, consuming more memory compared to arrays.

As we already discussed in the "Python Linked Lists" article, the first thing we'd need to implement before the actual linked list is a class for a single node:

class Node: def __init__(self, data): self.data = data self.next = None

This implementation stores only two points of data - the value stored in the node (data) and the reference to the next node (next).

Our 3-part series about linked lists in Python:

• Part 1 - "Linked Lists in Detail with Python Examples: Single Linked Lists"
• Part 2 - "Sorting and Merging Single Linked List"
• Part 3 - "Doubly Linked List with Python Examples"

Now we can hop onto the actual stack class itself. The constructor will be a little different from the previous one. It will contain only one variable - the reference to the node on the top of the stack:

class Stack: def __init__(self): self.top = None

As expected, the push() method adds a new element (node in this case) to the top of the stack:

def push(self, item): node = Node(item) if self.top: node.next = self.top self.top = node

The pop() method checks if there are any elements in the stack and removes the topmost one if the stack is not empty:

def pop(self): if not self.top: raise Exception("Stack Underflow") item = self.top.data self.top = self.top.next return item

Finally, the peek() method simply reads the value of the element from the top of the stack (if there is one):

def peek(self): if not self.top: raise Exception("Stack is empty") return self.top.data

Note: The interface of both Stack classes is the same - the only difference is the internal implementation of the class methods. That means that you can easily switch between different implementations without the worry about the internals of the classes.

The choice between arrays and linked lists depends on the specific requirements and constraints of the application.

How to Implement a Stack using Python's Built-in Structures

For many developers, building a stack from scratch, while educational, may not be the most efficient way to use a stack in real-world applications. Fortunately, many popular programming languages come equipped with in-built data structures and classes that naturally support stack operations. In this section, we'll explore Python's offerings in this regard.

Python, being a versatile and dynamic language, doesn't have a dedicated stack class. However, its built-in data structures, particularly lists and the deque class from the collections module, can effortlessly serve as stacks.

Using Python Lists as Stacks

Python lists can emulate a stack quite effectively due to their dynamic nature and the presence of methods like append() and pop().

• Push Operation - Adding an element to the top of the stack is as simple as using the append() method:

  stack = [] stack.append('A') stack.append('B')

• Pop Operation - Removing the topmost element can be achieved using the pop() method without any argument:

  top_element = stack.pop() # This will remove 'B' from the stack

• Peek Operation Accessing the top without popping can be done using negative indexing:

  top_element = stack[-1] # This will return 'A' without removing it

Using deque Class from collections Module

The deque (short for double-ended queue) class is another versatile tool for stack implementations. It's optimized for fast appends and pops from both ends, making it slightly more efficient for stack operations than lists.

• Initialization:

  from collections import deque stack = deque()

• Push Operation - Similar to lists, append() method is used:

  stack.append('A') stack.append('B')

• Pop Operation - Like lists, pop() method does the job:

  top_element = stack.pop() # This will remove 'B' from the stack

• Peek Operation - The approach is the same as with lists:

  top_element = stack[-1] # This will return 'A' without removing it

When To Use Which?

While both lists and deques can be used as stacks, if you're primarily using the structure as a stack (with appends and pops from one end), the deque can be slightly faster due to its optimization. However, for most practical purposes and unless dealing with performance-critical applications, Python's lists should suffice.

Note: This section dives into Python's built-in offerings for stack-like behavior. You don't necessarily need to reinvent the wheel (by implementing stack from scratch) when you have such powerful tools at your fingertips.

Potential Stack-Related Issues and How to Overcome Them

While stacks are incredibly versatile and efficient, like any other data structure, they aren't immune to potential pitfalls. It's essential to recognize these challenges when working with stacks and have strategies in place to address them. In this section, we'll dive into some common stack-related issues and explore ways to overcome them.

Stack Overflow

This occurs when an attempt is made to push an element onto a stack that has reached its maximum capacity. It's especially an issue in environments where stack size is fixed, like in certain low-level programming scenarios or recursive function calls.

If you're using array-based stacks, consider switching to dynamic arrays or linked-list implementations, which resize themselves. Another step in prevention of the stack overflow is to continuously monitor the stack's size, especially before push operations, and provide clear error messages or prompts for stack overflows.

If stack overflow happens due to excessive recursive calls, consider iterative solutions or increase the recursion limit if the environment permits.

Stack Underflow

This happens when there's an attempt to pop an element from an empty stack. To prevent this from happening, always check if the stack is empty before executing pop or peek operations. Return a clear error message or handle the underflow gracefully without crashing the program.

In environments where it's acceptable, consider returning a special value when popping from an empty stack to signify the operation's invalidity.

Memory Constraints

In memory-constrained environments, even dynamically resizing stacks (like those based on linked lists) might lead to memory exhaustion if they grow too large. Therefore, keep an eye on the overall memory usage of the application and the stack's growth. Perhaps introduce a soft cap on the stack's size.

Thread Safety Concerns

In multi-threaded environments, simultaneous operations on a shared stack by different threads can lead to data inconsistencies or unexpected behaviors. Potential solutions to this problem might be:

• Mutexes and Locks - Use mutexes (mutual exclusion objects) or locks to ensure that only one thread can perform operations on the stack at a given time.
• Atomic Operations - Leverage atomic operations, if supported by the environment, to ensure data consistency during push and pop operations.
• Thread-local Stacks - In scenarios where each thread needs its stack, consider using thread-local storage to give each thread its separate stack instance.

While stacks are indeed powerful, being aware of their potential issues and actively implementing solutions will ensure robust and error-free applications. Recognizing these pitfalls is half the battle - the other half is adopting best practices to address them effectively.

Conclusion

Stacks, despite their seemingly simple nature, underpin many fundamental operations in the computing world. From parsing complex mathematical expressions to managing function calls, their utility is broad and essential. As we've journeyed through the ins and outs of this data structure, it's clear that its strength lies not just in its efficiency but also in its versatility.

However, as with all tools, its effectiveness depends on how it's used. Just make sure you have a thorough understanding of its principles, potential pitfalls, and best practices to ensure that you can harness the true power of stacks. Whether you're implementing one from scratch or leveraging built-in facilities in languages like Python, it's the mindful application of these data structures that will set your solutions apart.

Matplotlib is a versatile and widely-used library for data visualization in Python. While it excels at creating static plots and charts, it’s also capable of producing interactive visualizations. In this article, we are going to explore how to create such interactive plots in Matplotlib that allow users to actually interact with the graph, and drawn elements using the mouse cursor.

Let’s begin!

What are Interactive Plots?

Interactive Plots are special types of graphs which allow the user to interact with them, and making changes to the graph in real time. This is in contrast to static graphs, which only render once, and do not change over the duration of their lifetime.

There are two ways that interactive plots are typically created in matplotlib. The first method involves the use of “widgets” like buttons and checkboxes which can control and modify the graph.

The second method involves the use of “events” and the mouse pointer. The user performs an action using the mouse, like a click, which triggers an event, which in turn triggers a function which runs whatever code we want to. We can also incorporate complex actions, such as “mouse drag” and “scrolling” into our matplotlib applications with this method.

Both methods are equally effective, each with their own pros and cons. However, we will be focusing on the second method in this tutorial.

What is an Artist in Matplotlib?

This is a term that many who have been using matplotlib, likely do not know. This term will be used frequently through this tutorial, so it is important that we clarify its meaning right now.

In Matplotlib, an “artist” refers to a fundamental graphical object or entity that can be drawn on a canvas or axes, contributing to the visual representation of a plot or chart. These artists encompass a wide range of graphical elements, including shapes (e.g., rectangles, circles), lines, text, images, and more.

In simple words, an artist is anything that is drawn to the screen. A matplotlib shape, graph, even text.

“Selecting” an Artist using Mouse Pointer

In this section, we will cover how to use the mouse pointer to click on an artist, and retrieve its details (such as position, type, color, etc.). The technique that we will explore is a general approach, and will work on any, and all types of artists in matplotlib.

We will begin by introducing the code snippet-by-snippet, along with the appropriate explanations.

import matplotlib.pyplot as plt from matplotlib.patches import Rectangle, Circle

We have imported the plotting module in matplotlib, and also some “shapes” like a Rectangle and Circle.

Next, we create the matplotlib figure and axes object. The figure represents the window, whereas the axes represents the graph region where the plotting is done.

# Create a figure and axis fig, ax = plt.subplots()

Now we create some objects which we will be drawing to our matplotlib window. The first parameter in both of these classes controls the initial position, whereas the following parameters control the dimensions of the object, and other aspects like color.

# Draw some shapes for demonstration rect = Rectangle((0.1, 0.1), 0.3, 0.4, fc="blue", picker=True) circle = Circle((0.7, 0.7), 0.1, fc="red", picker=True)

The important thing to notice here is the picker=True parameter. This parameter enables the interactive feature that allows you to click on and select the drawn artists on the plot. This is normally kept off by default for performance reasons, as it is not needed in static graphs. Once this is turned on, matplotlib will begin tracking actions related to these objects (e.g. mouse clicks), and generate a pick event when an action occurs.

The next step is to add the created shapes to the plot:

# Add these shapes to the graph ax.add_patch(rect) ax.add_patch(circle)

This step is going to vary a bit depending on the type of object you are plotting. For example, scatter() would be used for a scatter plot, bar() would be used for a bar graph, etc.

Defining the onClick() Function

Before we proceed any further, we need to actually define a function which will execute when we interact (click) on any of our objects.

Here is our code for such a function, which performs the simple task of retrieving the artist behind the pick event (using .artist attribute), and prints out the relevant details based on which object it was (which we figure out using the isinstance method).

def onClick(event): artist = event.artist if isinstance(artist, Rectangle): print("Selected Artist Type: Rectangle") print(f"Artist Details: {artist.get_bbox()}") elif isinstance(artist, Circle): print("Selected Artist Type: Circle") center = artist.center radius = artist.radius print(f"Center: ({center[0]}, {center[1]}), Radius: {radius}")

In case you are wondering where the event parameter came from, this is automatically passed in by matplotlib when you connect a function to the pick event.

We will now connect this function to the pick event, and call the show() method to display our graph.

# Connect the pick event to the onClick function fig.canvas.mpl_connect('pick_event', onClick) plt.show()

With the pick event, we can trigger the onClick function by using any kind of mouse click (left/right/scroll).

Here is our output:

Clicking on the rectangle object here gives us the following output:

Selected Artist Type: Rectangle Artist Details: Bbox(x0=0.1, y0=0.1, x1=0.4, y1=0.5)

Similarly, clicking on the circle object gives us the followiing:

Selected Artist Type: Circle Center: (0.7, 0.7), Radius: 0.1

Pretty cool, right?

Interactive Plots where you can modify Artists

In this section, we will extend the previous example, and show you how to modify artists upon selecting them. Our goal in this section will be to create a system, where we have multiple objects in a matplotlib window, and can “select” one of them by clicking on it, upon which it will be highlighted (by changing the color of the object).

The base concept and code remains the same, we just need to change what we put into the onClick() function (or any function connected to the pick event). To simplify this example a bit, we will be using only rectangles.

Here is the base code, without any of extra interactivity stuff:

import matplotlib.pyplot as plt from matplotlib.patches import Rectangle, Circle # Create a figure and axis fig, ax = plt.subplots() # Draw some shapes for demonstration rect1 = Rectangle((0.1, 0.1), 0.3, 0.4, fc="blue", picker=True) rect2 = Rectangle((0.4, 0.6), 0.2, 0.3, fc="blue", picker=True) rect3 = Rectangle((0.8, 0.2), 0.1, 0.3, fc="blue", picker=True) # Add these shapes to the graph ax.add_patch(rect1) ax.add_patch(rect2) ax.add_patch(rect3) plt.show()

Next, we will define the onClick function.

def onClick(event): artist = event.artist if isinstance(artist, Rectangle): artist.set_facecolor('yellow') plt.draw()

There are two main lines of code here. Line 4, where we call the set_facecolor method on the artist object, and Line 5, where we call the plt.draw() method to update the matplotlib graph with our new changes.

We can then connect our onClick function, to the pick event, and display the graph.

fig.canvas.mpl_connect('pick_event', onClick) plt.show()

However, this code results in a problem which is illustrated by the following image of our current output (after all objects have been clicked once):

Managing de-selection

The current issue with the onClick function is that when we click on two objects consecutively, both remain highlighted. In an ideal scenario, clicking on, for example, rectangle-1 should highlight it, and when we subsequently click on rectangle-2, it should become highlighted while rectangle-1 becomes unhighlighted simultaneously.

What we need to do now, is setup a system which keeps tracks of the currently selected object (which is none by default). For this, we can create a global variable (or a variable creating using self if you are using OOP) called current_artist.

current_artist = None

We will now modify our onClick function to make use of this variable to add the concept of “de-selection” into our system. In simpler words, this will reset the color of the previously-selected object, before selecting a new one.

def onClick(event): global current_artist artist = event.artist if isinstance(artist, Rectangle): if current_artist: current_artist.set_facecolor("blue") current_artist = artist artist.set_facecolor('yellow') plt.draw()

Now our code will work flawlessly, and properly highlight, and un-highlight objects as we interact with the graphs.

This marks the end of the Interactive Plots in Matplotlib Tutorial. Any suggestions or contributions for Coderslegacy are more than welcome. Questions regarding the tutorial content can be asked in the comments section below.

The post Interactive Plots in Matplotlib: Mouse-Driven Selection of Artists appeared first on CodersLegacy.

In this tutorial, we’ll explore various ways to slice Python strings providing several code examples. We’ll also make a comparison of different slicing methods to help you choose the most suitable one for your needs. Slicing is one of the most effective ways to extract specific parts of a string. It’s a versatile technique used [...]

The post How to Slice a Python String with Examples appeared first on TechBeamers.

<strong>Topics covered in this episode:</strong><br> <ul> <li><a href="https://us.pycon.org/2024/"><strong>PyCon 2024 is up</strong></a><strong>?</strong></li> <li><a href="https://astral.sh/blog/the-ruff-formatter"><strong>Ruff formatter is production ready</strong></a></li> <li><a href="https://mastodon.social/@hugovk/111293506058914553"><strong>gil--;</strong></a></li> <li><a href="https://www.coderedcorp.com/blog/why-is-the-django-admin-ugly/"><strong>Why is the Django Admin “Ugly”?</strong></a></li> <li><strong>Extras</strong></li> <li><strong>Joke</strong></li> </ul><a href='https://www.youtube.com/watch?v=D8jTg_vfCZ8' style='font-weight: bold;'>Watch on YouTube</a><br> <p><strong>About the show</strong></p> <p>Sponsored by <a href="https://pythonbytes.fm/scout"><strong>Scout APM</strong></a></p> <p><strong>Connect with the hosts</strong></p> <ul> <li>Michael: <a href="https://fosstodon.org/@mkennedy"><strong>@mkennedy@fosstodon.org</strong></a></li> <li>Brian: <a href="https://fosstodon.org/@brianokken"><strong>@brianokken@fosstodon.org</strong></a></li> <li>Show: <a href="https://fosstodon.org/@pythonbytes"><strong>@pythonbytes@fosstodon.org</strong></a></li> </ul> <p>Join us on YouTube at <a href="https://pythonbytes.fm/stream/live"><strong>pythonbytes.fm/live</strong></a> to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too.</p> <p><strong>Michael #1:</strong> <a href="https://us.pycon.org/2024/"><strong>PyCon 2024 is up</strong></a><strong>?</strong></p> <ul> <li>May 15 - May 23, 2024 - Pittsburgh, Pennsylvania</li> <li>Conference breakdown: <ul> <li>Tutorials: May 15 - 16, 2024</li> <li>Main Conference and Online: May 17 - 19, 2024</li> <li>Job Fair: May 19, 2024</li> <li>Sprints: May 20 - May 23, 2024</li> </ul></li> <li>Tickets aren’t on sale yet</li> <li>Unfortunately, I’m not going (see <a href="https://us.pycon.org/2024/about/health-safety-guidelines/"><strong>health and safety guidelines</strong></a>)</li> <li>Attendance numbers over time <a href="https://en.wikipedia.org/wiki/Python_Conference"><strong>on Wikipedia</strong></a></li> </ul> <p><strong>Brian #2:</strong> <a href="https://astral.sh/blog/the-ruff-formatter"><strong>Ruff formatter is production ready</strong></a></p> <ul> <li><a href="https://pythonbytes.fm/episodes/show/353/hatching-another-episode">We reported the alpha release in September</a></li> <li>It’s fast, 30x faster than Black</li> <li>Provides &gt;99.9% compatibility with Black, with a <a href="https://github.com/astral-sh/ruff/blob/main/docs/formatter/black.md">list of known deviations</a></li> <li>More configurable</li> <li>Bundled with ruff, <code>ruff format</code></li> <li>Still in Beta, but considered production-ready</li> <li>Integration extensions for VSCode and PyCharm</li> </ul> <p><strong>Michael #3:</strong> <a href="https://mastodon.social/@hugovk/111293506058914553"><strong>gil--;</strong></a></p> <ul> <li>The Python Steering Council has now formally accepted <a href="https://peps.python.org/pep-0703/">PEP 703</a> ("Making the Global Interpreter Lock Optional in CPython")</li> <li>The <strong>global interpreter lock will remain the default for CPython</strong> builds and python.org downloads.</li> <li>A new <strong>build</strong> configuration flag, --disable-gil will be added to the configure script that will build CPython with support for running without the global interpreter lock.</li> <li>"In short, the SC accepts PEP 703, but with clear provisio: <ul> <li>that the rollout be gradual and break as little as possible,</li> <li>that we can roll back any changes that turn out to be too disruptive – which includes potentially rolling back all of PEP 703 entirely if necessary (however unlikely or undesirable we expect that to be)."</li> </ul></li> <li>Removing the global interpreter lock requires substantial changes to CPython internals, but relatively few changes to the public Python and C APIs.</li> <li>The implementation changes can be grouped into the following four categories: <ul> <li>Reference counting</li> <li>Memory management</li> <li>Container thread-safety</li> <li>Locking and atomic APIs</li> </ul></li> </ul> <p><strong>Brian #4:</strong> <a href="https://www.coderedcorp.com/blog/why-is-the-django-admin-ugly/"><strong>Why is the Django Admin “Ugly”?</strong></a></p> <ul> <li>Vince Salvino</li> <li>Some great quotes from the article: <ul> <li>"The Django admin is not ugly, rather, no effort was made to make it a beautiful end-user tool.” - Ken Whitesell</li> <li>“The admin’s recommended use is limited to an organization’s internal management tool. It’s not intended for building your entire front end around.” - Django docs</li> <li>“The Django admin was built for Phil.” - Jacob Kaplan-Moss</li> <li>“Even in the 0.9x days we used to have a image that said “Admin: it’s not your app”.” - Curtis Maloney</li> </ul></li> <li>As Curtis put it, “encouraging people to build their own management interface, and treat admin as a DB admin tool, has saved a lot of people pain... the effort to customise it grows far faster than the payoffs.”</li> </ul> <p><strong>Extras</strong> </p> <p>Brian:</p> <ul> <li><a href="https://overtag.dk/v2/blog/local-conferences-big-potential/"><strong>Local Conferences: Big Potential</strong></a></li> </ul> <p>Michael:</p> <ul> <li><a href="https://training.talkpython.fm/courses/data-science-jumpstart-with-10-projects"><strong>Data Science Jumpstart with 10 Projects course</strong></a> is out!</li> <li><a href="https://mastodon.social/@andymckay/111293685334582922"><strong>PSF is X-ed out</strong></a> (or are they?)</li> <li><a href="https://gpt4all.io/index.html"><strong>GPT4All</strong></a> is pretty excellent</li> <li><a href="https://fosstodon.org/invite/fmM5PebT"><strong>Fosstodon invites</strong></a> from us (expires Nov 7 2023)</li> </ul> <p><strong>Joke:</strong> <a href="https://www.reddit.com/r/programminghumor/comments/16065hs/this_is_me_right_now/">Searching YouTube for bug fixes</a></p>

In this episode, we cleaned up the email sending job. I had to make some changes based on some discoveries that I made while researching how to track responses and associate the journal entries back to the user. While doing this clean up, we added a new Prompt model to make the email sending job idempotent for each day.

The big news for October is that The Python Coding Place is now live and my first video course is available for beginners starting to learn to code–well, the first few chapters anyway.

This is big news! The Python Coding Place has been a gradual project that started with this site–The Python Coding Book. The Book has now been around for several years and has gained a strong and faithful following.

After a while, I had starting writing articles and tutorials on the blog on this site–which is what you’re reading now. And earlier this year, the Python articles and tutorials moved to The Python Coding Stack, where I now publish a lot more regularly than I did before.

The Python Coding Place is the hub which connects them all, plus the new services that I have started rolling out this week, with many more to come from January. Here’s what The Place looks like in terms of colour and style:

The Python Coding Place website is now live at thepythoncodingplace.com and the first video course for beginners has also started rolling out: A Python Tale • A Relaxed Stroll Through Programming Basics. Here’s the cover image for A Python Tale:

You can also find The Python Coding Place on YouTube, but if I have to be honest, you’re better off going directly to The Place’s own learning platform if you want to enroll on this free course since you’ll get notes, exercises, and a lot more, and a better experience than on YouTube. Here’s the link directly to A Python Tale on The Place’s learning platform.

And there will be many more courses coming over the coming weeks and months, at all levels. And in January, the Members’ Area of The Python Coding Place will launch with premium courses, live workshops, a forum, and lots more. But I’ll say more about this in a future post, including the pre-launch special offer to become a member–there are no subscriptions for membership of The Python Coding Place, only a one-time fee to become a lifetime member and get everything, forever.

Anyway, in addition to all this excitement, I also published my usual articles on The Python Coding Stack and Breaking the Rules.

I published a three-part series called Monty and The White Room analogy which explores the structure and functioning of a Python program through Monty’s adventures in Python City. This is my favourite coding analogy of all time.

I also published a post for paid subscribers of The Python Coding Stack talking about Python’s None object. Who would have thought there’s so much to say about nothing! The article is This Page Is Intentionally Left Blank • The Story of None. I quite like the artwork for this article, too:

I’ll stop there for this October review. And in case you haven’t realised, I’m quite excited about the new video course, A Python Tale, so if you’re a beginner or you know a beginner, this is the course to try out!

The post The Python Coding Place is Live! | October in Review appeared first on The Python Coding Book.

Many people learn basic language and word skills by playing the game of hangman. First mentioned in print in the late nineteenth century and popularized in international media as the game show Wheel of Fortune, hangman appeals to many people. For beginning programmers who want a challenge or for more experienced coders looking for a bit of fun, writing hangman in Python is a rewarding endeavor.

Throughout this tutorial, you’ll build the hangman game in Python in a series of steps. The game will work as a command-line application. Throughout the process, you’ll also learn the basics of building computer games.

In this tutorial, you’ll:

• Learn the common elements of a computer game
• Track the state of a computer game
• Get and validate the user’s input
• Create a text-based user interface (TUI) for your game
• Figure out when the game is over and who the winner is

To get the most from this tutorial, you should be comfortable working with Python sets and lists. You don’t need to have any previous knowledge about game writing.

All the code that you’ll write in this tutorial is available for download at the link below:

Get Your Game: Click here to download the free code that you’ll write to create a hangman game with Python.

Demo: Command-Line Hangman Game in Python

In this tutorial, you’ll write a hangman game for your command line using Python. Once you’ve run all the steps to build the game, then you’ll end up with a text-based user interface (TUI) that will work something like the following:

Throughout this tutorial, you’ll run through several challenges designed to help you solve specific game-related problems. At the end, you’ll have a hangman game that works like the demo above.

Project Overview

The first step to building a good computer game is to come up with a good design. So, the design for your hangman game should begin with a good description of the game itself. You also need to have a general understanding of the common elements that make up a computer game. These two points are crucial for you to come up with a good design.

Description of the Hangman Game

While many people know the game well, it’s helpful to have a formal description of the game. You can use this description to resolve programming issues later during development. As with many things in life, the exact description of a hangman game could vary from person to person. Here’s a possible description of the game:

• Game setup: The game of hangman is for two or more players, comprising a selecting player and one or more guessing players.
• Word selection: The selecting player selects a word that the guessing players will try to guess.
  • The selected word is traditionally represented as a series of underscores for each letter in the word.
  • The selecting player also draws a scaffold to hold the hangman illustration.
• Guessing: The guessing players attempt to guess the word by selecting letters one at a time.
• Feedback: The selecting player indicates whether each guessed letter appears in the word.
  • If the letter appears, then the selecting player replaces each underscore with the letter as it appears in the word.
  • If the letter doesn’t appear, then the selecting player writes the letter in a list of guessed letters. Then, they draw the next piece of the hanged man. To draw the hanged man, they begin with the head, then the torso, the arms, and the legs for a total of six parts.
• Winning conditions: The selecting player wins if the hanged man drawing is complete after six incorrect guesses, in which case the game is over. The guessing players win if they guess the word.
  • If the guess is right, the game is over, and the guessing players win.
  • If the guess is wrong, the game continues.

A game in progress is shown below. In this game, the word to be guessed is hangman:

In this tutorial, to write the hangman game in Python, you’ll make a few additional design decisions:

• The game will be between the computer and one human player.
• The computer will act as the selecting player and will select the word to guess, process human input, and handle all output.
• The human player is the guessing player, hereafter simply referred to as the player. When the player knows the word, they continue to guess correct letters until the word is complete.

With this basic understanding of the game and some design decisions for the computer version, you can begin creating the game. First, however, you need to know the common elements of computer games and understand how they interact to produce the desired result.

Common Elements of a Computer Game

Every game, whether it’s a computer game or not, has a set of elements that make it a game. Without these elements, there’s no game:

• Initial setup: You get the game ready to play. This might mean setting the pieces on a chess board, dealing out cards, or rolling dice to see who goes first.
• Gameplay: When people think of the game, this is what they normally think of. The flow of gameplay is controlled by a game loop, which keeps the game moving and doesn’t end until the game is over. The game loop ensures the following happens:
  • The user input is gathered and processed.
  • The game state is updated based on reactions to the user input. This may include checking for end-of-game conditions.
  • Any required output changes are made to reflect the new game state.
  • The game loop repeats.
• Game ending: How the game ends depends on the game itself. Whether capturing the enemy king in checkmate, achieving a certain score in a card game, or having your piece cross the line in a board game, certain conditions determine the end of the game and the winner.

All games have a game loop. For example, in a chess game, you first set the pieces on the board and declare the current player. Then the game loop begins:

Read the full article at https://realpython.com/python-hangman/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Searching for ways to rename a column using Pandas? This is a common operation you might have to perform while using a data frame. This tutorial will walk you through several methods to rename one or more columns in Pandas, providing examples and a comparison of each method to help you choose the most suitable [...]

The post 4 Different Ways to Rename Columns in Pandas appeared first on TechBeamers.

Welcome back to the Pybites Podcast. We’ve been teasing this for a while, and it’s finally here: our Python Developer Certificates. 

Listen here:

Or watch it on YouTube:

Our certificates are not just shiny badges. They’re the real deal, representing solid Python skills, experience with frameworks like Django/FastAPI, debugging, testing, professional developer workflow / collaboration, deployment, and so much more (see assessment criteria below). 

Unlike other theoretical (rote learning) certifications ours cover MVP-ready + deployed projects. In this episode we talk about our vision with this and address some common concerns like pricing, time commitment, how long they are valid for, and more.

As our programs and certifications are so unique and individualized, if interested in this, we encourage you to hop on a call with us to further discuss how this can help you in your career.

Links:
– Certification homepage
– PCPD-A (associate)
– PCPD-P (professional)
– Assessment criteria

Book a free call Learn more

Books mentioned:
– Steal Like an Artist
– War and Peace
– The Courage to Be Disliked

Chapters:
00:00 Intro
01:27 Announcing Pybites developer certification
02:38 What?! Pybites do certifications? How ours is different
04:30 Different certificates and our vision
06:22 Assessment criteria, we emulate a real developer environment
08:22 Our certificates are timeless, they don’t expire
09:58 Why did we make this? Industry and our people want this!
13:14 It’s an add-on to PDM
14:33 Some concerns and questions people might have
17:25 This is very unique and individualized, get on a call
20:46 Books – Austin Kleon, War and Peace, Courage to be Disliked
24:14 Wrap up and outro

Book a free call Learn more

In this tutorial, we’ll explore how to read CSV files in Python using the Pandas library with 7 unique examples. Pandas is a powerful data manipulation and analysis library that provides easy-to-use functions for working with structured data, such as CSV files. We will cover various methods for reading CSV files, and at the end, [...]

The post How to Read CSV Files in Python using Pandas appeared first on TechBeamers.

Python has several other Python files besides the traditional *.py file that it supports. In this tutorial, you will learn about the PYD file.

A PYD file is a dynamic link library (DLL) written in Python that can be imported like a Python file. However, if you attempt to open a PYD file, you will find it is an unreadable binary file. When you create a PYD file, you will save it with a .pyd extension, such as hello.pyd

The PYD file is to be used on the Windows operating system. The source code is not accessible inside of a PYD file.

Let’s get started and learn how to create a PYD file!

Getting Dependencies

You will need to install a couple of other modules to create PYD files successfully. Specifically, you will need the following:

• setuptools
• Cython

Technically, you can use distutilsinstead of setuptools, however, distutilsis no longer included with Python, starting in version 3.12. Since you are creating a DLL, you will need Cython to compile your Python code to C and then into the DLL.

If you don’t already have the two packages listed above, you can use pip to install them. Open up a terminal, PowerShell or cmd.exe and run the following command:

python -m pip install setuptools Cython

If everything worked correctly, you should now have setuptools and Cython installed on your machine!

You are now ready to learn how to create a PYD file.

Creating a PYD File

For this tutorial, you will be creating a Python script named api.py. Open up your favorite Python IDE or text editor and add the following code:

# api.py api_key = "MY_API_KEY" def PyInit_api(): pass

Note that when you create this file,

Here is the setup file, you need to create a specially named function: PyInit_CUSTOM_NAME(). In this example, your module is named api.py, so you’ll want to name your special function PyInit_api(). Your special function does not need any code inside it. Instead, this function will remain empty.

When the api module is called, it will execute the PyInit_api() function, will run automatically, and initialize all the variables and other pieces of code in the module so that you can import them.

In this example, you create module-level variable called api_key that you can import and access in your other Python code.

Now that the module is ready, you must turn it into a PYD file. To create your PYD file, you need to create a setup.pyscript.

Create a new file in your Python IDE or text editor and enter the following code:

# setup.py from Cython.Distutils import build_ext from setuptools import Extension, setup ext_modules = [Extension("api", ["api_key.py"])] setup( name="Test Program", cmdclass={"build_ext": build_ext}, ext_modules=ext_modules, )

Here, you create a list of extension modules. In this case, you only create one extension. You pass in the extension’s name and a list of sources relative to the distribution root. There are additional arguments you could pass in, but you don’t need them for this example.

Next, you configure setup() so it will build your PYD file. Since you want to build one or more extensions, you must set the cmdclass to build_ext and pass in Cython’s special extension builder module. You also pass in your custom Extensions list.

Now you’ll need to run setup to build your PYD file. Open your terminal back up and navigate to your source code’s folder. Then run the following command:

python setup.py build_ext --inplace

When you run this command, you will see a bunch of text output  to the screen that will look similar to the following:

running build_ext Compiling api_key.py because it changed. [1/1] Cythonizing api_key.py C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\Cython\Compiler\Main.py:381: FutureWarning: Cooks\code_play\api_key.py tree = Parsing.p_module(s, pxd, full_module_name) creating build\temp.win-amd64-cpython-311 creating build\temp.win-amd64-cpython-311\Release "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.37.32822\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\Mike\AppData\Local\Programs\Python\Python311\include -IC:\Users\Mike\AppDSVC\14.37.32822\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" -IC:\PROGRA~1\IBM\SQLLIB\INCLUDE -IC:\PROGRA~1\IBM\SQLLIB\LIB /Tcapi_key.c /Fobuild\temp.win-amd64-cpython-311\Release\api_key.obj api_key.c "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.37.32822\bin\HostX86\x64\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\Mike\AppData\Local\Programs\Python\Python311\libs /LIBPATH:C:\Users\Mike\AppData\Local\Programs\Python\Python311 /LIBPATH:C:\Users\Mike\AppData\Local\Programs\Python\Python311\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.37.32822\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64" /LIBPATH:C:\PROGRA~1\IBM\SQLLIB\LIB /EXPORT:PyInit_api build\temp.win-amd64-cpython-311\Release\api_key.obj /OUT:C:\books\code_play\api.cp311-win_amd64.pyd /IMPLIB:build\temp.win-amd64-cpython-311\Release\api.cp311-win_amd64.lib Creating library build\temp.win-amd64-cpython-311\Release\api.cp311-win_amd64.lib and object build\temp.win-amd64-cpython-311\Release\api.cp311-win_amd64.exp

If everything worked correctly, you should now have a file named something like this: api.cp311-win_amd64.pyd

You will also have the following files and folders:

• build (a folder)
• __pycache__ (a folder)
• api_key.c  (C file)

You should rename your PYD file to something else to import it. For example, you can rename it to api.pyd

Now that it’s renamed run Python in your terminal and make sure it works:

$ python Python 3.11.5 (tags/v3.11.5:cce6ba9, Aug 24 2023, 14:38:34) [MSC v.1936 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import api >>> api.api_key 'MY_API_KEY'

Great! You just created a PYD file!

Wrapping Up

You can use PYD files as a type of obfuscation. You may be able to get some speed increases using PYD files, if you are careful.

Probably the best use case for PYD files is allowing programs other than Python to load them. Give it a try and see if you can find a good use for these types of files!

Links

• SetupTools – Build Extension Modules

The post How to Create a pyd File in Python appeared first on Mouse Vs Python.