Planet Python
Django Weblog: DjangoCon US and Europe 2022 Talks Available Online
The organizers of DjangoCon US and DjangoCon Europe spend a tremendous amount of energy both hosting the conferences and putting all the talks online after. The complete playlists of talks from each conference are now available on YouTube:
PyCon: PyCon US 2023 Registration Launch!
The news you’ve been waiting for is finally here - registration for PyCon US 2023 is officially open!
PyCon US will take place April 19 - April 27, 2023, online and in Salt Lake City, Utah. The core of the conference, April 21 - April 23, 2023, packs in three days worth of our community’s 95 best talks, amazing keynote speakers, and our famed lightning talks to close out each day, but it is much more than that.
It’s gathering together with the members of our community, to learn from and share with. It’s joining a conversation in the hallway with the creators of open source projects. It’s taking yourself from beginner to intermediate, or intermediate to advanced. For some, it’s getting started with Python for the first time.
We hope you’ll join us for this special anniversary year, celebrating 20 years of PyCon.
Health & Safety GuidelinesPyCon US is committed to protecting the health and safety of our community. To ensure that we are gathering safely, we have implemented COVID-19 guidelines and protocols to be followed by all attendees during the event. We ask that you please review these guidelines prior to registration.
Please note: We will not be rolling back our mask requirement. If the situation changes and there are additional ways to keep attendees safe, we may incorporate those strategies closer to the event. Please check back regularly to stay updated on current guidelines.
How to RegisterOnce you have created an account on the PyCon US 2023 conference website, you can register via the registration button on your dashboard. Head over to our Registration Information page to get all the details on how to register.
Early Bird Registration RatesPyCon US is providing discounted rates for Corporate, Student, and Individual tickets for the first 500 tickets sold. Register now to get your discount while supplies last!
- Corporate - $650
- Individual - $350
- Student - $75
- Corporate - $750 USD
- Individual - $400 USD
- Student - $100 USD
- Online - $100 USD
- Patron / Patron Online - $1200 USD
Tutorials will be presented Wednesday, April 19, 2023, and Thursday, April 20, 2023. We are accepting proposals for tutorials through December 9, 2022. Find more information and submit a proposal here. Once our program committee has scheduled the selected tutorials, you will be able to add them to your conference registration.
Watch for tutorial launch in February 2023. Opt-in for PyCon US News and follow us on Twitter and Mastodon for the announcement.Sponsorship and Sponsor Presentations
Sponsorship for PyCon US 2023 is open now, and you can apply directly on our Sponsorship Application page. We’re grateful to all of our sponsors, who make PyCon US possible.
Sponsor Presentations will take place Thursday, April 20, 2023. To reserve a slot for an hour-long Sponsor Presentation on the topic of your choice, please apply for Partner Level Sponsorship or higher and select the check mark next to “Sponsor Presentation.” Slots are limited and typically sell out, so please submit your request soon. Contact sponsors@python.org with any questions.
HotelPyCon US has contracted special rates with nearby hotels. When you complete your registration for PyCon US 2023, you will be able to book a hotel reservation on your dashboard through our official housing bureau, Orchid Events. This is the only way to get the conference rates so book now while supplies last!
More information can be found on the Venue and Hotels page.
Note: Beware of Housing Pirates! PyCon US or Orchid Events will not be calling delegates to sell rooms. If you are contacted by an agency other than Orchid Events offering to make your hotel reservations, we urge you to not use their services. We cannot protect you against them if you do book a reservation.
Cancellation FeesRegistration cancellations must be submitted in writing and received by April 3, 2023, in order to receive a refund minus the $50 cancellation fee ($25 for students; waived for cancellation due to health reasons). No refunds will be granted for cancellations received after April 3, 2023, unless you must cancel for any health-related reasons (see more details under Attendance in the Health & Safety Guidelines).
In lieu of cancellation, you are able to transfer your registration to another person. For details about transferring your registration, visit the registration page.
Attendees traveling to PyCon US internationally are encouraged to review our International Travel Refund Policy as well as their countries' international travel guidelines prior to registering. This is especially important for recipients of Travel Grants attending from abroad. PyCon US strives to support the Python community in attending, no matter where they are traveling from.
Travel GrantsCheck out the Travel Grant page to learn more about the support we provide for travel, hotel, and registration to ensure that everyone has an opportunity to attend PyCon US. We actively encourage people to apply for travel grants and welcome applications from attendees who otherwise would not be able to attend. Our goal is to support diversity and provide equity to attendees and attract Python developers at all experience levels from around the world. For questions during the application process, visit the Travel Grant FAQ page.
Real Python: Python Basics: Dictionaries
In plain English, a dictionary is a book containing the definitions of words. Each entry in a dictionary has two parts: the word being defined, and its definition.
Python dictionaries, like lists and tuples, store a collection of objects. However, instead of storing objects in a sequence, dictionaries hold information in pairs of data called key-value pairs. That is, each object in a dictionary has two parts: a key and a value. Each key is assigned a single value, which defines the relationship between the two sets.
In this video course, you’ll learn:
- What a dictionary is and how it’s structured
- How dictionaries differ from other data structures
- How to define and use dictionaries in your own code
This video course is part of the Python Basics series, which accompanies Python Basics: A Practical Introduction to Python 3. You can also check out the other Python Basics courses.
Note that you’ll be using IDLE to interact with Python throughout this course.
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
PyPy: PyPy v7.3.10 release
The PyPy team is proud to release version 7.3.10 of PyPy. We have some nice speedups and bugfixes we wish to share. The release includes three different interpreters:
PyPy2.7, which is an interpreter supporting the syntax and the features of Python 2.7 including the stdlib for CPython 2.7.18+ (the + is for backported security updates)
PyPy3.8, which is an interpreter supporting the syntax and the features of Python 3.8, including the stdlib for CPython 3.8.15.
PyPy3.9, which is an interpreter supporting the syntax and the features of Python 3.9, including the stdlib for CPython 3.9.15. We have gained confidence in the stability of this version, and are removing the "beta" label.
The interpreters are based on much the same codebase, thus the multiple release. This is a micro release, all APIs are compatible with the other 7.3 releases. Highlights of the release, since the release of 7.3.9 in March 2022 include:
A release of Apple Silicon M1 arm64 versions. This work was sponsored by an anonymous donor and is tested on our buildbots.
Many improvements to the basic interpreter to make it 15-20% faster
The conda-forge community has built over 1000 packages for PyPy3.8 and 3.9, making it easier than ever to use PyPy.
Update the packaged OpenSSL to 1.1.1s, sqlite3 to 3.39.4, and apply applicable security fixes from CPython 3.9.15 to PyPy2.7
Update the HPy backend in PyPy3.8 and PyPy3.9 to 0.0.4
We recommend updating. You can find links to download the v7.3.10 releases here:
https://pypy.org/download.html
We would like to thank our donors for the continued support of the PyPy project. If PyPy is not quite good enough for your needs, we are available for direct consulting work. If PyPy is helping you out, we would love to hear about it and encourage submissions to our blog via a pull request to https://github.com/pypy/pypy.org
We would also like to thank our contributors and encourage new people to join the project. PyPy has many layers and we need help with all of them: bug fixes, PyPy and RPython documentation improvements, or general help with making RPython's JIT even better. Since the previous release, we have accepted contributions from five new contributors, thanks for pitching in, and welcome to the project!
If you are a python library maintainer and use C-extensions, please consider making a HPy / CFFI / cppyy version of your library that would be performant on PyPy. In any case, both cibuildwheel and the multibuild system support building wheels for PyPy.
What is PyPy?PyPy is a Python interpreter, a drop-in replacement for CPython 2.7, 3.8 and 3.9. It's fast (PyPy and CPython 3.7.4 performance comparison) due to its integrated tracing JIT compiler.
We also welcome developers of other dynamic languages to see what RPython can do for them.
We provide binary builds for:
x86 machines on most common operating systems (Linux 32/64 bits, Mac OS 64 bits, Windows 64 bits)
64-bit ARM machines running Linux (aarch64).
Apple M1 arm64 machines (macos_arm64).
s390x running Linux
PyPy support Windows 32-bit, Linux PPC64 big- and little-endian, and Linux ARM 32 bit, but does not release binaries. Please reach out to us if you wish to sponsor binary releases for those platforms. Downstream packagers provide binary builds for debian, Fedora, conda, OpenBSD, FreeBSD, Gentoo, and more.
What else is new?For more information about the 7.3.10 release, see the full changelog.
Please update, and continue to help us make pypy better.
Cheers, The PyPy Team
Django Weblog: Django bugfix release: 4.1.4
Today we've issued the 4.1.4 bugfix release.
The release package and checksums are available from our downloads page, as well as from the Python Package Index. The PGP key ID used for this release is Carlton Gibson: E17DF5C82B4F9D00.
Python Bytes: #313 Programming Robots With a Marker
Read the Docs: Read the Docs newsletter - December 2022
This newsletter contains the first features and updates that have hatched since we announced a Q4 focus on core platform features in the previous newsletter.
News and updatesHere are the latest updates from our team:
⚙️ The entire build process can now be overridden (if you need it). See the Announcement.
💡️ This is useful if you want to publish outputs generated from a framework that isn’t supported by default. We wrote some examples for Pelican and Docsify.
💡️ You can also extend the build process for instance you can skip a build.
🐘️ We started tooting in the Fediverse (Mastodon). Follow us at @readthedocs@fosstodon.org
🛳️ Server-Side Search API v3 has been released.
🎬️ We are preparing to redirect our existing front pages to our new marketing site: about.readthedocs.com. Let us know know if you have comments or ideas for it.
⏳️ We organized a large refactor of our user documentation to comply with the Diátaxis methodology framework. So far, we broke it down into 73 tasks and counting.
✅️ We added a Policy for Unofficial and Unmaintained Projects that will help official maintainers to de-list forks among others.
✅️ We added an additional auditing feature, whereby invitations are added to the Security Log. The feature is available for users of Read the Docs for Business.
✅️ We found, fixed and disclosed a security issue, XSS: Allow serving of arbitrary HTML files from main domain.
You can always see the latest changes to our platforms in our Read the Docs Changelog.
Upcoming featuresWe’re working on improving our integration with Material for MkDocs, which is a great theme for MkDocs documentation projects.
Many improvements to our URL handling code, which will allow us to support more flexible URL configurations for projects.
A search redesign to make it nicer across our dashboard and in-doc search experiences.
404 pages are being improved by contextualization the user message, giving relevant guidance to readers and project owners.
Sphinx 6 is coming very soon. For those of you planning to upgrade immediately, we are preparing a blog post with a couple of considerations in that regard.
If you find regressions in any new releases of the sphinx-rtd-theme, please don’t hesitate to open an issue on GitHub.
Our Q4 focus also includes deprecating old and outdated approaches to using our platform. We don’t have anything firm to announce here yet, but we do plan to be more active in removing these features in the coming months.
Tip of the monthDo you need to document past or upcoming events? Chris Sewell has created sphinx-timeline for exactly this.
See sphinx-timeline in full action on the main page of AiiDA (which by the way is built with Read the Docs).
Awesome Project of the monthNautobot is a great example of a documentation landing page with several subprojects. See all the highlights in the following Twitter thread:
Nautobot is the project of our today’s 💫 Showcase the Docs 🌟 edition!
“An extensible and flexible Network Source of Truth and Network Automation Platform that is the cornerstone of any network automation architecture” #Nautobot #documentation pic.twitter.com/icp2q2Epty
Looking for more inspiration? Check out our new list: Awesome Read the Docs Projects.
Considering using Read the Docs for your next documentation project? Check out our documentation to get started!
Questions? Comments? Ideas for the next newsletter? Contact us!
Real Python: Python News: What's New From November 2022
The Python world never stops spinning! After the release of Python 3.11 in October, the Python release team is already launching the first alpha versions of Python 3.12. The excitement of trying shiny new things shouldn’t distract you from being cautious while coding, though. Recently, researchers found more malicious packages on PyPI, and lawyers raised concerns about license violations when using GitHub Copilot to generate code.
Let’s dive into the biggest Python news from the past month!
Join Now: Click here to join the Real Python Newsletter and you'll never miss another Python tutorial, course update, or post.
Python 3.12 Alpha ReleasedOne of the most newsworthy events from October was the release of Python 3.11. While many of us are exploring the cool new features of Python 3.11, others are already hard at work on the next release.
Right on track with the Python 3.12 release schedule, the Python release team unveiled Python 3.12 alpha 2 in the middle of November. The core team is still early in the development cycle, but so far, the list of new features seems exciting.
Here’s some of what’s coming in Python 3.12:
- Even more improved error messages
- Support for the Linux perf profiler
- Deprecation of old functions, classes, and modules
While Python 3.11 already improved error messages, the next feature release will offer even better suggestions for fixing errors. For example, check out what it does when you forget to import a module or order your import statements wrong:
>>>>>> sys.version_info Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'sys' is not defined. Did you forget to import 'sys'? >>> import pi from math File "<stdin>", line 1 import pi from math ^^^^^^^^^^^^^^^^^^^ SyntaxError: Did you mean to use 'from ... import ...' instead?Especially when you’re learning Python, constructive error messages can point you in the right direction to improve your code. But also, if you’re a seasoned Python developer, then the upcoming Python release will have something in stock for your code improvements:
Read the full article at https://realpython.com/python-news-november-2022/ »[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
Python for Beginners: Create Index in a Pandas Series
Pandas series objects are used to store data when we need to access it using its position as well as labels. In this article, we will discuss different ways to create index in a pandas series.
Table of Contents- Create Index in a Pandas Series Using the Index Parameter
- Create Index in a Pandas Series Using the Index Attribute
- Create an Index in a Pandas Series Using the set_axis() Method
- Create Index Inplace in a Pandas Series
- Conclusion
When we create a pandas series, it has a default index starting from 0 to the length of the series-1. For instance, consider the following example.
import pandas as pd import numpy as np letters=["a","b","c","ab","abc","abcd","bc","d"] series=pd.Series(letters) print("The series is:") print(series)Output:
The series is: 0 a 1 b 2 c 3 ab 4 abc 5 abcd 6 bc 7 d dtype: objectIn the above example, we have created a series of 8 elements. You can observe that the indices of the elements in the series are numbered from 0 to 7. These are the default indices.
If you want to assign a custom index to the series, you can use the index parameter in the Series() constructor. The index parameter in the Series() constructor takes a list having an equal number of elements as the elements in the series and creates a custom index for the series as shown below.
import pandas as pd import numpy as np letters=["a","b","c","ab","abc","abcd","bc","d"] numbers=[3,23,11,14,16,2,45,65] series=pd.Series(letters,index=numbers) print("The series is:") print(series)Output:
The series is: 3 a 23 b 11 c 14 ab 16 abc 2 abcd 45 bc 65 d dtype: objectIn the above example, we have passed the python list [3, 23, 11, 14, 16, 2, 45, 65] to the index parameter of the Series() constructor. After the execution of the Series() constructor, the elements of this list are assigned as the indices of the elements in the series.
Create Index in a Pandas Series Using the Index AttributeYou can also create a new index for a series after creating the series. For instance, if you want to assign other values as indices in the series, you can use the index attribute of the series object. To create a new custom index, you can assign a list of values to the index attribute as shown below.
import pandas as pd import numpy as np letters=["a","b","c","ab","abc","abcd","bc","d"] numbers=[3,23,11,14,16,2,45,65] series=pd.Series(letters) series.index=numbers print("The series is:") print(series)Output:
The series is: 3 a 23 b 11 c 14 ab 16 abc 2 abcd 45 bc 65 d dtype: objectIn this example, we have assigned the list [3, 23, 11, 14, 16, 2, 45, 65] to the index attribute of the series after creating the series. Hence, the elements of this list are assigned as the indices of the elements in the series.
Here, the list passed to the index attribute must have a length equal to the number of elements in the series. Otherwise, the program will run into a ValueError exception. You can observe this in the following example.
import pandas as pd import numpy as np letters=["a","b","c","ab","abc","abcd","bc","d"] numbers=[3,23,11,14,16,2,45,65,117] series=pd.Series(letters) series.index=numbers print("The series is:") print(series)Output:
ValueError: Length mismatch: Expected axis has 8 elements, new values have 9 elementsIn the above example, you can observe that the list "letters" has only 8 elements. As a result, the series contains only 8 elements. On the other hand, the list "numbers" has 9 elements. Hence, when we assign the "numbers" list to the index attribute of the series, the program runs into a ValueError exception.
Suggested Reading: If you are into machine learning, you can read this MLFlow tutorial with code examples. You might also like this article on clustering mixed data types in Python.
Create an Index in a Pandas Series Using the set_axis() MethodInstead of using the index attribute, we can use the set_axis() method to create an index in a pandas series.
The set_axis() MethodThe set_axis() method has the following syntax.
Series.set_axis(labels, *, axis=0, inplace=_NoDefault.no_default, copy=_NoDefault.no_default)Here,
- The labels parameter takes a list-like object containing index values. You can also pass an Index object to the labels parameter. The number of elements in any object passed to the labels parameter should have the same number of elements as the series on which the set_axis() method is invoked.
- The axis parameter is used to decide if we want to create the index for rows or columns. As a Series has only one column, the axis parameter is unused.
- After creating a new index, the set_axis() method returns a new Series object. If you want to modify the original Series object, you can set the inplace parameter to True.
- The copy parameter is used to decide whether to make a copy of the underlying data instead of modifying the original series. By default, it is True.
To create an index using the set_axis() method, we will invoke this method on the original series object. We will pass a list containing the new index values to the set_axis() method as an input argument. After execution, the set_axis() method will return a new series having a modified index. You can observe this in the following example.
import pandas as pd import numpy as np letters=["a","b","c","ab","abc","abcd","bc","d"] numbers=[3,23,11,14,16,2,45,65] series=pd.Series(letters) series=series.set_axis(labels=numbers) print("The series is:") print(series)Output:
The series is: 3 a 23 b 11 c 14 ab 16 abc 2 abcd 45 bc 65 d dtype: objectIn this example, we have first created a series containing 8 elements. Then, we used the set_index() method to assign new indices to the elements in the series. You can observe that the set_index() method returns a new series. Hence, the original series isn’t modified. To modify the original series by assigning new indices instead of creating a new one, you can create an index in place in the series.
Create Index Inplace in a Pandas SeriesTo create an index inplace in a pandas series, you can assign the new index to the index attribute of the series object as shown in the following example.
import pandas as pd import numpy as np letters=["a","b","c","ab","abc","abcd","bc","d"] numbers=[3,23,11,14,16,2,45,65] series=pd.Series(letters) series.index=numbers print("The series is:") print(series)Output:
The series is: 3 a 23 b 11 c 14 ab 16 abc 2 abcd 45 bc 65 d dtype: objectYou can also use the set_axis() method to create an index inplace in a series. For this, you can pass the list containing the new index values to the set_axis() method and set the inplace parameter to True while invoking the set_axis() method on the original series object. After execution, you will get the modified series object as shown below.
import pandas as pd import numpy as np letters=["a","b","c","ab","abc","abcd","bc","d"] numbers=[3,23,11,14,16,2,45,65] series=pd.Series(letters) series.set_axis(labels=numbers,inplace=True) print("The series is:") print(series)Output:
The series is: 3 a 23 b 11 c 14 ab 16 abc 2 abcd 45 bc 65 d dtype: objectIn this example, we used the set_index() method to assign new indices to the elements in the series. You can observe that we have set the inplace parameter to True in the set_index() method. Hence, the new indices are assigned in the original series object itself.
While using the inplace parameter you will get a FutureWarning stating "FutureWarning: Series.set_axis 'inplace' keyword is deprecated and will be removed in a future version. Use obj = obj.set_axis(..., copy=False) instead". It means that the inplace parameter has been deprecated. Hence, if the same code is used in future versions of pandas, the program may run into an error. To avoid this, you can use the copy parameter.
By default, the copy parameter is set to True. Hence, the set_axis() method uses a copy of the original series and modifies it. If you want to modify the original series, you can set the copy parameter to False in the set_axis() method.
ConclusionIn this article, we discussed different ways to create the index in a pandas series in Python. To know more about the pandas module, you can read this article on how to sort a pandas dataframe. You might also like this article on how to drop columns from a pandas dataframe.
The post Create Index in a Pandas Series appeared first on PythonForBeginners.com.
Mike Driscoll: PyDev of the Week: Miroslav Šedivý
This week we welcome Miroslav Šedivý as our PyDev of the Week! Miro has been a speaker at several different Python conferences. You can also catch up with Miro on his website or by checking out Miro's GitHub Profile.
Let's take some time to get to know Miro better!
Can you tell us a little about yourself (hobbies, education, etc):My name is Miroslav Šedivý, but most people call me Miro, which allows them to avoid typing some letters they do not have on their keyboard. I was born in Czechoslovakia (yes, I am over 30 now), studied computer sciences in France and Germany, and now I am living in Austria.
Computers and programming were the hobbies that became my occupation. Another hobby, which is difficult to combine professionally with the first one, is human languages. As a Central European fascinated by travel, I speak a bunch of languages and enjoy using them every day.
Apart from this, I love spending time with my family, hiking, biking, fixing OpenStreetMap, camping, and woodworking.
You can find me at https://mas.to/@eumiro and occasionally at some Python events.
Why did you start using Python?After I have developed a quite complex power forecasting system for wind turbines in Perl using PDL (Perl equivalent to NumPy), my colleagues started using Python for new projects. It was around 2008, in the age of Python 2.5 with some rumors of Python 3. I got myself a book called “Perl to Python Migration” and tried to wrap my head around the new concepts.
The worst thing I remember from my beginnings was the necessity to "import re" each time I wanted to work with regular expressions. In Perl it was second nature to do almost anything with regular expressions. Not it Python. And this is beautiful about different languages: the way of thinking in them. Just like in French one does not simply count to seventy, nor in Czech say “one beer” (“one” is actually a synonym for “beer”), in Python one, can use other tools to dissect strings.
What other programming languages do you know and which is your favorite?There were some BASIC and Pascal somewhere in the past century, but now my number one is Python, accompanied by Shell. I remember some Perl, Java, C, and PHP from more or less serious projects in the past, and I have played with Go and Rust.
The choice of language usually depends on the task. I also do not use English to speak about food and cuisine, or Polish to comment a Python code review. I am using Python only for stuff it fits well. Luckily, it fits most of the stuff I am working on.
What projects are you working on now?Apart from my work, I have some sleeping projects at https://github.com/eumiro, which are currently waiting for another burst of enthusiasm or external input. A day has only 24±1 hours, but I wish it had more.
Which Python libraries are your favorite (core or 3rd party)?From standard lib, I am always happy if I can reach into the itertools module and simplify my code. Migration to Python 3 brings pathlib, which is a complete game changer when working with file system. I am also happy to see some gaps closing between the different modules, so you do not have to import anything else to obtain Unix timestamp of a datetime object, for instance.
There are plenty of wonderful third-party libraries I have worked with. The most important and complex one is probably pandas.
How did you end up contributing to open source projects?At my first Python conference EuroPython 2015 in Bilbao, I met Francesc Alted from the PyTables team and as an active user of the library I had a long discussion with him at the end, we even did a sprint and with his help, I removed one obsolete exception-raiser from the codebase that was bugging me since some time.
Later I occasionally contributed to some projects I was using, but then around Christmas 2020 I had quite a lot of free time and I discovered outdated CI/CD configurations in dozens of Python projects, so I started systematically helping them to fix the pipelines and also to modernize their code. This is where one of my talks “There Are Python 2 Relics In Your Code!” were born. Python 2 hacks like `int(math.floor(x))` work in Python 3 but they do not make sense in modern code and should be refactored.
What are the top three things you've learned as a contributor?Make small steps. Even smaller. If you think you've done a great bunch of work and submit a huge PR, the maintainers will have a very difficult time reviewing it and any rebase in an active project will probably stop their interest in your contribution.
Maintainers are humans with jobs paying their bills. Respect their time and energy and do not expect they're here only for you.
Comment on the project. Show how you're using it in your own work to make the maintainers feel their endeavor matters.
Is there anything else you’d like to say?Please do not use backslashes to deliberately break lines. If there's some line length limit (whether it is 78, 80, 88, 100, or 120), it makes your code more readable. There's virtually always a possibility to reformat your code to respect this limit and avoid horizontal scrolling in editor. Every such formatting allows you to break the line at a suitable position. Breaking lines with backslashes tells the reader “you have to join these lines in your head into a very long one to understand it.”
Finally, I'd like to thank Mike for his engagement and the whole Python community for collaborating on such a marvelous set of products and the whole ecosystem around it. So happy to be a part of it!
Thanks for doing the interview, Miro!
The post PyDev of the Week: Miroslav Šedivý appeared first on Mouse Vs Python.
Zato Blog: LDAP and Active Directory as Python API Services
LDAP and Active Directory often play key a role in the management of a company’s network resources yet it is not always very convenient to query a directory directly using the LDAP syntax and protocol that few people truly specialize in. This is why in this article we are using Zato to offer a REST API on top of directory services so that API clients can use REST and JSON instead.
Installing ZatoStart off by installing Zato - if you are not sure what to choose, pick the Docker Quickstart option and this will set up a working environment in a few minutes.
Creating connectionsOnce Zato is running, connections can be easily created in its Dashboard (by default, http://127.0.0.1:8183). Navigate to Connections -> Outgoing -> LDAP ..
.. and then click Create a new connection which will open a form as below:
The same form works for both regular LDAP and Active Directory - in the latter case, make sure that Auth type is set to NTLM.
The most important information is:
- User credentials
- Authentication type
- Server or servers to connect to
Note that if authentication type is not NTLM, user credentials can be provided using the LDAP syntax, e.g. uid=MyUser,ou=users,o=MyOrganization,dc=example,dc=com.
Right after creating a connection be sure to set its password too - the password asigned by default is a randomly generated one.
PingingIt is always prudent to ping a newly created connection to ensure that all the information entered was correct.
Note that if you have more than one server in a pool then the first available one of them will be pinged - it is the whole pool that is pinged, not a particular part of it.
Active Directory as a REST service
As the first usage example, let’s create a service that will translate JSON queries into LDAP lookups - given username or email the service will basic information about the person’s account, such as first and last name.
Note that the conn object returned by client.get() below is capable of running any commands that its underlying Python library offers - in this case we are only using searches but any other operation can also be used, e.g. add or modify as well.
# -*- coding: utf-8 -*- # stdlib from json import loads # Bunch from bunch import bunchify # Zato from zato.server.service import Service # Where in the directory we expect to find the user search_base = 'cn=users, dc=example, dc=com' # On input, we are looking users up by either username or email search_filter = '(&(|(uid={user_info})(mail={user_info})))' # On output, we are interested in username, first name, last name and the person's email query_attributes = ['uid', 'givenName', 'sn', 'mail'] class ADService(Service): """ Looks up users in AD by their username or email. """ class SimpleIO: input_required = 'user_info' output_optional = 'message', 'username', 'first_name', 'last_name', 'email' response_elem = None skip_empty_keys = True def handle(self): # Connection name to use conn_name = 'My AD Connection' # Get a handle to the connection pool with self.out.ldap[conn_name].conn.client() as client: # Get a handle to a particular connection with client.get() as conn: # Build a filter to find a user by user_info = self.request.input['user_info'] user_filter = search_filter.format(user_info=user_info) # Returns True if query succeeds and has any information on output if conn.search(search_base, user_filter, attributes=query_attributes): # This is where the actual response can be found response = conn.entries # In this case, we expect at most one user matching input criteria entry = response[0] # Convert it to JSON for easier handling .. entry = entry.entry_to_json() # .. and load it from JSON to a Python dict entry = loads(entry) # Convert to a Bunch instance to get dot access to dictionary keys entry = bunchify(entry['attributes']) # Now, actually produce a JSON response. For simplicity's sake, # assume that users have only one of email or other attributes. self.response.payload.message = 'User found' self.response.payload.username = entry.uid[0] self.response.payload.first_name = entry.givenName[0] self.response.payload.last_name = entry.sn[0] self.response.payload.email = entry.mail[0] else: # No business response = no such user found self.response.payload.message = 'No such user'After creating a REST channel, we can invoke the service from command line, thus confirming that we can offer the directory as a REST service:
$ curl "localhost:11223/api/get-user?user_info=MyOrganization\\MyUser" ; echo { "message": "User found", "username": "MyOrganization\\MyUser", "first_name": "First", "last_name": "Last", "email": "address@example.com" } $ Next steps-
Start the tutorial to learn more technical details about Zato, including its architecture, installation and usage. After completing it, you will have a multi-protocol service representing a sample scenario often seen in banking systems with several applications cooperating to provide a single and consistent API to its callers.
-
Check more resources for developers and screenshots.
-
Para aprender más sobre las integraciones de Zato y API en español, haga clic aquí
Podcast.__init__: Declarative Machine Learning For High Performance Deep Learning Models With Predibase
This is a cross-over episode from our new show The Machine Learning Podcast, the show about going from idea to production with machine learning.
SummaryDeep learning is a revolutionary category of machine learning that accelerates our ability to build powerful inference models. Along with that power comes a great deal of complexity in determining what neural architectures are best suited to a given task, engineering features, scaling computation, etc. Predibase is building on the successes of the Ludwig framework for declarative deep learning and Horovod for horizontally distributing model training. In this episode CTO and co-founder of Predibase, Travis Addair, explains how they are reducing the burden of model development even further with their managed service for declarative and low-code ML and how they are integrating with the growing ecosystem of solutions for the full ML lifecycle.
Announcements- Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great!
- When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With their managed Kubernetes platform it’s easy to get started with the next generation of deployment and scaling, powered by the battle tested Linode platform, including simple pricing, node balancers, 40Gbit networking, dedicated CPU and GPU instances, and worldwide data centers. And now you can launch a managed MySQL, Postgres, or Mongo database cluster in minutes to keep your critical data safe with automated backups and failover. Go to pythonpodcast.com/linode and get a $100 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show!
- Your host is Tobias Macey and today I’m interviewing Travis Addair about Predibase, a low-code platform for building ML models in a declarative format
- Introduction
- How did you get involved in machine learning?
- Can you describe what Predibase is and the story behind it?
- Who is your target audience and how does that focus influence your user experience and feature development priorities?
- How would you describe the semantic differences between your chosen terminology of "declarative ML" and the "autoML" nomenclature that many projects and products have adopted?
- Another platform that launched recently with a promise of "declarative ML" is Continual. How would you characterize your relative strengths?
- Can you describe how the Predibase platform is implemented?
- How have the design and goals of the product changed as you worked through the initial implementation and started working with early customers?
- The operational aspects of the ML lifecycle are still fairly nascent. How have you thought about the boundaries for your product to avoid getting drawn into scope creep while providing a happy path to delivery?
- Ludwig is a core element of your platform. What are the other capabilities that you are layering around and on top of it to build a differentiated product?
- In addition to the existing interfaces for Ludwig you created a new language in the form of PQL. What was the motivation for that decision?
- How did you approach the semantic and syntactic design of the dialect?
- What is your vision for PQL in the space of "declarative ML" that you are working to define?
- Can you describe the available workflows for an individual or team that is using Predibase for prototyping and validating an ML model?
- Once a model has been deemed satisfactory, what is the path to production?
- How are you approaching governance and sustainability of Ludwig and Horovod while balancing your reliance on them in Predibase?
- What are some of the notable investments/improvements that you have made in Ludwig during your work of building Predibase?
- What are the most interesting, innovative, or unexpected ways that you have seen Predibase used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on Predibase?
- When is Predibase the wrong choice?
- What do you have planned for the future of Predibase?
- tgaddair on GitHub
- @travisaddair on Twitter
- From your perspective, what is the biggest barrier to adoption of machine learning today?
- Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. The Machine Learning Podcast helps you go from idea to production with machine learning.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@podcastinit.com) with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers
- Predibase
- Horovod
- Ludwig
- Support Vector Machine
- Hadoop
- Tensorflow
- Uber Michaelangelo
- AutoML
- Spark ML Lib
- Deep Learning
- PyTorch
- Continual
- Overton
- Kubernetes
- Ray
- Nvidia Triton
- Whylogs
- Weights and Biases
- MLFlow
- Comet
- Confusion Matrices
- dbt
- Torchscript
- Self-supervised Learning
The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
Brian Okken: Testing with Python 3.12
John Ludhi/nbshare.io: ERROR Could not find a version that satisfies the requirement numpy==1 22 3
You might run in to following error if you try to install numpy version 1.22+.
The reason is mostly the wrong Python version as shown below.
Install Python3.8 or Python3.8+
In [1]: !python --version Python 3.8.0 In [14]: !pip install numpy==1.22.3 Collecting numpy==1.22.3 Using cached numpy-1.22.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.8 MB) Installing collected packages: numpy Successfully installed numpy-1.22.3 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venvJohn Ludhi/nbshare.io: Understand Python Slicing
Python slicing is used to slice lists or tuples. Let us learn Python slicing through examples.
Let us declare a Python list.
In [1]: x = [1,2,3,4,5,6,7,8,9]Python slicing can be used in two ways...
x[start:stop:step] -- Start through stop-1
x[slice(start,stop,step) -- Start through stop-1
Note step parameter is optional, by default it is 1.
Python slice all but first element In [6]: x[1:] Out[6]: [2, 3, 4, 5, 6, 7, 8, 9] Python slice all but last element In [7]: x[:8] Out[7]: [1, 2, 3, 4, 5, 6, 7, 8] Python slice and copy the whole list In [8]: x[:] Out[8]: [1, 2, 3, 4, 5, 6, 7, 8, 9] Python slice every other element in the listThe below command means start from index 0 and extract every 2nd element.
index 0 , index (2) 0 +2, index (4) 0 + 2 +2, index (6) 0 + 2 + 2 +2 , index (8) 0 + 2 + 2 + 2 +2
Note two colons in the below syntax. There is nothing between Ist and 2nd colon that means evreything from start to end of list.
In [9]: x[0::2] Out[9]: [1, 3, 5, 7, 9] Python Slicing - Negative and Positive step valuesNegative step of -1 means go from right to left
Positive step of 1 means go from left to right
Negative number -1 means last element, -2 means seconda last element so on and so forth.
In [10]: x[-1] Out[10]: 9The below syntax will print nothing, because index is starting from -1 which means last element in the list then it goes to index 0 because by default step is 1 but our "stop" is -3. To fix this we will have to give step of -1.
In [11]: x[-1:-3] Out[11]: []As we can see, the index will start from -1, the last element, then goes to -2 because step is -1, then stops at -3
In [12]: x[-1:-3:-1] Out[12]: [9, 8] Python Reverse all the items in the array using Slicing In [13]: x[::-1] Out[13]: [9, 8, 7, 6, 5, 4, 3, 2, 1]Note in the above, since step is -1, Python considers start:stop as -1:-10, we can re-write above as shown below.
In [14]: x[-1:-10:-1] Out[14]: [9, 8, 7, 6, 5, 4, 3, 2, 1]print last 3 elements. Note below syntax means -3 to last element (right to left) and step is 1 by default
In [15]: x[-3:] Out[15]: [7, 8, 9]Below syntax means, start at -3 index, then go to (-5) -3-2 so on and so forth to all the way to start of the list (right to left) because step is -1.
In [16]: x[-3::-2] Out[16]: [7, 5, 3, 1] Python Slicing - Reverse first two items In [17]: x[1::-1] Out[17]: [2, 1]Let us see how did the above slicing syntax works.
The index will start at...
index 1, index (0) 1-1
Note -1 tells it go from right to left
Note all the above can be used with slice method. The different is instead of ":" (colon) use None to indicate end or start of list index.
In [20]: x[slice(-3,None,-1)] Out[20]: [7, 6, 5, 4, 3, 2, 1]Peter Hoffmann: beautiful leaflet markers with folium and fontawesome
Folium is a Python library that allows users to create and display interactive maps. The library uses the Leaflet.js library and is capable of creating powerful and visually appealing maps. Folium can be used to visualize geographical data by adding markers, polygons, heatmaps, and other geographical elements onto a map. The library is easy to use and offers a range of options for customizing the maps and the elements that are displayed on them.
Minimal markerThe minimal example just adds a marker at a specific location:
import folium loc = [45.957916666667, 7.8123888888889] m = folium.Map(location=loc, zoom_start=13) folium.Marker( location=loc ).add_to(m) m Marker with a bootstrap iconMarkers can be customized through providing a Icon instance. As a default you can use bootstrap glyphicons that provide over 250 glyphs for free:
In addition you can colorize the marker. Available color names are red blue green purple orange darkred lightred beige darkblue darkgreen cadetblue darkpurple white pink lightblue lightgreen gray black lightgray .
m = folium.Map(location=loc) folium.Marker( location=loc, icon=folium.Icon(icon="home", color="purple", icon_color="blue") ).add_to(m) Marker with a fonteawesome iconFont Awesome is a collection of scalable vector icons that can be customized and used in a variety of ways, such as in graphic design projects, websites, and applications. The icons are available in different styles, including Solid, Regular, and Brands, and can be easily integrated by adding the fa prefix
m = folium.Map(location=loc) folium.Marker( location=loc, icon=folium.Icon(icon="tents", prefix='fa') ).add_to(m) Extended Marker Customization with BeautifulIconsThe Leaflet Beautiful Icons is lightweight plugin that adds colorful iconic markers without images for Leaflet by giving full control of style to end user ( i.e. unlimited colors and many more...).
It ist exposed to folium via the Beautiful Icon plugin
Supported icon shapes are circle circle-dot doughnut rectangle rectangle-dot marker and the color be either one of the predefined or any valid hex code.
import folium.plugins as plugins folium.Marker( location=loc, icon=plugins.BeautifyIcon( icon="tent", icon_shape="circle", border_color='purple', text_color="#007799", background_color='yellow' ) ).add_to(m)Declassed Art: Declassed Plausible Deniability Toolkit
Python for Beginners: Rename Index in a Pandas Series
We use pandas series objects for various data processing tasks in python. In this article, we will discuss how to rename the index in a pandas series.
Table of Contents- Rename Index in a Pandas Series Using the index Attribute
- Rename the Index of a Series Using the rename_axis() Method
- Rename Index in a Series Inplace in Python
- Conclusion
When a series is created, the name of the index is empty. To rename the index of the series, you can use the name attribute of the series index object. You can assign the new index name to the name attribute of the index object to rename the series index as shown below.
import pandas as pd import numpy as np letters=["a","b","c","ab","abc","abcd","bc","d"] numbers=[3,23,11,14,16,2,45,65] series=pd.Series(letters,index=numbers) series.index.name="Numbers" print("The series is:") print(series)Output:
The series is: Numbers 3 a 23 b 11 c 14 ab 16 abc 2 abcd 45 bc 65 d dtype: objectIn the above example, We have first created a pandas series using the Series() constructor. Then, we assigned the string "Numbers" to the index.name attribute of the pandas series. Hence, the series index is renamed to "Numbers".
To rename the index of a series, you can also use the rename_axis() method.
Rename the Index of a Series Using the rename_axis() MethodThe rename_axis() method has the following syntax.
Series.rename_axis(mapper=_NoDefault.no_default, *, inplace=False, **kwargs)Here,
- The mapper parameter takes the new name of the index as its input argument.
- By default, the rename_axis() method returns a new series object. To modify the original series on which the rename_axis() method is invoked, you can set the inplace parameter to True.
After execution, the rename_axis() method returns a new series with renamed index as shown below.
import pandas as pd import numpy as np letters=["a","b","c","ab","abc","abcd","bc","d"] numbers=[3,23,11,14,16,2,45,65] series=pd.Series(letters,index=numbers) series=series.rename_axis("Numbers") print("The series is:") print(series)Output:
The series is: Numbers 3 a 23 b 11 c 14 ab 16 abc 2 abcd 45 bc 65 d dtype: objectIn the above example, we first created a series. Then, we used the rename_axis() method to rename the index column of the series. Here, the rename_axis() method returns a new series instead of modifying the original series.
Suggested Reading: If you are into machine learning, you can read this MLFlow tutorial with code examples. You might also like this article on clustering mixed data types in Python.
Rename Index in a Series Inplace in PythonYou can also modify the original series instead of creating a new series object after renaming the index. For this, you can set the inplace parameter to True in the rename_axis() method as shown below.
import pandas as pd import numpy as np letters=["a","b","c","ab","abc","abcd","bc","d"] numbers=[3,23,11,14,16,2,45,65] series=pd.Series(letters,index=numbers) series.rename_axis("Numbers",inplace=True) print("The series is:") print(series)Output:
The series is: Numbers 3 a 23 b 11 c 14 ab 16 abc 2 abcd 45 bc 65 d dtype: objectIn this example, we have set the inplace parameter to True in the rename_axis() parameter. Hence, the index of the original series has been renamed instead of creating a new series.
ConclusionIn this article, we have discussed how to rename the index in a pandas series using the index attribute and the renam_axis() method. To know more about the pandas module, you can read this article on how to sort a pandas dataframe. You might also like this article on how to drop columns from a pandas dataframe.
The post Rename Index in a Pandas Series appeared first on PythonForBeginners.com.
Real Python: The Real Python Podcast – Episode #135: Preparing Data to Measure True Machine Learning Model Performance
How do you prepare a dataset for machine learning (ML)? How do you go beyond cleaning the data and move toward measuring how the model performs? This week on the show, Jodie Burchell, developer advocate for data science at JetBrains, returns to talk about strategies for better ML model performance.
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
Lucas Cimon: Animated one-page-dungeon : Escape of the Torment
Last week, while translating John Harper's micro-TTRPG World of Dungeons: Turbo Breakers, I discovered the wonderful world of one page dungeons, starting with Michael Prescott splendid production at trilemma.com and also the yearly One Page Dungeon Context.
While crawling through the OPDC 2021 entries, I discovered a great map …
— Permalink