Planet Python

Syndicate content
Planet Python -
Updated: 1 day 3 hours ago

Robin Parmar: Arduino IDE: Best practices and gotchas

Fri, 2017-05-26 00:17
Programming for the Arduino is designed to be easy for beginners. The Integrated Development Environment (IDE) provides a safe place to write code, and handles the make and compiler steps that are required to create processor instructions from your C++ code.

This is fine for trivial applications and school exercises. But as soon as you try to use structured code (including classes and custom libraries) on a larger project, mysterious errors and roadblocks become the order of the day.

This article will consider best practices for working within the IDE. I will document a number of common errors and their workarounds. My perspective is of an experienced Python coder who finds C++ full of needless obfuscation. But we can make it work!

Why not switch?On encountering limitations with the Arduino IDE, the natural thing to do is switch to a mature development environment. For example, you could use Microsoft Visual Studio by way of Visual Micro, a plugin that enables Arduino coding. Or, use Eclipse with one of several available plugins: Sloeber, PlatformIO, or AVR-eclipse.

But there are cases when it is advantageous to stick with the Arduino IDE. For example, I might be working on a team with other less-experienced developers. While I might wish to carry the cognitive burden of Eclipse plus plugins plus project management, they might not.

Or I could be in a teaching environment where my code must be developed with the same tools my students will be using.

Language features... and what's missingThe Arduino IDE gives you many basic C++ language features plus hardware-specific functions. Control structures, values, and data types are documented in the Reference.

But you don't get modern features such as the Standard Template Library (STL). If you want to use stacks, queues, lists, vectors, etc. you must install a library. Start with those by Marc Jacobi (last updated 2 years ago) and Andy Brown (updated 1 year ago). I am sure there are plenty of articles discussing the relative merits of these or other solutions.

You also don't get new and delete operators, and there's good reason. Dynamic memory management is discouraged on microprocessor boards, since RAM and other resources are limited. There are libraries that add these to your toolkit, but the IDE encourages us to use C++ as though it was little more than plain vanilla C. It can be frustrating, but my advice is to adapt.

Code structureAs you know, when using the Arduino IDE you start coding with a sketch that is your application's entry point. As an example, I'll use project.ino.

Inside this file are always two functions, setup() and loop(). These take no parameters and return no values. There's not much you can do with them... except populate them with your code. These functions are part of an implicit code structure that could be written as follows:

void main() {

// declaration section

setup(); // initialisation (runs once)

while (true) {
loop(); // process-oriented code (runs forever)
In the IDE you never see the main() function and neither can you manipulate it.

Declaration sectionThe declaration section comes at the top of your project.ino. It is effectively outside any code block. Yes, even though it is in an implicit main() function. This means that only declarations and initializations are valid here. You cannot call methods of a class, nor access properties. This is our first rule:

Rule 1. The declaration section should contain only includes, initialisations of variables, and instantiations of classes.

This restriction can result in subtle errors when using classes. The declaration section is naturally where you will be instantiating classes you wish to use throughout the sketch. This means that the same restrictions just stated must apply to each and every class constructor. For this reason, you cannot use instances or methods of other classes in a constructor. No, not even for built-in libraries like Serial or Wire. Because the order of instantiation of classes is non-deterministic. All instances must be constructed before any instances are used.

Rule 2. A class constructor should have no arguments and do nothing but set default values for any properties.

Follow the example of the library classes for your own custom classes. Provide a begin() method that does take needed parameters and performs any initialization tasks. In other words, begin() should do everything you might otherwise expect the constructor to do. Call this method in the setup() block.

By the way, this solves another problem. A class that might be passed to another class requires a constructor that takes no parameters. Normally you would provide this in addition to another constructor template. But if you follow the rule two, this condition is already met.

Care with instantiationThe next discussion will prevent a syntax error. When instantiating a class with a constructor, you would normally do something like the following, assuming class Foo is defined elsewhere.

const byte pin = 10;
Foo bar(pin);

void setup() {}

void loop() {
int result =;
}But following our previous rule, constructors will never have arguments. You might quite naturally write this instead:

const byte pin = 10;
Foo bar();

void setup() {
void loop() {
int result =;
}This generates the error "request for member 'read' in 'bar' which is of non-class type 'Foo'. That appears nonsensical, because Foo is most definitely a class. Spot the syntax error?

To the compiler, bar() looks like a function call. You need to rewrite that line as:

Foo bar;

Abandoning the sketchBefore you even get to this point of sophistication in your code, you will be seeing all sorts of mystifying compiler output. "Error: 'foo' has not been declared" for a foo that most certainly has been declared. "Error: 'foo' does not name a type" for a foo that is definitively a type. And so on.

These errors occur because the compiler is generating function prototypes for you, automatically, even if you don't need them. These prototypes will even over-ride your own perfectly good code. The only thing to do is abandon the sketch! Move to the lifeboats! Compiler error! Danger, Will Robinson!


Do the following:

1. Create a new .cpp file, ensuring it is not named the same as your sketch, and also not named main.cpp. These are both name conflicts. As an example, let's call it primary.cpp.

2. Copy all the code from project.ino to primary.cpp.

3. Add #include <Arduino.h> to the top of primary.cpp, before your other includes. This ensures that your code can access the standard prototypes.

4. In project.ino leave only your #include statements. Delete everything else.

This will solve all those mysterious issues. You can now prototype your own functions and classes without the IDE getting in your way. You will, however, need to remember that every time you add an #include to primary.cpp, you need to also add it to project.ino. But it's a small price to pay.

Rule 3. Use a top-level C++ file instead of a sketch file.

Simple includesIt's easy to get confused about include files. But all an include represents is a copy and paste operation. The referenced code is inserted at the point where the include directive is located.

Here are the rules.

1. You need a .h header file for each .cpp code file.

2. The .cpp should have only one include, that being its corresponding header (a file with the same name but different extension).

3. The header file must have all the includes necessary for the correct running of the .h and .cpp code. And, in the correct order, if there are dependencies.

4. A header guard is required for each .h file. This prevents the header from being included in your project multiple times. It doesn't matter what variable name you choose for the test, so long as it is unique.
#ifndef LIB_H
#define LIB_H

// everything else

#endif5. If you have any sort of complex chaining, with circular pointer referencing, you may have to use forward referencing. But you should be avoiding this complexity in the sort of projects likely to run on an Arduino. So I won't count this rule in our next meta-rule.

Rule 4. Follow the four rules of correct header use.

Using librariesThe IDE limits how you use libraries to the very simplest case. Libraries get installed in one standard location across all your projects. You can put a library nowhere else. Why might you want to?

I am currently developing three modules as part of a single project. The code for each module is in its own folder. They have shared library code that I would like to put in a parallel folder, so I would have a folder hierarchy something like this:

/commonThen I could easily archive "myproject" into a ZIP file to share with the rest of the team.

Can I do this? No. It is not possible, since relative paths cannot be used in the IDE. And absolute paths are evil.

Rule 5. There is no rule to help manage libraries. Sorry.

Final WordsI have personally wasted dozens of hours before discovering these tips and working methods. It has been an enormous process of trial-and-error. If you are lucky enough to read this article first, you will never know the pain.

I have a donate button in the sidebar, in case you wish to thank me with a coffee.

In turn I'd like to thank Nick Gammon for an article I wish I'd read a bit sooner.

If there's interest, I might follow up with some words about general C++ syntax and issues that are not so Arduino-centric.
Categories: FLOSS Project Planets

Programming Ideas With Jake: Lots of Programming Videos!

Thu, 2017-05-25 17:53
A whole bunch of videos have recently dropped from programmer conferences. Like, a LOT!
Categories: FLOSS Project Planets

PyBites: How to Write a Python Class

Thu, 2017-05-25 09:44

In this post I cover learning Python classes by walking through one of our 100 days of code submissions.

Categories: FLOSS Project Planets

Python Bytes: #27 The PyCon 2017 recap and functional Python

Thu, 2017-05-25 04:00
<ul> <li>All videos available: <a href=""></a></li> <li>Lessons learned: <ul> <li>pick up swag on day one. vendors run out.</li> <li>take business cards with you and keep them on you</li> <li>Not your actual business cards unless you are representing your company.</li> <li>Cards that have your social media, github account, blog, or podcast or whatever on them.</li> <li>3x3 stickers are too big. 2x2 plenty big enough</li> <li>lightening talks are awesome, because they are a lot of ranges of speaking experience</li> <li>will definitely do that again</li> <li>try to go to the talks that are important to you, but don’t over stress about it, since they are taped. However, it would be lame if all the rooms were empty, so don’t everybody ditch.</li> <li>lastly: everyone knows Michael. </li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>How to Create Your First Python 3.6 AWS Lambda Function</strong></a></p> <ul> <li>Tutorial from <a href="">Full Stack Python</a></li> <li>Walks you through creating an account</li> <li>Select your Python version (3.6, yes!)</li> <li><code>def lambda_handler(event, context): …</code> # write this function, done!</li> <li>Set and read environment variables (could be connection strings and API keys)</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>How to Publish Your Package on PYPI</strong></a></p> <ul> <li>jetbrains article <ul> <li>structure of the package</li> <li>oops. doesn't include src, see</li> <li>decent discussion of a the contents of the file (but interestingly absent is an example file)</li> <li>good discussion of .pypirc file and links to the test and production PyPi</li> <li>example of using twine to push to PyPI</li> <li>overall: good discussion, but you'll still need a decent example.</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Coconut: Simple, elegant, Pythonic functional programming</strong></a></p> <ul> <li>Coconut is a functional programming language that compiles to Python. </li> <li>Since all valid Python is valid Coconut, using Coconut will only extend and enhance what you're already capable of in Python.</li> <li><code>pip install coconut</code> <ol> <li>Some of Coconut’s major features include built-in, syntactic support for:</li> <li>Pattern-matching,</li> <li>Algebraic data-types,</li> <li>Tail call optimization,</li> <li>Partial application,</li> <li>Better lambdas,</li> <li>Parallelization primitives, and</li> <li>A whole lot more, all of which can be found in <a href="">Coconut’s detailed documentation</a>.</li> </ol></li> <li>Talk Python episode coming in a week</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Choose a licence</strong></a></p> <ul> <li>MIT : simple and permissive</li> <li>Apache 2.0 : something extra about patents.</li> <li>GPL v3 : this is the contagious one that requires derivitive work to also be GPL v3</li> <li>Nice list with overviews of what they all mean with color coded bullet points: <a href=""></a></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Python for Scientists and Engineers</strong></a></p> <ul> <li><strong>Table of contents</strong>:</li> <li><strong>Beginners Start Here:</strong> <ul> <li><a href=""><strong>Create a Word Counter in Python</strong></a></li> <li><a href=""><strong>An introduction to Numpy and Matplotlib</strong></a></li> <li><a href=""><strong>Introduction to Pandas with Practical Examples (New)</strong></a></li> </ul></li> <li><strong>Main Book</strong> <ul> <li><a href=""><strong>Image and Video Processing in Python</strong></a></li> <li><a href=""><strong>Data Analysis with Pandas</strong></a></li> <li><a href=""><strong>Audio and Digital Signal Processing (DSP)</strong></a></li> <li><a href=""><strong>Control Your Raspberry Pi From Your Phone / Tablet</strong></a></li> </ul></li> <li><strong>Machine Learning Section</strong> <ul> <li><a href=""><strong>Machine Learning with an Amazon like Recommendation Engine</strong></a></li> <li><a href=""><strong>Machine Learning For Complete Beginners:</strong></a> <em><em></em></em>Learn how to predict how many Titanic survivors using machine learning. No previous knowledge needed!</li> <li><a href=""><strong>Cross Validation and Model Selection</strong></a>: In which we look at cross validation, and how to choose between different machine learning algorithms. Working with the Iris flower dataset and the Pima diabetes dataset.</li> </ul></li> <li><strong>Natural Language Processing</strong> <ul> <li><a href=""><strong>Introduction to NLP and Sentiment Analysis</strong></a></li> <li><a href=""><strong>Natural Language Processing with NTLK</strong></a></li> <li><a href=""><strong>Intro to NTLK, Part 2</strong></a></li> <li><a href=""><strong>Build a sentiment analysis program</strong></a></li> <li><a href=""><strong>Sentiment Analysis with Twitter</strong></a></li> <li><a href=""><strong>Analysing the Enron Email Corpus</strong></a>: The Enron Email corpus has half a million files spread over 2.5 GB. When looking at data this size, the question is, where do you even start?</li> <li><a href=""><strong>Build a Spam Filter using the Enron Corpus</strong></a></li> </ul></li> </ul> <p><strong>In other news</strong>:</p> <ul> <li><a href="">Python Testing with pytest</a> Beta release and initial feedback is going very well.</li> </ul>
Categories: FLOSS Project Planets

Experienced Django: Return of pylint

Wed, 2017-05-24 21:22

Until last fall I was working in python 2 (due to some limitations at work) and was very happy to have the Syntastic module in my Vim configuration to flag error each time I save a python file.  This was great, especially after writing in C/C++ for years where there is no official standard format and really poor tools to enforce coding standards.

Then, last fall when I started on Django, I made the decision to move to Python 3.  I quickly discovered that pylint is very version-dependent and running the python2.7 version of pylint against Python3 code was not going to work.

I wasn’t particularly familiar with virtualenv at the time, so I gave up and moved on with other things at the time.  I finally got back to fixing this and thus getting pylint and flake8 running again on my code.


I won’t cover the details of how to install Syntastic as it depends on how you manage your plugins in Vim and is well documented.  I will only point out here that Syntastic isn’t a checker by itself, it’s merely a plugin to run various checkers for you directly in Vim.  It run checkers for many languages, but I’m only using it for Python currently as the C code I use for work is so ugly that it will never pass.

Switching versions

The key to getting pylint to run against different versions of python is to not install pylint on a global level, but rather to install it in each virtualenv.  This seems obvious now that I’m more familiar with virtualenv, but I’ll admit it wasn’t at the time I first ran into the problem.

The other key to getting this to work is to only initiate Vim from inside the virtualenv.  This hampers my overall workflow a bit, as I tend to have gVim up and running for the long-term and just add files in new tabs as I go.  To get pylint to work properly, I’ll need to restart Vim when I switch python versions (at a minimum).  This shouldn’t be too much of a problem, however, as I’m doing less and less python2x coding these days.

Coding Style Thoughts

As I beat my head against horrible C code on a daily basis at work, I find myself appreciating more-and-more the idea of PEP-8 and having good tools for coding style enforcement.  While I frequently find some of the rules odd (two spaces here, but only one space there?) I really find it comforting to have a tool which runs, and runs quickly, to keep the code looking consistent.  Now if I could only get that kind of tool for C…….


Categories: FLOSS Project Planets

Daniel Bader: In Love, War, and Open-Source: Never Give Up

Wed, 2017-05-24 20:00
In Love, War, and Open-Source: Never Give Up

I’ll never forget launching my first open-source project and sharing it publicly on Reddit…

I had spent a couple of days at my parents’ place over Christmas that year and decided to use some of my spare time to work on a Python library I christened schedule.

The idea behind schedule was very simple and had a narrow focus (I find that that that’s always a good idea for libraries by the way):

Developers would use it like a timer to periodically call a function inside their Python programs.

The kicker was that schedule used a funky “natural sounding” syntax to specify the timer interval. For example, if you wanted to run a function every 10 minutes you’d do this:


Or, if you wanted to run a particular task every day at 10:30 in the morning, you’d do this:


Because I was so frustrated with Cron’s syntax I thought this approach was really cool. And so I decided this would be the first Python module I’d release as open-source.

I cleaned up the code and spent some time coming up with a nice README file—because that’s really the first thing that your potential users will see when they check out your library.

Once I had my module available on PyPI and the source code on GitHub I decided to call some attention to the project. The same night I posted a link to the repository to Reddit and a couple of other sites.

I still remember that I had shaky hands when I clicked the “submit” button…

It’s scary to put your work out there for the whole world to judge! Also, I didn’t know what to expect.

Would people call me stupid for writing a “simple” library like that?

Would they think my code wasn’t good enough?

Would they find all kinds of bugs and publicly shame me for them? I felt almost a physical sense of dread about pushing the “submit” button on Reddit that night!

The next morning I woke up and immediately checked my email. Were there any comments? Yes, about twenty or so!

I started reading through all of them, faster and faster—

And of course my still frightful mind immediately zoomed in on the negative ones, like

“Cool idea, but not particularly useful”,


“The documentation is not enough”,


“Not a big fan of the pseudo-english syntax. Way too clever and gimmicky.”

At this point I was starting to feel a little discouraged… I’d never really shared my code publicly before and to be honest I my skin receiving criticism on it was paper thin. After all, this was just something I wrote in a couple of hours and gave away for free.

The comment that really made my stomach churn was one from a well known member of the Python community:

“And another library with global state :-( … Such an API should not even exist. It sets a bad example.”

Ouch, that stung. I really looked up to that person and had used some of their libraries in other projects… It was almost like my worst fears we’re confirmed and we’re now playing out in front of me!

I’d never be able to get another job as a Python developer after this…

At the time I didn’t see the positive and supportive comments in that discussion thread. I didn’t see the almost 70 upvotes. I didn’t see the valuable lessons hidden in the seemingly rude comments. I dwelled on the negative and felt terrible and depressed that whole day.

So how do you think this story ends?

Did I delete the schedule repo, switched careers and never looked at Reddit again?


schedule now has almost 3,000 stars on GitHub and is among the top 70 Python repositories (out of more than 215,000). When PyPI’s download statistics we’re still working I saw that it got several thousand downloads per month. I get emails every week from people asking questions about it or thanking me for writing it…

Isn’t that crazy!? How’s that possible after all of these disheartening comments?

My answer is “I don’t know”—and I also don’t think that schedule is a particularly great library that deserves all this attention, by the way.

But, it seems to solve a problem for some people. It also seems to have a polarizing effect on developers who see it—some love it, some hate it.

Today I’m glad I shipped schedule that night.

Glad because it was helpful to so many people over the years and glad because it helped me develop a thicker skin when it comes to sharing and launching things publicly.

I’m partly writing this meandering post because not very long ago I found this comment buried in my Reddit message history:

As someone who has posted a number of projects and blog posts in r/Python, just wanted to drop you a line and encourage that you don’t let the comments in your thread get you down. You see all those upvotes?

Those are people that like your library, but don’t really have a comment to make in the thread proper. My biggest issue with /r/Python is that it tends towards cynicism and sometimes cruelty rather than encouragement and constructive criticism.

Keep up the great work,


Wow! What a positive and encouraging comment!

Back when I felt discouraged by all of these negative comments I must’ve missed it. But reading it a few years later made me re-live that whole situation and it showed me how much I’d grown as a developer and as a person in the meantime.

If you find yourself in a similar situation, maybe feeling bogged down by the developer community who can be unfiltered and pretty rude sometimes, don’t get discouraged.

Even if some people don’t like what you did there can be thousands who love your work.

It’s a big pond, and sometimes the best ideas are polarizing.

The only way to find out is to ship, ship, ship.

Categories: FLOSS Project Planets

Filipe Saraiva: LaKademy 2017

Wed, 2017-05-24 16:37

LaKademy 2017 group photo

Some weeks ago we had the fifth edition of the KDE Latin-America summit, LaKademy. Since the first edition, KDE community in Latin-America has grown up and now we has several developers, translators, artists, promoters, and more people from here involved in KDE activities.

This time LaKademy was held in Belo Horizonte, a nice city known for the amazing cachaça, cheese, home made beers, cheese, hills, and of course, cheese. The city is very cosmopolitan, with several options of activities and gastronomy, while the people is gentle. I would like to back to Belo Horizonte, maybe in my next vacation.

LaKademy activites were held in CEFET, an educational technological institute. During the days of LaKademy there were political demonstrations and a general strike in the country, consequence of the current political crisis here in Brazil. Despite I support the demonstrations, I was in Belo Horizonte for event. So I focused in the tasks while in my mind I was side-by-side with the workers on the streets.

Like in past editions I worked a lot with Cantor, the mathematical software I am the maintainer. This time the main tasks performed were an extensive set of reviews: revisions in pending patches, in the bug management system in order to close very old (and invalid) reports, and in the task management workboard, specially to ping developers with old tasks without any comment in the last year.

There were some work to implement new features as well. I finished a backends refactoring in order to provide a recommended version of the programming language for each backend in Cantor. How each programming language has its own planning and scheduling, it is common some programming language version not be correctly supported in a Cantor backend (Sage, I am thinking you). This feature presents a “recommended” version of the programming language supported for the Cantor backend, meaning that version was tested and it will work correctly with Cantor. It is more like a workaround in order to maintain the sanity of the developer while he try to support 11 different programming languages.

Other feature I worked but it is not finished is a option to select different LaTeX processors in Cantor. Currently there are several LaTeX processors available (like pdflatex, pdftex, luatex, xetex, …), some of them with several additional features. This option will increased the versatility of Cantor and will allow the use of moderns processors and their features in the software.

I addition to these tasks I fixed some bugs and helped Fernando Telles, my past SoK student, with some tasks in Cantor.

(Like in past editions)², in LaKademy 2017 I also worked in other set of tasks related to the management and promotion of KDE Brazil. I investigated how to bring back our unified feed with Brazilian blogs posts as in the old Planet KDE Português, utilized to send updates about KDE in Brazil to our social networks. Fred implemented the solution. So I updated this feed in social networks, updated our e-mail contact utilized in this networks, and started a bootstrap version of LaKademy website (but the team is migrating to WordPress, I think it will not be used). I also did a large revision in the tasks of KDE Brazil workboard, migrated past year from the TODO website. Besides all this we had the promo meeting to discuss our actions in Latin-America – all the tasks were documented in the workboard.

Of course, just as we worked intensely in those days, we also had a lot of fun between a push and other. LaKademy is also a opportunity to find old friends and make new ones. It is amazing see again the KDE fellows, and I invite the newcomers to stay with us and go to next LaKademy editions!

This year we had a problem that we must to address in next edition – all the participants were Brazilians. We need to think about how to integrate people from other Latin-America countries in LaKademy. It would be bad if the event become only an Akademy-BR.

Filipe and Chicão

So, I give my greetings to the community and put myself in the mission to continue to work in order to grown the Latin-America as an important player to the development and future of KDE.

Categories: FLOSS Project Planets

EuroPython: EuroPython 2017: Social event tickets available

Wed, 2017-05-24 10:43

After trainings and talks, EuroPython is going (Coco)nuts! Join us for the EuroPython social event in Rimini, which will be held in the Coconuts Club on Thursday, July 13th. 

Tickets for the social event are not included in the conference ticket. They are now available in our ticket store (listed under ‘Goodies’) for the price of 25 €. The social event ticket includes an aperitivo buffet of Italian specialties, a choice of two drinks and a reserved area in the club from 19:00 to 22:00. The club will open to the general public after that. 

Leave the conference tickets fields blank if you only want to purchase social event tickets.

Take this opportunity to network and socialize with other Python attendees and buy your social event ticket now on the registration page.


EuroPython 2017 Team
EuroPython Society
EuroPython 2017 Conference

Categories: FLOSS Project Planets

Enthought: Enthought Receives 2017 Product of the Year Award From National Instruments LabVIEW Tools Network

Wed, 2017-05-24 09:42

Python Integration Toolkit for LabVIEW recognized for extending LabVIEW connectivity and bringing the power of Python to applications in Test, Measurement and the Industrial Internet of Things (IIoT)

AUSTIN, TX – May 24, 2017 Enthought, a global leader in scientific and analytic computing solutions, was honored this week by National Instruments with the LabVIEW Tools Network Platform Connectivity 2017 Product of the Year Award for its Python Integration Toolkit for LabVIEW.

First released at NIWeek 2016, the Python Integration Toolkit enables fast, two-way communication between LabVIEW and Python. With seamless access to the Python ecosystem of tools, LabVIEW users are able to do more with their data than ever before. For example, using the Toolkit, a user can acquire data from test and measurement tools with LabVIEW, perform signal processing or apply machine learning algorithms in Python, display it in LabVIEW, then share results using a Python-enabled web dashboard.

Click to see the webinar “Using Python and LabVIEW to Rapidly Solve Engineering Problems” to learn more about adding capabilities such as machine learning by extending LabVIEW applications with Python.

“Python is ideally suited for scientists and engineers due to its simple, yet powerful syntax and the availability of an extensive array of open source tools contributed by a user community from industry and R&D,” said Dr. Tim Diller, Director, IIoT Solutions Group at Enthought. “The Python Integration Toolkit for LabVIEW unites the best elements of two major tools in the science and engineering world and we are honored to receive this award.”

Key benefits of the Python Integration Toolkit for LabVIEW from Enthought:

  • Click to see the webinar “Introduction to the Python Integration Toolkit for LabVIEW” to learn more about the fast, two-way communication between Python and LabVIEW.

    Enables fast, two-way communication between LabVIEW and Python

  • Provides LabVIEW users seamless access to tens of thousands of mature, well-tested scientific and analytic software packages in the Python ecosystem, including software for machine learning, signal processing, image processing and cloud connectivity
  • Speeds development time by providing access to robust, pre-developed Python tools
  • Provides a comprehensive out-of-the box solution that allows users to be up and running immediately

“Add-on software from our third-party developers is an integral part of the NI ecosystem, and we’re excited to recognize Enthought for its achievement with the Python Integration Toolkit for LabVIEW,” said Matthew Friedman, senior group manager of the LabVIEW Tools Network at NI.

The Python Integration Toolkit is available for download via the LabVIEW Tools Network, and also includes the Enthought Canopy analysis environment and Python distribution. Enthought’s training, support and consulting resources are also available to help LabVIEW users maximize their value in leveraging Python.

For more information on Enthought’s Python Integration Toolkit for LabVIEW, visit


Additional Resources Product Information

Python Integration Toolkit for LabVIEW product page

Download a free trial of the Python Integration Toolkit for LabVIEW


Webinar: Using Python and LabVIEW to Rapidly Solve Engineering Problems | Enthought
April 2017

Webinar: Introducing the New Python Integration Toolkit for LabVIEW from Enthought
September 2016

About Enthought

Enthought is a global leader in scientific and analytic software, consulting, and training solutions serving a customer base comprised of some of the most respected names in the oil and gas, manufacturing, financial services, aerospace, military, government, biotechnology, consumer products and technology industries. The company was founded in 2001 and is headquartered in Austin, Texas, with additional offices in Cambridge, United Kingdom and Pune, India. For more information visit and connect with Enthought on Twitter, LinkedIn, Google+, Facebook and YouTube.

About NI

Since 1976, NI ( has made it possible for engineers and scientists to solve the world’s greatest engineering challenges with powerful platform-based systems that accelerate productivity and drive rapid innovation. Customers from a wide variety of industries – from healthcare to automotive and from consumer electronics to particle physics – use NI’s integrated hardware and software platform to improve the world we live in.

About the LabVIEW Tools Network

The LabVIEW Tools Network is the NI app store equipping engineers and scientists with certified, third-party add-ons and apps to complete their systems. Developed by industry experts, these cutting-edge technologies expand the power of NI software and modular hardware. Each third-party product is reviewed to meet specific guidelines and ensure compatibility. With hundreds of products available, the LabVIEW Tools Network is part of a rich ecosystem extending the NI Platform to help customers positively impact our world. Learn more about the LabVIEW Tools Network at

LabVIEW, National Instruments, NI and and NIWeek are trademarks of National Instruments. Enthought, Canopy and Python Integration Toolkit for LabVIEW are trademarks of Enthought, Inc.

Media Contact

Courtenay Godshall, VP, Marketing, +1.512.536.1057,

The post Enthought Receives 2017 Product of the Year Award From National Instruments LabVIEW Tools Network appeared first on Enthought Blog.

Categories: FLOSS Project Planets

Caktus Consulting Group: PyCon 2017 Recap

Wed, 2017-05-24 09:30

Caktus attended PyCon 2017 in Portland from May 18-21. It was the first PyCon for some, while others were PyCon veterans. All of us were looking forward to the opportunity to hear some great talks and make or renew connections in the Python community.

Getting Set Up

Right after arriving from the East Coast on Thursday it was time to set up for Day 1 of the expo. We eagerly prepared for the meet and greet opening night.

Our Ultimate Tic Tac Toe game made a comeback this year with a new AI feature. We only had 2 winners, although one of them beat the AI four times! (Our developers think it’s time to turn the difficulty up to 11.)

Meetings and Giveaways

As expected, booth time was busy. It was exciting to welcome so many members of the Python community. We enjoyed chatting about what we do at Caktus and our work building web apps with Django, as well as making connections or meeting with people we haven’t seen in awhile. Mark was excited to catch up with Tom Christie of Django REST Framework.

Best part of PyCon is being a shameless fanboy for those whose work makes my job easier ❤

— Mark Lavin (@DrOhYes) May 20, 2017

In addition to Caktus swag, we had two prize giveaways on offer this year. The first one was a social contest on Twitter. Christine was our winner by random selection - congratulations!

The second giveaway was a random drawing of people who signed up for our newsletter. Congratulations to Mike for winning that drawing.

Lots to Talk About

Six Caktus developers attended a variety of talks and came out of each of them feeling inspired by fresh ideas.

Interesting idea from @_tomchristie: API mocking. Front End devs can get started immediately & this promotes API design first. #PyCon2017

— Erin Mullaney (@_erin_rachel) May 20, 2017

I'm feeling inspired to contribute to some #OpenScience projects after outstanding keynotes by @jakevdp and @katyhuff at #PyCon2017

— Jeff Bradberry (@jeff_bradberry) May 20, 2017

.@jsaryer demonstrating how mutation testing can prove that your "covered" code is actually being tested #pycon2017

— Charlotte Mays (@charlottecodes) May 19, 2017

We can’t wait to see how they apply what they’ve learned to their work at Caktus.

After-hours Event

This year, Caktus hosted an exclusive, invite-only event at Fuse Bar. We were excited to welcome clients and special guests for sushi, nibbles, and refreshments.

Some of our clients are based across the country from us, so it was nice to have the opportunity to catch up in person and mingle.

Job Fair

We spoke to lots of interested developers at the PyCon job fair on May 21st. Some of the most common questions we received - and their answers - are below.

Question 1: Where is Caktus based?

Our office is in a lovely brick building in downtown Durham, North Carolina.

Question 2: What does Caktus do / What projects have you worked on?

Caktus is a consultancy and development agency. Work primarily focuses on building custom web and SMS apps in Python, using the Django framework. Some of our projects include building the world’s first SMS-based voter registration system, a live event management app, and responsive, database-driven websites for clients in industries like food and beverage, travel and entertainment.

We also do team augmentation and offer discovery workshops to help our clients strategically plan out the user experience and technical development aspects of their projects. To find out more about our specialties at Caktus, visit our Services page.

Question 3: Do you have opportunities for remote work?

We hire remotely for some contract work, but full-time positions must be based out of our Durham office.

Question 4: Where can I find your current openings / How can I apply?

If you’re interested in joining us at Caktus, check out our Careers page. We’ll need a resume and cover letter, and encourage you to send a link to your GitHub, portfolio, or website as well.

Caktus is growing fast, and we’re pleased to offer great benefits and a fair, equal-opportunity working environment. Why not grow with us?

Until next year!

As always, we had a great time meeting and mingling with our fellow Pythonistas. Thanks to the organizers for putting on another fantastic event, our fellow sponsors for supporting it, the volunteers who kept things moving, and of course, all the attendees for your energy and enthusiasm. See you next year!

Categories: FLOSS Project Planets

PyBites: PyCon 2017 - Digest, Impressions, Reflection

Wed, 2017-05-24 06:00

PyCon 2017 was such a great conference, I made so many good connections, got to see many good things the community is working on. It was very inspiring. In this article a digest.

Categories: FLOSS Project Planets

NumFOCUS: Welcome Nancy Nguyen, the new NumFOCUS Events Coordinator!

Tue, 2017-05-23 12:48
NumFOCUS is pleased to announce Nancy Nguyen has been hired as our new Events Coordinator. Nancy has over five years of event management experience in the non-profit and higher education sectors. She graduated from The University of Texas at Austin in 2011 with a BA in History. Prior to joining NumFOCUS, Nancy worked in development and fundraising […]
Categories: FLOSS Project Planets

PyCharm: PyCharm 2017.1.3 Out Now!

Tue, 2017-05-23 12:44

We’re happy to announce that PyCharm 2017.1.3 is now available. You can get it now from our website, or using the updater in PyCharm.

Improvements since PyCharm 2017.1.2:

  • Test runner issues: django-nose tests, stdout from tests, and various other fixes
  • Pyramid project creation with Chameleon templates
  • The issue which caused our community edition installer for macOS to be unsigned has been fixed
  • An issue that caused PyCharm to freeze on startup has been resolved
  • Several bugs related to React support have been fixed
  • macOS terminal locale detection
  • As always, you can read the further details in the release notes

We’d like to thank the users who tried the EAP versions of this release for helping us make sure that this version of PyCharm is of the quality you expect from a JetBrains IDE.

If you’d like to see new PyCharm features before we release them, try out the EAP version of our next release.

PyCharm Team
-The Drive to Develop

Categories: FLOSS Project Planets

codeboje: This Problem with PySide, Py2App and Brew drove me Nuts

Tue, 2017-05-23 11:33

I build several desktop applications using PySide. It works like a charm on Windows. However, I found a nasty bug in the Mac version which I could not reproduce on my local machine. This one drove me nuts. It was hard to analyze, I own just one Mac and do not have access to another machine. Luckily, with the help of some very long googling (guessing symptoms) and the help of a user of one of my applications, I could fix the issue. As a reminder for myself and help for you, I document it.


I build the application using PySide, Py2App and Python 3 on a Mac using a virtualenv. I used Brew for installing PySide and Python, and PySide was also installed inside my virutaenv.

The application builds and runs fine on my machine.

However, on a different Mac, the application crashed without any useful traces in the system logs.

My application also writes a separate log file, but this was empty. So it seemed like the application crashed before even reaching my code.

After some google and adjusting my log outputs, however, I found out that it started to execute my code and then crashed when initializing PySide.

First, I thought it was just a problem on this single users machine, but I got a similar bug report which gave me some hints it might be a general problem.

Now, better equipped I found the culprit after some additional googling.

The application could not load the QT binaries.

The problem is a combination of installing PySide and with Brew and using PySide inside a virtualenv. While building the application Py2App can't pick up the correct library path, and the generated application includes the libraries but with an incorrect path in the Mac specific descriptors.

otool -L <my-app>/Contents/Resources/lib/python3.4/lib-dynload/PySide/

It should list the file with @executable_path and not with an absolute path only valid for your machine.

In my case, the binaries were referenced by absolute paths which were of course not valid on someone else machine.


I do not know who exactly is causing this issue, but the following workaround did help me.

  • Deinstall Python and PySide
  • Deinstall Brew
  • Install Macport
  • Install Python and PySide with Macport
  • PySide is installed in the main python site-packages
  • Create a new virutalenv and let it use the main site-packages

And then continue building the application as before. This time the lib references were fine in the application.

Guess what?

It worked now. Yippie.

Categories: FLOSS Project Planets

Chris Moffitt: How Accurately Can Prophet Project Website Traffic?

Tue, 2017-05-23 08:05

In early March, I published an article introducing prophet which is an open source library released by Facebook that is used to automate the time series forecasting process. As I promised in that article, I’m going to see how well those predictions held up to the real world after 2.5 months of traffic on this site.

Getting Started

Before going forward, please review the prior article on prophet. I also encourage you to review the matplotlib article which is a useful starting point for understanding how to plot these trends. Without further discussion, let’s dive into the code. If you wish to follow along, the notebook is posted on github.

First, let’s get our imports setup, plotting configured and the forecast data read into our DataFrame:

import pandas as pd import numpy as np import matplotlib.pyplot as plt %matplotlib inline'ggplot') proj = pd.read_excel('') proj[['ds', 'yhat', 'yhat_lower', 'yhat_upper']].head()

The projected data is stored in the proj DataFrame. There are many columns but we only care about a couple of them:

ds yhat yhat_lower yhat_upper 0 2014-09-25 3.294797 2.770241 3.856544 1 2014-09-26 3.129766 2.564662 3.677923 2 2014-09-27 3.152004 2.577474 3.670529 3 2014-09-28 3.659615 3.112663 4.191708 4 2014-09-29 3.823493 3.279714 4.376206

All of the projections are based on the log scale so we need to convert them back and filter through May 20th:

proj["Projected_Sessions"] = np.exp(proj.yhat).round() proj["Projected_Sessions_lower"] = np.exp(proj.yhat_lower).round() proj["Projected_Sessions_upper"] = np.exp(proj.yhat_upper).round() final_proj = proj[(proj.ds > "3-5-2017") & (proj.ds < "5-20-2017")][["ds", "Projected_Sessions_lower", "Projected_Sessions", "Projected_Sessions_upper"]]

Next, I’ll read in the actual traffic from March 6th through May 20th and rename the columns for consistency sake:

actual = pd.read_excel('Traffic_20170306-20170519.xlsx') actual.columns = ["ds", "Actual_Sessions"] actual.head() ds Actual_Sessions 0 2017-03-06 2227 1 2017-03-07 2093 2 2017-03-08 2068 3 2017-03-09 2400 4 2017-03-10 1888

Pandas makes combining all of this into a single DataFrame simple:

df = pd.merge(actual, final_proj) df.head() ds Actual_Sessions Projected_Sessions_lower Projected_Sessions Projected_Sessions_upper 0 2017-03-06 2227 1427.0 2503.0 4289.0 1 2017-03-07 2093 1791.0 3194.0 5458.0 2 2017-03-08 2068 1162.0 1928.0 3273.0 3 2017-03-09 2400 1118.0 1886.0 3172.0 4 2017-03-10 1888 958.0 1642.0 2836.0 Evaluating the Results

With the predictions and actuals in a single DataFrame, let’s see how far our projections were off from actuals by calculating the difference and looking at the basic stats.

df["Session_Delta"] = df.Actual_Sessions - df.Projected_Sessions df.Session_Delta.describe() count 75.000000 mean 739.440000 std 711.001829 min -1101.000000 25% 377.500000 50% 619.000000 75% 927.000000 max 4584.000000

This gives us a basic idea of the errors but visualizing will be more useful. Let’s use the process described in the matplotlib article to plot the data.

# Need to convert to just a date in order to keep plot from throwing errors df['ds'] = df['ds'] fig, ax = plt.subplots(figsize=(9, 6)) df.plot("ds", "Session_Delta", ax=ax) fig.autofmt_xdate(bottom=0.2, rotation=30, ha='right');

This visualization is helpful for understanding the data and highlights a couple of things:

  • Most of the variance shows the actual traffic being higher than projected
  • There were two big spikes in April which correspond to publish dates for articles
  • The majority of the variance was less than 1000

On the surface this may seem a little disappointing. However, we should not look at the predicted value as much as the predicted range. Prophet gives us the range and we can use the fill_between function in matplotlib to display the range around the predicted values:

fig, ax = plt.subplots(figsize=(9, 6)) df.plot(kind='line', x='ds', y=['Actual_Sessions', 'Projected_Sessions'], ax=ax, style=['-','--']) ax.fill_between(df['ds'].values, df['Projected_Sessions_lower'], df['Projected_Sessions_upper'], alpha=0.2) ax.set(title='Pbpython Traffic Prediction Accuracy', xlabel='', ylabel='Sessions') fig.autofmt_xdate(bottom=0.2, rotation=30, ha='right'

This view restores some more confidence in our model. It looks like we had a big over prediction at the beginning of the time frame but did not predict the impact of the two articles published in the subsequent weeks. More interestingly, the majority of the traffic was right at the upper end of our projection and the weekly variability is captured reasonably well.

Final Thoughts

So, how good was the model? I think a lot depends on what we were hoping for. In my case, I was not making any multi-million dollar decisions based on the accuracy. Additionally, I did not have any other models in place so I have nothing to compare the prediction to. From that perspective, I am happy that I was able to develop a fairly robust model with only a little effort. Another way to think about this is that if I were trying to put this model together by hand, I am sure I would not have come up with a better approach. Additionally, the volume of the views with the April 25th article is nearly impossible to predict so I don’t worry about that miss and the subsequent uptick in volume.

Predictive models are rarely a one shot affair. It takes some time to understand what makes them tick and how to interpret their output. I plan to look at some of the tuning options to see which parameters I could tweak to improve the accuracy for my use case.

I hope this is useful and would definitely like to hear what others have found with prophet or other tools to predict this type of activity. For those of you with experience predicting website traffic, would this have been a “good” outcome?

Categories: FLOSS Project Planets

Simple is Better Than Complex: How to Deploy a Django Application on RHEL 7

Tue, 2017-05-23 05:02

In this tutorial, you will learn how to deploy a Django application with PostgreSQL, Nginx, Gunicorn on a Red Hat Enterprise Linux (RHEL) version 7.3. For testing purpose I’m using an Amazon EC2 instance running RHEL 7.3.

Recently I had to deploy an existing Django project running on Ubuntu 16.04 to a new environment, RHEL 7.3. It gave me some headache because I don’t have much server administration skills and I wasn’t familiar with Security Enhanced Linux (SELinux) distributions, so I thought about sharing the details of the deployment so it could help someone in the same position I was.

If you are just getting started with Django deployment, and doesn’t have a good reason to be using RHEL, I suggest you use Ubuntu instead. It requires less configuration, and the process is fairly easier than using RHEL. Perhaps you could check this past tutorial: How to Deploy a Django Application to Digital Ocean.

Anyway, for this tutorial I will deploy the following Django application: It is just an empty Django project to demonstrate the deployment process. So, every time you see urban-train, change it for your project name.

Initial Setup

First, let’s install all the needed resources and applications. Get started by installing git, gcc and python-virtualenv. Everything should be available in the yum repository.

sudo yum -y install git gcc python-virtualenv

Create a system user for the application:

sudo groupadd --system urbantrain sudo useradd --system --gid urbantrain --shell /bin/bash --home /opt/urban-train urbantrain

Create the Django project home inside /opt:

sudo mkdir /opt/urban-train

Give the permissions to the urbantrain user:

sudo chown urbantrain:urbantrain /opt/urban-train PostgreSQL Server

Now install PostgreSQL 9.6 server and development tools:

sudo yum -y install sudo yum -y install postgresql96-server postgresql96-contrib postgresql96-devel

Initialize the database:

sudo /usr/pgsql-9.6/bin/postgresql96-setup initdb

Start and enable the PostgreSQL 9.6 service:

sudo systemctl start postgresql-9.6 sudo systemctl enable postgresql-9.6

Log in with the postgres user:

sudo su - postgres

Create a database user, set a password (save it for later) and create a database for the Bootcamp application:

createuser u_urban psql -c "ALTER USER u_urban WITH PASSWORD '123';" createdb --owner u_urban urban_prod

Now we have to update the authentication method of the database user in the file pg_hba.conf:

vi /var/lib/pgsql/9.6/data/pg_hba.conf

Go to the bottom of the file, find this snippet:

# TYPE DATABASE USER ADDRESS METHOD # "local" is for Unix domain socket connections only local all all peer # IPv4 local connections: host all all ident # IPv6 local connections: host all all ::1/128 ident # Allow replication connections from localhost, by a user with the # replication privilege. #local replication postgres peer #host replication postgres ident #host replication postgres ::1/128 ident

Change the method from ident to md5 on the IPv4 and IPv6 rows:

# TYPE DATABASE USER ADDRESS METHOD # "local" is for Unix domain socket connections only local all all peer # IPv4 local connections: host all all md5 # <- here # IPv6 local connections: host all all ::1/128 md5 # <- and here # Allow replication connections from localhost, by a user with the # replication privilege. #local replication postgres peer #host replication postgres ident #host replication postgres ::1/128 ident

Save the file and exit.

Now, log out from the postgres session:


Restart the PostgreSQL 9.6 server:

sudo systemctl restart postgresql-9.6 Python Virtual Environment

First, log in with the urbantrain system user:

sudo su - urbantrain

Start a new python-virtualenv inside the /opt/urban-train directory:

virtualenv venv

Activate the python-virtualenv:

source venv/bin/activate

Create a directory named logs that will be used by Gunicorn and Nginx to write the logs:

mkdir logs

Clone your project’s repository inside the /opt/urban-train directory:

git clone

Now we have to install the Python dependencies. But first, add the PostgreSQL to the path. The psycopg2 will need it to install:

export PATH=$PATH:/usr/pgsql-9.6/bin/

Upgrade the Python package manager:

pip install pip --upgrade

Install the dependencies (/opt/urban-train/urban-train/requirements.txt inside the repository):

pip install -r requirements.txt

Migrate the database:

python migrate

Collect the static assets (css, javascripts, images, etc.):

python collectstatic --noinput Gunicorn

Still logged in with the urbantrain user, let’s create a gunicorn_start file to startup the application server.

vi /opt/urban-train/gunicorn_start

Use the structure below, change the paths, user/groups etc accordingly to your environment/project:

#!/bin/bash NAME="urban_train" DJANGODIR=/opt/urban-train/urban-train USER=urban GROUP=urban WORKERS=3 BIND=unix:/opt/urban-train/run/gunicorn.sock DJANGO_SETTINGS_MODULE=urban_train.settings DJANGO_WSGI_MODULE=urban_train.wsgi LOGLEVEL=error cd $DJANGODIR source venv/bin/activate export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE export PYTHONPATH=$DJANGODIR:$PYTHONPATH exec venv/bin/gunicorn ${DJANGO_WSGI_MODULE}:application \ --name $NAME \ --workers $WORKERS \ --user=$USER \ --group=$GROUP \ --bind=$BIND \ --log-level=$LOGLEVEL \ --log-file=-

Make the gunicorn_start file executable:

chmod u+x gunicorn_start

Create a directory named run, for the unix socket file:

mkdir run Gunicorn Systemd Service

Now let’s create a systemd service file for gunicorn server to manage

First, exit the urbantrain user. Create the following systemd service file:

sudo vi /etc/systemd/system/gunicorn.service

Insert the following in the file:

[Unit] Description=gunicorn daemon [Service] User=urbantrain Group=urbantrain WorkingDirectory=/opt/urban-train ExecStart=/opt/urban-train/gunicorn_start [Install]

Start the gunicorn systemd service we created and enable it so that it starts at boot:

sudo systemctl start gunicorn sudo systemctl enable gunicorn Nginx

The application must be served behind a proxy server. First, create a yum repo file:

sudo vi /etc/yum.repos.d/nginx.repo

Add repository information:

[nginx] name=nginx repo baseurl=$basearch/ gpgcheck=0 enabled=1

Save and exit.

Now install nginx:

sudo yum -y install nginx

Because of the security policies of the SELinux, we need to manually add the httpd_t to the list of permissive domains, run this command:

sudo semanage permissive -a httpd_t

Now let’s create a .conf file for the project. Go to the conf.d directory:

cd /etc/nginx/conf.d/

Remove the default.conf file, and create a new one for our project:

sudo rm default.conf sudo vi urban-train.conf

Inside of the urban-train.conf file, insert the new server block:

upstream app_server { server unix:/opt/urban-train/run/gunicorn.sock fail_timeout=0; } server { listen 80; server_name IP_ADDRESS_OR_DOMAIN_NAME; # <- insert here the ip address/domain name keepalive_timeout 5; client_max_body_size 4G; access_log /opt/urban-train/logs/nginx-access.log; error_log /opt/urban-train/logs/nginx-error.log; location /static/ { alias /opt/urban-train/static/; } location /media/ { alias /opt/urban-train/media/; } location / { try_files $uri @proxy_to_app; } location @proxy_to_app { proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_redirect off; proxy_pass http://app_server; } }

Test the nginx.conf:

sudo nginx -t

Start the nginx service and enable it so that it starts at boot:

sudo systemctl start nginx sudo systemctl enable nginx Final Remarks

Everything should be working now. Do a final test, reboot the server and check if everything starts up normally:

sudo reboot

Some things that may cause trouble:

Categories: FLOSS Project Planets

S. Lott: Python under XCode?!? Cool. Sort of.

Tue, 2017-05-23 04:00
Dan Bader (@dbader_org)5/5/17, 10:47 PM"Running Python in Xcode: Step by Step"…

Thanks for the link. There's a fair amount of "this doesn't seem to be what Xcode was designed to do." But it does seem to work.

I'll stick with Komodo Edit.
Categories: FLOSS Project Planets

Daniel Bader: The Meaning of Underscores in Python

Mon, 2017-05-22 20:00
The Meaning of Underscores in Python

The various meanings and naming conventions around single and double underscores (“dunder”) in Python, how name mangling works and how it affects your own Python classes.

Single and double underscores have a meaning in Python variable and method names. Some of that meaning is merely by convention and intended as a hint to the programmer—and some of it is enforced by the Python interpreter.

If you’re wondering “What’s the meaning of single and double underscores in Python variable and method names?” I’ll do my best to get you the answer here.

In this article I’ll discuss the following five underscore patterns and naming conventions and how they affect the behavior of your Python programs:

  • Single Leading Underscore: _var
  • Single Trailing Underscore: var_
  • Double Leading Underscore: __var
  • Double Leading and Trailing Underscore: __var__
  • Single Underscore: _

At the end of the article you’ll also find a brief “cheat sheet” summary of the five different underscore naming conventions and their meaning, as well as a short video tutorial that gives you a hands-on demo of their behavior.

Let’s dive right in!

1. Single Leading Underscore: _var

When it comes to variable and method names, the single underscore prefix has a meaning by convention only. It’s a hint to the programmer—and it means what the Python community agrees it should mean, but it does not affect the behavior of your programs.

The underscore prefix is meant as a hint to another programmer that a variable or method starting with a single underscore is intended for internal use. This convention is defined in PEP 8.

This isn’t enforced by Python. Python does not have strong distinctions between “private” and “public” variables like Java does. It’s like someone put up a tiny underscore warning sign that says:

“Hey, this isn’t really meant to be a part of the public interface of this class. Best to leave it alone.”

Take a look at the following example:

class Test: def __init__(self): = 11 self._bar = 23

What’s going to happen if you instantiate this class and try to access the foo and _bar attributes defined in its __init__ constructor? Let’s find out:

>>> t = Test() >>> 11 >>> t._bar 23

You just saw that the leading single underscore in _bar did not prevent us from “reaching into” the class and accessing the value of that variable.

That’s because the single underscore prefix in Python is merely an agreed upon convention—at least when it comes to variable and method names.

However, leading underscores do impact how names get imported from modules. Imagine you had the following code in a module called my_module:

# This is def external_func(): return 23 def _internal_func(): return 42

Now if you use a wildcard import to import all names from the module, Python will not import names with a leading underscore (unless the module defines an __all__ list that overrides this behavior):

>>> from my_module import * >>> external_func() 23 >>> _internal_func() NameError: "name '_internal_func' is not defined"

By the way, wildcard imports should be avoided as they make it unclear which names are present in the namespace. It’s better to stick to regular imports for the sake of clarity.

Unlike wildcard imports, regular imports are not affected by the leading single underscore naming convention:

>>> import my_module >>> my_module.external_func() 23 >>> my_module._internal_func() 42

I know this might be a little confusing at this point. If you stick to the PEP 8 recommendation that wildcard imports should be avoided, then really all you need to remember is this:

Single underscores are a Python naming convention indicating a name is meant for internal use. It is generally not enforced by the Python interpreter and meant as a hint to the programmer only.

2. Single Trailing Underscore: var_

Sometimes the most fitting name for a variable is already taken by a keyword. Therefore names like class or def cannot be used as variable names in Python. In this case you can append a single underscore to break the naming conflict:

>>> def make_object(name, class): SyntaxError: "invalid syntax" >>> def make_object(name, class_): ... pass

In summary, a single trailing underscore (postfix) is used by convention to avoid naming conflicts with Python keywords. This convention is explained in PEP 8.

3. Double Leading Underscore: __var

The naming patterns we covered so far received their meaning from agreed upon conventions only. With Python class attributes (variables and methods) that start with double underscores, things are a little different.

A double underscore prefix causes the Python interpreter to rewrite the attribute name in order to avoid naming conflicts in subclasses.

This is also called name mangling—the interpreter changes the name of the variable in a way that makes it harder to create collisions when the class is extended later.

I know this sounds rather abstract. This is why I put together this little code example we can use for experimentation:

class Test: def __init__(self): = 11 self._bar = 23 self.__baz = 23

Let’s take a look at the attributes on this object using the built-in dir() function:

>>> t = Test() >>> dir(t) ['_Test__baz', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_bar', 'foo']

This gives us a list with the object’s attributes. Let’s take this list and look for our original variable names foo, _bar, and __baz—I promise you’ll notice some interesting changes.

  • The variable appears unmodified as foo in the attribute list.
  • self._bar behaves the same way—it shows up on the class as _bar. Like I said before, the leading underscore is just a convention in this case. A hint for the programmer.
  • However with self.__baz, things look a little different. When you search for __baz in that list you’ll see that there is no variable with that name.

So what happened to __baz?

If you look closely you’ll see there’s an attribute called _Test__baz on this object. This is the name mangling that the Python interpreter applies. It does this to protect the variable from getting overridden in subclasses.

Let’s create another class that extends the Test class and attempts to override its existing attributes added in the constructor:

class ExtendedTest(Test): def __init__(self): super().__init__() = 'overridden' self._bar = 'overridden' self.__baz = 'overridden'

Now what do you think the values of foo, _bar, and __baz will be on instances of this ExtendedTest class? Let’s take a look:

>>> t2 = ExtendedTest() >>> 'overridden' >>> t2._bar 'overridden' >>> t2.__baz AttributeError: "'ExtendedTest' object has no attribute '__baz'"

Wait, why did we get that AttributeError when we tried to inspect the value of t2.__baz? Name mangling strikes again! It turns out this object doesn’t even have a __baz attribute:

>>> dir(t2) ['_ExtendedTest__baz', '_Test__baz', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_bar', 'foo', 'get_vars']

As you can see __baz got turned into _ExtendedTest__baz to prevent accidental modification:

>>> t2._ExtendedTest__baz 'overridden'

But the original _Test__baz is also still around:

>>> t2._Test__baz 42

Double underscore name mangling is fully transparent to the programmer. Take a look at the following example that will confirm this:

class ManglingTest: def __init__(self): self.__mangled = 'hello' def get_mangled(self): return self.__mangled >>> ManglingTest().get_mangled() 'hello' >>> ManglingTest().__mangled AttributeError: "'ManglingTest' object has no attribute '__mangled'"

Does name mangling also apply to method names? It sure does—name mangling affects all names that start with two underscore characters (“dunders”) in a class context:

class MangledMethod: def __method(self): return 42 def call_it(self): return self.__method() >>> MangledMethod().__method() AttributeError: "'MangledMethod' object has no attribute '__method'" >>> MangledMethod().call_it() 42

Here’s another, perhaps surprising, example of name mangling in action:

_MangledGlobal__mangled = 23 class MangledGlobal: def test(self): return __mangled >>> MangledGlobal().test() 23

In this example I declared a global variable called _MangledGlobal__mangled. Then I accessed the variable inside the context of a class named MangledGlobal. Because of name mangling I was able to reference the _MangledGlobal__mangled global variable as just __mangled inside the test() method on the class.

The Python interpreter automatically expanded the name __mangled to _MangledGlobal__mangled because it begins with two underscore characters. This demonstrated that name mangling isn’t tied to class attributes specifically. It applies to any name starting with two underscore characters used in a class context.

Now this was a lot of stuff to absorb.

To be honest with you I didn’t write these examples and explanations down off the top of my head. It took me some research and editing to do it. I’ve been using Python for years but rules and special cases like that aren’t constantly on my mind.

Sometimes the most important skills for a programmer are “pattern recognition” and knowing where to look things up. If you feel a little overwhelmed at this point, don’t worry. Take your time and play with some of the examples in this article.

Make these concepts sink in enough so that you’ll recognize the general idea of name mangling and some of the other behaviors I showed you. If you encounter them “in the wild” one day, you’ll know what to look for in the documentation.

⏰ Sidebar: What’s a “dunder” in Python?

I’ve you’ve heard some experienced Pythonistas talk about Python or watched a few conference talks you may have heard the term dunder. If you’re wondering what that is, here’s your answer:

Double underscores are often referred to as “dunders” in the Python community. The reason is that double underscores appear quite often in Python code and to avoid fatiguing their jaw muscles Pythonistas often shorten “double underscore” to “dunder.”

For example, you’d pronounce __baz as “dunder baz”. Likewise __init__ would be pronounced as “dunder init”, even though one might think it should be “dunder init dunder.” But that’s just yet another quirk in the naming convention.

It’s like a secret handshake for Python developers

Categories: FLOSS Project Planets

NumFOCUS: NumFOCUS Awards Small Development Grants to Projects

Mon, 2017-05-22 17:52
This spring the NumFOCUS Board of Directors awarded targeted small development grants to applicants from or approved by our sponsored and affiliated projects. In the wake of a successful 2016 end-of-year fundraising drive, NumFOCUS wanted to direct the donated funds to our projects in a way that would have impact and visibility to donors and […]
Categories: FLOSS Project Planets

Curtis Miller: The End of the Honeymoon: Falling Out of Love with quantstrat

Mon, 2017-05-22 11:00
Introduction I spent good chunks of Friday, Saturday, and Sunday attempting to write another blog post on using R and the quantstrat package for backtesting, and all I have to show for my work is frustration. So I’ve started to fall out of love with quantstrat and am thinking of exploring Python backtesting libraries from…Read more The End of the Honeymoon: Falling Out of Love with quantstrat
Categories: FLOSS Project Planets