Feeds

The highlight of the month: our annual retreat.

Between February 21-24, Team Axelerant traveled to beautiful Goa, India from their homes all over the world to attend our annual retreat. We had almost 60 of us there on the beach.

Sometimes it seems that we have nearly infinite memory resources, especially compared to the tiny 48K RAM of yesteryear’s 8-bit computers. But today’s complex applications can soak up megabytes before you know it. While it would be great if developers planned their memory management for all applications, thinking through a memory management strategy is crucial for applications with especially RAM intensive features like image/video processing, massive databases, and machine learning.

How do you plan a memory management strategy? It’s very dependent on your application and its requirements, but a good start is to work with your operating system instead of against it. That’s where memory mapping comes in. mmap can make your application’s performance better while also improving its memory profile by letting you leverage the same virtual memory paging machinery that the OS itself relies on. Smart use of the memory mapping API (Qt, UNIX, Windows) allows you to transparently handle massive data sets, automatically paging them out of memory as needed – and it’s much better than you’re likely to manage with a roll-your-own memory management scheme.

Here’s a real-life use case of how we used mmap to optimize RAM use in QiTissue, a medical image application. This application loads, merges, manipulates, and displays highly detailed microscope images that are up to gigabytes in size. It needs to be efficient or risks running out of memory even on desktops loaded with RAM.

QiTissue highlighting tumor cell division on a digital microscopic image

 

The above image is stitched together from many individual microscope images, and the algorithm needs access to many large bitmaps to do that – a pretty memory-intensive process. Capturing a memory snapshot of the application in action shows that memory use grows as the application pulls in the images required for the stitching algorithm. The top purple line, Resident Set Size (RSS), is the amount of RAM that belongs to the application and that physically resides in memory. That curve reveals that memory use fluctuates but tops out over 6GB.

Memory use before mmap optimization

 

We used mmap extensively in our rewrite of QiTissue to help its overall memory profile. In this case, we decompress the microscope images into files and then memory map those files into memory. Note that using memory-mapped files won’t eliminate the RAM needed to load, manipulate, or display the images. However, it does allow the OS to do what it does best – intelligently manage memory so that our application operates effectively with the RAM that’s available.

 

Memory use after mmap optimization

The post-mmap optimization memory diagram looks similar, so what are the practical differences?

• Heap consumption drops. For our stitching algorithm, the heap size drops from around 500MB down to around 35MB, which is a fraction of its original size. The memory hasn’t disappeared; it’s still being used and counts against the application’s RSS total. However, because the memory used isn’t part of the heap, it doesn’t come out of the OS’s virtual memory paging file. That’s important because there’s a cap on the total amount of RAM that can be allocated. The size of the paging file is usually set to be around 1.5 to 2 times the physical RAM, and this size limit prevents the system from wasting all its time swapping memory in and out from disk. That means that even when using virtual memory, the total amount of memory a program can access is limited. By mmapping memory, you can intelligently exceed that cap if you need to.
• Less dirty memory. Dirty memory is memory that has been written to and as a result, no longer reflects the copy of the memory paged in from disk. If a memory block is dirty, the OS has to write it back out to disk where it’s paged out, and that write introduces a huge performance hit. Why does our dirty memory drop? The heap is actually a living data structure that manages all the news/deletes/mallocs/frees that your application makes. As normal C++ code lives and breathes, the C++ malloc libraries must maintain its linked lists of memory blocks by updating heap structures. Of course once touched, that memory must be flushed back to disk. Moving our big data out of the heap and into mmapped files prevents dirty flushes of those large memory structures, saving a lot of unnecessary CPU cycles.
• Better RSS profile. Moving the majority of application data out of the heap and into mmapped files also trims down the structures needed to maintain that data. An operating system’s raw memory paging, which mmap leverages, may be limited to 4096 byte blocks, but it’s pretty efficient in managing those blocks. That is reflected in the memory required when we convert over to a mmap-based architecture. Not only do we need a smaller amount of resident RAM (6.2GB vs 5.8GB), but also our overall RAM consumption profile peaks less and recovers faster when it does peak. Meaning that there is more memory left for all the other OS tasks and more RAM that can be used before paging memory back in from disk is required.

Best of all, incorporating mmap isn’t too difficult to do. Less memory, faster, and easy: what’s not to like? I’ve put together a small example on GitHub if you want to experiment around with your own mmap application: https://github.com/milianw/mmap_demo/.

The post Making the Most of your Memory with mmap appeared first on KDAB.

A cheat sheet for sets in Python. What is the 'set' datatype in Python? How is it used?

Jump to the cheat sheet
Key points

• Unlike lists and tuples, a set is an unordered collection of unique elements.
• A set can be created using the set() constructor. It takes an optional iterable as input.
• Sets with some elements can also be initialized using the curly braces (e.g., {1, 2, 3}).
• However, curly braces cannot be used to create an empty set. Using empty curly braces, i.e., {}, creates a dictionary datatype in Python.
• 'in' operator can be used to check the presence of an element in a set.
• Sets support the mathematical set operations like union, intersection, difference, symmetric difference, etc.
• Sets are mutable, we can add or remove elements from them.

Cheat sheetUseful resources and references

• Python documentation for sets

• cel-go

  The Go implementation of the Common Expression Language (CEL). CEL is a non-Turing complete language designed to be portable and fast. It is well suited to embedded applications expression evaluation with familiar syntax and features, protocol buffer support, and not needing the sandboxing needed for a runtime like JavaScript or Lua.

  (tags: scripting golang go cel languages coding configuration config embedded)

• XXH3

  ‘a cross-over inspired by many other great hash algorithms, which proves substantially faster than existing variants of xxHash, across basically all dimensions.’

  (tags: hashing algorithms xxhash xxh3 checksums performance)

Another update is now due from what we’ve been doing at the Foundation, and we’ve been busy!

As you may have seen, we’ve hired three excellent people over the past couple of months. Kristi Progri has joined us as Program Coordinator, Bartłomiej Piorski as a devops sysadmin, and Emmanuele Bassi as our GTK Core developer. I hope to announce another new hire soon, so watch this space…

There’s been quite a lot of discussion around the Google API access, and GNOME Online Accounts. The latest update is that I submitted the application to Google to get GOA verified, and we’ve got a couple of things we’re working through to get this sorted.

Events all round!

Although the new year’s conference season is just kicking off, it’s been a busy one for GNOME already. We were at FOSDEM in Brussels where we had a large booth, selling t-shirts, hoodies and of course, the famous GNOME socks. I held a meeting of the Advisory Board, and we had a great GNOME Beers event – kindly sponsored by Codethink.

We also had a very successful GTK Hackfest – moving us one step closer to GTK 4.0.

Coming up, we’ll have a GNOME booth at:

• SCALEx17 – Pasadena, California (7th – 10th March)
• LibrePlanet – Boston Massachusetts (23rd – 24th March)
• FOSS North – Gothenburg, Sweden (8th – 9th April)
• Linux Fest North West – Bellingham, Washington (26th – 28th April)

If you’re at any of these, please come along and say hi! We’re also planning out events for the rest of the year. If anyone has any particularly exciting conferences we may not have heard of, please let us know.

Discourse

It hasn’t yet been announced, but we’re trialling an instance of Discourse for the GTK and Engagement teams. It’s hopeful that this may replace mailman, but we’re being quite careful to make sure that email integration continues to work. Expect more information about this in the coming month. If you want to go have a look, the instance is available at discourse.gnome.org

tl;dr how to clean up your Python3 homebrew install

MetroSnek — snek on Metro M0 Express

When I first mentioned Snek a few months ago, Phillip Torrone from Adafruit pointed me at their Metro M0 board, which uses an Arduino-compatible layout but replaces the ATMega 328P with a SAMD21G18A. This chip is an ARM Cortex M0 part with 256kB of flash and 32kB of RAM. Such space!

Even though there is already a usable MicroPython port for this board, called CircuitPython, I figured it would be fun to get Snek running as well. The CircuitPython build nearly fills the chip, so the Circuit Python boards all include an off-chip flash part for storing applications. With Snek, there will be plenty of space inside the chip itself for source code, so one could build a cheaper/smaller version without the extra part.

UF2 Boot loader

I decided to leave the existing boot loader in place instead of replacing it with the AltOS version. This makes it easy to swap back to CircuitPython without needing any custom AltOS tools.

The Metro M0 Express boot loader is reached by pressing the reset button twice; it's pretty sweet in exposing a virtual storage device with a magic file, CURRENT.UF2, into which you write the ROM image. You write a UF2 formatted file to this name and the firmware extracts the data on the fly and updates the flash in the device. Very slick.

To make this work with AltOS, I had to adjust the start location of the operating system to 0x2000 and leave a bit of space at the end of ROM and RAM clear for the boot loader to use.

Porting AltOS

I already have an embedded operating system that works on Cortex M0 parts, AltOS, which I've been developing for nearly 10 years for use in rocketry and satellite applications. It's also what powers [ChaosKey])(http://altusmetrum.org/ChaosKey/).

Getting AltOS running on another Cortex M0 part is a simple matter of getting clocks running and writing drivers.

What I haven't really settled on is whether to leave this code as a part of AltOS, or to pull the necessary bits into the Snek repository and doing a bare-metal implementation.

I've set up the Snek distribution to make integrating it into another operating system simple; that's how the NuttX port works, for instance. It does make the build process more complicated as you have to build and install Snek, then build AltOS for the target device.

SAMD21 Clocks

Every SoC has a different way of configuring and wiring clocks within the system. Most that I've used have a complex clock-tree that you plug various configuration values into to generate clocks for the processor and peripherals.

The SAMD21 is simpler in offering a set of general-purpose clock controllers that can source a variety of clock signals and divide them by an integer. The processor uses clock controller 0; all of the other peripherals can be configured to use any clock controller you like.

The Metro M0 express and Feather M0 express have only a 32.768kHz crystal; they don't have a nice even-MHz crystal connected to the high-speed oscillator. As a result, to generate a '48MHz' clock for the processor and USB controller, I ended up multiplying the 32.768kHz frequency by 1464 using a PLL to generate a 47.972352MHz signal, which is about 0.06% low. Close enough for USB to work.

At first, I typo'd a register value leaving the PLL un-locked. The processor still ran fine, but when I looked at the clock with my oscilloscope, it was very ragged with a mean frequency around 30MHz. It took a few hours to track down the incorrect value, at which point the clock stabilized at about 48MHz.

SAMD21 USART

Next on the agenda was getting a USART to work; nothing terribly complicated there, aside from the clock problem mentioned above which generated a baud rate of around 6000 instead of 9600.

I like getting a USART working because it's usually (always?) easier than USB, plus demonstrates that clocking is working as expected. I can debug serial data with a simple logic analyzer. This time, the logic analyzer is how I discovered the clocking issue -- a bit time of 166µs does not equal 9600 baud.

SAMD21 USB

While I like having USB on-chip in the abstract, the concrete adventure of implementing USB for a new chip is always fraught with peril. In this case, the chip documentation was missing a couple of key details that I had to discover experimentally.

I'm still trying to come up with an abstraction for writing USB drivers for small systems; every one is different enough that I keep using copy&paste instead of building a driver core on top of hardware-specific primitives. In this case, the USB driver is 883 lines of code; the second shortest in AltOS with the ATMega32u4 driver being slightly smaller.

TODO

The only hardware that works today is one USARTs and USB. I also go Snek compiled and running. Left to do:

• Digital GPIO controls. I've got basic GPIO functionality available in the underlying operating system, but it isn't exposed through Snek yet.

• Analog outputs. This will involve hooking timers to outputs so that we can PWM them.

• Analog inputs. That requires getting an ADC driver written and then hooking that into Snek.

• On-board source storage. I think the ATMega model of storing just one set of source code on the device and reading that at boot time is clean and simple, so I want to do the same here. I think it will be simpler to use the on-chip flash instead of the external flash part. That means reserving a specific chunk of that for source code.

• Figure out whether this code is part of AltOS, or part of Snek.

Links

• snek
• altos
• altos repo with samd21 branch containing this stuff

How did I not know about Settings Sync for keeping the configuration of of VisualStudio Code on different machines in sync?

Thanks @holtbt for the info!

#360 – MARCH 19, 2019
View in Browser »

How to Build a Python GUI Application With wxPython

In this step-by-step tutorial, you’ll learn how to create a cross-platform graphical user interface (GUI) using Python and the wxPython toolkit. A graphical user interface is an application that has buttons, windows, and lots of other widgets that the user can use to interact with your application.
REAL PYTHON

Simplify Your Python Developer Environment

How three tools (pyenv, pipx, pipenv) make for smooth, isolated, reproducible Python developer and production environments.
MASON EGGER • Shared by Mason Egger

Automated Code Reviews for Python

Take the hassle out of code reviews—Codacy flags errors so you can fix them quickly. Address security concerns, code duplication, code complexity and drops in coverage, directly from your workflow. Click here to get started →
CODACY sponsor

When C Extensions Crash: Easier Debugging for Your Tests

Learn how to prepare for crashes in advance, so when they do occur you can quickly figure out which part of the codebase caused them: The standard library’s faulthandler, verbose test runs, package listing, and catchsegv on Linux.
ITAMAR TURNER-TRAURING

Why Operators Are Useful

Latest blog post from Guido, related to the recent discussion about Python getting + and - operators for merging dictionaries (PEP 584).
GUIDO VAN ROSSUM • Shared by Ricky White

Python Decorators 101

See step-by-step what Python decorators are and how you can use them to make your own code more Pythonic and expressive.
REAL PYTHON video

Tuple Ordering and Deep Comparisons in Python

Nice deep dive on comparison operators in Python.
TREY HUNNER • Shared by Ricky White

urllib CRLF Injection Vulnerability

The current Python 2.x and 3.x implementation of urllib does not encode the \r\n sequence in the query string, which allows an attacker to manipulate a HTTP header with the \r\n sequence in it, so the attacker can insert arbitrary content to the new line of the HTTP header.
PYTHON.ORG

Give Me Back My Monolith

“It feels like we’re starting to pass the peak of the hype cycle of microservices.” Interesting counterpoint to the “everything should be broken down into microservices” hypetrain. Not Python-specific, but worth a read nonetheless.
CRAIG KERSTIENS opinion

Discussions Guido Explains Why Python Uses 0-Based Indexing

Google Plus is shutting down soon, so here’s a final hurrah.
GOOGLE.COM

There Are Some Huge Speedups in Python 3.8. Will They Also Be Put Into the Older Versions?

REDDIT

Python Jobs Brought to you by Indeed Primesponsor

Apply today to see what job opportunities are waiting for you!

Machine Learning and Data Science Developer (Austin, TX)

Protection Engineering Consultants LLC

Lead Python Developer (Toronto, Canada)

Kognitiv

Senior Systems Engineer (Hamilton, Canada)

Preteckt

Python Software Engineer (Berlin, Germany)

Wooga

Computer Science Teacher (Pasadena, CA)

ArtCenter College of Design

Senior Python Engineer (New York, NY)

15Five

More Python Jobs >>>

Articles & Tutorials Understanding the Python Mock Object Library

In this tutorial, you’ll learn how to use the Python mock object library, unittest.mock, to create and use mock objects to improve your tests. Obstacles like complex logic and unpredictable dependencies make writing valuable tests difficult, but unittest.mock can help you overcome these obstacles.
REAL PYTHON

How I Translated a Mathematical Algorithm Into Code: TF-IDF to Python

Does your brain short-circuit when you see a mathematical algorithm? Don’t worry, you’re not alone. In this post you’ll see how the author worked her way through an algorithm, namely TF-IDF, and got it up and running in Python. Nice writeup!
SILKE HENDERICKX • Shared by Silke Henderickx

Take ActiveState’s ~7 Min Survey for a Chance to Win a Lego Star Wars TIE Fighter

We want to know your pain points building & programming open source languages. We’ve got 3 cool prizes too. Click to speed through this ~7min survey →
ACTIVESTATE sponsor

10 Python Image Manipulation Tools

Nice overview of Python libraries that provide an easy and intuitive way to transform images and make sense of the underlying data.
PARUL PANDEY

Tips and Tricks to Write LaTeX Papers in With Figures Generated in Python

Some nice tips and example code for writing scientific papers in LaTeX, with figures generated in Python.
VINCENT ETTER

Make Python Delete Your Unwanted Emails Periodically

How to use the Gmail API to create a Python script which will automatically search & delete the messages matching your query.
UDIT VASHISHT • Shared by Udit Vashisht

Python’s except Quirk

I don’t know how Alex comes up with these, but that’s a fun one :)
ALEX BECKER

How to Distribute a wxPython Application

You finished up a wonderful GUI application using wxPython. How do you share it with the world? Read Mike’s article to find out.
MIKE DRISCOLL

Django: An Unofficial Opinionated FAQ

KRISTIAN GLASS

Python Standard Library Gems: collections.Counter

IVAN SAGALAEV

How to Use Grouping Sets in Django

“How we cut a heavy admin dashboard response time in half with advanced SQL and some Django hackery.”
HAKI BENITA

Projects & Code OWASP/CheatSheetSeries: High Value Information on Specific Application Security Topics

GITHUB.COM/OWASP

orm: An Async ORM

GITHUB.COM/ENCODE

namedzip: Extends zip() and itertools.zip_longest() to Generate Named Tuples

GITHUB.COM/ERBERLIN

transistor: A Python Web Scraping Framework for Structured Web Pages

GITHUB.COM/BOMQUOTE

instaviz: Instant Visualization of Python AST and Code Objects

ANTHONY SHAW

Events PyData Bristol Meetup

March 21, 2019
MEETUP.COM

Python Northwest

March 21, 2019
PYNW.ORG.UK

PyLadies Dublin

March 21, 2019
PYLADIES.COM

IndyPy Automate Conf 2019

March 22 to March 23, 2019
INDYPY.ORG

PyCon SK 2019

March 22 to March 25, 2019
PYCON.SK

Inland Empire Pyladies (CA, USA)

March 25, 2019
MEETUP.COM

Happy Pythoning!
This was PyCoder’s Weekly Issue #360.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

In "Install fests: What to do about the deal with the devil," Richard Stallman issues both a caveat to free software novices who would like to transition to using free software exclusively via an install-fest, and a plea to install-fest organizers and volunteers not to make injurious ethical decisions for the people availing themselves of their help.

Stallman explains that, because of obstacles deliberately devised to thwart back engineering, not all computers can function properly with a completely free distro. And that a choice, therefore, often has to be made, between freedom and convenience, between installing a fully free distro that won't function as intended, and installing a nonfree distro that will. He argues that this choice should be made by the informed user alone, not silently by the install-fest volunteer.

Stallman appeals to install fests to forgo the "tacit deal with the devil" that suppresses the free software movement's message about freedom and justice, and to take advantage of the teachable moment, to introduce the user to the "moral dimension" of their computing choices. He suggests a number of things an install-fest could do (implement visual demarcations that help users understand when they're about to "forfeit their freedom," give technical advice regarding free software and free hardware, encourage users to lobby offending manufacturers) in order to "retain full moral authority when it talks about the imperative for freedom." Better the devil you know than the devil you don't, and, ultimately, better no devil at all.

In "Install fests: What to do about the deal with the devil," Richard Stallman issues both a caveat to free software novices who would like to transition to using free software exclusively via an install-fest, and a plea to install-fest organizers and volunteers not to make injurious ethical decisions for the people availing themselves of their help.

Stallman explains that, because of obstacles deliberately devised to thwart back engineering, not all computers can function properly with a completely free distro. And that a choice, therefore, often has to be made, between freedom and convenience, between installing a fully free distro that won't function as intended, and installing a nonfree distro that will. He argues that this choice should be made by the informed user alone, not silently by the install-fest volunteer.

Stallman appeals to install fests to forgo the "tacit deal with the devil" that suppresses the free software movement's message about freedom and justice, and to take advantage of the teachable moment, to introduce the user to the "moral dimension" of their computing choices. He suggests a number of things an install-fest could do (implement visual demarcations that help users understand when they're about to "forfeit their freedom," give technical advice regarding free software and free hardware, encourage users to lobby offending manufacturers) in order to "retain full moral authority when it talks about the imperative for freedom." Better the devil you know than the devil you don't, and, ultimately, better no devil at all.

Thanks for joining me on 2019-03-19 at Spark BCN Meetup 2019 Barcelona, Spain for Contributing to Spark 3.

The slides are at http://bit.ly/2HwlXFf.

Contributing to Apache Spark 3 from Holden Karau Comment bellow to join in the discussion :).Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback

Let’s say you finished up a wonderful GUI application using wxPython. How do you share it with the world? This is always the dilemma when you finish an amazing program. Fortunately, there are several ways you can share your code. If you want to share your code with other developers, than Github or a similar website is definitely a good way to do. I won’t be covering using Git or Mercurial here. Instead what you will learn here is how to turn your application into an executable.

By turning your code into an executable, you can allow a user to just download the binary and run it without requiring them to download Python, your source code and your dependencies. All of those things will be bundled up into the executable instead.

There are many tools you can use to generate an executable:

• py2exe
• py2app
• PyInstaller
• cx_Freeze
• bbfreeze
• Nuitka
• Briefcase

You will be using PyInstaller in this tutorial. The main benefit to using PyInstaller is that it can generate executables for Windows, Mac and Linux. Note that it does not support cross-compiling. What that means is that you cannot run PyInstaller on Linux to create a Windows executable. Instead, PyInstaller will only create an executable for the OS that it is ran on. In other words, if you run PyInstaller on Windows, it will create a Windows executable only.

Installing PyInstaller

Installing the PyInstaller package is nice and straightforward. All you need is pip.

Here is how you would install PyInstaller to your system Python:

pip install pyinstaller

You could also install PyInstaller to a virtual Python environment using Python’s venv module or the virtualenv package.

Generating an Executable

The nice thing about PyInstaller is that it is very easy to use out of the box. All you need to do is run the `pyinstaller` command followed by the path to the main file of the application that you want to convert to an executable.

Here is a non-working example:

pyinstaller path/to/main/script.py

If the PyInstaller application is not found, you may have to specify a full path to it. By default, PyInstaller installs to Python’s **Scripts** sub-folder, which is going to be in your system Python folder or in your virtual environment.

Let’s take one of the simple applications from my upcoming book and turn it into an executable. For example, you could use image_viewer_slideshow.py from chapter 3:

# image_viewer_slideshow.py   import glob import os import wx   class ImagePanel(wx.Panel):   def __init__(self, parent): super().__init__(parent) self.max_size = 240 self.photos = [] self.current_photo = 0 self.total_photos = 0 self.layout()   self.slideshow_timer = wx.Timer(self) self.Bind(wx.EVT_TIMER, self.on_next, self.slideshow_timer)   def layout(self): """ Layout the widgets on the panel """   self.main_sizer = wx.BoxSizer(wx.VERTICAL) btn_sizer = wx.BoxSizer(wx.HORIZONTAL)   img = wx.Image(self.max_size, self.max_size) self.image_ctrl = wx.StaticBitmap(self, wx.ID_ANY, wx.Bitmap(img)) self.main_sizer.Add(self.image_ctrl, 0, wx.ALL|wx.CENTER, 5) self.image_label = wx.StaticText(self, label="") self.main_sizer.Add(self.image_label, 0, wx.ALL|wx.CENTER, 5)   btn_data = [("Previous", btn_sizer, self.on_previous), ("Slide Show", btn_sizer, self.on_slideshow), ("Next", btn_sizer, self.on_next)] for data in btn_data: label, sizer, handler = data self.btn_builder(label, sizer, handler)   self.main_sizer.Add(btn_sizer, 0, wx.CENTER) self.SetSizer(self.main_sizer)   def btn_builder(self, label, sizer, handler): """ Builds a button, binds it to an event handler and adds it to a sizer """ btn = wx.Button(self, label=label) btn.Bind(wx.EVT_BUTTON, handler) sizer.Add(btn, 0, wx.ALL|wx.CENTER, 5)   def on_next(self, event): """ Loads the next picture in the directory """ if not self.photos: return   if self.current_photo == self.total_photos - 1: self.current_photo = 0 else: self.current_photo += 1 self.update_photo(self.photos[self.current_photo])   def on_previous(self, event): """ Displays the previous picture in the directory """ if not self.photos: return   if self.current_photo == 0: self.current_photo = self.total_photos - 1 else: self.current_photo -= 1 self.update_photo(self.photos[self.current_photo])   def on_slideshow(self, event): """ Starts and stops the slideshow """ btn = event.GetEventObject() label = btn.GetLabel() if label == "Slide Show": self.slideshow_timer.Start(3000) btn.SetLabel("Stop") else: self.slideshow_timer.Stop() btn.SetLabel("Slide Show")   def update_photo(self, image): """ Update the currently shown photo """ img = wx.Image(image, wx.BITMAP_TYPE_ANY) # scale the image, preserving the aspect ratio W = img.GetWidth() H = img.GetHeight() if W > H: NewW = self.max_size NewH = self.max_size * H / W else: NewH = self.max_size NewW = self.max_size * W / H img = img.Scale(NewW, NewH)   self.image_ctrl.SetBitmap(wx.Bitmap(img)) self.Refresh()   def reset(self): img = wx.Image(self.max_size, self.max_size) bmp = wx.Bitmap(img) self.image_ctrl.SetBitmap(bmp) self.current_photo = 0 self.photos = []     class MainFrame(wx.Frame):   def __init__(self): super().__init__(None, title='Image Viewer', size=(400, 400)) self.panel = ImagePanel(self) self.create_toolbar() self.Show()   def create_toolbar(self): """ Create a toolbar """ self.toolbar = self.CreateToolBar() self.toolbar.SetToolBitmapSize((16,16))   open_ico = wx.ArtProvider.GetBitmap( wx.ART_FILE_OPEN, wx.ART_TOOLBAR, (16,16)) openTool = self.toolbar.AddTool( wx.ID_ANY, "Open", open_ico, "Open an Image Directory") self.Bind(wx.EVT_MENU, self.on_open_directory, openTool)   self.toolbar.Realize()   def on_open_directory(self, event): """ Open a directory dialog """ with wx.DirDialog(self, "Choose a directory", style=wx.DD_DEFAULT_STYLE) as dlg:   if dlg.ShowModal() == wx.ID_OK: self.folderPath = dlg.GetPath()   photos = glob.glob(os.path.join(self.folderPath, '*.jpg')) self.panel.photos = photos if photos: self.panel.update_photo(photos[0]) self.panel.total_photos = len(photos) else: self.panel.reset()     if __name__ == '__main__': app = wx.App(redirect=False) frame = MainFrame() app.MainLoop()

If you wanted to turn it into an executable, you would run the following:

pyinstaller image_viewer_slideshow.py

Make sure that when you run this command, your current working directory is the one that contains the script you are converting to an executable. PyInstaller will be creating its output in whatever the current working directory is.

When you run this command, you should see something like this in your terminal:

PyInstaller will create two folders in the same folder as the script that you are converting called **dist** and **build**. The **dist** folder is where you will find your executable if PyInstaller completes successfully. There will be many other files in the **dist** folder besides your executable. These are files that are required for your executable to run.

Now let’s try running your newly created executable. When I ran my copy, I noticed that a terminal / console was appearing behind my application.

Image Viewer with Console in Background

This is normal as the default behavior of PyInstaller is to build your application as if it were a command-line application, not a GUI.

You will need to add the –noconsole flag to remove the console:

pyinstaller image_viewer_slideshow.py --noconsole

Now when you run the result, you should no longer see a console window appearing behind your application.

It can be complicated to distribute lots of files, so PyInstaller has another command that you can use to bundle everything up into a single executable. That command is `–onefile`. As an aside, a lot of the commands that you use with PyInstaller have shorter aliases. For example, there is a shorter alias for `–noconsole` that you can also use called: -w. Note the single dash in `-w`.

So let’s take that information and have PyInstaller create a single file executable with no console:

[python] pyinstaller image_viewer_slideshow.py --onefile -w

You should now have just one file in the dist folder.

The spec file

PyInstaller has the concept of specification files. They are kind of like a setup.py script, which is something that you use with Python’s distutils. These spec files tell PyInstaller how to build your executable. PyInstaller will generate one for you automatically with the same name as the passed in script, but with a .spec extension. So if you passed in image_viewer_slideshow.py, then you should see a image_viewer_slideshow.spec file after running PyInstaller. This spec file will be created in the same location as your application file.

Here is the contents of the spec file that was created from the last run of PyInstaller above:

# -*- mode: python -*-   block_cipher = None     a = Analysis(['image_viewer.py'], pathex=['C:\\Users\\mdriscoll\\Documents\\test'], binaries=[], datas=[], hiddenimports=[], hookspath=[], runtime_hooks=[], excludes=[], win_no_prefer_redirects=False, win_private_assemblies=False, cipher=block_cipher, noarchive=False) pyz = PYZ(a.pure, a.zipped_data, cipher=block_cipher) exe = EXE(pyz, a.scripts, a.binaries, a.zipfiles, a.datas, [], name='image_viewer', debug=False, bootloader_ignore_signals=False, strip=False, upx=True, runtime_tmpdir=None, console=False )

While PyInstaller worked fine with the image viewer example, you may find that it won’t work out of the box if you had other dependencies, such as NumPy or Pandas. If you run into issues with PyInstaller, it has very verbose logs that you can use to help you figure out the issue. One good location is the `build/cli/warn-cli.txt` file. You may also want to rebuild without the `-w` command so that you can see what is being printed to stdout in the console window.

There are also options for changing the log level during building that may help you uncover issues.

If none of those work, try Google or go to PyInstaller’s support page and get help there.

Creating Executables for Mac

While the same commands should work on Mac OSX as it does on Windows, I found that I needed to run the following command to generate a working executable:

pyinstaller image_viewer_slideshow.py --windowed

The output that PyInstaller generates will be slightly different and the result is an application file.

Another popular option for generating applications on Mac is a Python package called py2app.

Creating Executables for Linux

For Linux, it is usually recommended that you build the executable with an old version of glibc because the newer glibc versions are backwards compatible. By building with an old version of Linux, you can usually target a wider variety of Linux versions. But your mileage may vary.

After the files are generated, you can just tar them up into a gzipped tarball (.tax.gz). You could even using the archiving application you created in this book to do that for you, if you wanted.

An alternative would be to learn how to create a .deb or related file that most Linux versions can install.

Learning More About PyInstaller

This article is not meant to be an in-depth guide to PyInstaller. It will likely change much faster than wxPython, so it is recommended that you read the documentation for PyInstaller instead. It will always be the most up-to-date location to get the information you need on the project.

What About Installers?

Windows users know that most of the time you have an installer application that you can run to install your application on your computer and put some shortcuts here and there. There are several useful free programs that you can use to create a Windows Installer as well as some paid ones

Here are the two freeware applications I see mentioned the most:

• NSIS
• Inno Setup

I have used Inno Setup to create a Windows installer on several occasions. It is easy to use and requires only a little reading of its documentation to get it working. I haven’t used NSIS before, but I suspect it is quite easy to use as well.

Let’s use Inno Setup as an example and see how to generate an installer with it.

Creating an Installer with Inno Setup

Inno Setup is a nice freeware application that you can use to create professional looking installer programs. It works on most versions of Windows. I personally have used it for quite a few years. While Inno Setup is not open source, it is still a really nice program. You will need to download and install it from there website.

Once installed, you can use this tool to create an installer for the executable you created earlier in this chapter.

To get started, just run Inno Setup and you should see the following:

Inno Setup’s Startup Page

While Inno Setup defaults to opening an existing file, what you want to do is choose the second option from the top: “Create a new script file using the Script Wizard”. Then press **OK**.

You should now see the first page of the Inno Setup Script Wizard. Just hit **Next** here since there’s nothing else you can really do.

Now you should see something like this:

Inno Setup Script Wizard Application Information Page

This is where you enter your applications name, its version information, the publisher’s name and the application’s website. I pre-filled it with some examples, but you can enter whatever you want to here.

Go ahead and press Next and you should see page 3:

Inno Setup Script Wizard Application Folder Page

This page of the wizard is where you can set the application’s install directory. On Windows, most applications install to **Program Files**, which is also the default here. This is also where you set the folder name for your application. This is the name of the folder that will appear in Program Files. Alternatively, you can check the box at the bottom that indicates that your application doesn’t need a folder at all.

Let’s go to the next page:

Inno Setup Script Wizard Application Files Page

Here is where you will choose the main executable file. In this case, you want to choose the executable you created with PyInstaller. If you didn’t create the executable using the –onefile flag, then you can add the other files using the Add file(s)… button. If your application requires any other special files, like a SQLite database file or images, this is also where you would want to add them.

By default, this page will allow the user to run your application when the installer finishes. A lot of installers do this, so it’s actually expected by most users.

Let’s continue:

Inno Setup Script Wizard Application Shortcuts Page

This is the Application Shortcuts page and it allows you to manage what shortcuts are created for your application and where they should go. The options are pretty self-explanatory. I usually just use the defaults, but you are welcome to change them however you see fit.

Let’s find out what’s on the documentation page:

Inno Setup Script Wizard Application Documentation Page

The Documentation Page of the wizard is where you can add your application’s license file. For example, if you were putting out an open source application, you can add the GPL or MIT or whatever license file you need there. If this were a commercial application, this is where you would add your End-Users License Agreement (EULA) file.

Let’s see what’s next:

Inno Setup Script Wizard Setup Languages Page

Here you can set up which setup languages should be included. Inno Setup supports quite a few languages, with English as the default choice.

Now let’s find out what compiler settings are:

Inno Setup Script Wizard Compiler Settings Page

The Compiler Settings page let’s you name the output setup file, which defaults to simply **setup**. You can set the output folder here, add a custom setup file icon and even add password protection to the setup file. I usually just leave the defaults alone, but this is an opportunity to add some branding to the setup if you have a nice icon file handy.

The next page is for the preprocessor:

Inno Setup Script Wizard Preprocessor Page

The preprocessor is primarily for catching typos in the Inno Setup script file. It basically adds some helpful options at compile time to your Inno Setup script.

Check out the documentation for full details.

Click Next and you should see the last page of the wizard:

Inno Setup Script Wizard End Page

Click Finish and Inno Setup will generate an Inno Setup Script (.iss) file. When it is finished, it will ask you if you would like to compile the file.

Go ahead and accept that dialog and you should see the following:

Inno Setup Script

This is the Inno Setup Script editor with your newly generated script pre-loaded into it. The top half is the script that was generated and the bottom half shows the compiler’s output. In this screenshot, it shows that the setup file was generated successfully but it also displays a warning that you might want to rename the setup file.

At this point, you should have a working installer executable that will install your program and any files it depends on to the right locations. It will also create shortcuts in the Windows Start menu and whichever other locations you specified in the wizard.

The script file itself can be edited. It is just a text file and the syntax is well documented on Inno Setup’s website.

Code Signing

Windows and Mac OSX prefer that applications are signed by a corporation or the developer. Otherwise you will see a warning that you are using an unsigned piece of code or software. The reason this matters is that it protects your application from being modified by someone else. You can think of code signing as a kind of embedded MD5 hash in your application. A signed application can be traced back to whomever signed it, which makes it more trust-worthy.

If you want to sign code on Mac OSX, you can use XCode

Windows has several options for signing their code. Here is a URL for getting your application certified for Windows

You can also purchase a certificate from various companies that specialize in code signing, such as digicert.

There is also the concept of self-signed certificates, but that is not for production or for end users. You would only self-sign for internal testing, proof-of-concept, etc. You can look up how to do that on your own.

Wrapping Up

You have now learned how to generate executables using PyInstaller on Windows, Mac and Linux. The command to generate the executable is the same across all platforms. While you cannot create a Windows executable by running PyInstaller on Linux, it is still quite useful for creating executable for the target operating system.

You also learned how to use Inno Setup to create an installer for Windows. You can now use these skills to create executables for your own applications or for some of the other applications that you created in this book!

Further Reading

• A bbfreeze Tutorial – Build a Binary Series!
• Packaging wxPyMail for Distribution

The Board of the Django Software Foundation is pleased to announce that the 2018 Malcolm Tredinnick Memorial Prize has been awarded to Kojo Idrissa.

Kojo has been active in the Django community since at least 2015, if not earlier. He's been a DjangoCon US organizer since 2016, former DEFNA board member, and current DEFNA North American Ambassador.

Kojo has hosted an orientation for first-time DjangoCon US attendees for the last several years, which could not be a better example of Malcolm's friendly spirit to new users.

Ken Whitesell, who nominated Kojo, also noted many Kojo's other contributions:

Kojo is a very active member of the weekly CodeNewbie chats. Hosts the DjangoCon new-user orientation session. Very visible presence at DjangoCon, always seems to be focused on ensuring first time attendees have the best possible experience.

The other nominees this year were:

• Anna Makarudze
• Daniel Joey Darko
• Humphrey Butau
• Jani Tiainen
• Jeff Triplett
• Kenneth Love
• Simon Charette

Every year we receive many nominations and it's always hard to pick the winner. In fact, some have been nominated in multiple years. Malcolm would be very proud of the legacy he has fostered in our community!

Congratulations Kojo on the well deserved honor!

In 2018 we updated our list of free GNU/Linux distributions to add a "Historical" section. We retired BLAG Linux and GNU at that time, as it was no longer maintained. We are sad to announce today that Musix will also being moving to the Historical section, as it is likewise no longer maintained. Founded in 2004, Musix was on the list of free GNU/Linux distributions for over a decade. The list helps users to find operating systems that come with only free software and documentation, and that do not promote any nonfree software. Being added to the list means that a distribution has gone through a rigorous screening process, and is dedicated to diligently fixing any freedom issues that may arise.

Musix was maintained by a sole developer, Marcos Guglielmetti, as a volunteer effort, a truly impressive accomplishment. Maintaining a distribution is a difficult task. Dealing with technical and security issues across an entire system, as well as upholding the ethical standards required for inclusion on our list, takes a great deal of effort.

While it is sad that Musix will now reside in our Historical section, we can all still be thankful for the maintainer's work over the years, and for the fact that there are still many endorsed distributions available. Users of Musix should consider switching to another distro on our list to ensure that the security and freedom of their system is up to date.

In 2018 we updated our list of free GNU/Linux distributions to add a "Historical" section. We retired BLAG Linux and GNU at that time, as it was no longer maintained. We are sad to announce today that Musix will also being moving to the Historical section, as it is likewise no longer maintained. Founded in 2004, Musix was on the list of free GNU/Linux distributions for over a decade. The list helps users to find operating systems that come with only free software and documentation, and that do not promote any nonfree software. Being added to the list means that a distribution has gone through a rigorous screening process, and is dedicated to diligently fixing any freedom issues that may arise.

Musix was maintained by a sole developer, Marcos Guglielmetti, as a volunteer effort, a truly impressive accomplishment. Maintaining a distribution is a difficult task. Dealing with technical and security issues across an entire system, as well as upholding the ethical standards required for inclusion on our list, takes a great deal of effort.

While it is sad that Musix will now reside in our Historical section, we can all still be thankful for the maintainer's work over the years, and for the fact that there are still many endorsed distributions available. Users of Musix should consider switching to another distro on our list to ensure that the security and freedom of their system is up to date.

The 2019 incarnation of foss-north is less than a month away. This year we’re extending the conference in two directions: a training day and a community day. This time, I wanted to write about the training day.

The training day, April 10, is an additional day for those who want to extend
the conference with a day of dedicated training. I’m very happy to have two experienced and well known trainers on side: Michael Kerrisk and Chris Simmonds. Both has years of training experience.

Michael will teach about the details in dynamic linking. The topic may seem trivial, but when you start scratching the surface, there are a lot of details to discover such as how to handle version compatibility, how symbol resolution really works, and so on. You can read more about the Building and Using Shared Libraries on Linux training here.

Chris will run a getting started with embedded Linux training. Using BeagleBone Black devices the participants will learn how to build linux for the target, how to get devices such as GPIO and i2c working and more. You can read more about the Fast Track to Embedded Linux training here.

The best part of enrolling for training at foss-north is that you also get full access to the two conference days, and that you help fund the conference itself. If you are interested, check out the tickets page.

In this course on Python decorators, you’ll learn what they are and how to create and use them. Decorators provide a simple syntax for calling higher-order functions in Python. By definition, a decorator is a function that takes another function and extends the behavior of the latter function without explicitly modifying it.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Richard Stallman décrira les buts et la philosophie du mouvement des logiciels libres et évoquera les enjeux majeurs du monde numérique d'aujourd'hui. Il abordera les Droits de l'Homme des utilisateurs d'un logiciel : liberté de contrôler ce qu'il fait pour eux, égalité dans la communauté des développeurs et des utilisateurs, fraternité entre eux. Richard Stallman traitera enfin la question des modèles économiques pour l'utilisation et le développement des logiciels libres.

Ce discours de Richard Stallman ne sera pas technique, l'entrée sera libre, et tout le monde est invité à assister.

Lieu: Fablac, Maison des Assocations, 2 place de l'église, 74200 Anthy-sur-Léman, France

Veuillez complèter notre formulaire de contact, pour que nous puissions vous annoncer de futurs événements dans la région de Anthy-sur-Léman.

Drupal 8.6 became one of the most interesting releases in Drupal 8’s history. It brought us the oEmbed feature, the Media Library, the Workspaces module, and more. But it’s time to move forward, and in May 2019 we expect Drupal 8.7. Its “alpha” version has just been released. Although an alpha version is not a final one, we will gladly take a look at it and discuss what to expect in Drupal 8.7.

Drupal 8.7: the alpha version

Drupal 8.7.0-alpha1 has come out on March 14, 2019. Alpha versions are far from being ready for production sites. They are just preliminary releases that allow developers to do a good testing, receive feedback, make final preparations, and fix bugs.