FLOSS Project Planets

Learn PyQt: Embedding PyQtGraph (or any other custom PyQt5 widgets) from Qt Designer

Planet Python - Tue, 2019-08-20 15:48

Qt Designer is a great tool for designing PyQt5 GUIs, allowing you to use the entire range of Qt5 widgets and layouts to construct your apps. As your applications get more complex however you may find yourself creating custom widgets, or using PyQt5 libraries such as PyQtGraph, who's widgets are not available within Designer.

Helpfully, Qt Designer supports a mechanism for using placeholder widgets to represent your custom or external widgets in your design. This tutorial will walk you through the process of using placeholders to include a PyQtGraph plot in your app from within Qt Designer.

Promoting Widgets

The principle of using placeholders in Qt Designer is quite straightforward — 

  1. Create a UI as normal in Qt Designer.
  2. Add a placeholder widget to represent the custom widget you're adding.
  3. Tell Qt to replace your placeholder with your actual widget when building the UI.

In Qt this final step is referred to as promoting (as in promoting a base class).

If the custom widget you are adding is a subclass of an existing Qt widget, you may want to use the base class as your placeholder to promote from. For example, if you have a custom MyAwesomeButton button widget subclassed from QPushButton use QPushButton as the placeholder and promote it to MyAwesomeButton. This gives you access to the base class properties, events and actions from within Qt Designer.

If you don't have an obvious base class to use, then you can use QWidget, the common base class of all Qt widgets.

PyQtGraph

Data science is one of the post popular uses of Python, and building dashboards and analysis tools is a common use case of PyQt5. For all of these being able to add plots to your UI is very useful — and being able to do this from Qt Designer even more so.

There are a number of plotting libraries available in Python, with matplotlib being the most popular and offering some basic support for PyQt5. PyQtGraph is an popular alternative which uses Qt's native QGraphicsScene to provide fast zooming, scaling, drag-drop behaviour that feels a natural part of your application.

Whether you're using PyQtGraph or maplotlib for your plotting needs, the plot canvas widgets are not available from within Qt Designer. In this tutorial I'll walk you through the process of using these custom widgets in your apps.

If you don't have PyQtGraph installed already, you can install it using:

pip install pyqtgraph

The instructions below aren't specific to PyQtGraph, and you can use the same process to add matplotlib or any other custom widgets to your app.

Qt Designer

We will be using Qt Designer to create a simple UI design, and adding a placeholder for our PyQtGraph widget. First open Qt Designer and create a new QMainWindow as normal.

Qt Creator — Select MainWindow for widget type

We next need to add the placeholder widget. As there is no suitable baseclass for the PyQtGraph plot widget, we'll use the basic QWidget as our placeholder. Select the Widget from the left sidebar and place it in the centre of your window.

Give the widget a name, "graphWidget" will do. This is just a tag to reference the element in code.

Add a widget to the window. Name the widget as "graphWidget"

Right click on the widget and select Promote to from the widget's context menu.

Promoting a QWidget indicates that it should be replaced with the specified subclass, in our case the PyQtGraph plot widget.

A promoted widget can be reverted back to its base class by right-clicking and choosing Demote to from the widget's context menu.

Right click to show the promotion menu

You will be presented with a dialog to specify the custom widget class the placeholder widget will become.

The header file is the name of the Python module used to import the class, which is pyqtgraph. Specify PlotWidget as the class name of the widget to replace it with.

Promote the widget by specifying the class name as PlotWidget and header file as pyqtgraph.

The name you use for the file doesn't matter, but it's usually a good idea to name it after the class you're going to create with it.

Voila! The widget is now promoted to a canvas to plot. But you won't be able to see any changes within Qt Designer. Save the window as mainwindow.ui in the same directory as your PyQt app.

For a complete guide to using Qt Designer .ui files from Python check out First steps with Qt Creator.

Loading the .ui file

We now have the mainwindow.ui file containing our UI definition. We can load this from Python to show the window and our custom widget.

Let's start from a basic app template.

python from PyQt5 import QtWidgets, uic import pyqtgraph as pg import sys class MainWindow(QtWidgets.QMainWindow): def __init__(self, *args, **kwargs): super(MainWindow, self).__init__(*args, **kwargs) #Load the UI Page uic.loadUi('mainwindow.ui', self) def main(): app = QtWidgets.QApplication(sys.argv) main = MainWindow() main.show() sys.exit(app.exec_()) if __name__ == '__main__': main()

Save the code above in the same folder as your mainwindow.ui file, and run it as normal.

python3 my_app.py Your graph is now embedded

You should see a window with your widget transformed into a PyQtGraph plotting widget.

Let's now create a function to make a simple plot of x and y data.

python from PyQt5 import QtWidgets, uic from pyqtgraph import PlotWidget, plot import pyqtgraph as pg import sys # We need sys so that we can pass argv to QApplication import os class MainWindow(QtWidgets.QMainWindow): def __init__(self, *args, **kwargs): super(MainWindow, self).__init__(*args, **kwargs) #Load the UI Page uic.loadUi('mainwindow.ui', self) self.plot([1,2,3,4,5,6,7,8,9,10], [30,32,34,32,33,31,29,32,35,45]) def plot(self, hour, temperature): self.graphWidget.plot(hour, temperature) def main(): app = QtWidgets.QApplication(sys.argv) main = MainWindow() main.show() sys.exit(app.exec_()) if __name__ == '__main__': main()

So we added the plot() method which accepts two arrays, temp Temperature and hour Hour, then plots the data using the graph widget .plot() method.

Run the code, you should see the following.

The custom PyQtGraph widget showing dummy data.

That's it! You have just embedded your first plot with PyQtGraph.

The default PyQtGraph plot isn't very pretty, however can play around with the .plot() call to change the data shown.

We'll cover more complex PyQtGraph plots and plot customization, including line colours, styles and alternative types of plots in an upcoming tutorial.

Categories: FLOSS Project Planets

Mediacurrent: Open Waters Podcast Ep. 3: Improving Drupal's Admin UI With Cristina Chumillas

Planet Drupal - Tue, 2019-08-20 15:36

Welcome to Mediacurrent’s Open Waters, a podcast about open source solutions. In this episode, we catch up with Cristina Chumillas. Cristina comes from the design world and is passionate about front-end development. She works at Lullabot (though when we recorded this, she worked at Ymbra) and has been involved in the Drupal community for years, contributing with code, design, and organizing events. Her contributions to Drupal Core are mainly focused on front-end, design and UX. Nowadays, she's a co-organizer of the Drupal Admin UI & JS Modernization Initiative and a Drupal core UX maintainer.



Audio Download Link

Project Pick

 Claro

Interview with Cristina Chumillas
  1. Tell us about yourself: What is your role, who do you work for, and where are you from?
  2. You are a busy woman, what events have you recently attended and/or are scheduled to attend in the near future?
  3. Which Drupal core initiatives are you currently contributing to?
  4. How does a better admin theme UI help site owners?  
  5. What are the main goals?
  6. Is this initiative sponsored by anyone? 
  7. Who is the target for the initiative? 
  8. How is the initiative organized? 
  9. What improvements will it bring in a short/mid/long term?
  10. How can people get involved in helping with these initiatives?
Quick-takes
  •  Currently contributing to The Out of the Box initiative for a while, together with podcast co-host Mario
  • 3 reasons why Drupal needs a better admin theme UI: Content Productivity, savings, less frustration
  • Main goals: We have 2 separate paths: the super-fancy JS app that will land in an undefined point in the future and Claro as the new realistic & releasable short term work that will introduce improvements on each release.
  • Why focus on admin UI?  We’re focusing on the content author's experience because that’s one of the main pain points mentioned in an early survey we did last year.)
  • How is the initiative organized? JS, UX&User studies, New design system (UI), Claro (new theme)
  • What improvements will it bring in a short/mid/long term? Short: New theme/UI, Mid: editor role with specific features, autosave, Long: JS app. 

That’s it for today’s show, thanks for joining us!  Looking for more useful tips, technical takeaways, and creative insights? Visit mediacurrent.com/podcast for more episodes and to subscribe to our newsletter.

Categories: FLOSS Project Planets

PyCoder’s Weekly: Issue #382 (Aug. 20, 2019)

Planet Python - Tue, 2019-08-20 15:30

#382 – AUGUST 20, 2019
View in Browser »

An Effective Python Environment: Making Yourself at Home

This guide will walk you through the decisions you need to make when customizing your development environment for working with Python: shells, terminal emulators, version management (pyenv), virtual environments & pipenv, package management, and more.
REAL PYTHON

Python Static Analysis at Scale: An Instagram Story

Instagram’s back-end is a massive Django app with several million lines of code. This post is about how they’ve used linting and automated refactoring to help manage the scale of their Python codebase.
BENJAMIN WOODRUFF (INSTAGRAM)

Automated Python Code Reviews, Directly From Your Git Workflow

Take the hassle out of code reviews - Codacy flags errors so you can fix them quickly. Address security concerns, duplication, complexity, drops in coverage, and style violations before you merge. Integrates seamlessly with GitHub, Bitbucket, and GitLab →
CODACY sponsor

Writing Custom Profilers for Python

In this article you’ll learn how to write custom profilers, and in particular profilers that will help you pinpoint the places in your code where it just sits there waiting.
ITAMAR TURNER-TRAURING

Python Web Frameworks Overview List

Django, Flask, Pyramid, and many, many others—get an overview of what’s available so you can choose the right Python framework for your web development project.
STXNEXT.COM

Designing Continuous Build Systems: Handling Webhooks With Sanic

How to use Python and Sanic to handle webhook events from GitHub in continuous build systems.
CHRISTIAN MEDINA • Shared by Cristian Medina

PyCon US 2020 Website is Live

PyCon 2020 takes place April 15–23, 2020 in Pittsburgh, PA.
PYCON.ORG

Discussions PyPI Close to 200,000 Packages

RAYMOND HETTINGER

Reformatting Python Code Using Black Without Messing Up git blame

TWITTER.COM/LLANGA

Why Have I Heard the Phrase “Pure Python” but Not “Pure Java”, “Pure C”, etc?

REDDIT

Python Jobs Senior Backend Software Engineer (Remote)

Close

Senior Python Developer (Austin, TX)

InQuest

Backend and DataScience Engineers (London, Relocation & Visa Possible)

Citymapper Ltd

Software Engineering Lead, Python (Houston, TX)

SimpleLegal

Software Engineer (Multiple US Locations)

Invitae

Python Software Engineer (Munich Germany)

Stylight GmbH

Senior Software Developer (Edmonton, AB)

Levven Electronics Ltd.

Lead Data Scientist (Buffalo, NY)

Utilant LLC

More Python Jobs >>>

Articles & Tutorials Data Cleaning Using Python & Pandas

Learn how to do simplify your data preprocessing work using the Pyjanitor package. More specifically, you’ll learn how to: Add a column to a Pandas dataframe, remove missing values, remove an empty column, and clean up column names.
ERIK MARSJA

Python Histogram Plotting: NumPy, Matplotlib, Pandas & Seaborn

Get equipped to make production-quality, presentation-ready Python histogram plots with a range of choices and features. It’s your one-stop shop for constructing and manipulating histograms with Python’s scientific stack.
REAL PYTHON video

Slack <3 Python

Slack Python SDK v2 is now available to help you build Slack apps faster with less complexity. Built for Python 3, you can now use new features (like type hints and return types) to help you accomplish more while coding less. Looking for a tutorial or migration guide? It all lives here →
SLACK sponsor

Python for LEGO EV3

“You can now use your EV3 Brick to unleash the power of Python programming using MicroPython. Simply install the EV3 MicroPython image onto any micro SD card and boot up your EV3 Brick from it to start programming straight away.”
LEGO.COM • Shared by Python Bytes FM

How to Make a Discord Bot in Python

Learn how to make a Discord bot in Python and interact with several APIs. See how to handle events, accept commands, validate and verify input, and all the basics that can help you create useful and exciting automations!
REAL PYTHON

Bridging Node.js and Python With PyNode to Predict Home Prices

“In this article I demonstrate a novel Node.js package named PyNode used to invoke Python code within a Node.js application and, more importantly, receive Python return types in the calling Node.js application.”
ADAM MCQUISTAN

Python Now Supported in Azure Functions

Python support for Azure Functions is now generally available and ready to host your production workloads across data science and machine learning, automated resource management, and more.
MICROSOFT.COM

150+ Business Data Science Application Ideas in Python

A curated list of practical business machine learning (BML) and business data science (BDS) applications for Accounting, Customer, Employee, Legal, Management and Operations.
GITHUB.COM/FIRMAI

Super Easy Python CLIs With Click

How to give your Python scripts a full-featured Command-Line Interface (CLI) using the click library.
RICHARD BARELLA

Simple Scene Boundary/Shot Transition Detection With OpenCV

Learn how to implement a simple scene boundary and shot transition detector with OpenCV and Python.
ADRIAN ROSEBROCK

Software Defined Network and the OpenFlow Protocol

Explains a simple implementation of a switching hub using software defined networking with Python.
REBECA SARAI

Python 3 at Mozilla

How far along has Mozilla come in the Python 3 migration?
ANDREW HALBERSTADT • Shared by Python Bytes FM

The nonlocal Statement in Python

ABHILASH RAJ • Shared by Abhilash Raj

What Are Some Open-Source Python Projects That Are Really Well Written?

REDDIT

Scala’s Case Class in Python With Case Matching

BRANKO BLAGOJEVIC

The Future of Qt for Python

QT.IO

Projects & Code dry-python: Set of Libraries for Pluggable Business Logic Components

GITHUB.COM/DRY-PYTHON

pytest-mypy: mypy Static Type Checker Plugin for Pytest

PYPI.ORG

flake8-mypy: A Plugin for Flake8 Integrating mypy

PYPI.ORG

homer: Make Your Text More Clear, Simple and Useful for the Reader

GITHUB.COM/WYOUNAS

starlark: Python-Like Configuration Language

GITHUB.COM/BAZELBUILD

sneklang: Experimental Restricted Subset of Python for Safe Evaluation

GITHUB.COM/READEVALPRINT

opendrop: Apple AirDrop Implementation Written in Python

GITHUB.COM/SEEMOO-LAB

strictyaml: Type-Safe YAML Parser and Validator

GITHUB.COM/CRDOCONNOR

memory-profiler: Monitor the Memory Usage of a Python Program

PYPI.ORG • Shared by Mike Driscoll

Events Kiwi PyCon X

August 23 to August 26, 2019
PYTHON.NZ

IndyPy Web Conf 2019

August 23 to August 24, 2019
INDYPY.ORG

Inland Empire Pyladies (CA, USA)

August 26, 2019
MEETUP.COM

Python Sheffield

August 27, 2019
GOOGLE.COM

PyCon Latam 2019

August 29 to September 1, 2019
PYLATAM.ORG

Happy Pythoning!
This was PyCoder’s Weekly Issue #382.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

Categories: FLOSS Project Planets

Shubham (shubham)

Planet KDE - Tue, 2019-08-20 14:46
Third month progressHello visitors!! I am here presenting you with my final month GSoC project report. I will be providing the links to my work at the end of the section.Final month of the work period was much more hectic and tiring than the first couple of months. I had been busy more than I had anticipated. Nonetheless, I had to write code which I enjoyed writing : ) . In the first half of this work period, I was focused on completing the left-over QDBus communication from the phase 2, which I did successfully. But as when I thought my task was all over, I was faced with some regression in the code, which I utilised my rest half a month to fix it. I will talk about it later in the post. Coming to the progress made during this period, I have done the following:Complete left-over QDBus communication from Phase 2:During the first 2 months of the work period, I have completed the rest of the QDBus communication from Helper to Application. This was pretty easy to do, just had to follow the same as done by the Application when it contacts the helper. Earlier, it was QDBus which was used from Application towards Helper and KAuth from Helper towards Application. Now it is QDBus both ways.
Tried kick starting the helper:
As I had said above in the intro, I was faced with some real difficulty during the second half of the work period. As soon as I finished up QDBus thing, a regression was caused (Which I should have noticed before, my bad), helper was no longer started by the main application. I spent rest of the days brain-storming the issue but due to shortage of time, could not fix it. I plan to try fixing it in the next few days before GSoC ends(26th August), if I successfully do that, I will update the status here as well .

Things yet to work:
There are couple of things which are yet to work before we reach complete independence from KAuth:
1. Helper as a standalone application not able to start from the main application
2. Scanning issue: This is an issue in which after successfully authenticating, KDE Partition Manager is stuck on the scanning window. I believe this issue is directly related to the one mentioned above, but can't say for sure.

The above mentioned issues are obstacles in between achieving a KAuth free KDE Partition Manager.

Plans after GSoC:

I plan to complete the left over parts and issues after the GSoC ends. I look forward to contributing to KPM and KPMCore in future as well.


Links to my patches:1. QDBus communication from Helper towards Application:Link to cgit repository: 
Categories: FLOSS Project Planets

Fuse Interactive: What does Drupal 7 End of Life mean for your business?

Planet Drupal - Tue, 2019-08-20 14:08
What does Drupal 7 End of Life mean for your business? It's been a great run, but it will soon be time to say goodbye to an old friend. Over the last 8+ years, Drupal 7 has served our clients well. During that time we're thankful to have worked on 100+ Drupal 7 websites for some great organizations from non-profits, to telecoms. While we have been building all our projects on Drupal 8 for the last couple of years, Drupal 7 has continued to be a stable and effective business tool for many of our clients. In an announcement by Dries Buytaert at Drupal Europe (September 2018), Drupal 7 (and 8) will reach End of Life in November 2021 while Drupal 9 is scheduled to be released in 2020. In this post we hope to answer some of the questions you may have as Drupal 7 or 8 site owners / managers regarding the implications of this End of Life date. Greg Gillingham Tue, 08/20/2019 - 11:08
Categories: FLOSS Project Planets

FSF Events: Coding workshop (Ballarat, VIC, Australia)

GNU Planet! - Tue, 2019-08-20 11:54

This workshop, presented by Sturm Software Engineering, will teach you how to code on a free software project, and guide you through the whole process.

On the Friday evening you'll meet the mentors and participants, go through some preliminaries and then head out for dinner with the group. Dinner not included.

You'll spend a full day on Saturday working with the mentors and participants, with the aim of making your first contribution. Lunch and snacks will be provided.

It's a lot to do in one day, so a mentor will be in touch with you a week before and again two weeks after the event to help you get set up and to overcome any hurdles.

This event is suitable for tertiary students, hobbyists and technology professionals. We won't be teaching programming as such, so basic coding experience is required. We'll aim to find you a project that suits your skills and experience.

Please provide your own laptop.

Places limited to 16 participants

Location: 136 Albert St, Ballarat, Central Highlands of Victoria 3350, Australia

See here for registration information.

Please fill out our contact form, so that we can contact you about future events in and around Ballarat.

Categories: FLOSS Project Planets

Real Python: Python Histogram Plotting: NumPy, Matplotlib, Pandas &amp; Seaborn

Planet Python - Tue, 2019-08-20 10:00

In this course, you’ll be equipped to make production-quality, presentation-ready Python histogram plots with a range of choices and features.

If you have introductory to intermediate knowledge in Python and statistics, then you can use this article as a one-stop shop for building and plotting histograms in Python using libraries from its scientific stack, including NumPy, Matplotlib, Pandas, and Seaborn.

A histogram is a great tool for quickly assessing a probability distribution that is intuitively understood by almost any audience. Python offers a handful of different options for building and plotting histograms. Most people know a histogram by its graphical representation, which is similar to a bar graph:

This course will guide you through creating plots like the one above as well as more complex ones. Here’s what you’ll cover:

  • Building histograms in pure Python, without use of third party libraries
  • Constructing histograms with NumPy to summarize the underlying data
  • Plotting the resulting histogram with Matplotlib, Pandas, and Seaborn

Free Bonus: Short on time? Click here to get access to a free two-page Python histograms cheat sheet that summarizes the techniques explained in this tutorial.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Categories: FLOSS Project Planets

PSF GSoC students blogs: Weekly Check-in #13 : ( 16 Aug - 22 Aug )

Planet Python - Tue, 2019-08-20 09:10
What did you do this week?
  • I worked on getting Travis to push releases automatically to PyPI, adding a new `ROBOTSTXT_USER_AGENT` setting in Scrapy, and improvements to SitemapSpider.
What is coming up next?
  • I am going to work on the final PSF blog post in which I will focus on my experience of GSoC 2019 working with my awesome mentors, and Scrapy.
  • Next, I will write a final report for third evaluation of Google Summer of Code.
  • Next up, I will work on the changes suggested on my this week's work.
Did you get stuck anywhere?
  • Nothing Major.
Categories: FLOSS Project Planets

PyCon: PyCon 2020 Conference Site is here!

Planet Python - Tue, 2019-08-20 08:00


After 2 successful years in Cleveland, OH, PyCon 2020 and PyCon 2021 will be
moving to Pittsburgh, PA!

Head over to us.pycon.org/2020 to check out the look for PyCon 2020.

Our bold design includes the Roberto Clemente Bridge, also known as the Sixth Street Bridge, which spans the Allegheny River in downtown Pittsburgh. The Pittsburgh Steelmark, was originally created for United States Steel Corporation to promote the attributes of steel: yellow lightens your work; orange brightens your leisure; and blue widens your world. The PPG Building, is a complex in downtown Pittsburgh, consisting of six buildings within three city blocks and five and a half acres. Named for its anchor tenant, PPG Industries, who initiated the project for its headquarters, the buildings are all of matching glass design consisting of 19,750 pieces of glass. Also included in the design are a fun snake, terminal window, and hardware related items.
Sponsor OpportunitiesSponsors help keep PyCon affordable and accessible to the widest possible audience. Sponsors are what make this conference possible. From low ticket prices to financial aid, to video recording, the organizations who step forward to support PyCon, in turn, support the entire Python community. They make it possible for so many to attend, for so many to be presenters, and for the people at home to watch along.

As with any sponsorship, the benefits go both ways. Organizations have many options for sponsorship packages, and they all benefit from exposure to an ever growing audience of Python programmers, from those just getting started to 20 year veterans and every walk of life in between. If you're hiring, the Job Fair puts your organization within reach of a few thousand dedicated people who came to PyCon looking to sharpen their skills.

For more details on sponsorship opportunities go to the Sponsor Prospectus. If you are interested in becoming a PyCon sponsor go to the application form.

We look forward to sharing more news on the call for proposals, financial aid applications, registration, and more, so stay tuned! Also follow us here on the PyCon Blog and @PyCon on Twitter.
Categories: FLOSS Project Planets

Catalin George Festila: Python Qt5 - the QTimer class.

Planet Python - Tue, 2019-08-20 07:49
I haven't written about PyQt5 in a while and today I decided to add a short tutorial on this python module. The QTimer class is a high-level programming interface for timers and provides repetitive and single-shot timers. I this example I call a method every second with these lines: self.timer = QTimer() self.timer.timeout.connect(self.handleTimer) self.timer.start(1000)
Categories: FLOSS Project Planets

Bits from Debian: salsa.debian.org: Postmortem of failed Docker registry move

Planet Debian - Tue, 2019-08-20 07:20

The Salsa admin team provides the following report about the failed migration of the Docker container registry. The Docker container registry stores Docker images, which are for example used in the Salsa CI toolset. This migration would have moved all data off to Google Cloud Storage (GCS) and would have lowered the used file system space on Debian systems significantly.

The Docker container registry is part of the Docker distribution toolset. This system supports multiple backends for file storage: local, Amazon Simple Storage Service (Amazon S3) and Google Cloud Storage (GCS). As Salsa already uses GCS for data storage, the Salsa admin team decided to move all the Docker registry data off to GCS too.

Migration and rollback

On 2019-08-06 the migration process was started. The migration itself went fine, although it took a bit longer than anticipated. However, as not all parts of the migration had been properly tested, a test of the garbage collection triggered a bug in the software.

On 2019-08-10 the Salsa admins started to see problems with garbage collection. The job running it timed out after one hour. Within this timeframe it not even managed to collect information about all used layers to see what it can cleanup. A source code analysis showed that this design flaw can't be fixed.

On 2019-08-13 the change was rolled back to storing data on the file system.

Docker registry data storage

The Docker registry stores all of the data sans indexing or reverse references in a file system-like structure comprised of 4 separate types of information: Manifests of images and contents, tags for the manifests, deduplicaed layers (or blobs) which store the actual data, and lastly links which show which deduplicated blogs belong to their respective images, all of this does not allow for easy searching within the data.

The file system structure is built as append-only which allows for adding blobs and manifests, addition, modification, or deletion of tags. However cleanup of items other than tags is not achievable within the maintenance tools.

There is a garbage collection process which can be used to clean up unreferenced blobs, however according to the documentation the process can only be used while the registry is set to read-only and unfortunately it cannot be used to clean up unused links.

Docker registry garbage collection on external storage

For the garbage collection the registry tool needs to read a lot of information as there is no indexing of the data. The tool connects to the storage medium and proceeds to download … everything, every single manifest and information about the referenced blobs, which now takes up over 1 second to process a single manifest. This process will take up a significant amount of time, which in the current configuration of external storage would make the clean up nearly impossible.

Leasons learned

The Docker registry is a data storage tool that can only properly be used in append-only mode. If you never cleanup, it works well.

As soon as you want to actually remove data, it goes bad. For Salsa clean up of old data is actually a necessity, as the registry currently grows about 20GB per day.

Next steps

Sadly there is not much that can be done using the existing Docker container registry. Maybe GitLab or someone else would like to contribute a new implementation of a Docker registry, either integrated into GitLab itself or stand-alone?

Categories: FLOSS Project Planets

Continuum Analytics Blog: Enterprises Need to Think Differently about Data Science. Here’s How.

Planet Python - Tue, 2019-08-20 07:00

Companies that are data science literate make and communicate decisions on the basis of real data models, and not merely instinct or tradition. They welcome new data science technologies as opportunities for potential innovation, rather…

The post Enterprises Need to Think Differently about Data Science. Here’s How. appeared first on Anaconda.

Categories: FLOSS Project Planets

Rapha&#235;l Hertzog: Promoting Debian LTS with stickers, flyers and a video

Planet Debian - Tue, 2019-08-20 06:45

With the agreement of the Debian LTS contributors funded by Freexian, earlier this year I decided to spend some Freexian money on marketing: we sponsored DebConf 19 as a bronze sponsor and we prepared some stickers and flyers to give out during the event.

The stickers only promote the Debian LTS project with the semi-official logo we have been using and a link to the wiki page. You can see them on the back of a laptop in the picture below. As you can see, we have made two variants with different background colors:

The flyers and the video are meant to introduce the Debian LTS project and to convince companies to sponsor the Debian LTS project through the Freexian offer. Those are short documents and they can’t explain the precise relationship between Debian LTS and Freexian. We try to show that Freexian is just an intermediary between contributors and companies, but some persons will still have the feeling that a commercial entity is organizing Debian LTS.

Check out the video on YouTube:

The inside of the flyer looks like this:

Click on the picture to see it full size

Note that due to some delivery issues, we have left-over flyers and stickers. If you want some to give out during a free software event, feel free to reach out to me.

No comment | Liked this article? Click here. | My blog is Flattr-enabled.

Categories: FLOSS Project Planets

Rapha&#235;l Hertzog: Freexian’s report about Debian Long Term Support, July 2019

Planet Debian - Tue, 2019-08-20 05:38

Like each month, here comes a report about the work of paid contributors to Debian LTS.

Individual reports

In July, 199 work hours have been dispatched among 13 paid contributors. Their reports are available:

  • Adrian Bunk got 8h assigned but did nothing (plus 10 extra hours from June), thus he is carrying over 18h to August.
  • Ben Hutchings did 18.5 hours (out of 18.5 hours allocated).
  • Brian May did 10 hours (out of 10 hours allocated).
  • Chris Lamb did 18 hours (out of 18 hours allocated).
  • Emilio Pozuelo Monfort did 21 hours (out of 18.5h allocated + 17h remaining, thus keeping 14.5 extra hours for August).
  • Hugo Lefeuvre did 9.75 hours (out of 18.5 hours, thus carrying over 8.75h to Augustq).
  • Jonas Meurer did 19 hours (out of 17 hours allocated plus 2h extra hours June).
  • Markus Koschany did 18.5 hours (out of 18.5 hours allocated).
  • Mike Gabriel did 15.75 hours (out of 18.5 hours allocated plus 7.25 extra hours from June, thus carrying over 10h to August.).
  • Ola Lundqvist did 0.5 hours (out of 8 hours allocated plus 8 extra hours from June, then he gave 7.5h back to the pool, thus he is carrying over 8 extra hours to August).
  • Roberto C. Sanchez did 8 hours (out of 8 hours allocated).
  • Sylvain Beucler did 18.5 hours (out of 18.5 hours allocated).
  • Thorsten Alteholz did 18.5 hours (out of 18.5 hours allocated).
Evolution of the situation

July was different than other months. First, some people have been on actual vacations, while 4 of the above 14 contributors met in Curitiba, Brazil, for DebConf19. There, a talk about LTS (slides, video) was given, followed by a Q&ligA session. Also a new promotional video about Debian LTS, aimed at potential sponsors was shown there for the first time.

DebConf19 was also a success in respect to on-boarding of new contributors, we’ve found three potential new contributors, one of them is already in training.

The security tracker (now for oldoldstable as Buster has been released and thus Jessie became oldoldstable) currently lists 51 packages with a known CVE and the dla-needed.txt file has 35 packages needing an update.

Thanks to our sponsors

New sponsors are in bold.

No comment | Liked this article? Click here. | My blog is Flattr-enabled.

Categories: FLOSS Project Planets

Test and Code: 84: CircuitPython - Scott Shawcroft

Planet Python - Tue, 2019-08-20 03:00

Adafruit enables beginners to make amazing hardware/software projects.
With CircuitPython, these projects can now use Python.

The combination of Python's ease of use and Adafruit's super cool hardware and a focus on a successful beginner experience makes learning to write code that controls hardware super fun.

In this episode, Scott Shawcroft, the project lead, talks about the past, present, and future of CircuitPython, and discusses the focus on the beginner.

We also discuss contributing to the project, testing CircuitPython, and many of the cool projects and hardware boards that can use CircuitPython, and Blinka, a library to allow you to use "CircuitPython APIs for non-CircuitPython versions of Python such as CPython on Linux and MicroPython," including Raspberry Pi.

Special Guest: Scott Shawcroft.

Sponsored By:

Support Test & Code - Python Testing & Development

Links:

<p>Adafruit enables beginners to make amazing hardware/software projects.<br> With CircuitPython, these projects can now use Python.</p> <p>The combination of Python&#39;s ease of use and Adafruit&#39;s super cool hardware and a focus on a successful beginner experience makes learning to write code that controls hardware super fun.</p> <p>In this episode, Scott Shawcroft, the project lead, talks about the past, present, and future of CircuitPython, and discusses the focus on the beginner.</p> <p>We also discuss contributing to the project, testing CircuitPython, and many of the cool projects and hardware boards that can use CircuitPython, and Blinka, a library to allow you to use &quot;CircuitPython APIs for non-CircuitPython versions of Python such as CPython on Linux and MicroPython,&quot; including Raspberry Pi.</p><p>Special Guest: Scott Shawcroft.</p><p>Sponsored By:</p><ul><li><a href="https://www.patreon.com/testpodcast" rel="nofollow">Patreon Supporters</a>: <a href="https://www.patreon.com/testpodcast" rel="nofollow">Help support the show with as little as $1 per month and be the first to know when new episodes come out.</a></li></ul><p><a href="https://www.patreon.com/testpodcast" rel="payment">Support Test & Code - Python Testing & Development</a></p><p>Links:</p><ul><li><a href="https://circuitpython.org/" title="CircuitPython" rel="nofollow">CircuitPython</a></li><li><a href="https://circuitpython.org/downloads" title="Downloads" rel="nofollow">Downloads</a> &mdash; All the products that run CircuitPython</li><li><a href="https://learn.adafruit.com/category/circuitpython" title="Learning Guides with CircuitPython" rel="nofollow">Learning Guides with CircuitPython</a></li><li><a href="https://learn.adafruit.com/adafruit-circuit-playground-express/circuitpython-quickstart" title="Loading CircuitPython on Circuit Playground Express" rel="nofollow">Loading CircuitPython on Circuit Playground Express</a></li><li><a href="https://adafru.it/discord" title="Adafruit Discord Server" rel="nofollow">Adafruit Discord Server</a></li><li><a href="https://twitter.com/tannewt" title="Scott Shawcroft (@tannewt) / Twitter" rel="nofollow">Scott Shawcroft (@tannewt) / Twitter</a></li><li><a href="https://pypi.org/project/Adafruit-Blinka/" title="Adafruit-Blinka " rel="nofollow">Adafruit-Blinka </a></li></ul>
Categories: FLOSS Project Planets

PSF GSoC students blogs: weeklyCheckIn[12]

Planet Python - Tue, 2019-08-20 01:43
What did I do this week?

Last week, me and my mentor started testing mscolab. At first, I exposed my port through ngrok. But the network I/O were extremely slow. I couldn't demonstrate all of mscolab's features. Then we setup an AWS instance with mscolab server deployed. We could test and improve on many of its features. Some things I worked on this week:

  1. Adding support to use Postgres
  2. Refactoring tests, so that the library code is independent of test configurations.
  3. Adding newly created projects to project-list, in real time
  4. Add newly permitted collaborators to users-list in project window, in real time.
  5. Fix autosave inconsistency when projects are switched
What's coming up next?

There are very few development tasks remaining. Since this is the last week of GSoC, I'd be working on documenting the code I've written, externally. That would take most of my time this week.

Did I get stuck anywhere?

Yes, I got stuck in a bad place for some-time this week. While setting up Postgres, I had to manually autoincrement sequence to assign 'id' to tables, if I explicitly insert rows. I have documented the issue and solution in this blogpost if you're interested in learning more about it.

Categories: FLOSS Project Planets

Codementor: Top 9 Django Concepts - Part 2 : 5 Mins

Planet Python - Mon, 2019-08-19 22:58
The 2nd part of a 3 part series on 9 concepts of Django to help any aspiring Django developer to accelerate their learnings
Categories: FLOSS Project Planets

PSF GSoC students blogs: Week 10 &amp; 11

Planet Python - Mon, 2019-08-19 22:56

These two weeks, I've been working on fixing bug #1. I think my algorithm will work in general, but right now the get_cves function doesn't work as I expected, which would cause some problem with data format in futher. Previously, the function will run the query that selects all the version of a product_vendor pair, and then save the result in memory, and using Python code to check whether the version we want is in it. This is actually inefficient and memory-consuming. Therefore, I refactored the code for conducting the query. Right now, the query will only select the version we want and return it to memory. The time for running the whole test cases reduced from 83 seconds to 61 second. So right now, I will try to implement the dynamic programming method and see if the time complexity is not too high.

Categories: FLOSS Project Planets

PSF GSoC students blogs: Working towards pre-planned transforms

Planet Python - Mon, 2019-08-19 21:44
What did you do this week?

The main feature I've been working on this week has been adding pre-planned transforms to pypocketfft in (pypocketfft!28). This will allow users to speed-up cases where many transforms of the same shape are done, without needing to rely on any internal caches. This gives the users a small performance improvement in pypocketfft but adding this to the scipy.fft interface will also allow other backends to benefit. e.g. pyFFTW can have a significantly more expensive planning phase.

Plans have now been added into the c++ interface of pypocketfft but I have still got to expose these in the python interface. Additionally, I've made a number of other contributions in several other places:

  • Refactored the real transform tests to use pytest and to test all the transforms in long double precision as well (scipy#10669)
  • Fixed a race condition in my custom threadpool code that was causing crashes on macOS (pypocketfft!27)
  • Added benchmarks to my multithreading PR (scipy#10614)
  • Opened an issue in mkl_fft about adding a scipy.fft interface (mkl_fft#42). Unfortunately, mkl_ffl doesn't seem to be very active and the issue hasn't received any response yet.
  • Fixed a compile warning in uarray (uarray#195)
What is coming up next?

The coding period has officially ended and it's now time to "submit the code". Most of my code has already been merged into the SciPy master branch so this will just involve writing a summary of my work over the whole GSoC project.

I would also like to finish up the pre-planned transform code but it's likely too late to get this merged in time for the evaluation.

Did you get stuck anywhere?

Fixing the race condition in pypocketfft was quite challenging. The only symptom was a SEGFAULT that occurred only rarely and only on macOS, a platform which I don't use and so couldn't reproduce the issue locally. This made diagnosing and solving the issue very tricky. Thankfully, Martin (pypocketfft's author) had access to apple hardware and was eventually able to give me enough details that I found an issue from another project that had run into almost exactly the same issue.

 

Categories: FLOSS Project Planets

Dirk Eddelbuettel: RcppQuantuccia 0.0.3

Planet Debian - Mon, 2019-08-19 20:45

A maintenance release of RcppQuantuccia arrived on CRAN earlier today.

RcppQuantuccia brings the Quantuccia header-only subset / variant of QuantLib to R. At the current stage, it mostly offers date and calendaring functions.

This release was triggered by some work CRAN is doing on updating C++ standards for code in the repository. Notably, under C++11 some constructs such ptr_fun, bind1st, bind2nd, … are now deprecated, and CRAN prefers the code base to not issue such warnings (as e.g. now seen under clang++-9). So we updated the corresponding code in a good dozen or so places to the (more current and compliant) code from QuantLib itself.

We also took this opportunity to significantly reduce the footprint of the sources and the installed shared library of RcppQuantuccia. One (unexported) feature was pricing models via Brownian Bridges based on quasi-random Sobol sequences. But the main source file for these sequences comes in at several megabytes in sizes, and allocates a large number of constants. So in this version the file is excluded, making the current build of RcppQuantuccia lighter in size and more suitable for the (simpler, popular and trusted) calendar functions. We also added a new holiday to the US calendar.

The complete list changes follows.

Changes in version 0.0.3 (2019-08-19)
  • Updated Travis CI test file (#8)).

  • Updated US holiday calendar data with G H Bush funeral date (#9).

  • Updated C++ use to not trigger warnings [CRAN request] (#9).

  • Comment-out pragmas to suppress warnings [CRAN Policy] (#9).

  • Change build to exclude Sobol sequence reducing file size for source and shared library, at the cost of excluding market models (#10).

Courtesy of CRANberries, there is also a diffstat report relative to the previous release. More information is on the RcppQuantuccia page. Issues and bugreports should go to the GitHub issue tracker.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Categories: FLOSS Project Planets

Pages