FLOSS Project Planets

Introducing new KCM for network configuration

Planet KDE - Mon, 2016-11-28 07:45

After several attempts trying to write a new KCM for network configuration and actually not finishing any of them, I decided to start one more time, but this time my goal was to simply transform the old editor into a bit nicer KCM and place it into system settings where this was missing for very long time. You can see my current result below.

This is still same editor as it was existing before as a standalone application, except the list of connections is now written in QML and is similar to the applet we have in systray. I also had to rewrite the editor widget a bit because it’s currently implemented as a dialog with a tabwidget inside where each tab is represented by one setting widget (e.g. Ipv4SettingWidget), For the new KCM we now have ConnectionEditorBase widget doing all the logic behind, like creating specific setting widgets based on connection type and so on. This widget alone doesn’t display anything and you have to actually subclass it and reimplement method taking care of layouting. This allows me to have e.g. ConnectionEditorTabWidget which just subclasses ConnectionEditorBase and reimplements addWidget() method to place setting widgets into QTabWidget. In future we can also simply write a new UI/layout on top ConnectionEditorBase widget and get rid of the tab layout.

Regarding functionality, it should be already almost on par with functionality of the editor. There are still some missing features (like import/export of VPN), but besides that I think everything else is going well. With the new KCM there are also some minor improvements, like you can now reset your not-saved changes you made to a connection. My plan is to get this into Plasma 5.9 which is supposed to be released in january so I still have plenty of time to finish missing features and address issues I made during this transition and of course time to take your comments into account and make this KCM as most usable for everyone I can :).

Categories: FLOSS Project Planets

Nuvole: Configuration Split: first beta release at Drupal IronCamp

Planet Drupal - Mon, 2016-11-28 06:30
Solving one of Drupal 8 core's development and deployment work-flow issues.

One of the nice things that happened during Drupal Ironcamp in Prague last week was the first beta release of the Configuration Split that I started developing a few months ago.

As explained in the previous blog post about Configuration Split, the typical use case of Configuration Split is to be able to maintain a "development" configuration and a "production" configuration, where some of the modules or settings you use in development are not copied to production upon configuration export/import.

A typical use case

We assume that you have a production and development version of the same site and that they are aligned.

Step 1: Enable anything you need in development and work normally

In the development environment we enable Devel and UI modules enabled and we can create new content types, add fields and do all things we like to do while developing a Drupal site. Some of this configuration work is meant to go to the production site and some (e.g., the fact that Devel is enabled and related configuration) is not. We thus need to split our configuration.

Step 2: Create a split definition

Choose a name for your "local" configuration and select all the configuration you don't want to have on the live site (even though, if you wish, you can still share it with colleagues). This can include modules you would not want to be enabled or configuration that is for development only. The path here is a folder next to the normal config sync directory outside of the web root. In the example above, we are splitting out to a configuration set named "Development" the Devel, Fields UI, Views UI modules and the system.menu.devel configuration.

Step 3: Export your configuration with config_split

$ drush config-split-export

In the most basic use case we just replace config-export with config-split-export in our workflow. This exports the elements we selected above to ../config/dev and the rest (the configuration we want to deplay) to our "standard" configuration folder, in this case ../config/sync.

Step 4: Import the "clean" configuration in production

On all environments we use the configuration import of config_split:

$ drush config-split-import

So in our usual workflow we replace config-import with config-split-import

In order not to have different workflows for development and production we simply deactivate the split on production by overriding the configuration in settings.php on production: $config['config_split.config_split.development']['status'] = FALSE;

This will result in the split definition to be deactivated. And consequently the import will have the configuration blacklisted in the split removed and development modules deactivated.

What's new in the beta version

The following features are new or changed in the beta release:

  • The drush command works and has some text to warn users to which directories configuration will be written to.
  • The drupal console command is disabled, but there is a patch/branch you can use or help to make it functional again.
  • Split entities now have a status and the default command only uses the active splits. Of course you can override that in settings.php as with almost all configuration.
  • You can switch out the default config.storage.sync service in your services.yml file so that the default configuration sync UI uses the split import. Details are in the README.txt file.
More information and acknowledgments

On Friday I gave a presentation about my favorite topic in Drupal: Configuration Management, attached here are the slides.

I wish to thank IronCamp participants and especially Swentel and mr.baileys who worked on patches, and I'd like to thank IronCamp organizers for a very well organized event where I met a lot of old and new friends. Swentel also took care of documenting more advanced use cases of Configuration Split on his blog.

For any further information see the module page and its issue queue.

Tags: Drupal PlanetDrupal 8Code Driven DevelopmentAttachments:  Configuration Management - slides Ironcamp Prague
Categories: FLOSS Project Planets

Stefano Zacchiroli: last week to take part in the Debian Contributors Survey

Planet Debian - Mon, 2016-11-28 05:27
Debian Contributors Survey 2016

About 3 weeks ago, together with Molly and Mathieu, we launched the first edition of the Debian Contributors Survey. I won't harp on it any further, because you can find all relevant information about it on the Debian blog or as part of the original announcement.

But it's worth noting that you've now only one week left to participate if you want to: the deadline for participation is 4 December 2016, at 23:59 UTC.

If you're a Debian contributor and would like to participate, just go to the survey participation page and fill in!

Categories: FLOSS Project Planets

qed42.com: REST API Explorations in Drupal 8 - Primer

Planet Drupal - Mon, 2016-11-28 05:26
REST API Explorations in Drupal 8 - Primer Body

This article assumes you are familiar with what RESTful is & what do we mean when we use the term REST API. Some of you might have already worked with RESTful Web Services module in D7, it exposes all entity types as web services using REST architecture. Drupal 8 out of the box is RESTful with core support. All entities (provided by core + ones created using Entity API) are RESTful resources.

To explore the RESTful nature of Drupal 8, we will need to enable the following modules:

In Core
  • HAL - Serializes entities using Hypertext Application Language.
  • HTTP Basic Authentication - Provides the HTTP Basic authentication provider.
  • RESTful Web Services - Exposes entities and other resources as RESTful web API
  • Serialization - Provides a service for (de)serializing data to/from formats such as JSON and XML.
  • REST UI - Provides a user interface to manage REST resources.
RESTful Resources

Every entity in D8 is a resource, which has an end point. Since, its RESTful, the same end-point is used for CRUD (Create, Read, Update, Delete) operations with different HTTP verbs. Postman is an excellent tool to explore / test RESTful services.  Drupal 8 allows you to selectively choose & enable a REST API. e.g., we can choose to expose only nodes via a REST API & not other entities like users, taxonomy, comments etc.

After enabling REST_UI module we can see list of all RESTful resources at /admin/config/services/rest. In addition to ability to choose the entity one can enable, we can also choose the authentication method per resource & enable specific CRUD operations per resource.

Let us take a look at what the REST APIs for User entity would be after we save the configuration in the above screenshot.



http://domain.com/entity/user?_format=hal_json {  "_links": {    "type": {      "href": "http://domain.com/rest/type/user/user"    }  },  "name": {    "value":"testuser"  },  "mail":{    "value":"testuser@mailserver.com"  },  "pass":{    "value":"testpass"  },  "status": {    "value": 1  } }


X-CSRF-Token: Get from http://domain.com/rest/session/token Content-Type: application/hal+json Accept: application/hal+json Authorization: Basic (hashed username and password)

Note: Drupal 8 doesn't allow anonymous user to send a POST on user resource. It is already fixed in 8.3.x branch but for now we can pass the credentials of the user who have permission to create users. If you are interested in taking a deeper look at the issue, you can follow https://www.drupal.org/node/2291055.

Response: You will get a user object with "200 OK" response code



http://domain.com/user/{uid}?_format=hal_json {  "_links": {    "type": {      "href": "http://domain.com/rest/type/user/user"    }  },  "pass":[{"existing":"testpass"}],  "mail":{    "value":"updatedtestuser@mailserver.com"  } }

Note: Now as user have permission to update his own profile so we can pass current user's credentials in authentication header.

Response: You will get "204 No Content" in response code.




Response: You will get a user object with "200 OK" response code.




Response: You will get "204 No Content" in response code.

RESTful Views and Authentication

Drupal 8 also allows us to export views as a REST service. It allows you to use all the available authentication mechanism in views itself.


JSON API module provides a new format called "api_json" which is soon becoming the de-facto standard for Javascript Frontend frameworks, If you plan to use completely de-coupled Drupal with frontend framework like Angular / React / Ember then its worth a look. To read more about JSON API you can visit the site.

SUMIT MADAN Mon, 11/28/2016 - 15:56
Categories: FLOSS Project Planets

Codementor: How to Build a Python Media Player using LibVLC and GTK+

Planet Python - Mon, 2016-11-28 05:22

In this tutorial, we shall learn how to create a multimedia app in Python with the help of LibVLC and GTK+ which plays media when its starts. App users will also be able to control basic playback options.

This tutorial assumes that the readers know how to create and run Python files; understand basic interpreter errors; working with strings, floats and booleans; object orientation; and idea of threads in Python. Here are other neat Python tips worth knowing about.

The tutorial is suited for novice and intermediate programmers. This tutorial covers GTK+ version 3 and LibVLC media framework; and has been created and tested on Linux.


It is the external programming interface of the VLC media player. LibVLC (VLC SDK) is the core engine and interface to the multimedia framework on which VLC media player is based. It can be embedded into applications to get multimedia capabilities. Therefore, the application should have the same features that VLC media player has.


VLC is developed in C language, therefore install it before using it through Python.The Python bindings have complete coverage of the LibVLC API and the generated module is in pure Python. It only depends on ctypes.

Download the vlc.py module from the Git repository and put the module in a place accessible by Python.


A toolkit for creating graphical user interfaces written in the C programming language.

You should also know about these things:

  • GDK: the GIMP Drawing Kit lies between the xlib and the GTK+ library, handling basic renderings such as drawing primitives, raster graphics (bitmaps), cursors, fonts, as well as window events and drag-and-drop functionality. It contains back-ends to a couple of windowing systems, like the X11.

  • GdkX11: Also a part of GObject package, for X Window System Interaction, and X backend-specific functions.

  • PyGObject: A Python module that enables developers to access GObject-based libraries such as GTK+ ,Gdk, GdkX11, etc within Python.


PyGObject and its dependencies are packaged by all major Linux distributions. So, install the package from the official repository of the distribution.

The code block below shows how to import Gtk+ library from the GI (gobject-introspection) package. Since users can have multiple versions of GTK+ on their systems, import GTK that it refers to GTK+ 3 and not any other version of the library, which is the purpose of statement gi.require_version('Gtk', '3.0').

import gi gi.require_version('Gtk', '3.0') from gi.repository import Gtk Application Components I. Application Window

It is a sub-class of Gtk.Window to define our own ApplicationWindow class to create a top-level window that contains other widgets.

In the class’s constructor, call the constructor of the super class and set the value of the property title to Python-Vlc Media Player.

class ApplicationWindow(Gtk.Window): def __init__(self): Gtk.Window.__init__(self, title="Python-Vlc Media Player") Media control buttons self.playback_button = Gtk.Button() self.stop_button = Gtk.Button() Images to be displayed on buttons

We use built-in images provided by the GTK+ library.

self.play_image = Gtk.Image.new_from_icon_name( "gtk-media-play", Gtk.IconSize.MENU ) self.pause_image = Gtk.Image.new_from_icon_name( "gtk-media-pause", Gtk.IconSize.MENU ) self.stop_image = Gtk.Image.new_from_icon_name( "gtk-media-stop", Gtk.IconSize.MENU )

Set the images on the buttons.

self.playback_button.set_image(self.play_image) self.stop_button.set_image(self.stop_image)

events and signals in GTK+
  • GTK+ is an event-driven system. All GUI applications are event-driven.
  • The applications start on a main loop, which continuously checks for newly generated events. If there is no event, the application waits and does nothing.
  • In GTK+, an event is a message from the X server. When the event reaches a widget, it may react to this event by emitting a signal.
  • The GTK+ programmer can connect a specific callback to the signal. The callback is a handler function that reacts to the signal.
  • The handler function takes two parameters :
    The first parameter is the object which emitted the signal; in our case, it is the Play/Pause button. The second parameter is optional where one can send events/data generated when the signal is fired.
    For example: data type can be a Gdk.EventType.

Here is the link to get to know more about Event-Driven programming.

Connect buttons to clicked signals. self.playback_button.connect("clicked", self.toggle_player_playback) self.stop_button.connect("clicked", self.stop_player)

The function toggle_player_playback handles switching between play/pause options and sets playback_button image accordingly.

def toggle_player_playback(self, widget, data=None): """ Handler for Player's Playback Button (Play/Pause). """ if self.is_player_active == False and self.player_paused == False: self.player.play() self.playback_button.set_image(self.pause_image) self.is_player_active = True elif self.is_player_active == True and self.player_paused == True: self.player.play() self.playback_button.set_image(self.pause_image) self.player_paused = False elif self.is_player_active == True and self.player_paused == False: self.player.pause() self.playback_button.set_image(self.play_image) self.player_paused = True else: pass

The Function stop_player stops media playback and sets the playback button to play the image.

def stop_player(self, widget, data=None): self.player.stop() self.is_player_active = False self.playback_button.set_image(self.play_image) Media Rendering Surface

To draw on the screen, we use the DrawingArea widget. A drawing area widget is essentially an X window and nothing more. It’s a blank canvas in which we can draw whatever we like. A drawing area is created using the call below.

The default size can be overridden, as is true for all widgets, by calling set_size_request_, and that, in turn, can also be overridden if the user manually resizes the window containing the drawing area.

self.draw_area = Gtk.DrawingArea() self.draw_area.set_size_request(300,300)

Note: This tutorial does not cover input signals from users on GtkDrawingArea. To capture events from input devices like mouse, for these widgets, we need to use an EventBox widget.

Connect the rendering area to its realize signal. The “realize” signal is to take any necessary actions when the widget is instantiated on a particular display. I shall talk about the _realize handler later on in this tutorial.


Before discussing LibVLC, lets pack widgets in Gtk.Box widgets known as Gtk Containers.

What are boxes? Boxes are invisible containers into which we can pack our widgets. And when packing widgets into a horizontal box, the objects are inserted horizontally from left to right or right to left depending on whether Gtk.Box.pack___start() or Gtk.Box.pack_end() is used.

In a vertical box, widgets are packed from top to bottom or vice versa. We can use any combination of boxes inside or beside other boxes to create the desired effect.

Create a horizontally-orientated box container where 6 pixels are placed between children. Here, the children are our two media control buttons.

We pack from the start of the box. The pack start takes four parameters:
1. The GtkWidget to be added to the box.
2. Boolean expand: TRUE if the new child is to be given extra space allocated to the box.
3. Boolean fill: TRUE if the space given to child by the expand option is actually allocated to child, rather than just padding it.
4. padding: extra space in pixels to put between this child and its neighbors, over and above the global amount specified by “spacing” property.

self.hbox = Gtk.Box(spacing=6) self.hbox.pack_start(self.playback_button, True, True, 0) self.hbox.pack_start(self.stop_button, True, True, 0)

Similarly, we create a vertically-oriented box and it as the child of the top-level window. Next, we pack the rendering surface first and then the horizontal box.

self.vbox = Gtk.Box(orientation=Gtk.Orientation.VERTICAL) self.add(self.vbox) self.vbox.pack_start(self.draw_area, True, True, 0) self.vbox.pack_start(self.hbox, False, False, 0)

Show the window recursively and any child widgets.


Below is the screenshot of our resultant GUI of the application.

II. Playing Media using the LibVLCvlc framework import vlc

- The drawing area has to be realized before to get its Gdk.Window.
- We can’t get the window property directly.
- We need to import GdkX11 for the xid method because the rendering surface’s window is an instance of GdkX11.X11Window.
- GObject.Object -> Gdk.Window -> GdkX11.X11Window
- Hencem import GdkX11.

gi.require_version('GdkX11', '3.0') from gi.repository import GdkX11

As soon as the rendering surface is realized, its handler function performs the following operations:

  • Create and initialize a LibVLC instance and pass "--no-xlib" as parameter.

  • Create an empty Media Player object.

self.vlcInstance = vlc.Instance("--no-xlib") self.player = self.vlcInstance.media_player_new()

Why is the "--no-xlib" passed to vlcInstance?

Generally, when multiple threads are using Xlib (also known as libX11) concurrently, it is necessary to call the XInitThreads() function, as we are not dealing with threads using xlib here. Thus, notifying LibVLC that the Xlib is not initialized for threads.

  • Get the window ID of the Media Rendering Surface’s X11 window and hook the player to the window for media output.
win_id = widget.get_window().get_xid() self.player.set_xwindow(win_id)
  • Set theMRL(Media Resource Locator — used by LibVLC) to play.

###Putting it altogether

import sys import gi gi.require_version('Gtk', '3.0') from gi.repository import Gtk gi.require_version('GdkX11', '3.0') from gi.repository import GdkX11 import vlc MRL = "" class ApplicationWindow(Gtk.Window): def __init__(self): Gtk.Window.__init__(self, title="Python-Vlc Media Player") self.player_paused=False self.is_player_active = False self.connect("destroy",Gtk.main_quit) def show(self): self.show_all() def setup_objects_and_events(self): self.playback_button = Gtk.Button() self.stop_button = Gtk.Button() self.play_image = Gtk.Image.new_from_icon_name( "gtk-media-play", Gtk.IconSize.MENU ) self.pause_image = Gtk.Image.new_from_icon_name( "gtk-media-pause", Gtk.IconSize.MENU ) self.stop_image = Gtk.Image.new_from_icon_name( "gtk-media-stop", Gtk.IconSize.MENU ) self.playback_button.set_image(self.play_image) self.stop_button.set_image(self.stop_image) self.playback_button.connect("clicked", self.toggle_player_playback) self.stop_button.connect("clicked", self.stop_player) self.draw_area = Gtk.DrawingArea() self.draw_area.set_size_request(300,300) self.draw_area.connect("realize",self._realized) self.hbox = Gtk.Box(spacing=6) self.hbox.pack_start(self.playback_button, True, True, 0) self.hbox.pack_start(self.stop_button, True, True, 0) self.vbox = Gtk.Box(orientation=Gtk.Orientation.VERTICAL) self.add(self.vbox) self.vbox.pack_start(self.draw_area, True, True, 0) self.vbox.pack_start(self.hbox, False, False, 0) def stop_player(self, widget, data=None): self.player.stop() self.is_player_active = False self.playback_button.set_image(self.play_image) def toggle_player_playback(self, widget, data=None): """ Handler for Player's Playback Button (Play/Pause). """ if self.is_player_active == False and self.player_paused == False: self.player.play() self.playback_button.set_image(self.pause_image) self.is_player_active = True elif self.is_player_active == True and self.player_paused == True: self.player.play() self.playback_button.set_image(self.pause_image) self.player_paused = False elif self.is_player_active == True and self.player_paused == False: self.player.pause() self.playback_button.set_image(self.play_image) self.player_paused = True else: pass def _realized(self, widget, data=None): self.vlcInstance = vlc.Instance("--no-xlib") self.player = self.vlcInstance.media_player_new() win_id = widget.get_window().get_xid() self.player.set_xwindow(win_id) self.player.set_mrl(MRL) self.player.play() self.playback_button.set_image(self.pause_image) self.is_player_active = True if __name__ == '__main__': if not sys.argv[1:]: print "Exiting \nMust provide the MRL." sys.exit(1) if len(sys.argv[1:]) == 1: MRL = sys.argv[1] window = ApplicationWindow() window.setup_objects_and_events() window.show() Gtk.main() window.player.stop() window.vlcInstance.release() Gtk.main()

It is the GTK+ processing loop which is running inside our main application thread.

  • When the user is doing nothing, GTK+ sits in the main loop and waits for input. If the user performs some action — say, a mouse click — then the main loop “wakes up” and delivers an event to GTK+.
  • GTK+ is “thread aware” but not thread safe — it provides a global lock controlled by gdk_threads_enter() and gdk_threads_leave() which protects all use of GTK+. That is, only one thread can use GTK+ at any given time.
  • Calling Gtk.main_quit() makes the main loop inside of Gtk.main() return.
  • Thus, connecting the “destroy” signal to Gtk.main_quit() will destroy the window and also terminate our application.

Below is the screenshot showing a media playing in our media player.

Its seems that we have reached end of this tutorial. However, This application can be extended to add more multimedia capabilities and UI features.
Therefore, you may explore both libraries to add more robust features in your app.

Thanks! I hope you enjoyed the tutorial. Please leave your questions in the comment section below.

Categories: FLOSS Project Planets

Kristof De Jaeger: Configuration split: deploy subsets of configuration in Drupal 8

Planet Drupal - Mon, 2016-11-28 05:01
Written on November 28, 2016 - 11:01

From the Configuration split project page: "The Drupal 8 configuration management works best when importing and exporting the whole set of the sites configuration. However, sometimes developers like to opt out of the robustness of CMI and have a super-set of configuration active on their development machine and deploy only a subset. The canonical example for this is to have the devel module installed or having a few block placements or views in the development environment and then not export them into the set of configuration to be deployed, yet still being able to share the development configuration with colleagues."

This small utility module for Drupal 8 allows you to exactly achieve this use case (and many others). I will cover two common scenarios, explaining how to configure your site and which commands you need to run:

  1. Have (development/ui) modules installed on your dev environment, uninstalled on production
  2. Have modules installed on your production environment, but not on development

I've also recorded a screencast in which I configure and run all the commands explained underneath. The best way in the end is to play around with the module itself of course.

Configuration split 101

Configuration split exposes a configuration entity which controls what you want to split off. Currently you can

  • blacklist modules: any configuration that this module owns will automatically be blacklisted too.
  • blacklist configuration: settings or configuration entities. These will be removed from the active sync directory.
  • graylist configuration: settings or configuration entities. These will not be removed if they are in the active sync directory, but also not exported if they are not there yet. I won't cover this functionality in this article post, that will be for another time.

After you configured one or more configurations split entities, you can use the drush commands to export and import configuration, based on one or more of those config entities. When it comes to exporting configuration, you can not use the existing drush core command (config-export / cex). Using drush config-import (cim) or the UI to import may still be used, but this depends on your setup. Drupal Console commands are under revision.

Each config split entity defines the directory in which the splitted configuration will live in case there are some. Not all modules define configuration, so there's a possibility that this directory is empty after you do an export.

An important technical aspect is that the module does not interfere with the active configuration but instead filters on the import/export pipeline. What simple happens is this: just before the actual writing, it will check the configuration and remove any entries that you don't want to be there. Then your active configuration is written away. Config that doesn't belong into the active configuration will be moved to separate directory. On import, the opposite happens: it will merge settings back in and set modules to the installed state just before core configuration then starts importing.

Last, but not least: you can swap the config.storage.sync service so that the synchronize screen will use the config split config entities for importing. More information is in the README file and in the video.

Scenario 1: modules installed on dev, not on production

Step 1

After you installed the module, go to 'admin/config/development/configuration/config-split' and click on 'Add configuration split'. Naming is important, so we'll enter 'Dev split' as the name for this configuration. We'll also create a directory called 'sync-dev-split'. Now toggle the modules that you don't want to have installed on production. In this case, we're going to blacklist Devel, Devel Kint, Field UI, Views UI and database logging. Additionally, also toggle system.menu.devel. This configuration is owned by the system module, so there's no dependency on the devel module. There's no problem having this config on your production site though, so it's not required to blacklist it.

Optionally, you can also blacklist the configuration split module because it doesn't necessarily have to be installed on production. We're not doing that here, because in the second scenario, we're building further on this one and we want to have it installed on the production environment as well.

Step 2

Go to your command line interface and run following command:

swentel@dev:/home/drupal/drupal-core$ drush csex --split=dev_split

The split option is not required, but when starting to work with the module, it helps you to know which config split will be used for this command. It's possible to override the status of any config split config entity so that eventually, you can omit the split option. Sadly enough, you don't get any feedback (yet), but after the command has run you should see various files in your sync-dev-split directory:

swentel@dev:/home/drupal/drupal-core$ ls sync-dev-split/
dblog.settings.yml  devel.settings.yml  field_ui.settings.yml  system.menu.devel.yml

Step 3

You can now commit your active sync and go to production. You don't necessarily have to put the sync-dev-live directory in your version control because production doesn't need to know about these files at all. Once you have pulled on production, go to the synchronize screen to verify what is staged. You will see that the setting files will be removed and core.extension will be changed uninstalling the development modules and installing configuration split. Since we only have one config split configuration, you can hit 'Import all' here or run drush config-import, (which is the drush core command). Note that if you don't use the UI, you can blacklist the configuration manager module as well.

You might wonder why we don't use the 'config-split-import' command from the module itself: this would import the active sync directory and also include the modules and settings again that we have blacklisted. And that's not what we want. This seems confusing at first, but ultimately, if this is your setup, you just keep on using the core / drush commands to import your staged configuration on production.

# This command can be used now, but not anymore further on when we will add scenario 2.
swentel@live:/home/drupal/drupal-core$ drush cim Scenario 2: modules installed on production, not on dev

Step 1

This scenario builds further on the first one. Config split is now installed on our live website. Create a new config split configuration called 'Live split'. You will need a new directory for this second config split, so make sure it's there. On production we still want to log events and for that we're going to use the syslog module. Install the module, so that we can now blacklist it for the live split configuration.

Step 2

Go to your command line interface and run following commands:

swentel@live:/home/drupal/drupal-core$ drush csex --split=live_split

Again, no feedback here, but after the command has run you should see the syslog settings file in your sync-live-split directory:

swentel@live:/home/drupal/drupal-core$ ls sync-live-split/
syslog.settings.yml Doing exports and imports on dev or live

From now on, if you want to import new settings whether it's on your dev or live environment, you can not use the drush core commands anymore. Use following commands:

# to export on your dev environment
swentel@dev:/home/drupal/drupal-core$ drush csex --split=dev_split
# to import on your dev environment
swentel@dev:/home/drupal/drupal-core$ drush csim --split=dev_split
# to export on your live environment
swentel@live:/home/drupal/drupal-core$ drush csex --split=live_split
# to import on your live environment
swentel@live:/home/drupal/drupal-core$ drush csim --split=live_split The future

Please join us in the issue queue because there's still some work:

  • Optimize the user interface and allow wildcards
  • Add confirmation and feedback in the drush commands

Ultimately, this functionality should live in core. A core issue to discuss this is at https://www.drupal.org/node/2830300.

Categories: FLOSS Project Planets

S. Lott: Handling Irregular File Formats

Planet Python - Mon, 2016-11-28 03:05
This is a common issue. We have a file which was printed for human consumption. Consequently, it has many different kinds of lines.

These are the two kinds of lines of interest:

900296268 4/9/16 Mobility, Data Mining and Privacy Expired

900295204 4/1/16 Pro .NET Best Practices

The first is a single physical line.  It has four data elements. The second is two physical lines. The first has three data elements.

There are a number of other noise lines in the file which must be filtered out.

The first "solution" pitched to me could be summarized with this:

Move "Expired" on a line by itself to the previous line

That was part of the email subject line. The body of the email was some whining about regular expressions. Which I mostly ignored. Multiline regular expressions are their own kind of challenge.

We (should) all know this: https://blog.codinghorror.com/regular-expressions-now-you-have-two-problems/

Let's do this without regular expressions. There are two things we need to know. One is buffering, and the other is the best way to split each line. It turns out that there are spaces as well as tabs, and can can, by splitting on tabs, make a lot of progress.

Instead of the good approach, I'll pick the other approach that doesn't involve splitting on tabs.

Here's the simulated file, with data lightly redacted.

sample_text = '''
"Your eBooks"

Show 200

Page: 1

Order # Date Title Formats Status Download
xxx315605 9/30/16 R for Cloud Computing Available

xxx304790 6/21/16 Java XML and JSON Available
xxx304790 6/21/16 Accelerated DOM Scripting with Ajax, APIs, and Libraries Available

xxx291633 2/28/16 Practical Google Analytics and Google Tag Manager for Developers

It's not perfectly obvious (because of line wrapping) but there are three examples of the "all-complete-in-one-line" records. There's one example of the "two-lines" record.

Rather than mess with the file, we'll build a file-like object with our sample data.

import io
file_like_object = io.StringIO(sample_text)

I like this because it lets me write proper unit test cases.

The file has four kinds of lines:

  • Complete Records
  • Record Headers (without Available/Expired)
  • Record Trailers (only Available/Expired)
  • Noise

We'll create some decision rules for the two obvious kinds of file lines: complete records and trailers. We can deduce the headers based on a simple adjacency rule: they precede a trailer. The fourth kind of lines are those which are possible headers but are not immediately prior to a trailer.

def complete(words):
    return len(words) > 3 and words[-1] in ('Available', 'Expired')

def trailer(words):
    return len(words) == 1 and words[0] in ('Available', 'Expired')    

We can spot these two kinds of lines easily. The other kinds require a Buffered Generator.

def emit_clean(source):
    header = None
    for line in (line.strip() for line in source):
        words = [w.strip() for w in line.split()]
        if len(words) == 0: continue
        if complete(words):
            header = None
        elif trailer(words) and header:
            yield(header + '\t\t' + line)
            header = None
            # Possible header
            # print('??', line)
            header = line

The Buffered Generator is a way to implement a "look ahead one item" (LA1) algorithm. We do this by buffering rows. When we get to the next row we can use the buffered row and the current row to implement the look-ahead logic.

The actual implementation uses a look-behind buffer, header.

The (line.strip() for line in source) generator expression strips away leading and trailing spaces. This gets rid of the newline characters at the end of each input line.

The default behavior of split() is to split on whitespace. In this case, it will create a number of words for complete records or header records, and a single word for a trailer record. If we had split on tab characters, some of this logic would be simplified.

That's left as an exercise for the reader.

If the len(words) is zero, the line is blank.

If the line matches the complete() function, we can yield it as one of the iterable results of the generator function. We also clear out the look-behind buffer, header.

If the line is a trailer and we have a buffered look-behind line, this is the two-physical-line case. We can assemble a complete record and emit it.

Otherwise, we don't know what the line is. It's a possible header line, so we'll save it for later examination.

This algorithm involves no regular expressions.

With Regular Expressions
An alternative would use three regular expressions to match the three kinds of lines.

import re
all_one_pat = re.compile("(.*)\t(.*)\t(.*)\t\t((?:Available)|(?:Expired))")
header_pat = re.compile("(.*)\t(.*)\t(.*)")
trailer_pat = re.compile("((?:Available)|(?:Expired))")

This has the advantage that we can then use the groups() method of each successful match to emit useful data instead of text which needs subsequent parsing. This leads to a slightly more robust process.
def emit_clean2(source):    header = None    for line in (line.strip() for line in source):        if len(line) == 0: continue        all_one_match = all_one_pat.match(line)        header_match = header_pat.match(line)        trailer_match = trailer_pat.match(line)        if all_one_match:            yield(all_one_match.groups())            header = None        elif header_match and not header:            header = header_match.groups()        elif trailer_match and header:            yield header + trailer_match.groups()            header = None        else:            pass # noise
The essential processing involves seeing which of the regular expressions match the line at hand. If it's all-in-one, this is good. We can yield the groups of meaningful data. If it's a header, we can save the groups. If it's a trailer, we can combine header and trailer groups and yield the composite.

This has the advantage of explicitly rejecting noise lines instead of treating each noise line as a possible header.
Categories: FLOSS Project Planets

ADCI Solutions: Drush

Planet Drupal - Mon, 2016-11-28 02:44


Drush is a command line utility for Drupal that allows to avoid or at least to speed up many kinds of routine tasks. Usually a developer deals with these Drush parts: Drush make, adding the extra Drush commands and tuning Drush and Drupal via configuration files. In this article we listed the main Drush commands, thoroughly described the Drupal website configuration process with the help of Drush, told about shell and Drupal site aliases and learned how working with Drush has changed in Drupal 8. We supplemented each description of working with a particular Drush part with code samples: please feel free to use these examples in your development workflow.


Drush (The Drupal Shell) is a command line utility for Drupal, which provides many commands to interact with Drupal, its modules, themes, etc. Drush is a great developer’s helper and it allows to avoid or at least to speed up many kinds of routine tasks.

There are many articles telling about different Drush commands and how they helpful are, so I'd like to focus on Drush configuration possibilities and do a quick overview on Drush commands and Drush make. When you use Drush all the interactions with a Drupal site happen by executing commands, but there are some other Drush parts you may deal with:

  • Drush make which is combination of a special definition stored in the (make) file and a few specific commands to build a Drupal site codebase;
  • possibilities to add your own Drush commands by writing code in the files called in a special way and placed properly;
  • tuning Drush and Drupal via configuration files and more.

Drush processes described below are not necessarily should be implemented together. Since it’s not an algorithm of working with Drush, but a descriptive compilation, you may refer to the any article’s part when needed.

Drush commands

As you already know,  an interaction with a Drupal site happens via Drush commands. The Drush core provides many commands for many different purposes, but I believe the most used Drush commands are:

  • drush pm-download (dl) - download one or several projects (the Drupal core, module, theme, distribution), so you don't have to visit drupal.org to download and then unpack projects one by one;

  • drush pm-enable (en) - enable (install, if wasn't enabled before) one or several modules or themes;
  • drush cache-clear (cc) - clear cache (routing, theme, all, Drush, etc. cache).

With Drush you also can:

  • make a full site backup;
  • update the Drupal core, contrib modules and themes;
  • run cron jobs;
  • execute SQL query;
  • block user account(s);
  • change Drupal settings;

and other.

Drush make

Drush make allows to create a ready-to-use Drupal site codebase including the Drupal core, modules, themes, etc., (but without installation) from a definition stored in the just one file. Drush make does the work for you:

  • downloads the Drupal core, modules, themes;
  • fetches code from Git, SVN repositories;
  • downloads and applies patches;
  • downloads .tar.gz and .zip archives in general;
  • downloads libraries in particular.

The Drush make file can be written in 2 formats: YAML and the older Drupal .info INI format similar to modules' .info. YAML is recommended though, since Drupal is switched to YAML for configurations (since Drupal 8).

There are 2 main commands to work with Drush make:

  • drush make example.make.yml docroot - make a Drupal codebase using the definition stored in the example.make.yml file;
  • drush make-generate example.make.yml - generate the make file from a current Drupal site.

In case you develop Drupal distributions with following publishing at drupal.org, you have to deal with Drush make anyway.

Drush make file example (in YAML):

core: 7.x api: 2 projects: drupal: type: core admin_menu: subdir: contrib ctools: version: "1.7" tao: type: theme download: url: "git://github.com/developmentseed/tao.git"Extending the list of Drush commands

New commands can be easily added to Drush, and many contrib modules provide their own Drush commands: there is the modulename.drush.inc file in the module containing Drush integration code in this case. For example:

  • drush fn-hook (Devel module) - shows a list of implementations for the particular hook;
  • drush generate-content (Devel module) - generates dummy content;
  • drush features-revert-all (Features module) - reverts all the configurations exported into modules via the Features module to the code state;
  • drush views-list (Views module) - lists all the views on the site.

It's also possible to add Drush commands outside of modules, for example, to perform some specific actions for deployment, so you don't have to create and install a module on the site just to be able to invoke these Drush commands.

Drush configuration

Drush as well as a Drupal site can be configured in the context of running Drush commands. Needed configurations are defined in the drushrc.php file (DrushRC = Drush Runtime Config). For example, it's possible to force some global options for the Drush commands like always answering "yes" to all the confirmation prompts, force some of the command-specific options, change some of the Drupal settings (like an emails handler class and so on).

Drush looks for the drushrc.php file in the following directories, one by one as listed below. The order is basically the same when Drush looks for the files containing additional Drush commands (if the commands aren't related to the particular module):

  • sites/{default|example.com}/drushrc.php - a site specific directory;
  • Drupal /drush and sites/all/drush directories;
  • /drush directory 1 level above the Drupal root;
  • as specified by --config (-c) option when running a Drush command;
  • User's .drush folder (~/.drush/drushrc.php);
  • /etc/drush/drushrc.php;
  • the Drush installation folder.

Let's see some examples (see the code comments for more info on what each setting means):

/** * Drush commands configuration. */ // Force verbose mode for all commands. $options['v'] = 1; // Answer "yes" to all confirmation prompts. $options['y'] = 1; // Force --notes option for pm-update command. $command_specific['pm-update'] = ['notes' => TRUE]; // Use admin/admin as superuser credentials when installing Drupal site. $command_specific['site-install'] = [ 'account-name' => 'admin', 'account-pass' => 'admin', ]; /** * Site configuration. */ // Change site name. $options['variables']['site_name'] = 'My Drupal site'; // Use different theme. $options['variables']['theme_default'] = 'minnelli'; // Disable clean URLs. $options['variables']['clean_url'] = 0; // Use different directory for temporary. $options['variables']['file_temporary_path'] = '/home/vagrant/tmp'; /** * SQL dump and structure tables. */ // List of tables to export structure only without data on dump. $options['structure-tables']['common'] = [ 'cache', 'cache_*', 'history', 'search_*', 'sessions', ]; // Force "common" structure tables list by default on sql-dump command. $command_specific['sql-dump'] = [ 'structure-tables-key' => 'common', ]; Shell aliases

Drush also allows to define short aliases for a command or a number of commands, potentially used with a number of set options, via drushrc.php file, and this is not limited by Drush commands only.

// drush wipe $options['shell-aliases']['wipe'] = 'cache-clear all'; // drush pulldb $options['shell-aliases']['pulldb'] = '!git pull && drush updatedb'; // drush offline $options['shell-aliases']['offline'] = 'variable-set -y --exact maintenance_mode 1'; // drush online $options['shell-aliases']['online'] = 'variable-delete -y --exact maintenance_mode'; // drush unsuck $options['shell-aliases']['unsuck'] = 'pm-disable -y overlay, dashboard'; The lines starting with ! are not Drush commands, the lines without ! - are supposed to be Drush commands and the leading drush word is not needed in this case.

As it was shown, Drush allows to shorten the entered commands (executed at the command line) via different configurations, and to properly configure a Drupal site when you interact with it via Drush. All of these decrease an amount of routine actions and speed up the process of dealing with Drupal as a result.

Drush site aliases

Let’s expand on an interaction with Drupal via Drush. Usually, the interaction and an execution of some commands require that you have to navigate to the Drupal root directory first. If it’s a Drupal multi-site you need to navigate deeper to the site directory (sites/site-name), or explicitly set --uri (a site address when the site is opened in a browser) and --root (a path to the Drupal root directory) options when executing a Drush command. With the help of Drush site aliases all of these can be shortened to drush @example.com status with no difference from where the command is executed.

Drush site aliases can be defined in the file similar to drushrc.php, and Drush looks for this file in the same places it looks for drushrc.php. To define the site alias you just need to set uri and root properties.

// drush @example status $aliases['example'] = [ 'root' => '/var/www/example.dev/docroot', 'uri' => 'http://dev.example.com', ];

There are 3 ways to name the file where the site aliases are defined:

  • ALIASNAME.alias.drushrc.php - commands are executed like drush @ALIASNAME status;
  • aliases.drushrc.php - commands are executed like drush @ALIASNAME status;
  • GROUPNAME.aliases.drushrc.php - commands are executed like drush @GROUPNAME.ALIASNAME status.

The 3rd way is convenient for defining several instances / environments of the same site:

/** * @file * example.aliases.drushrc.php */ // drush @example.dev status $aliases['dev'] = [ 'root' => '/var/www/example.dev/docroot', 'uri' => 'http://dev.example.com', ]; // drush @example.stage status $aliases['stage'] = [ 'root' => '/var/www/example.stage/docroot', 'uri' => 'http://stage.example.com', ];

It's also possible to execute a command for all sites in a group simultaneously:

drush @example status

Actually, Drush allows to interact via the site aliases not with local sites only, but also with remote sites. In this case, you need to set a few more properties, at least remote-host and remote-user (but there are more options available).

$aliases['example.stage'] = [ 'remote-host' => 'server.example.com', 'remote-user' => 'www-admin', 'root' => '/var/www/example.stage/docroot', 'uri' => 'http://stage.example.com', ];

When you deal with the remote sites, you definitely want to avoid any mistakes: this can be done by adding a validation via Drush command hooks:

/** * @file * policy.drush.inc */ /** * Implements drush_hook_COMMAND_validate(). */ function drush_policy_sql_sync_validate($source = NULL, $destination = NULL) { // Deny sync to prod/production. if (preg_match("/^@.*prod/i", $destination)) { return drush_set_error('POLICY_DENY', dt('Nope.')); } }

Basically this is the same way how new Drush commands are added, so file naming is the same. In this particular case there is a convention to call this "Policy".

Drush contains a number of special commands for usage with the remote sites and the sites aliases.

drush core-rsync @site.remote:db.sql @site.local:backup/ drush core-rsync @site.remote:%files/ @site.local:%files/

- to rsync the Drupal tree to/from another server using ssh. %files is a special token pointing to a public file uploads directory, but there are more tokens available and custom ones may be defined via path-aliases site alias property.

drush sql-sync @site.remote @site.local

- to copy the database content from a source site to a target site, a database dump is transferred via rsync.

To connect to the Drupal site's server via SSH for an interactive session:

drush @site.remote site-ssh

or to run a shell command:

drush @site.remote site-ssh ls -la drush @site.remote core-execute ls -la drush @site.local core-execute ls -la Drush and Drupal 8

There are many changes in Drupal 8 in comparison with the previous version. When it comes to Drush, there are no big difference in dealing with a Drupal site. Probably the most noticeable change is that there is no more drush cache-clear all to flush all the caches. There is drush cache-rebuild instead. The another kinds of caches can still be cleared via drush cache-clear. An interaction with Drupal 8 via Drush requires Drush 8 version and newer.

Also there is few new Drush commands to interact with things specific to Drupal 8:

config-edit (cedit) config-export (cex) config-get (cget) ...

- to deal with configurations;

state-get (sget) state-set (sset) ...

- to deal with states.


Drush is a great tool which can help you with a Drupal site building, development and maintenance, and it noticeably speeds up these processes. The main part of  Drush is its commands, of course, but you can also tune Drush and Drupal (in context of interaction via Drush) for your own needs via configuration files, you can add custom commands like contrib modules often do, build a Drupal site codebase from a single file, deal with remote Drupal sites via Drush site aliases and more. Drush exists for a long time, but it's still a must-have: a very stable, well maintained and always evolving tool for Drupal. Stable and actively maintained Drush version for that moment is 8.x (you need this version or newer to deal with Drupal 8), Drush 9.x is in alpha and is being actively developed.

The observed Drush parts and processes are definitely going to cross your development and site-building path, so make sure to save and share this article with fellow Drupal developers. Good luck!


Categories: FLOSS Project Planets

Import Python: Awesome Django Admin

Planet Python - Mon, 2016-11-28 01:12
Just started with a curated list of awesome django resources aptly named Awesome Django Admin . If you have seen Awesome Python, it's on the same lines. I will be adding more to the list in the next couple of days. Feel free to send a pull request if you have any additions.
Categories: FLOSS Project Planets

Kushal Das: Updates from PyCon Pune

Planet Python - Sun, 2016-11-27 20:50

This will be the first year for PyCon Pune. This will give us a chance to meet our friends, discuss, and work on the language we all love :)

Event date: 16-19th February, 2017.

Location: Pune, India

Format: 2 days of main conference, 2 days of devsprints.

All Keynote speakers have been announced

We finally managed to announce all of our keynote speakers. I am putting up the names below. But you can learn more about them from the speakers page.

CFP is still open for two more days

We have our CFP still open for 2 more days, till 30th November, 2016. Feel free to submit any talk you think you want to present. The event is a single track event, means everyone will get a chance to see all the talks. This also means we have a tougher job in selecting talks :)

Registration is also open

The registration for the conference is also now open. As expected, the 4 days tickets (including devsprints) were sold out super fast. We still have main conference tickets left. Currently the only way to register for devsprints is through supporter ticket, which is also very limited in number. So remember to register fast :)

Categories: FLOSS Project Planets

PreviousNext: Presentation: At 16 years of age - does Drupal have an identity problem?

Planet Drupal - Sun, 2016-11-27 19:44

At Drupal South 2016, I attempted to summarize some of my pain points with Drupal core.

The session 'At 16 years of age, does Drupal have an identity problem' is the result of this.

You can watch the recording, or download the slides below.

Categories: FLOSS Project Planets

Desktops DevRoom @ FOSDEM 2017: you are still on time to submit a talk

Planet KDE - Sun, 2016-11-27 19:24

FOSDEM 2016 is going to be great (again!) and you still have the chance to be one of the stars.

Have you submitted your talk to the Desktops DevRoom yet?


Remember: we will only accept proposals until December 5th. After that, the Organization Team will get busy and vote and choose the talks.

Here is the full Call for Participation, in case you need to check the details on how to submit:

FOSDEM Desktops DevRoom 2017 Call for Participation

Topics include anything related to the Desktop: desktop environments, software development for desktop/cross-platform, applications, UI, etc

Categories: FLOSS Project Planets

Pau Garcia i Quiles: Desktops DevRoom @ FOSDEM 2017: you are still on time to submit a talk

Planet Debian - Sun, 2016-11-27 19:24

FOSDEM 2016 is going to be great (again!) and you still have the chance to be one of the stars.

Have you submitted your talk to the Desktops DevRoom yet?


Remember: we will only accept proposals until December 5th. After that, the Organization Team will get busy and vote and choose the talks.

Here is the full Call for Participation, in case you need to check the details on how to submit:

FOSDEM Desktops DevRoom 2017 Call for Participation

Topics include anything related to the Desktop: desktop environments, software development for desktop/cross-platform, applications, UI, etc

Categories: FLOSS Project Planets

Plasma 5.8.4, Applications 16.08.3 and Frameworks 5.28.0 available in Chakra

Planet KDE - Sun, 2016-11-27 16:43

This announcement is also available in Italian, Spanish and Taiwanese Mandarin.

As you have probably noticed, this move took a while to reach stable due to the issues with our main server, which resulted in a downtime of 2 days for our website and all the related services. There was nothing we could do, since our hosting provider experienced a major subsystem malfunction. The website might be a bit unstable or slow in the following days until the issue is properly fixed. We can only apologize for any inconvenience.

But the latest updates for KDE's Plasma, Applications and Frameworks series are now available to all Chakra users.

Plasma 5.8.4 includes three weeks worth of bugfixes and new translations, with changes mostly in the breeze theme, kwin and plasma workspace packages..

Applications 16.08.3 include more than 20 recorded bugfixes and improvements to ' kdepim, ark, okteta, umbrello, kmines, among others'. kdelibs was also updated to version 4.14.26.

Frameworks 5.28.0 include a new syntax-highlighting package, in addition to the usual bugfixes and improvements, mostly found in kio, plasma-framework, kwidgetsaddons and ktexteditor.

Other notable package upgrades and changes:


  • kirigami 1.1.0 has been added to the repos, a QtQuick based components set
  • openjdk 8.u112
  • cpupower 4.8.6
  • curl 7.51.0
  • dkms 2.3+git161025
  • eclipse-ecj 4.6.1
  • graphicsmagick 1.3.25
  • inetutils 1.9.4
  • libxi 1.7.8
  • ndiswrapper 1.61
  • net-tools 1.60.20160710git
  • pypy 5.6.0
  • rust 1.13.0
  • scons 2.5.1
  • sddm 0.14.0
  • tzdata 2016i

  • choqok 1.6.0
  • kdevelop 5.0.2
  • qtcreator 4.1.0

  • hugin 2016.2.0

  • wine 1.9.24
  • winetricks 20161107

    It should be safe to answer yes to any replacement question by Pacman. If in doubt or if you face another issue in relation to this update, please ask or report it on the related forum section.

    Most of our mirrors take 12-24h to synchronize, after which it should be safe to upgrade. To be sure, please use the mirror status page to check that your mirror synchronized with our main server after this announcement.
  • Categories: FLOSS Project Planets

    Dirk Eddelbuettel: anytime 0.1.1: More robust

    Planet Debian - Sun, 2016-11-27 16:09

    CRAN just accepted the newest release 0.1.1 of anytime, following the previous five releases since September.

    anytime is a very focussed package aiming to do just one thing really well: to convert anything in integer, numeric, character, factor, ordered, ... format to POSIXct (or Date) objects -- and to do so without requiring a format string.

    See the anytime page, or the GitHub README.md for a few examples, or just consider the following illustration:

    R> library(anytime) R> anytime("20161107 202122") ## all digits [1] "2016-11-07 20:21:22 CST" R> utctime("2016Nov07 202122") ## UTC parse example [1] "2016-11-07 14:21:22 CST" R>

    Release 0.1.1 robustifies two aspects. The 'digits only' input above extends what Boost Date_Time can parse and relies on simple-enough pre-processing. This operation is now more robust. We also ensure that input already of class Date is simply passed through by anydate() or utcdate(). Last but not least we added code coverage support, which oh-so-predictably lead us to game this metric to reach the elusive 100% coverage.

    The NEWS file summarises the release:

    Changes in anytime version 0.1.1 (2016-11-27)
    • Both anydate() and utcdate() no longer attempt to convert an input value that is already of type Date.

    • The string splitter (needed for the 'all-digits' formats extending Boost Date_time) is now more defensive about the input argument and more robust. Thanks to Bob Jansen for the heads-up (PR #30 closing issue #29).

    • Code coverage reporting has been added (PR #31).

    Courtesy of CRANberries, there is a comparison to the previous release. More information is on the anytime page.

    For questions or comments use the issue tracker off the GitHub repo.

    This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

    Categories: FLOSS Project Planets

    PythonClub - A Brazilian collaborative blog about Python: Debugging - logging

    Planet Python - Sun, 2016-11-27 14:48

    Achei algo interessante no livro que estou lendo (Automatize tarefas maçantes com Python) e resolvi compartilhar.

    Trata-se do Logging, que ajuda no debug do programa.

    Vejam o exemplo nesse programa, com falha:

    import logging logging.basicConfig(level=logging.DEBUG, format=' %(asctime)s - %(levelname)s - %(message)s') logging.debug('Start of program') def factorial(n): logging.debug('Start of factorial(%s%%)' % (n)) total = 1 for i in range(n+1): total *= i logging.debug('i is ' + str(i) + ', total is ' + str(total)) logging.debug('End of factorial(%s%%)' % (n)) return total print(factorial(5)) logging.debug('End of program')

    O programa retorna:

    2016-11-15 16:17:30,339 - DEBUG - Start of program 2016-11-15 16:17:30,340 - DEBUG - Start of factorial(5%) 2016-11-15 16:17:30,340 - DEBUG - i is 0, total is 0 2016-11-15 16:17:30,340 - DEBUG - i is 1, total is 0 2016-11-15 16:17:30,340 - DEBUG - i is 2, total is 0 2016-11-15 16:17:30,340 - DEBUG - i is 3, total is 0 2016-11-15 16:17:30,340 - DEBUG - i is 4, total is 0 2016-11-15 16:17:30,340 - DEBUG - i is 5, total is 0 2016-11-15 16:17:30,340 - DEBUG - End of factorial(5%) 2016-11-15 16:17:30,340 - DEBUG - End of program 0

    Dessa forma, podemos ver o passo a passo que o programa está realizando e identificar onde está o erro. No caso, vemos que para corrigir o problema, devemos alterar o for i in range(n+1): para for i in range(1, n+1):. Quando o desenvolvedor não quiser mais visualizar as mensagens de logging, basta chamar logging.disable(logging.CRITICAL) logo embaixo do import logging. Essa função faz com que não seja necessário alterar o programa removendo todas as chamadas de logging manualmente.

    Também é possível gravar as mensagens de log num arquivo, ao invés de mostrá-las na tela. A função aceita o argumento filename.

    import logging logging.basicConfig(filename='myProgramLog.txt', level=logging.DEBUG, format=' %(asctime)s - %(levelname)s - %(message)s')

    Lado negativo do uso dessa função: a leitura do código fica difícil, por causa desse monte de logging.debug no meio do código. Para evitar isso, pode-se usar um decorator.

    Categories: FLOSS Project Planets

    Marble Maps 1.0 has been released

    Planet KDE - Sun, 2016-11-27 14:20

    It’s finally done! I’m happy to tell you that Marble Maps version 1.0 has just landed in the Google Play Store (update: direct APK here if you are not using Google Play). We hope you like it as much as we do

    Many thanks to all contributors who made this possible. Thanks to a multitude of performance improvements all over the place, vector rendering has become very fast. And thanks to the ever-improving vector tile creation toolchain we are able to provide a lot more data than I anticipated some weeks ago. For the first version there are Germany and 200 cities world-wide in full detail, as well as most European countries and the USA in high detail (up to tile level 13 or 15). For the rest of the world we provide medium detail at least (up to tile level 9). The plan, of course, is to provide full vector data for the whole world in the near future.



    Categories: FLOSS Project Planets

    DataSmith: Inconsistent nofollow handling in Drupal input formats

    Planet Drupal - Sun, 2016-11-27 14:18
    Inconsistent nofollow handling in Drupal input formats The Problem

    Working on setting up commenting, which is highly suggested for sites who's content appears on Drupal Planet, I came across a bit of a confusing situation in regard to URLs in content. When using the "Limit allowed HTML tags and correct faulty HTML" filter, one of the option is to add rel="nofollow" attributes to anchor tags. However, in the default Plain Text format, the "Convert URLs into links" filter does not provide that option. So if a user types in an HTML anchor, nofollow gets added. But if they type in a plain URl, it gets converted to an HTML anchor without the nofollow.

    To illustrate, if I allow anchor links to be entered as html and set the option to add rel=nofollow and I also enable the filter to convert URLs to links, if a user enters:

    www.nytimes.com Another NY Times link

    The output HTML in the comment is:

    Another NY Times link

    For commenting, I really want to tighten permissions down as far as I can to avoid potential security risks, so the Plain Text format with the "Display any HTML as plain text" filter is the best choice1. However, for usability I do want URLs converted to links. But I also want those links set to nofollow for link fraud prevention2.

    By playing with format filter configurations and ordering I was able to make a solution that works (albeit a little janky-ly), but it sure feels like this is an area where a core patch could improve the situation. If I have time one day maybe I'll work on that3.


    The solution I came up with is to set the following filters on the input format (the order is significant):

    1. Display any HTML as plain text
    2. Convert URLs into links
    3. Convert line breaks into HTML (i.e.


    4. Limit allowed HTML tags and correct faulty HTML

    Then for the allowed HTML tags, I allowed <a href hreflang> <p> <br> and checked the `Add rel="nofollow" to all links` option.

    The result is that user entered HTML is rendered as plain text, then URLs and line brakes get converted to HTML, and finally the Limit allowed HTML filter double checks the markup and adds `rel="nofollow"` to anchor tags.  So given a user input comment like in the screen shot below, the resulting HTML is:

    <h2>This should not be displayed as an h2 element.</h2>
    <a href="If" rel="nofollow">www.example.com">If this is a link to example.com and not www.nytimes.com, you've failed.</a>
    Comment preview showing the user entered comment, the resulting comment, and the help text.

    Now, this solution is not perfect. Mostly, it's hinky to set up and I hate that I have to allow any HTML, even if user input is first stripped to plain text. Secondly though, it's also a user experience problem. As you can see in the picture above, the help text says that no html is allowed and that the anchor, break, and paragraph tags are allowed.


    1. using the core commenting facility at least. Add on tools like Disqus obviate the issue but I don't want to go that route. I also don't want to require (or even allow) users to register before commenting. And yes, I do require approval of comments before they are visible, but I don't want to have to remember to add rel=nofollow in links.

    2. Yes. I want to eat my cake and have it too.

    3. I was put on this earth to achieve certain things. At this point I'm so far behind I'll never die.

    Barrett Sun, 11/27/2016 - 13:18 Tags Add new comment
    Categories: FLOSS Project Planets

    Eriberto Mota: Debian with three monitors under low cost graphics interface

    Planet Debian - Sun, 2016-11-27 13:27

    Since 2008 I use two monitors in my desktop. Yesterday I bought a new graphics interface and a third monitor. Some time I was looking for a low cost graphics interface. Ok, I am using GeForce GT 740 which has three output ports: VGA, DVI and HDMI. In Brazil this interface card can be found around R$ 400 (US$ 117, but my card was US$ 87 in Brazilian Black Friday). In Amazon.com, it is between US$ 51 and US$ 109. The chosen manufacturer was Zotac, but all GT 740 and 750 will work fine (I tested the GT 750 too).

    The GeForce GT 740 was imediatelly recognised by Debian Jessie with kernel Linux 4.7.0 from Backports (it is my default, so I didn't test with original 3.16 kernel). The driver used was the default X.Org Nouveau. I use KDE and the management was easy.

    I hope this post can help people interested in use 3 monitors. Enjoy!


    Categories: FLOSS Project Planets

    Drupal Modules: The One Percent: Drupal Modules: The One Percent — Sticky Navigation (video tutorial)

    Planet Drupal - Sun, 2016-11-27 12:39
    Drupal Modules: The One Percent — Sticky Navigation (video tutorial) NonProfit Sun, 11/27/2016 - 11:39 6

    Here is where we look at Drupal modules running on less than 1% of reporting sites. Today we'll consider Sticky Navigation, a module which allows you to easily ensure your navigation remains within the viewport.

    Categories: FLOSS Project Planets
    Syndicate content