Feeds

part 1 part 2 part 3

Introduction

I'm going to be talking about None in a series of short articles. It is intended for less experienced developers. We're going to talk about some basic patterns that can clean up your code. Welcome!

If you're an experienced developer what I'm going to say is probably obvious to you. Nothing I'm going to say is particularly new or innovative, so this series of articles is probably not for you. But feel free to follow along and comment anyway!

While I'll focus on Python as an example language, most of this stuff is also applicable to many other programming languages as well. So if you use, say, JavaScript, don't go away, this may be useful to you too! Python is pretty easy to read so you should be able to follow along.

None

What is None? Python has a special value called None. None in Python is the standard sentinel, though of course other objects could also be used (and sometimes are). Other languages use NULL or null or nil; JavaScript confusingly has two values along these lines, null and undefined.

So what do we use None for? It turns out that when we have some value (attribute, return value, function argument), in many cases we want to be able to signal that the value is not there at all. In other words, the value is maybe there, and maybe not. To signal this we typically use None in Python.

If None is a possible value, it becomes important to make sure you handle the None case. Handling None is what this is all about.

Examples

Let's get a bit more concrete and give some simple examples of where None may come in.

A form in a user interface (for instance a web form) could have some fields that are not required to be filled in by the user. These empty fields can be represented as None by the application.

None is also often the default fallback value when no value can be found: the get method on the Python dictionary has such behavior.

Some functions have optional arguments. If they are not given, the value is some default. Often None is used for this default. The function's implementation can then detect this and deal with it accordingly.

A detailed example

Let's look at an example in more detail. This is a validation function:

def validate_end_date_later_than_start(start_date, end_date): if end_date <= start_date: raise ValidationError( "The end date should be later than the start date.")

The idea is that the function passes silently if the arguments (start_date and end_date) are valid, but will fail with a ValidationError if not.

If start_date and end_date may be omitted (for instance in a user interface), this ommission can be represented as None. In this case, the function may be called with either two dates as arguments, or None in either one or both of them.

But if the arguments can be None, the implementation of this validation function is buggy, because we've neglected to consider the None case for the arguments. If one of the arguments is None we'll see this:

TypeError: can't compare datetime.date to NoneType

That's not what we want!

So next we will talk about how to recognize None properly in the first place.

In response to Mark's post, Converting Views List Into Responsive Stacked Columns, we've teamed up to create a module for just this sort of thing. Views Responsive Grid is a views display format plugin designed to give you the proper HTML structure for creating CSS grids. Although still a sandbox project, we're working to get it through the application queue quickly.... Read more

There are several steps depending on how much involved the tests are, what parts are tested and, of course, how long it takes to run the tests.

For unit testing we use rspec puppet, and we can check that our manifests and modules compile and contain the expected values. It can be used to test that specific types, classes or definitions are in the compiled catalog and that the parameters math the expectations.

Later on we can do some integration testing starting a new VM with Vagrant and checking that there are no errors in the provisioning, as well as checking that some conditions are met.

For rspec-puppet, PuppetLabs has created a project called puppetlabs_spec_helper that let’s us avoid writing a bunch of boilerplate. A missing point though is that it only allows to use modules for testing from git. If you’re already using librarian-puppet (and you should!) you can easily use the same Puppetfile for deploying modules and to test them. Doing otherwise sounds like a bit of useless testing, you could end with different versions in different development machines, CI server, puppet master,… So just add a call to librarian puppet in your rakefile to populate the rspec-puppet fixtures before running the specs.

Unfortunately rspec-puppet doesn’t work with Puppet 3.0.x and  at least Puppet 3.1.0-rc1 is required. It was a bit of a setback when we moved to Puppet 3 and started using hiera, which is proving to be very useful to have simpler manifests and external data injected for our Maestro installations with Puppet from scratch.

You can also use the same Puppetfile to start Vagrant boxes with the exact same version of the modules. We are using Cucumber and Aruba to execute vagrant, provision the VM with puppet and check several things, like open ports, services up,… but that’s a different story

Example

In this puppet-for-java-devs project you will find the bits that showcase all these tools integrated. It includes definition of a 3-tier system with Puppet definitions for a postgresql database, tomcat nodes with a war installed and apache nodes fronting them.

Install all required gems

bundle install

Install all Puppet modules with Puppet Librarian

librarian-puppet install

Run the specs with puppet-rspec

bundle exec rake

Start all the vms with Vagrant

vagrant up Rakefile

require 'bundler' Bundler.require(:rake) require 'rake/clean' require 'puppetlabs_spec_helper/rake_tasks' CLEAN.include('modules', 'spec/fixtures/', 'doc') CLOBBER.include('.tmp', '.librarian') task :librarian_spec_prep do sh "librarian-puppet install" end task :spec_prep => :librarian_spec_prep task :default => [:spec]

Puppetfile for librarian-puppet

forge 'http://forge.puppetlabs.com' mod 'puppetlabs/java', '0.1.6' mod 'puppetlabs/apache', '0.4.0' mod 'inkling/postgresql', '0.2.0' mod 'puppetlabs/firewall', '0.0.4' mod 'tomcat', :git => 'https://github.com/carlossg/puppet-tomcat.git', :ref => 'centos' mod 'maestrodev/maven', '1.x' mod 'stahnma/epel', '0.0.2' mod 'maestrodev/avahi', '1.x' mod 'other', :path => 'mymodules/other'

tomcat_spec.rb with rspec-puppet

require 'spec_helper' describe 'tomcat1.acme.com' do let(:facts) { {:osfamily => 'RedHat', :operatingsystem => 'CentOS', :operatingsystemrelease => 6.3} } it { should contain_class('java').with_distribution /openjdk/ } it "configure webapp" do should contain_maven('/srv/tomcat/appfuse/webapps/ROOT.war') should contain_maven('/srv/tomcat/appfuse/webapps/ROOT/WEB-INF/lib/postgresql-9.1-901.jdbc4.jar') end end

Vagrantfile

Vagrant::Config.run do |config| config.vm.box = "CentOS-6.3-x86_64-minimal" config.vm.box_url = "https://dl.dropbox.com/u/7225008/Vagrant/CentOS-6.3-x86_64-minimal.box" config.vm.customize ["modifyvm", :id, "--rtcuseutc", "on"] # use UTC clock https://github.com/mitchellh/vagrant/issues/912 # db server config.vm.define :db do |config| config.vm.host_name = "db.acme.local" config.vm.customize ["modifyvm", :id, "--name", "db"] # name for VirtualBox GUI config.vm.forward_port 5432, 5432 config.vm.network :hostonly, "192.168.33.10" config.vm.provision :puppet do |puppet| puppet.module_path = "modules" puppet.manifest_file = "site.pp" end end # tomcat server config.vm.define :tomcat1 do |config| config.vm.host_name = "tomcat1.acme.local" config.vm.customize ["modifyvm", :id, "--name", "tomcat1"] # name for VirtualBox GUI config.vm.forward_port 8080, 8081 config.vm.network :hostonly, "192.168.33.11" config.vm.provision :puppet do |puppet| puppet.module_path = "modules" puppet.manifest_file = "site.pp" end end # web server config.vm.define :www do |config| config.vm.host_name = "www.acme.local" config.vm.customize ["modifyvm", :id, "--name", "www"] # name for VirtualBox GUI config.vm.forward_port 80, 8080 config.vm.network :hostonly, "192.168.33.12" config.vm.provision :puppet do |puppet| puppet.module_path = "modules" puppet.manifest_file = "site.pp" end end end

At the moment there are several Evennia projects churning along behind the scenes, none of which I've yet gotten to the point of pushing into a finished state.  Apart from bug fixes and other minor things happening, these are the main updates in the pipeline at the moment.
Multiple Characters per Player/SessionEvennia has for a long time enforced a clean separation between the Player and the Character. It's a much appreciated feature among our users. The Player is "you", the human playing the game. It knows your password, eventual user profile etc. The Character is your avatar in-game. This setup makes it easy for a Player to have many characters, and to "puppet" characters - all you need to do is "disconnect" the Player object from the Character object, then connect to another Character object (assuming you are allowed to puppet that object, obviously). So far so good. 
What Evennia currently doesn't support is being logged in with different client sessions to the same Player/account while puppeting multiple characters at the same time. Currently multiple client sessions may log into the same Player account, but they will then all just act as separate views of the same action (all will see the same output, you can send commands from each but they will end up with the same Character). 
Allowing each session to control a separate Character suggests changing the way the session is tracked by the player and Character. This turns out to be more work than I originally envisioned when seeing the feature request in the issue tracker. But if my plan works out it will indeed become quite easy to use Evennia to both allow multi-play or not as you please, without having to remember separate passwords for each Character/account.
Webserver change to Server levelEvennia consists of two main processes, the Portal and the Server. The details of those were covered in an earlier blog post here. Evennia comes with a Twisted-based webserver which is currently operating on the Portal level. This has the advantage of not being affected by Server-reboots. The drawback is however that being in a different process from the main Server, accessing the database and notably its server-side caches becomes a problem - changing the database from the Portal side does not automatically update the caches on the Server side, telling them that the database has changed. Also writing to the database from two processes may introduce race conditions. 
For our simple default setup (like a website just listing some database statistics) this is not a terrible problem, but as more users start to use Evennia, there is a growing interest in more advanced uses of the webserver. Several developers want to use the webserver to build game-related rich website experiences for their games - online character generation, tie-in forums and things like that. Out-of-sync caches then becomes a real concern. 
One way around this could be to implement a framework (such as memcached) for homogenizing caches across all Evennia processes. After lots of IRC discussions I'm going with what seems to be the more elegant and clean solution though - moving the webserver into the Server process altogether. The Portal side will thus only hold a web proxy and the webclient protocol. This way all database access will happen from the same process simplifying things a lot. It will make it much easier for users to use django to create rich web experiences without having to worry about pesky behind the scenes things like caches and the like. 
Out-of-band communication This has been "brewing" for quite some time, I've been strangely unmotivated to finalize it. Out of band communication means the MUD client can send and receive data to/from the server directly, without the player having to necessesarily enter an active command or see any immediate effect. This could be things like updating a health bar in a client-side GUI, redirect text to a specific client window but also potentially more advanced stuff. I created the Evennia-side oob-handler over Christmas; it allows for client sessions to "sign up" for "listening" to attribute updates, do scheduled checks and so on. It's already in the codebase but is not activated nor tested yet.

On the protocol side (for serializing data to the client) I have a MSDP implementation ready for telnet subnegotiation, it should be simple to add also GMCP once everything is tested. A JSON-based side channel for the webclient is already in place since a long time if I remember correctly, it just need to be connected to the server-side oob-handler once that's finished. 

Working with Lightbox2 and Colorbox

In this series, Lightboxes and Drupal 7, Kyle Hofmeyer walks you through working with two popular lightbox modules in Drupal 7: Lightbox2 and Colorbox. The first video in the series is free, and explains what a lightbox is, and some things to consider when selecting which module to use.

Finally, phpMyAdmin packages for Debian and Ubuntu do not lag much behind upstream. Today, I've prepared packages for yesterday released bug fix release 3.5.6.

For Debian users, the package should be soon available in experimental (sorry no uploads to unstable during freeze).

Ubuntu users, can use my phpMyAdmin PPA. After dozens of comments and no help offered, I've still decided to be nice to Ubuntu users and adjusted the package so that it should work on Lucid as well. The downside is that, unlike in Debian, the package includes bundled copy of many PHP and javascript libraries.

PS: As soon as Debian is not frozen or 4.0 is officially released, I will start uploading 4.0 to experimental. My current bet is that 4.0 release will come earlier.

Filed under: Debian English Phpmyadmin | 0 comments | Flattr this!

Sebastian wrote a pair of blog entries in the last week about where we are heading with Plasma in the near future. The first was an overview of the pathway to Frameworks 5 and what we're provisionally referring to as Plasma Workspaces 2. The second entry covered his work on making it possible to write widget layouts (aka Containments) in QML.

For me, the key sentence in that entry was this:
The work that is upcoming in Plasma Desktop is further bridging the gap between Plasma’s interfaces for different devices and formfactors. I'd like to share a little bit more about what this means long-term, and share a few more nuggets of where we are headed with the various Plasma workspaces in the process.
One Shell To Bind ThemIn the early days of Plasma, the idea was to write one binary per form factor and share components between them. This would allow us to sculpt the overall interaction pattern of the shell while sharing the bulk of actual code, particularly the code that took the most effort such as network management. The shells take care of creating and managing the windows that make up the shell: panels, desktop layers, configuration dialogs and controls.
The desktop shell has around 9,300 lines of code, the netbook shell has under 3,000 and plasma-device (used in Plasma Active) has a little under 4000. Given that the rest of the Plasma code base ends up closer to a quarter million lines, this has been a pretty effective strategy.
There a couple limitations here, however. First, you need to write a shell if you want to approach a completely different type of device or interactive pattern. Experience shows that it's only 3-10 thousand lines to do so, but it would be nicer if that number was closer to zero.
Secondly, we've offered the ability for some time to switch at runtime between the Netbook and Desktop form factors. This is a little messy in that we have to stop one process and start the other. Not particularly pretty. With Plasma Active, we also started really fleshing out how much could be done if we also tweaked the settings of the window manager as well as various applications. What we'd far prefer is to be able to change the workspace layout entirely, including the bits that the shells have been doing up until now, without restarting processes and also alter other runtime settings such as changing the window decorations (or even removing them entirely in the case of Plasma Active).
We also learned in Plasma Active that we could write a rather generic shell and populate it entirely with QML. Sebastian's efforts bring those lessons deep into the core of Plasma. So in Plasma Workspaces 2, we want to provide a single shell binary with desktop, netbook, tablet and other form factors all being serviced by it. This will also give us the infrastructure to easily and elegantly switch between interaction concepts, e.g. between a mouse+keyboard-centric Desktop to a touch driven tablet experience. These transitions could be triggered by hardware events (docking, screen connected via HDMI, etc) or user configuration.
The result is that we'll share even more code between the different workspaces, be able to target new form factors even faster and produce a more consistent experience with cutting edge features.
One use case I'm personally looking forward to is plugging my laptop into the T.V. and having Plasma Desktop be replaced with Plasma Mediacenter automatically.Plasma FrameworkIn coordination with the efforts to turn KDE's libraries into the highly componentized KDE Frameworks, Plasma's libraries and runtime parts are going to go into their own self-contained repository as well that follows the Frameworks policies. This means that all the components that are currently in kde-runtime will be moving into the same repository as libplasma or into kde-workspace, depending on where they best belong.
This isn't just an exercise in practicing how to use the "mv" command, however. The biggest shift here is that all the user interface bits are going QML. In libplasma2 there is no more QGraphicsView support. While DataEngines, Services, Runners and other data-centric components will continue to be written in C++, Javascript, Ruby or Python, the UI will all be done in QML. This is why we've been porting components at such a pace to QML.
In KDE Workspaces 4.11 we'll have some of the remaining big pieces in place: the window list (tasks) widget, the containments, krunner, etc. So when we make the move to libplasma2 as part of Frameworks 5, we'll be ready for the move away from QGraphicsView and into QML scene graph without hiccups for the user or our development teams. Yes, this means that the next major version of Plasma is going to be somewhat unexciting. We can't promise the fireworks of the 4.0 release, and I'm sure you're all disappointed by that news. ;)
For those who have OpenGL capable hardware and reasonable drivers, this will give them a fully hardware accelerated desktop. In such environments, we'll also have the ability to use fancy things like OpenGL shaders to achieve visual effects and usability concepts we can't do right now.
Oh, and the library is now a fraction of its previous size. More with less.Plasma Active 5We're hoping to make Plasma Active 4 the last release based on KDE Platform 4. This is a very important release, if only because it's going to be the version that ships on the Vivaldi tablet. However, we're also looking forward here too.
There are still variables in the equation, but our current goal is to make Plasma Active 5 a slightly longer than usual (for Plasma Active) development cycle and re-base on KDE Frameworks 5. This means Qt 5 and QML Scene Graph. It also means the possibility of using Wayland on devices. The KWin hackers have been insanely busy working on the changes needed to make this possible, doubly so since Martin was recently sponsored to work on KWin full time.
If things go according to plan, we'll be able to use the improved QML shell with libplasma2 and provide a very modern experience in terms of performance and interaction. Things like per-window transformations (important for supporting device rotation, among other things) become within our reach.
We also want to move Plasma Active's code closer to the Plasma Workspaces repositories. In fact, we'd like to create one Workspaces area on projects.kde.org under which the code in kde-workspaces, kdeplasma-addons, the left-overs from kde-runtime/plasma, networkmanagement, bluedevil and new repositories such as kscreen can all come together .. without coming together in one git repository. This will make building everything easier (if you use kdesrc-build, at least) while making it easier for us to manage the various assets.
Of course, it isn't all under the hood work.Expanding on the Active experienceSomething we've striven to do with Plasma Active is create an environment that is productive. Right out of the box you have eBook readers (and books!), office document viewers, a file manager (and even a terminal ;), data sync and much more. Combined with Activities, Plasma Active is really about getting useful things done (where you define what "useful" means) with your device.
We've been working lately on extending that from consumption to production. If you noticed, most of the items in the last paragraph were about using data: viewing, sharing, organizing. What about manipulating and creating? We really would like to make it dead simple to do things like take a photo, touch it up a bit and share it without requiring an Instagram in the way. We'd like to make it as easy to open a spreadsheet and view it as it is to create a new one.
However, in Plasma Active we've made two purposeful decisions: do not expose the hierarchical file system (unless the use case dictates that as a requirement) and do not expose details that are not relevant to the usage of the device (e.g. I care that I can open that spreadsheet, it's less important at that moment that the application is Calligra Sheets). Thus to open a spreadsheet one opens the file manager and goes to Documents, or simply searches for the document from the Launch area directly. No file system, no application launchers.
Creation should be just as straight forward. So we will be providing an interface that focuses on creating new things, just as the Files app focuses on managing existing things. In this model, you will simply tell the device that you'd like to start making a spreadsheet. Oh, and that you'd like to email it to your colleagues when your done.
Combined with an expanded Share Like Connect system and data sync services, publish and subscribe for activities and various other tweaks we have in the pipeline, 2013 will be a really great time for Plasma Active.The Desktop too!This will also be a time of significant upgrades to the desktop as well. The QML work is one major thing, but  Share Like Connect will also be featured in the desktop along with improved activity management and widget (Plasmoid) handling.
We have a steady stream of improvements coming starting with 4.10's QML containment support, continuing on to things like kscreen so we have a first-rate hotplug-driven and low-to-no-configuration screen management system (hopefully included with 4.11; many of us are already using it now!) and the bigger UI work that will come with the move to libplasma2 and Frameworks 5.
The desktop changes will not be very radical, but then our focus is to improve performance and expose features like Activities more effectively while also tieing into the multi-form-factor-but-one-shell idea.Ok, we get it: you're doing lots of stuff.I could continue to write about what we're up to right now and what that leads to next .. but I fear this is already long enough. I hope, along with Sebastian's and Marco's posts, it gives everyone a slightly clearer idea of where we're heading. We have a lot of work to do to get where we want to go, but we've already accomplished so much in the last few years that we know it is not only possible, but that we have the people who can make it happen.
Of course, one of those people could also be you. :)

For contributing to the Drupal community by developing your own module or theme you first have to contribute "a useful and working module or theme as a sandbox project, and go through a one-time approval process to get permission to promote it (and future projects) to a full project".

Weiterlesen

At the end of March, we will be releasing Plasma Active 4. Since Plasma Active 3, We've made improvements to the Files, eBook reader (Okular), Settings and Alarms applications along with a large number of bug fixes and performance improvements. We're also in the middle of moving to the KDE Platform 4.10 release as well as getting closer in line with other Mer based efforts, such as sharing the description files with Nemo that are used when building images for different device targets and adapting to a systemd driven user session. These two changes introduced a relatively large number of regressions that are being ironed out. In fact, we paused on the feature development and turned focus to fit and finish work before continuing on.

Marco posted a really nice blog entry on how you can help Plasma Active 4 shine. There are many small and large things yet to do, so there are tasks that should fit just about everyone. The easiest way to find these tasks is to start using Plasma Active and find what you feel is missing. One developer did exactly that, appearing on the mailing list with a KSnapshot interface written in QML. We're around most of the time online helping others find their feet and start working on the code, so don't hesitate to visit us and ask questions or share ideas. You can find us on irc.freenode.net in #active and #plasma, the active at kde.org and plasma-devel mailing lists and even in the new Google+ Plasma Active community.

Which brings me to a very nice anecdote: A couple of people have been working on bringing Plasma Active to the Nexus 7 device. I noted this on the Google+ community page and a person responded saying that if there was documentation on how to have both Android and Plasma Active on the device that would make it a lot more attractive to them. Alan Pope from Canonical showed up and shared a link to a page in the Ubuntu wiki showing how this can be done. I shared that on irc with Ruediger Gad, who jumped on Google+ to respond directly saying that they will add this to the documentation when he has a chance. At which point Bill Newton popped up saying he has dual-booting working and by that evening he had updated the wiki showing how to accomplish this. I love that kind of dynamic in the Free software world: from problem to solution in a span of a day with the help of multiple hands.

Tuesday, 29th January 2013drop.coop podcast episode 1 - Drupal Drop In Sprint London Saturday 26th January 2013

drop coop podcast episode 1 - Drupal Drop In Sprint London Saturday 26th January 2013 from Steve Purkiss on Vimeo.

Audio & Video tracks  also available on Archive.org

drop.coop podcast episode 1 - Drupal Drop In Sprint London

 

1. Robert Castelo
   • History of Drupal London group
   • Drupal Drop In Sprints
   • Drupal maturing as a product
   • Global Sprint Weekend 9/10 March inc. London & Brighton
   • Drupal for NGOs
   • Core Office Hours
   • DrupalCon Prague
2. Stefan van Hooft
   • Drupal Somerset
3. Pedro Cambra
   • Commerce Guys
   • Drupal Association
   • DevDays Dublin in June
   • CxO Barcelona in March
   • FrontEnd United in London
4. Steve Parks
   • Wunderkraut & Wunderroot
   • Drupal Radar
   • High Profile UK Sites this year!
5. Leon Tong
   • DrupalCamp London in March - a marriage made in…
   • drop.coop - Europe vs. U.S. ways of doing business

tags: Drupal PlanetPlanet Drupalcooperativesprintlondoninterviewsdrupalcondrupalcamp

While digiKam is first and foremost an application for processing and organizing digital photos, it also features tools for working with film negatives.

Continue to read

The Shield module is a quick and simple solution for locking down websites with an Apache authentication username and password. It requires no changes to the .htaccess file (.htaccess modification is required if running on CGI or FastCGI) or configuration to an Apache vhost. Just install the module and configure it from Drupal's administration section. This module is very useful for locking down development and staging versions of a website.

In this short video, we'll cover the following:

• How to configure Shield
• Demonstration on how to use the module

Episode Number:  96

Spam is always something you need to consider when developing a Drupal website. One tool that makes spam detection and prevention extremely easy is the Drupal 7 Mollom module. This module uses the Mollom service to give your Drupal website capabilities to add captchas to forms, or even analyze text for content that looks like Spam. The service has a completely free version so you can easily get started preventing Spam on your Drupal 7 website.

In this episode you will learn:

DDoD Video: 

read more

Indeed, two days was enough time to recover from an afternoon of social, despite having an hour and a half of meetings today. But today was a day of catching up, sorting issues in JIRA, doing lots of planning meetings, and doing some debugging, so I don't have anything new code-wise to give to the world. So have a photograph.

The product backlog is now all sorted out for the Account Services project into the new phases that will define the work through pretty much the end of the calendar year, I'm guessing. And everything is now a story, not a bug or enhancement or something else without story points, and a few things are shuffled into a more reasonable order.

I'm probably going to find a few hours this week to go play video games or work on personal projects, given that I've been working well over 40 hours a week since the start of the year.

Testing network applications is hard: the order of events is unpredictable, the passage of time is important, and the sources of errors are many. At PyCon this year I will be teaching a three-hour tutorial on test-driven development with Twisted, demonstrating how to build well-tested network applications.

(As a reminder, I'm also teaching a two day Twisted class in San Francisco with Jean-Paul Calderone, on the Monday and Tuesday before PyCon; early bird pricing expires Feb 15th. The material in the testing tutorial is also included in the longer class.)

In the lecture part of the tutorial I will cover:

• Testing network protocols in a deterministic manner (no need for actual TCP connections).
• Testing the passage of time (no need to wait 2 hours in your test to prove that a timeout is hit).
• Twisted's testing infrastructure for running the reactor and handling Deferreds.

Once that is done, students will begin a hands-on lab, implementing a HTTP server from scratch. I will provide pre-written unit tests, and students will write code to make these tests pass, with help from me (and perhaps an assistant or two depending on number of students).
The lessons here are implicit in the design of the tests, and the design of the server as shaped by the tests. If anything these lessons are more important than understanding Twisted's testing APIs:

• The scoping of tests into small units of work.
• Separation of concerns - parsing/generating bytes vs. business logic.
• Design patterns for Deferred APIs.
• Building robust network applications, including dealing with bad input and timeouts.
• Separation of library code and application configuration.

Students who finish early can move on to a more difficult exercise, implementing both the tests and logic for an HTTP client, but benefiting from the ability to ask for in-person help.
You can sign up at the PyCon website as part of registration, or read more on the tutorial's PyCon web page.

Let's send an email! Here's the steps involved in what from the outside looks like a simple function call (and this is of course a very high-level view):

1. Look up the IP of our SMTP server's domain using DNS. This may involve a series of UDP messages to one or more servers, which may do further work on our behalf.
2. Establish a TCP connection with the server.
3. Exchange a series of commands with the server over the TCP connection, some of which may involve arbitrarily complex processing on the server-side.

Obviously a lot can go wrong here, from communication problems to hardware failures to software bugs. The fact it usually works is an impressive engineering feat. For our purposes the interesting point is that failure can take an arbitrary amount of time.

The necessary and obvious solution is a timeout: if enough time has passed without getting a response, abort the operation and consider it to have failed. We may end up sending duplicate emails if we retry, but that is a business logic decision tied to the specifics of an application, so not something I'll be talking about. Now, we could have timeouts on each step of the process (DNS lookup, TCP connection, each command), and in fact may want timeouts for each of these. But from the point of view of the email sending API, the time it takes to do the underlying steps is irrelevant, except perhaps for debugging or performance: if we want to send an email within 5 seconds, we want it to take 5 seconds, and don't care which step happens to be the slow one.

This is where Deferred cancellation comes in. We want to make sure each step along the way has a cancellation function registered, if possible, but that's not strictly necessary. Our code looks something like this:
def sendmail(from, to, data, smtphost, smtpport=25):
endpoint = TCPv4ClientEndpoint(smtphost, smtpport)
d = endpoint.connect(SMTPFactory())
def gotProtocol(smtpProtocol)
return smtpProtocol.send(from, to, data)
d.addCallback(gotProtocol)
return d
The wonderful thing about Deferreds is that, as with results, cancellation also gets automatically chained. Thus if we call cancel() on the result of sendmail(), it will cancel the Deferred connecting to the server if that's where we are in the process, or the Deferred return from SMTPProtocol.send() if that's what we're waiting for. So if want to time out sending an email after 5 seconds... all we have to do is cancel the Deferred returned by the sendmail() function after 5 seconds if we haven't gotten a result! The following utility function, soon to be part of Twisted (ticket #5786), does just that:
def timeoutDeferred(deferred, timeout):
delayedCall = reactor.callLater(timeout, deferred.cancel)
def gotResult(result):
if delayedCall.active():
delayedCall.cancel()
return result
deferred.addBoth(gotResult)
And now, we can send an email with a timeout of our choice, e.g. 5 seconds:
sent = sendmail("from@example.net", "to@example.net",
"An email message.", "smtp.example.net")
timeoutDeferred(sent, 5)
The nice thing about this API is that it doesn't require adding extra timeout arguments to every function. Instead, the highest-level caller just adds a timeout. And underlying callers (e.g. the TCP connect, its underlying DNS lookup, etc.) can have their own, more limited timeouts as well.

To summarize: supporting Deferred cancellation is a great way to make the integration points of your library code more useful, by allowing users of your code both ad-hoc and timeout-driven cancellation of your operations. And as the user of a Twisted library, timeouts can be easily added to any Deferred-returning API, in particular those that explicitly support cancellation for you.

Quick link: Proposal: Drupal PCI Compliance White Paper (the actual proposal).

In refererence to the loss of undocumented wisdom, Andrew Carnegie once stated that “it was one of the sins of the ages that this knowledge, gained at such a tremendous price, by so many men, was buried with their bones when they died. Nobody had ever organized it into a philosophy and made it available to the man of the street.”

I feel the exact same way when I think about the number Drupal developers that have suffered through the long, hard journey of achieving and maintaining PCI compliance for ecommerce websites. With over 67,000+ active Ubercart and Drupal Commerce websites (as reported by Drupal.org), one might assume there would be an abundance of quality resources out there (articles, blog posts, youtube videos, etc) to help others speed through this torturous learning curve.

Unfortunately, I didn't believe such a resource existed that was specifically tailored to the Drupal community. This inspired me to write a somehwat lengthy article to start that conversation. It touched on a lot of the major pain points, provided the pros and cons of each solution, and it also contained an aggregated list of resources that I had found over the last several years. The feedback was incredible. It had clearly struck a chord with a lot of developers and sparked a conversation.

The Next Step

In that article I had pitched the idea of taking it a step further and creating a white paper that was similar in form the Drupal Security Report. My intention was to go beyond a simple article and create something more definitive. I want to create a quality document that any developer or evaluator could read in a single sitting and get a solid high level overview of the issues at hand. And if it helps the next 67,000+ Drupal e-commerce sites achieve and maintain their PCI compliance, I’ll consider this a huge success!

The Proposal

Without further ado, you can access the proposal document here, which is the domain that will ultimately host this document as well as any future versions. My goal is to have this completed sometime before DrupalCon Portland, where I’m also submitting a session to give a talk on this same subject matter.

The Ask

A technical document of this type can take a lot of time to create and a lot of extra help to review it to ensure that all the technical information is sourced and as accurate as possible. Therefore you’ll see that I’m seeking volunteers (to help with the feedback/review process) and a modest level of sponsorship (to help move this project alone).

And as always, feel free to leave a (helpful) comment/suggestion below!

Tags: Drupal PlanetDrupalPCI compliance var switchTo5x = true;stLight.options({"publisher":"dr-8050e384-1b1-b22e-b17-7e272ca1dc8f"});

View the discussion thread.

I saw this lovely drawing over at Máirín Duffy's blog.  She works at RedHat.


Its from a post called FUDcon Lawrence: Overhauling the Fedora release model.  Note the little jumping person and the gap.  This indicates that Fedora understands that there is a barrier between using Fedora distro of GNU/Linux and contributing to it.  This is something they hope to address.

If I replace "Fedora" with "GNU" in the above picture, what happens?

• Hear about GNU
• Go to GNU website
• Download GNU
• Try out live media
• Install GNU
• Use GNU
• ???
• Contribute

Let's try it out and see what happens.

Step 1 - Hear about GNU.  Not a problem.  GNU gets mentioned fairly regularly in the press.

Step 2 - Go to GNU website.  There is a web site, www.gnu.org.  The main page is a bit cluttered, but, it exists.  There is a www.gnu.com, which is a snowboard company, but, there's little chance of confusing the two.

Step 3 - Download GNU.  From the main page, there is "What is GNU" and a big green "Download GNU Now" button.  Visually it can get lost because the page is very text-heavy and the button it is almost too big.  But overall, it is functional.  It takes you to the distros page.

This is where is gets more problematic.  First off, there is the big yellow italicized disclaimer "The FSF is not responsible for other web sites, or how up-to-date their information is".  It is highlighted so it must be important.  It sounds scary.  But what does it mean?  Up to this point, the term "FSF" has not been defined and its relationship to GNU has not been defined.

Also, there is no real distinction made between these distros or why or how one would use them.  Since they are treated equally, the first one, "BLAG" becomes the de-facto distro by virtue of its early appearance in alphabetical order.  So clicking on that takes us over to BLAG.
At the very top of Blag is a download link to Live CDs, and clicking on that link brings to an FTP server
Note that we've now implicitly answered the question of What is GNU OS?  GNU OS is Blag's 2011 fully free respin of Fedora's distro of GNU/Linux.

Step 4 - Try out live media.  So I've downloaded the live media.

Cool, but, how do I use it?  What do I do with it?  If I go back to the main Blag page and click on the documentation link, I get
This doesn't look good.  But if I allow the exception I end up at a perfectly respectable wiki.




Step 5 - Install GNU OS.  Installing GNU from live media become the same as installing Fedora from live media the way it was done back in 2011.

Step 6 - Use GNU OS.  Cool.  We're using GNU.

If I go back to the main GNU page to look for information on how to use the distro I've just downloaded, I get nothing general.  There is the documentation to many pieces of software that may or may not appear in that distro, though.

Step 7 - Contribute. 

On their part, Fedora knows that there is a big leap between using Fedora and contributing to Fedora. For GNU, it isn't just a leap from using GNU OS to contributing to GNU OS.  It is a complete disconnect.

Once someone has downloaded the Blag respin of the Fedora distro of GNU/Linux, what force is there to push them back to GNU the website, or the official GNU software projects?  What force is there to push them to contribute financially to the FSF?

Once someone had made the decision that he or she wants to contribute, he or she can end up back on gnu.org and see this.

Note the categories listed under the Take Action box or the yellow bar:  you can contribute by

• working on the FSF campaigns
• contributing to high-priority software projects
• or maintaining unmaintained software

Notice how everything on that list has either a very low or  a very high level of technical expertise required.  There isn't a good story on how to train up a programmer on the ways of free software.  It is a bit of all-or-nothing. Suppose you wanted to help with one of the technical projects: there isn't really a good tutorial story on learning how to code GNU-style.  There are projects like GNU Hello and documents like the Coding Standards and the Information for Maintainers that sketch out the concepts, but, you would have to know that they are there and where to find them, and the barrier to entry is pretty high.

So, all in all, the flow of moving people from curiosity about GNU OS to using GNU OS to eventually contributing to GNU needs work.

DISCLAIMER: I don't speak for GNU and have no influence over its policies, or management. I am only just barely a member of GNU by virtue of being the maintainer of the ncurses bindings for guile. Guile-ncurses is a project that is so niche and so obscure that it has never been packaged by any distro and never has been downloaded or used by anyone ever, other than me, as far as I can tell. I also used to be a committer on the main Guile project, during the 1.8 to 2.0 transition, but, I've retired from Guile development, mostly.

As many other components of the Plasma Workspaces, Plasma Desktop’s default Containment is being ported to QML. A technology preview of the containment is being demoed and can be tested by a wider audience now. While the port is mainly replicating the current desktop containment in QML, its interaction scheme to position and manipulate widgets on the desktop has been improved.

First of all, a note: The code presented in this article is not part of the upcoming Plasma Desktop 4.10. It can easily be installed on top of it, it might see inclusion in the summer 2013 release, however

In our Roadmap, you learn that KDE is currently porting many aspects of its workspaces to QML, with the goal to create a more modern user experience on top of state-of-the-art technologies such as Qt5, OpenGL scenegraph and Wayland. The move to QML is a gradual process, made possible by the componentized architecture of Plasma. Widgets and other components that make up the workspace are replaced with QML ports one by one. The premise is to not introduce regressions by shipping each component “when it’s ready”. Ultimately, we need a complete set of QML components to run the whole desktop (and at some point also apps) directly on top of the graphics hardware, leading to higher graphics performance and more available resources on the CPU.
One of the important pieces is the Desktop containment. This is the component in Plasma that is responsible for managing and laying out widgets on the desktop and creating the toolbox (which makes some “workspace actions” available to the user. In general, a “Containment” is an area on the screen (a panel, the desktop background, the dashboard, …), and it takes care of managing widgets, their position and sizing within. It also offers access to action specific to widgets, or the containment or workspace.
The currently shipped (also in 4.10) default Desktop containment is written in C++, using QGraphicsWidgets and offers free placing of widgets on the desktop, with a bit of edge and overlap detection and repositioning poured in.

Demo movie of the new QML Desktop containment

User interaction improvements

Most of the new containment is exactly the same as in the current default — this is done by design, we do not want to introduce radical changes to the workspace (and the users’ workflows), but rather improve gradually and in an iterative process. There are two areas (which in fact are closely related) where we did change a few things: positioning/sizing and visual cleanliness. These are expressed in two changes: integration of applet handle and positioning aids.
In order to reduce visual clutter, we integrated the applet handle into the applet’s background frame. Previously, it would be its own frame, and shift out as separate element from under the widget. Merging handle and background frame reduces the number of distinct elements on the screen and allows less intrusive transitions when the widget handle becomes visible.
The second important change is that we now provide helpers when the user moves and resizes a widget. When moving, we show a halo at the position the applet will snap to when dragged. This makes widget placement more predictable and allows the user to get it right in one go. We also align the widgets to an invisible grid, so applets automatically end up being aligned pixel-perfectly with each other, which leads to a more ergonomic workflow, cleaner appearance of the workspace, and again to less visual clutter.

Platform improvements: Bindings and Toolbox

An important aspect of the work on the QML containment, was to improve the bindings which load declarative code (QML) into Plasma shells, these improvements are included in Plasma 4.10, due to be released in early february. This includes the necessary platform features to allow running fully-featured QML containments, something which we have done in Plasma Active for a while, but within a more confined context.

As a result of this work, Plasma can now also load toolboxes written in QML. The Plasma Toolbox is the little widget with the Plasma icon you can see on top of many containments, and which gives access to actions such as add widgets, settings, shortcuts, etc.. The toolbox used with the containment shown is a 1:1 port of its counterpart in the current default (C++) toolbox. The name of the toolbox package is currently hard-coded in the bindings (it loads it from the org.kde.toolbox package and silently falls back to the C++ implementation if that isn’t found — a 4.10 feature), but it also opens up this part of the workspace to QtQuick goodness. The toolbox is basically a translucent layer on top of the desktop, so much freedom is given to other implementations).

A template and a bridge

The code is not only there to replace the current containment, it also serves as a template for new developments. With the new containment bindings in place, it is now very easy to create your own containment, modify someone else’s and publish them to share them. The containment shown is just one example for what we can do with the QML integration features in Plasma. As Plasmoid packages are architecture independent, this of course works across devices and different workspaces.

The work that is upcoming in Plasma Desktop is further bridging the gap between Plasma’s interfaces for different devices and formfactors. Some of its code has been introduced in Plasma Active, and is now available in a more generic fashion also for Plasma Desktop (and Netbook). This brings us closer to one of our goals, having only one shell that dynamically loads a suitable interface (Containment, Widgets) for a given formfactor, use case, output device, etc.

Give it a spin

If you’re interested and would like to try it (we appreciate feedback, it’s especially valuable in this phase!), there are two ways to get this containment. The minimal requirement for it is Plasma 4.10-beta1.
If you’re using git, you will find the code in the branch called “plasma/sebas/desktop-qml”, just check it out and build it, install it, run kbuildsycoca4, and you’re done.
If you are using the packages, you can easily install the following two Plasmoid packages to your system:

• desktop-git28012013.plasmoid
• toolbox-git28012013.plasmoid

If your system is using a version prior to KDE SC 4.10-beta1, the packages will install, but not work.

The following commands install the necessary Plasma packages into your home KDE install directory.

# create the package directory and go there mkdir -p `kde4-config --prefix`/share/apps/plasma/packages/org.kde.toolbox cd `kde4-config --prefix`/share/apps/plasma/packages/org.kde.toolbox # unpack the plasmoid package unzip ~/path/to/toolbox-git28012013.plasmoid # check if it's installed correctly, # this should list metadata.desktop and contents/ ls -la `kde4-config --prefix`/share/apps/plasma/packages/org.kde.toolbox

[Edit: changes --localprefix to --prefix, as we've found a bug in --localprefix code.]
Then install the desktop containment package (If you’re updating the containment at a later stage, use plasmapkg -u.):

plasmapkg -i desktop-git28012013.plasmoid

You can now choose the new containment from Layout dropdown in the Desktop Settings, pick “Default Desktop (QML)” there.

I would like to thank Blue Systems for supporting my work on the above topics.

RMS was at the "II Fórum Mundial de Educação Profissional e Tenológica: Democratização, Emancipaçao e Sustentabilidade" in Florianopolis, to deliver his speech "Free Software and Your Freedom," on May 30th, and, on the 31st, to speak on the "The Free Software Movement: Appropriation, Designing and Use" panel, in front of about 500 students, professors, researchers, government workers, and union members. His message was simple: "For ethical education, the software used must be livre, and the teaching materials must be livre too."

(Photos under CC BY-SA 3.0 and courtesy of Marcela Lin.)

RMS was in Porto Alegre, Rio Grande do Sul, Brazil, on June 4th, for a celebration to mark the first anniversary of the Digital Office of the State Government, the 40th of the Companhia de Processamento de Dados do RS, the 25th of the Science and Technology Department of Rio Grande do Sul, and 70th of the Science and Technology Foundation. He delivered his speech "A Free Digital Society," at the Palácio Piratini, the gubernatorial palace, to a room of about 200 students, bloggers, journalists, government and civil-society representatives, and about 500 more people who tuned in over a live stream.

(Photos under CC BY-SA 3.0 and courtesy of the Gabinete Digital.)

Many thanks to Prof. Golberi de Salvador Ferreira, in Florianopolis, and to Fabricio Solagna and Lincoln de Sousa, in Porto Alegre, for helping make this visit possible!

Please see www.fsf.org/events for a full list of all of RMS's confirmed engagements, and contact rms-assist@gnu.org if you'd like him to come speak.

Please fill out our contact form, so that we can inform you about future events in and around Campinas, Vila Velha, and Pirai do Sul, all of which RMS visited on his last trips.