Feeds

The second talk I went to on the first day was on the influence of TDD on software design. Keith Braithwaite did a really great job of first introducing the concept of cyclomatic complexity and than showing at the example of Hudson as well as many other open source Java projects that the average and mean cyclomatic complexity of all those projects actually is pretty close to one and when plotted for all methods pretty much follows a power law distribution. Comparing the properties of their specific distribution of cyclomatic complexities over projects he found out that the less steep the curve is, that is the more balance the distribution is, that is the less really complex pieces there are in the code the more likely are developers happy with the current state of the code. Not only that, also that distribution would be transformed into something more balanced after refactorings.

Now looking at a selection of open source projects he analyzed what the alpha of the distribution of cyclomatic complexity is for projects that have no tests at all, have tests and those that were developed according to TDD. Turns out that the latter ones were the ones with the most balanced alpha.

It’s release day today: This morning, ownCloud 4 was released! With a very cool set of features it’s even more useful and again more fun to use.

Along with that I am happy to let you know that we also released the ownCloud Desktop Client in version 1.0.2.

It is a maintenance release with a couple of important bugfixes, such as the cross platform filename encoding problem I was already talking about. Apart from that big blocker a couple of smaller, but annoying problems were fixed. Also the GUI was polished, text changes here and there and a new icon set that looks more cool and more like ownCloud.

Version 1.0.2 is also the version that is released on all big desktop platforms the first time. Now we also offer a dmg for MacOSX as well as a Windows Installer and packages for the major Linux distributions. Have fun!

If you want to talk about ownCloud, file synchronization or such, we have a booth on Linuxtag in Berlin and I will be there. I am looking forward to meeting you.

Third post about ganeti.

Ganet-debootstrap-instance contains a nice set of scripts to create a debian (or derivatives) image using debootstrap. Images can be configured and customized by writing simple hooks script to modify various aspects of the default installation. However writing these script is not really fun and pushing it too far can lead to long messy scripts, loosing the overall benefit of automatic configuration.

Puppet is my configuration management tool of choice, but installing puppet on a new machine requires few magic incantations that the user should perform manually, or in a semi automatic mode (autosign=true) to make it work. My goal is to install puppet automatically on the newly created instance so it will run and configure the new instance at the first boot. From that moment on I'll forget about ganeti and configure all remaining services of my new VM using puppet.

In order to do so, we need to install puppet (and apt-get update/upgrade...), create the ssl certificates for the client and enabling the puppet daemon on the client. We add another hook in /etc/ganeti/instance-debootstrap/hooks/ :

if [ -z "$TARGET" -o ! -d "$TARGET" ]; then
  echo "Missing target directory"
  exit 1
fi

LANG=C
chroot "$TARGET" apt-get -y --force-yes update
chroot "$TARGET" apt-get -y --force-yes upgrade

# install puppet on the client
chroot "$TARGET" apt-get -y --force-yes install puppet

DOMAIN=localnet.org
instance=$INSTANCE_NAME.$DOMAIN

echo "Installing puppet certificates for $instance"
puppetca clean $instance
puppetca -g $instance

mkdir -p $TARGET/etc/puppet
mkdir -p $TARGET/var/lib/puppet/ssl/private_keys/
mkdir -p $TARGET/var/lib/puppet/ssl/certs/

cp /var/lib/puppet/ssl/private_keys/$instance.pem $TARGET/var/lib/puppet/ssl/private_keys/
rm -f $TARGET/var/lib/puppet/ssl/public_keys/$instance.pem

cp /var/lib/puppet/ssl/certs/$instance.pem $TARGET/var/lib/puppet/ssl/certs/
cp /var/lib/puppet/ssl/certs/ca.pem $TARGET/var/lib/puppet/ssl/certs/

chown root. $TARGET/var/lib/puppet/ssl/private_keys/$instance.pem
chmod 0400 $TARGET/var/lib/puppet/ssl/private_keys/$instance.pem

chown root. $TARGET/var/lib/puppet/ssl/certs/$instance.pem
chmod 0640 $TARGET/var/lib/puppet/ssl/certs/$instance.pem

chown root. $TARGET/var/lib/puppet/ssl/certs/ca.pem
chmod 0641 $TARGET/var/lib/puppet/ssl/certs/ca.pem

#echo "server=puppet" >> /etc/puppet/puppet.conf

echo "START=yes" > $TARGET/etc/default/puppet
echo "DAEMON_OPTS=\"\"" >> $TARGET/etc/default/puppet

This script uses puppetca to create on the puppet (and ganeti) server the client key, sign it, and then copy it to the target machine. Notice that we create the certificate for a fqnd name $INSTANCE_NAME.$DOMAIN or otherwise puppet will complain loudly. This is not strictly needed, but if you want to do otherwise, you'll need to fiddle with the puppet configuration a bit more. The procedure to create a puppet certificate server-side is well documented on the puppet website, so if you are curious about the details duck-duck-it .

Furtively he looks round, then takes from the desk drawer a comic-book entitled ‘Thrills and Adventure’. We see the frames of the comic strip. A Superman-type character and a girl are shrinking from an explosion. She is saying ‘My God, his nose just exploded with enough force to destroy his kleenex’. In the next frame, the Superman character is saying ‘If only I had a kleenex to lend him – or even a linen handkerchief – but these trousers…!! No back pocket!’ In the frame beneath, he flies from side to side attempting to escape; finally he breaks through, bringing the two frames above down on himself. Cut to a picture of a safety curtain.

Last tutorial we covered ‘parsing’.  We broke a standard config file up into separate lines and then we broke each line into pairs, each pair having a ‘key’ and a ‘value’.   If we’re to edit the file, we need a way to edit the values (and after that we’ll write the edited values back to the config file).  We’re going to see how to use Tkinter to do that in this tutorial.

Let’s start thinking about how a single key/value pair will look.  I am thinking of having the key on the left hand side with a space to input the value on the right hand side.  To do this we will use the Label widget that we’ve met before and the Text widget.  Label is used to display static text – ie text that will not be edited, while the Text widget allows the user to edit the text which is displayed in the widget, and for the program to read what is entered in the widget.  The Text widget is the GUI equivalent of raw_input() that we met so long ago.

from Tkinter import * root = Tk() labelWidget = Label(root,text="A key:") textWidget = Text(root) textWidget.insert('1.0',"A Value") labelWidget.pack(side=LEFT) textWidget.pack(side=RIGHT) root.mainloop()

Here we first create a Tk() object, then create a label widget and a text widget in that object.   We pack the label first and we pack it on the LEFT side (LEFT is actually the name of a constant in Tk which Tk translates to ‘put this on the left’) and pack the text widget on the right side.  At location “1.0″ we add the text “A Value” to the Text widget.  Here the number “1.0″ means “row 1, at character position 0″, which is to say, at the very start of the text.

If I run this code I get something like this:

Which is sort of what I wanted – a label on the left, and an editable text box on the right (can you see the cursor in the screen shot?) – click the close window widget in the top right corner to close the window.

Exercise:  Type something into the text box.  See if you can do it to the label.

However, this isn’t really what I wanted.  I wanted a little text box, not the enormous one I’ve got here.  Since I didn’t specify a height and width for it, the Text widget used its default size (which is way too big).  The user also doesn’t have any way to tell the program to use (or cancel) the edit.  Let’s change the code to add an ok and cancel button, and to change the size of the text widget.

Here is a revised version which is nearer to what I was looking for:

from Tkinter import * def okClicked(): '''Get the edited values and write them to the file then quit''' #TODO: get values and write them to the file! exit() def cancelClicked(): '''Cancel edits and quit''' exit() root = Tk() labelWidget = Label(root,text="A key:") textWidget = Text(root, width=60, height = 1) okWidget = Button(root, text= "Ok", command = okClicked) cancelWidget = Button(root, text="Cancel", command = cancelClicked) textWidget.insert('1.0',"A Value") labelWidget.pack(side=LEFT) cancelWidget.pack(side=RIGHT) okWidget.pack(side=RIGHT) textWidget.pack(side=RIGHT) root.mainloop()

This gives:

I have added a couple of functions to be run when the ok and cancel buttons are clicked.  The ok button’s function is still a little empty at the moment though…  I have specified the width to be 60 characters and the height to be one row.  Note these are not pixel measurements.  If you change the size of the font the text box will also change.

Notice also the way the geometry is working.  The widgets which are pack(side=RIGHT) are added at the right in the order they are packed.  If the buttons were packed last they would be between the label and the text window.

Exercise: change the program so that the widgets are packed in a different order. What happens if you try side=TOP or side=BOTTOM?

The value can be edited by the user typing directly into the text box.  The text in the text box can also be edited programmatically, which is to say its contents can be changed by the program without the user typing.  See the Tkinter documentation for details – or clamour on this site and I’ll add a tute.

One thing that I don’t like about this layout is the fact the buttons are on the same line as the key label and value text.  When you did the exercise above, you should have noticed that side=TOP and side=BOTTOM don’t really help, since you can’t position the ok and cancel buttons on the same line.  What we need to use is the Frame widget.  Frames can be thought of as empty spaces in which you can group widgets together.  By treating the widgets within the Frame as a group, additional layouts can be achieved.   Frames can be packed inside other frames. We will use two frames, one on top of the other.  In the first frame we pack the label and text widgets.  In the second frame we pack the two buttons.

Here is the code:

from Tkinter import * def okClicked(): '''Get the edited values and write them to the file then quit''' #TODO: get values and write them to the file! exit() def cancelClicked(): '''Cancel edits and quit''' exit() root = Tk() topFrame = Frame(root) bottomFrame = Frame(root) labelWidget = Label(topFrame,text="A key:") textWidget = Text(topFrame, width=60, height = 1) okWidget = Button(bottomFrame, text= "Ok", command = okClicked) cancelWidget = Button(bottomFrame, text="Cancel", command = cancelClicked) textWidget.insert('1.0',"A Value") labelWidget.pack(side=LEFT) cancelWidget.pack(side=RIGHT) okWidget.pack(side=LEFT) textWidget.pack(side=RIGHT) topFrame.pack(side=TOP) bottomFrame.pack(side=BOTTOM) root.mainloop()

This is more like what I wanted (notice you can’t see the individual frames).  One might quibble with the location of the ok and cancel buttons.  Maybe they should be offset a little from the centre?  Maybe they should be off to one side.  In any event they are in the general layout that i was looking for: a key label and an editable value text above the ok and cancel buttons. Notice in the code that the widgets we used before have had their parent changed from root to either topFrame or bottomFrame?  However these Frame widgets have root as their parent.   So the original widgets we were using have effectively been pushed down one level in the hierarchy.

We still don’t know how to get what the user has typed into the text box, but maybe I can leave that as an exercise for the reader (try textWidget.get(“1.0″,END)).

Homework:  change the okClicked() function so that it prints the contents of the text box before exiting.  Use the hint in the previous paragraph.

Mr. Simpson:     Good. Well I have this large quantity of string, a hundred and twenty-two thousand miles of it to be exact, which I inherited, and I thought if I advertised it–
Wapcaplet:     Of course! A national campaign. Useful stuff, string, no trouble there.
Mr. Simpson:     Ah, but there’s a snag, you see. Due to bad planning, the hundred and twenty-two thousand miles is in three inch lengths. So it’s not very useful.

In the previous tutorial we learnt that when we pack Tkinter objects, the order in which we pack them affects how they are displayed in the GUI.  We used a Text widget to enable the user to edit text in the GUI.  We also learnt to use a Frame widget to help with the layout of our GUI.  In particular, we put a Label and a Text widget together into one Frame, and put an ok and cancel Button into another.   For homework you needed to get data from the Text widget.

In order to display all of the configuration options we are going to go a bit nutty using Frames.   We will eventually (but not today) use one Frame to hold all of the configuration options, and another Frame to hold the Ok and Cancel buttons.  But that’s not all!  We will also use a Frame to house each configuration option (ie key and value pair).  Before we do that though, we need to remember where we were up to reading and parsing the server.properties file.

Here is the code we finished with two tutorials ago, excluding the last couple of lines (which printed out the results) for your reference if you need it (click to expand):

'''Minecraft config editor: This is an editor for the Minecraft server.properties file. It: * opens the file server.properties * reads, then closes the file * parses each line by -- stripping leading and trailing whitespace -- if the line starts with "#", marks it as a comment -- splits the line into a key, value pair, with the pair separated by a "=" sign -- if the value of the pair is either "true" or "false", the entry is marked as a boolean (ie its only values are either true or false) * displays each key, value entry on the screen allowing you to edit it * renames the server.properties file to server.properties.bup (overwriting any existing file of that name from earlier edits) * opens a new file called server.properties * writes each of the entries to that new file * closes the server.properties file. ''' class configItem(object): # name of the class, it is based on an object called 'object' def __init__(self, line):# this is called each time an instance of the class is created line = line.strip() # this removes any white space at the start or end of the line # if it starts with a # it's a comment so check for it if line[:1] == "#": self.configKey = "#" self.configVal = line[1:] else: # otherwise assume it's of the form x = y spam = line.split("=") self.configKey = spam[0] self.configVal = spam[1] # now check to see whether the config item takes only the values "true" and "false" if self.configVal.lower() in ["true","false"]: self.isTrueFalse = True else: self.isTrueFalse = False # get data from the file fileName = "server.properties" fileObject = open(fileName,'rb') fileData = fileObject.read() fileObject.close() configLines = [] for line in fileData.split('\n'): # this splits it into individual lines if line.strip()=='': continue configLines.append(configItem(line))

If you remember we defined a class called configItem.  We read the lines from the config file and used each line to create instances of configItem.  We stored those in an array called configLines.  Each instance has two attributes – configKey and configVal (that is, the things on the left and right hand side of the equals respectively).  In the last tutorial for one key, value pair we:

• created a label and set it equal to configKey;
• created a text widget and set its value to configVal; and, finally,
• created a frame in which to pack each of these.

Now we have to do that for each and every entry in the array.  There are plenty of ways to do this.  However, I am going to do it by “subclassing” the configItem class.  That is, I am going to create a new class which is based on (“inherits from” or “is a subclass of”) the configItem class.  It has the features of the configItem class but will also store some stuff relating to the Tkinter widgets that we will need.  This is the new class which I’ve called guiConfigItem:

class guiConfigItem(configItem): def __init__(self,line): super(guiConfigItem,self).__init__(line) # run configItem's __init__ method self.frame = Frame() self.keyLabel = Label(self.frame, text = self.configKey) self.valueEntry = Entry(self.frame, width="60") self.valueEntry.insert("0",self.configVal) self.keyLabel.pack(side=LEFT ) self.valueEntry.pack(side=RIGHT) self.frame.pack(side=TOP)

Some things to note about this class:

• instead of the first line ending “(object):” like the other classes we’ve seen, this one ends “(configItem):”.  This means that guiConfigItem’s immediate parent is configItem.  However, since configItem is based on object, in the end, so is guiConfigItem.
• it takes the same initialisation parameters as configItem (that is, self and line)
• the first thing it does in initialising stuff is to call super(guiConfigItem,self).__init__(line).  This runs configItem’s __init__ method, so every guiConfigItem starts with the same initialisation that configItem would have
• it starts by creating a Frame, stores it in self.frame, then, inside the frame, it creates a label and an Entry widget.  An Entry widget is the single line version of the Text widget we used last time, and it should be good enough for our purposes.
• you can tell that the Label and Entry widgets are created inside the frame which has been created because the first parameter passed to them is self.frame.
• the Label widget is packed to the LEFT, and the Entry widget to the RIGHT.  the frame is also packed, but it is packed to TOP (ie it will make a list from top to bottom)

This class is added after the definition of the configItem class.  In order to get it working we just have to make a four changes to the program.  We will:

• import Tkinter – from Tkinter import *;
• create a root window in which to pack things - root =Tk();
• change the loop to create guiConfigItems rather than configItems – configLines.append(guiConfigItem(line=line)); and
• we will start the gui with a mainloop() – root.mainloop()

Here is the updated source code:

'''Minecraft config editor: This is an editor for the Minecraft server.properties file. It: * opens the file server.properties * reads, then closes the file * parses each line by -- stripping leading and trailing whitespace -- if the line starts with "#", marks it as a comment -- splits the line into a key, value pair, with the pair separated by a "=" sign -- if the value of the pair is either "true" or "false", the entry is marked as a boolean (ie its only values are either true or false) * displays each key, value entry on the screen allowing you to edit it * renames the server.properties file to server.properties.bup (overwriting any existing file of that name from earlier edits) * opens a new file called server.properties * writes each of the entries to that new file * closes the server.properties file. ''' from Tkinter import * class configItem(object): # name of the class, it is based on an object called 'object' def __init__(self, line):# this is called each time an instance of the class is created line = line.strip() # this removes any white space at the start or end of the line # if it starts with a # it's a comment so check for it if line[:1] == "#": self.configKey = "#" self.configVal = line[1:] else: # otherwise assume it's of the form x = y spam = line.split("=") self.configKey = spam[0] self.configVal = spam[1] # now check to see whether the config item takes only the values "true" and "false" if self.configVal.lower() in ["true","false"]: self.isTrueFalse = True else: self.isTrueFalse = False class guiConfigItem(configItem): def __init__(self,line): super(guiConfigItem,self).__init__(line) # run configItem's __init__ method self.frame = Frame() self.keyLabel = Label(self.frame, text = self.configKey) self.valueEntry = Entry(self.frame, width="60") self.valueEntry.insert("0",self.configVal) self.keyLabel.pack(side=LEFT ) self.valueEntry.pack(side=RIGHT) self.frame.pack(side=TOP) # get data from the file fileName = "server.properties" fileObject = open(fileName,'rb') fileData = fileObject.read() fileObject.close() root = Tk() configLines = [] for line in fileData.split('\n'): # this splits it into individual lines if line.strip()=='': continue configLines.append(guiConfigItem(line=line)) root.mainloop()

When I run this I get:

Wow, is that magic? The way we defined the class meant that each of the instances packed itself for us as we created them.  This is an example of why using classes can be so much fun.

Exercise: how might you do the same thing without using classes?

That said, the alignment is a little wonky.   This is because each of the individual frames (there is one on each line) are different sizes.  The overall window is big enough to fit the biggest, but that means that the smaller lines aren’t big enough.  This can be remedied by adding fill=”x” (that is, fill in the x (horizontal) direction if necessary to the pack command for each of the Frames:

self.frame.pack(side=TOP, fill="x")

Now the window looks much better:

Exercise: confirm that you can edit the values on the right.

Exercise 2: check through our docstring to see what we’ve done so far and what we’ve got left to do.

The complete source code with the final edit is below.

'''Minecraft config editor: This is an editor for the Minecraft server.properties file. It: * opens the file server.properties * reads, then closes the file * parses each line by -- stripping leading and trailing whitespace -- if the line starts with "#", marks it as a comment -- splits the line into a key, value pair, with the pair separated by a "=" sign -- if the value of the pair is either "true" or "false", the entry is marked as a boolean (ie its only values are either true or false) * displays each key, value entry on the screen allowing you to edit it * renames the server.properties file to server.properties.bup (overwriting any existing file of that name from earlier edits) * opens a new file called server.properties * writes each of the entries to that new file * closes the server.properties file. ''' from Tkinter import * class configItem(object): # name of the class, it is based on an object called 'object' def __init__(self, line):# this is called each time an instance of the class is created line = line.strip() # this removes any white space at the start or end of the line # if it starts with a # it's a comment so check for it if line[:1] == "#": self.configKey = "#" self.configVal = line[1:] else: # otherwise assume it's of the form x = y spam = line.split("=") self.configKey = spam[0] self.configVal = spam[1] # now check to see whether the config item takes only the values "true" and "false" if self.configVal.lower() in ["true","false"]: self.isTrueFalse = True else: self.isTrueFalse = False class guiConfigItem(configItem): def __init__(self,line): super(guiConfigItem,self).__init__(line) # run configItem's __init__ method self.frame = Frame() self.keyLabel = Label(self.frame, text = self.configKey) self.valueEntry = Entry(self.frame, width="60") self.valueEntry.insert("0",self.configVal) self.keyLabel.pack(side=LEFT ) self.valueEntry.pack(side=RIGHT) self.frame.pack(side=TOP, fill="x") # get data from the file fileName = "server.properties" fileObject = open(fileName,'rb') fileData = fileObject.read() fileObject.close() root = Tk() configLines = [] for line in fileData.split('\n'): # this splits it into individual lines if line.strip()=='': continue configLines.append(guiConfigItem(line=line)) root.mainloop()

Mr Mann     Ee ecky thump! (indicates more power)
Third Booth     Ee ecky thump!
Mr Mann     Excellent.
Third Booth     Thank you, sir. (puts earphones on, listens)
Mr Mann     It’s a really quick method of learning.

There are two more things to do with our Minecraft config file editor before we’ve got the main part of it working (we may do some tweaking later).  We need to:

• add the Ok and Cancel buttons back; and
• when someone clicks Ok, we need to update the server.properties file

We’re doing the first of these today. We saw earlier how to do the Ok and Cancel buttons, although at the time we didn’t actually put any meat in the functions they called.  So, let’s fill that out now.  For Cancel, we are just going to quit the editor without making any changes – that’s pretty easy.  For the Ok button though, we’re going to have to:

1. somehow read all of the values from the screen (since we don’t know which ones have been changed we need to read them all);
2. make a backup of the server.properties file
3. write all of the key:value pairs to the new server.properties file.

Unlike variables, widgets are not the same as what is stored in them.  If we have an Entry widget called E and we want to store what has been typed there in a variable called text we can’t just write text = E.   This is because E is not a variable as we understand it.  Actually, E is an instance of a class.  This would just make another reference to the same Entry widget with the name text.  Rather, we want to “get” the current value of the text entered into E.  It turns out that the Entry widget has a method (called get()) which gets that text for you.

>>> from Tkinter import * >>> E = Entry() # this should pop up a Tkinter window >>> E.pack() # the widget should appear in your Tkinter window now >>> type(E) <type 'instance'> >>> type(Entry) <type 'classobj'> >>> text = E >>> type(text) <type 'instance'> >>> print text .140543131462184 >>> # now type "Hi P4K!" in the entry widget ... >>> text = E.get() >>> print text Hi P4K! >>> # now add " - Again" to the end of the entry widget (leave the "Hi P4K!" there) ... >>> text = E.get() >>> print text Hi P4K! - Again >>> # you can also print the value which you get() without storing it first: ... >>> print E.get() Hi P4K! - Again

So what we’re going to do in our code is get() all these edited values when someone clicks “Ok”.  We could do that directly, for example by finding the relevant guiConfigItem and calling the get() method on the valueEntry attribute of that item.   That would also mean we’d have to make a copy of the key for that item and then combine them together with “=” before we wrote them to the server.properties file.  This would mean that logic which is relevant to the configItem class would be stored somewhere other than inside the class - which rather defeats the purpose of having a class to keep track of these things.  Instead, we’re going to add a method to the guiConfigItem class which updates the values it has stored.  That turns out to be pretty easy:

def update(self): ''' Get the value which is currently in the Entry widget and save it to configVal''' if self.isTrueFalse: '''if isTrueFalse is True, then we should only have the values 'true' and 'false' in this Entry. So, only update the configuration value if it is one of these two. Otherwise, ignore it. ''' spam = self.valueEntry.get() if spam in ['true','false']: self.configVal = spam else: '''this is not a variable which is limited to 'true' and 'false', so store the whole text''' self.configVal = self.valueEntry.get()

Note here that we are referencing the attribute configVal which is defined in the parent class.  Also note that we’ve included a bit of logic here to ensure that those configuration values which start as ‘true’ or ‘false’ can only be ‘true’ or ‘false’.  If you type something else into them it will be ignored.  It is sufficient here to just say if self.isTrueFalse rather than if self.isTrueFalse is True (the “is True” is redundant).

We also need a way to prepare the lines of the server.properties file to be printed or written to the file.  We do this by adding a method to the configItem class (since it doesn’t have anything to do with the graphical interface we don’t add it to the subclass):

def item2ConfigLine(self): if self.configKey=="#": '''If the key is '#' then this is a comment, so don't include an '=' sign''' return "%s%s"%(self.configKey, self.configVal) else: '''otherwise, it has the form key=value''' return "%s=%s"%(self.configKey,self.configVal)

See this tutorial for an explanation of the %s stuff…

We don’t have a way to test these methods out yet. So let’s hook up the Ok and Cancel buttons. The Ok button will run through each of the guiConfigItems and update it, then print out the configuration line. After all items have been processed this way, the program will exit. The cancel button will just exit without doing anything. So, we need to:
1. create callbacks for each of the buttons,
2. create a frame for the buttons to go in
3. create the buttons, hooking up each of the buttons up to the callback
4. pack the buttons, then, finally,
5. pack the frame.
These go before the root.mainloop() line.

# 1. create callbacks for each of the buttons, def okClicked(): '''Get the edited values and write them to the file then quit''' for c in configLines: c.update() print c.item2ConfigLine() # have updated and printed each line, now exit exit() def cancelClicked(): '''Cancel edits and quit''' exit() # 2. create a frame for the buttons to go in bottomFrame = Frame(root) # 3. create the buttons, hooking up each of the buttons up to the callback okWidget = Button(bottomFrame, text= "Ok", command = okClicked) cancelWidget = Button(bottomFrame, text="Cancel", command = cancelClicked) # 4. pack the buttons, then, finally, okWidget.pack(side=LEFT) cancelWidget.pack(side=RIGHT) # 5. then pack the frame: bottomFrame.pack(side=BOTTOM)

At the moment, the Ok button just prints out the values of the items. This is so that we can test how it is working before we let it go editing the actual file.

Here is the complete source code:

# -*- coding: utf-8 -*- '''Minecraft config editor: This is an editor for the Minecraft server.properties file. It: * opens the file server.properties * reads, then closes the file * parses each line by -- stripping leading and trailing whitespace -- if the line starts with "#", marks it as a comment -- splits the line into a key, value pair, with the pair separated by a "=" sign -- if the value of the pair is either "true" or "false", the entry is marked as a boolean (ie its only values are either true or false) * displays each key, value entry on the screen allowing you to edit it * renames the server.properties file to server.properties.bup (overwriting any existing file of that name from earlier edits) * opens a new file called server.properties * writes each of the entries to that new file * closes the server.properties file. ''' from Tkinter import * class configItem(object): # name of the class, it is based on an object called 'object' def __init__(self, line):# this is called each time an instance of the class is created line = line.strip() # this removes any white space at the start or end of the line # if it starts with a # it's a comment so check for it if line[:1] == "#": self.configKey = "#" self.configVal = line[1:] else: # otherwise assume it's of the form x = y spam = line.split("=") self.configKey = spam[0] self.configVal = spam[1] # now check to see whether the config item takes only the values "true" and "false" if self.configVal.lower() in ["true","false"]: self.isTrueFalse = True else: self.isTrueFalse = False def item2ConfigLine(self): if self.configKey=="#": '''If the key is '#' then this is a comment, so don't include an '=' sign''' return "%s%s"%(self.configKey, self.configVal) else: '''otherwise, it has the form key=value''' return "%s=%s"%(self.configKey,self.configVal) class guiConfigItem(configItem): def __init__(self,line): super(guiConfigItem,self).__init__(line) # run configItem's __init__ method self.frame = Frame() self.keyLabel = Label(self.frame, text = self.configKey) self.valueEntry = Entry(self.frame, width="60") self.valueEntry.insert("0",self.configVal) self.keyLabel.pack(side=LEFT ) self.valueEntry.pack(side=RIGHT) self.frame.pack(side=TOP, fill="x") def update(self): ''' Get the value which is currently in the Entry widget and save it to configVal''' if self.isTrueFalse: '''if isTrueFalse is True, then we should only have the values 'true' and 'false' in this Entry. So, only update the configuration value if it is one of these two. Otherwise, ignore it. ''' spam = self.valueEntry.get() if spam in ['true','false']: self.configVal = spam else: '''this is not a variable which is limited to 'true' and 'false', so store the whole text''' self.configVal = self.valueEntry.get() # get data from the file fileName = "server.properties" fileObject = open(fileName,'rb') fileData = fileObject.read() fileObject.close() root = Tk() configLines = [] for line in fileData.split('\n'): # this splits it into individual lines if line.strip()=='': continue configLines.append(guiConfigItem(line=line)) # 1. create callbacks for each of the buttons, def okClicked(): '''Get the edited values and write them to the file then quit''' for c in configLines: c.update() print c.item2ConfigLine() # have updated and printed each line, now exit exit() def cancelClicked(): '''Cancel edits and quit''' exit() # 2. create a frame for the buttons to go in bottomFrame = Frame(root) # 3. create the buttons, hooking up each of the buttons up to the callback okWidget = Button(bottomFrame, text= "Ok", command = okClicked) cancelWidget = Button(bottomFrame, text="Cancel", command = cancelClicked) # 4. pack the buttons, then, finally, okWidget.pack(side=LEFT) cancelWidget.pack(side=RIGHT) # 5. then pack the frame: bottomFrame.pack(side=BOTTOM) root.mainloop()

Exercise: run the code and confirm that: (a) your edits are captured and printed out; (b) if you enter anything but “true” or “false” for an item that takes only true and false, then the edit is ignored; and (c) that if you change a true to a false or vice versa, that that edit is captured.

Boss     (unfolding big map across table; talking carefully) Right … this is the plan then. … At 10:52, I shall approach the counter and purchase a watch costing £5.18.3d. I shall then give the watch to you, Vic. You’ll go straight to Norman’s Garage in East Street. You lads continue back up here at 10:56 and we rendezvous in the back room at the Cow and Sickle, at 11:15. All right, any questions?
Larry     We don’t seem to be doing anything illegal.
Boss     What do you mean?
Larry     Well … we’re paying for the watch.
Boss     (patiently) Yes…
Larry     (hesitating) Well… why are we paying for the watch?
Boss     (heavily) They wouldn’t give it to us if we didn’t pay for it, would they… eh?

This is our final instalment [sic] of our Minecraft config editor.   In the earlier tutorials we have done everything except actually updating the file.  Before we do update the file though, we need to make a backup of it, so it’s these two things that we’re going to do now.

All of the action will be in changing the behaviour of the ‘Ok’ button so that it makes a copy of server.properties into a new file called server.properties.bup and then writes the updated data to the server.properties file.  There is a small amount of work in making a copy of the file so we are going to do it in a separate function.  The function is not very smart.  It reads all of the data from the existing file and then just writes it out to the new file:

def backupFile(fileName): ''' Quick and dirty copy of file to fileName+".bup" - might also use os.rename(), but behaviour of os.rename is platform dependent ''' fileObject = open(fileName,'rb') fileData = fileObject.read() fileObject.close() fileObject= open(fileName+".bup",'wb') # will overwrite if it exists fileObject.write(fileData) fileObject.close()

The function could also use the rename() method from the os module.  Unfortunately, this behaves differently depending on the operating system you are using, so to keep it simple I have avoided it.

Exercise: Work out what this function is supposed to do, then confirm that it does it eg: start up a Python console, paste the definition in and then call the function with “server.properties” as a parameter.

Extra Points: If you are on a Unix based system, use the diff command to show the differences between the original and .bup (there shouldn’t be any).

With that done we can hook up the backup to the ‘Ok’ callback, and also write the new data:

def okClicked(): '''Get the edited values and write them to the file then quit''' #TODO: add a confirmation dialog global fileName backupFile(fileName) dataToWrite = [] for c in configLines: c.update() dataToWrite.append(c.item2ConfigLine()) # have updated and printed each line, now exit fileObject = open(fileName,'wt') fileObject.write('\n'.join(dataToWrite)+'\n') # '\n' is technically not the newline character on Windows # but by default Python converts \n to the correct character on write # not sure if minecraft needs a final '\n', so included one just in case fileObject.close() exit()

Here we have introduced an array called dataToWrite.  Where, in the last tute we just printed out the line, in this tute we are appending those lines to the dataToWrite array.  Then, once they have been accumulated, we open the server.properties file (clearing it) then write our new data into it.  Only one generation of backup is saved.

We have used the global statement here to use the value of the fileName variable.  This is a little messy, but is a consequence of how the program has evolved.

One of the things that you might include here is a confirmation step.  Before the data gets overwritten we might ask the user to confirm that they are going to write over their data, giving them a second chance if they clicked “Ok” by mistake.

Complete code here:

# -*- coding: utf-8 -*- '''Minecraft config editor: This is an editor for the Minecraft server.properties file. It: * opens the file server.properties * reads, then closes the file * parses each line by -- stripping leading and trailing whitespace -- if the line starts with "#", marks it as a comment -- splits the line into a key, value pair, with the pair separated by a "=" sign -- if the value of the pair is either "true" or "false", the entry is marked as a boolean (ie its only values are either true or false) * displays each key, value entry on the screen allowing you to edit it * renames the server.properties file to server.properties.bup (overwriting any existing file of that name from earlier edits) * opens a new file called server.properties * writes each of the entries to that new file * closes the server.properties file. ''' from Tkinter import * class configItem(object): # name of the class, it is based on an object called 'object' def __init__(self, line):# this is called each time an instance of the class is created line = line.strip() # this removes any white space at the start or end of the line # if it starts with a # it's a comment so check for it if line[:1] == "#": self.configKey = "#" self.configVal = line[1:] else: # otherwise assume it's of the form x = y spam = line.split("=") self.configKey = spam[0] self.configVal = spam[1] # now check to see whether the config item takes only the values "true" and "false" if self.configVal.lower() in ["true","false"]: self.isTrueFalse = True else: self.isTrueFalse = False def item2ConfigLine(self): if self.configKey=="#": '''If the key is '#' then this is a comment, so don't include an '=' sign''' return "%s%s"%(self.configKey, self.configVal) else: '''otherwise, it has the form key=value''' return "%s=%s"%(self.configKey,self.configVal) class guiConfigItem(configItem): def __init__(self,line): super(guiConfigItem,self).__init__(line) # run configItem's __init__ method self.frame = Frame() self.keyLabel = Label(self.frame, text = self.configKey) self.valueEntry = Entry(self.frame, width="60") self.valueEntry.insert("0",self.configVal) self.keyLabel.pack(side=LEFT ) self.valueEntry.pack(side=RIGHT) self.frame.pack(side=TOP, fill="x") def update(self): ''' Get the value which is currently in the Entry widget and save it to configVal''' if self.isTrueFalse: '''if isTrueFalse is True, then we should only have the values 'true' and 'false' in this Entry. So, only update the configuration value if it is one of these two. Otherwise, ignore it. ''' spam = self.valueEntry.get() if spam in ['true','false']: self.configVal = spam else: '''this is not a variable which is limited to 'true' and 'false', so store the whole text''' self.configVal = self.valueEntry.get() # get data from the file fileName = "ser_ver.properties" fileObject = open(fileName,'rb') fileData = fileObject.read() fileObject.close() root = Tk() configLines = [] for line in fileData.split('\n'): # this splits it into individual lines if line.strip()=='': continue configLines.append(guiConfigItem(line=line)) # 1. create callbacks for each of the buttons, def okClicked(): '''Get the edited values and write them to the file then quit''' #TODO: add a confirmation dialog global fileName backupFile(fileName) dataToWrite = [] for c in configLines: c.update() dataToWrite.append(c.item2ConfigLine()) # have updated and printed each line, now exit fileObject = open(fileName,'wt') fileObject.write('\n'.join(dataToWrite)+'\n') # '\n' is technically not the newline character on Windows # but by default Python converts \n to the correct character on write # not sure if minecraft needs a final '\n', so included one just in case fileObject.close() exit() def backupFile(fileName): ''' Quick and dirty copy of file to fileName+".bup" - might also use os.rename(), but behaviour of os.rename is platform dependent ''' fileObject = open(fileName,'rb') fileData = fileObject.read() fileObject.close() fileObject= open(fileName+".bup",'wb') # will overwrite if it exists fileObject.write(fileData) fileObject.close() def cancelClicked(): '''Cancel edits and quit''' exit() # 2. create a frame for the buttons to go in bottomFrame = Frame(root) # 3. create the buttons, hooking up each of the buttons up to the callback okWidget = Button(bottomFrame, text= "Ok", command = okClicked) cancelWidget = Button(bottomFrame, text="Cancel", command = cancelClicked) # 4. pack the buttons, then, finally, okWidget.pack(side=LEFT) cancelWidget.pack(side=RIGHT) # 5. then pack the frame: bottomFrame.pack(side=BOTTOM) root.mainloop()

Exercise: confirm that “Ok” saves your edits (open server.properties in a text editor or (extra points) write some Python to read and print the contents of the file) and that “Cancel” doesn’t.

Comments:

The code is a little messy because of how it has evolved in the course of explaining it.  Having code growing organically and getting messy is not unusual.  Every once in a while you need to stop and clean it up.  Cleaning it up can also allow you to restructure your code in ways you didn’t realise when you were writing it in the first place.

PS

My class names are naughty.  They should start with a capital letter.

Voice Over     This man is Ernest Scribbler… writer of jokes. In a few moments, he will have written the funniest joke in the world… and, as a consequence, he will die … laughing.
    Ernest stops writing, pauses to look at what he has written… a smile slowly spreads across his face, turning very, very slowly to uncontrolled hysterical laughter… he staggers to his feet and reels across room helpless with mounting mirth and eventually collapses and dies on the floor.

Summary: id(), copy, copy.copy(), copy.deepcopy()

A short tutorial this week on some oddities with the way Python stores and references (“binds to”) data.   You might remember, a long time ago, we talked about how, when we store data in a variable, it’s like putting your stuff in a bucket so that you can access it later.  Variables in Python actually turn out to be references to objects.   A side effect of this is that, in some cases, Python doesn’t work out how you think it will – typically this is where your object is a list or dictionary (actually any object, but you only notice this effect with compound objects) that you think you have copied, but you actually haven’t.

In particular, you can do this with ‘plain’ variables:

>>> a = 5 >>> b = a >>> a = 6 >>> b 5

You can also do this with lists:

>>> c = [1,2] >>> d= c >>> c =[5,6] >>> d [1, 2]

But there’s a gotcha with lists where you change one of the list’s entries:

>>> c= [1,2] >>> d = c >>> d [1, 2] >>> c[0]=3 >>> d [3, 2]

Can you see that, even though we only changed the first entry in the list c (that is, c[0]), the first entry of d has also changed?  That’s because there is an underlying list object that both c and d are pointing to.  That is, they are both pointing to the same thing.  In a sense they are both windows to the same room (the list object).  Looking in either window allows you to “see” the changes made in the room.   You can see that the objects are the same because you can check their location in memory using the id() function which is built in to Python (try help(id)):

>>> id(c) 139636641421288 >>> id(d) 139636641421288 >>> id(c) == id(d) True

The number (139636641421288) is where in the computer’s memory the object is stored.  It will change, probably each time you run the program.  If we assign a different list to d, it will have a different id, even though the values in the list are the same:

>>> d = [3,2]       # note this new list has the same values as the old one >>> id(c) == id(d) False >>> id(d) 139636640593824 >>>

We can see that this other list is stored in a different location because the id() of the lists is different.  It turns out that this referencing behaviour is actually what you want to happen in most cases.  However, every so often you want your lists to be separate.  For that there is a special module called copy.  The copy module has a method (also called copy) which allows you to copy across the values of an object, rather than simply referencing (called “binding“) to an existing object:

>>> import copy >>> d = copy.copy(c) >>> d [3, 2] >>> c[0]=1 >>> d [3, 2] >>> c [1, 2]

When you use copy.copy() the two objects will be separate and can be used independently.  Changes to one won’t show up in the other.  Where a compound object like a list or a dictionary has values which themselves are compound objects – for example a list where each entry in the list is itself a list – use the copy.deepcopy() method.   Depending on the complexity of your objects deepcopy() is not guaranteed to work (objects which refer to themselves somehow can cause a problem), but generally you will be fine.

I just finished upgrading GNU Bazaar from version 2.3.1 to version 2.5. Please submit support requests if you encounter any issues related to the upgrade.

Over the spring, I worked my way through two online cryptography classes:

• Professor Dan Boneh's course, hosted at Coursera
• Professor Dave Evans's course, hosted at Udacity

Taking both classes was not a waste; the two presentations nicely complemented and reinforced each other, and in the end I felt that I learned a lot from each.

But there were both differences and similarities between the courses, which I found interesting.

Both formats are broadly the same:

• Both classes are, roughly, upper-division undergraduate classes in modern cryptography, covering roughly the same material
• Both classes are presented as video lectures, in the style of allowing you to view the virtual whiteboard that the lecturer is drawing on, while listening to the lecture.
• Both classes augment the video lectures with quizzes, homework assignments, additional reading, and forums for students to gather and discuss the material

Some of the differences in the classes were mechanical:

• The videos for the Boneh class were generally 12-20 minutes in length, each, while the videos for the Evans class were generally 1-3 minutes in length, each
• The slides and quizzes for the Boneh class were typeset, and were supplemented by a certain amount of free-hand annotation, while the slides and quizzes for the Evans class were presented entirely in free-hand
• The classes present the material in a different order
• Although the materials used largely similar notation, there were notational differences (e.g., Boneh uses the terminology "public key/secret key" and the letters "p" and "s", where Evans generally uses the terminology "public key/private key" and the letters "u" and "r")

The most important differences, though, are a bit harder to describe.

I felt that the Boneh class was more theoretical, more rigorous, and more oriented around the mathematical aspects of cryptography. Boneh emphasized a fair amount of probability theory and number theory during the class, spent more time proving theorems during the videos, and used more structured techniques such as the probabilistic "adversary games" that I wrote about several weeks ago.

Meanwhile, I felt that the Evans class was more practical, more approachable, more intuitive, and more oriented around the applied aspects of cryptography. Evans uses many examples from real life, concentrates more on results and less on derivations, and presents the material in context of its use. Evans also spends a great deal more time discussing the history of cryptography research, with interesting illustrations of important milestones and events, pictures of the people and objects that have been part of the history of cryptography, and so forth.

In general, I was extremely impressed by both courses. The material was clear, accurate, relevant, well-presented, and thoroughly explained. A motivated student who approaches either of these courses with energy and time and commitment will reap significant rewards. After just a few months of serious study, I felt like I had substantially improved my knowledge of these areas, and felt both interested in and prepared to take my studies further.

I hope that this is the beginning of a widespread availability of educational materials of the finest quality on the Internet, and I hope that these courses both find the audience that they deserve.

I was reading Hynek Schlawack’s excellent article on becoming a Python core developer and decided to find out just how hard it would be to get set up on my machine so that I could be ready to do core development myself, should I ever get the honor of being a part of the team. Since I run on Windows the most, I’m just going to talk about how I got set up for that OS. I’ve been thinking about trying to help with core development for a while anyway, so now’s as good a time as any. Let’s find out just how easy or hard the setting up process is!

What You’ll Need

To get up and running as a developer of Python on Windows, you’ll need a Mercurial client to download Python, update and create patches. You can use a command-line tool or you can get TortoiseHg, a shell GUI. Once you have that configured correctly, you can do a

hg clone http://hg.python.org/cpython

Or use Tortoise to check out the repository. This will get you the latest Python 3.x version. If you want to help with a maintenance release, then you’ll want to read the documentation. The last major tool you’ll need is a compiler and the one that’s needed for the latest Python is Microsoft Visual Studio 2010. Fortunately, you don’t need to purchase the whole thing. In fact, you can just get the Express version of Visual C++ 2010 and you’ll be good to go. Note that this tool is not lightweight and ends up taking over one gigabyte of space on disc. Sad, but true.

Compiling Python on Windows 7

As you may have guessed, we’ll be doing the compiling of Python on Windows 7. I have Windows XP too, but that OS is practically dead now, so I’m not going to cover it. I doubt it’s much different anyway. Regardless, according to the dev guide documentation, you need to go into the repo that you just created on your machine and go into the PCBuild folder. Then find the pcbuild.sln file and run it with your new Visual Studio C++ application. You may see a warning from it about how the Express version doesn’t support Solutions, but just ignore that. Once the project is loaded, go into the Debug menu and select Build Solution. Oddly enough, the official docs say to go into the Build menu, NOT the Debug menu, but my copy doesn’t have a Build menu to choose from.

When I ran the build I got the following result at the end:

========== Build: 20 succeeded, 8 failed, 0 up-to-date, 3 skipped ==========

Looking through the log, it looks like it was unable to find the header files for sqlite3 and tcl and it had some issues with the bzip lib. It also complained that I don’t have ActivePerl / openssh installed. However, it still compiled Python and I had a fresh python_d.exe file in my PCBuild folder that I could run. I ran it by double-clicking it, but you can also run it from within Visual Studio using F5 or by going to the Debug menu and clicking Start Debugging.

I think at this point, I’m ready to figure out how to create a patch. If I do, I may try patching that messed up documentation so that people won’t spend time looking for a non-existent menu. Then I’ll write an article about how to submit a patch and use the issue tracker system for Python.

Related Reading

• Python 101: Setting up Python on Windows
• A (brief) introduction to Python-Core development
• Contribution article
• Python Core Mentorship program

Three years ago I wrote a post about my disappointment using SciPy with IronPython. A lot has changed since then, so I thought I’d write a short follow-up post.

To install NumPy and SciPy for use with IronPython, follow the instructions here. After installation, NumPy works as expected.

There is one small gotcha with SciPy. To use SciPy with IronPython, start ipy with the command line argument -X:Frames. Then you can use SciPy as you would from CPython. For example.

c:\> ipy -X:Frames >>> import scipy as sp >>> sp.pi 3.141592653589793

Without the -X:Frames option you’ll get an error when you try to import scipy.

AttributeError: 'module' object has no attribute '_getframe'

According to this page,

The issue is that SciPy makes use of the CPython API for inspecting the current stack frame which IronPython doesn’t enable by default because of a small runtime performance hit. You can turn on this functionality by passing the command line argument “-X:Frames” to on the command line.

John Scalzi recently wrote a piece on straight white male privilege. If you haven't read it already, go and do so. No rush. I'll wait.

So. Some facts:

• Women are underrepresented in free software development
  
• Those women who are involved in free software development are overwhelmingly more likely to have been subject to sexual harassment, belittling commentary or just plain ignored because of their gender
  
• When asked, women tend to believe that these two facts are fairly strongly related

(If you disagree with any of these then that's absolutely your right. You're wrong, but that's ok. But please do me a favour and stop reading here. Otherwise you'll just get angry and then you'll write something ill-tempered and still wrong in the comments and then I'll have to delete it and why not just save everybody the time and effort and go and eat ice cream or something instead)

I know I've said this before, but inappropriate and marginalising behaviour is rife in our community, and at all levels of our community. There's the time an open source evangelist just flat out told a woman that her experiences didn't match his so she must be an outlier. There's the time a leading kernel developer said that most rape statistics were basically made up. There's the time that I said the most useful thing Debian could do with its money would be to buy prostitutes for its developers, simultaneously sexualising the discussion, implying that Debian developers were all straight men and casting sex workers as property. These aren't the exceptions. It's endemic. Almost all of us have been part of the problem, and in doing so we've contributed to an environment that has at best driven away capable contributors. You probably don't want to know what it's done at worst.

But what people have done in the past isn't important. What's important is how we behave in the future. If you're not angry about social injustice like this then you're doing it wrong. If you're reading this then there's a pretty high probability that you're a white male. So, it's great that you're angry. You should be! As a straight white male born into a fairly well-off family, a native English speaker in an English speaking country, I have plenty of time to be angry before going back to my nice apartment and living my almost entirely discrimination-free life. So if it makes me angry, I have absolutely no way of comprehending how angry it must make the people who actually have to live with this shit on a daily basis.


(Were tampon mouse able to form and express coherent thoughts, tampon mouse would not put up with this shit)
The point isn't to be smugly self aware of our own shortcomings and the shortcomings of others. The point is to actually do something about it. If you're not already devoting some amount of your resources to improving fairness in the world, then why not? It doesn't have to be about women in technology - if you're already donating to charity or helping out at schools or engaging in local politics or any of the countless other ways an individual can help make the world a better place, large or small, then keep on doing that. But do consider that many of us have done things in the past that contributed to the alienation of an astounding number of potential community members, and if you can then please do do something to make up for it. It might be donating to groups like The Ada Initiative. It might be mentoring students for projects like the GNOME Outreach Program for Women, or working to create similar programs. Even just making our communities less toxic by pointing out unacceptable behaviour when you see it makes a huge difference.

But most importantly, be aware that it was people like me who were responsible for this problem in the first place and people like me who need to take responsibility for solving it. We can't demand the victims do that for us.

comments

A problem that sometimes comes up with source-controlled code is to find a revision in which some line was deleted, or otherwise modified in a way that blame can’t decipher. In other words, we want to grep over all revisions of some file to know which revisions contain a certain pattern. Note that the goal is not to search in the commit log (which is trivial), but rather in the code itself.

Well, if you’re using Mercurial or Git, you’re lucky because both provide built-in methods for doing this.

With Mercurial, use hg grep.

With Git, you can either use git grep in conjunction with git rev-list, or git log -S (more details in this SO thread).

What about Subversion, though? SVN, to the best of my knowledge, does not have this functionality built-in. Moreover, SVN’s design makes this task inherently slow because no revisions past the last one are actually kept on your machine (unless the repository is local) and you have to ask the server for each revision. That’s a lot of network traffic.

That said, if you’re willing to tolerate the slowness (and sometimes there’s no choice!), then the following script – svnrevgrep – makes it as simple as with Git or Mercurial:

import re, sys, subprocess def run_command(cmd): """ Run shell command, return its stdout output. """ return subprocess.check_output(cmd.split(), universal_newlines=True) def svnrevgrep(filename, s): """ Go over all revisions of filename, checking if s can be found in them. """ log = run_command('svn log ' + filename) for ver in re.findall('r\d+', log, flags=re.MULTILINE): cmd = 'svn cat -r %s %s' % (ver.rstrip('r'), filename) contents = run_command(cmd) print('%s: %s' % (ver, 'found' if re.search(s, contents) else 'not found')) if __name__ == '__main__': if len(sys.argv) != 3: print('Usage: %s <path> <regex>' % sys.argv[0]) else: svnrevgrep(sys.argv[1], sys.argv[2])

It basically goes over all revisions of the file starting with the most recent one and looks for the pattern.

Note that while one could imagine using some kind of binary searching to find the first revision in which the regex appears (or doesn’t), this won’t work in the general case because code sometimes is added, then deleted, then re-added, then deleted again (this happens when refactoring or when reverting problematic commits).

Finally, if you find yourself doing the above frequently for a given repository, you may be better off with:

git svn clone <path> git grep <...>

Related posts:

1. Python development switches to Mercurial source control The official CPython core development team has finally switched from...
2. Migrating my personal projects to Mercurial Introduction My first acquaintance with version control was soon...

Websites being built today need to be accessible on mobile devices such as smartphones and tablets. I will outline a prototyping process that we are experimenting with to efficiently create designs for responsive websites.

In Views 3.x, official support for SQL grouping was added, mostly by incorporating the functionality of the former views_groupby module. This can be a bit confusing, because this function really performs aggregation tasks like SUM, instead of grouping under a particular field. Views can group by a field though, allowing for things like quarterly reports sorted by year, with the year displayed at the top of each section. Here’s how in Views 7.x-3.x:

1. Create a view with fields
2. Add some fields that use the same data each time (date, title, price for example)
3. Under Format -> Format, choose Settings
4. Choose a Grouping field and Apply appropriately.

Done!

Author Lynette Miles

Lynette Miles is the co-author of Drupal Building Blocks, and has worked in the tech industry for her entire professional career. She became involved with Drupal in 2006, and participates in the documentation team as well as coordinating the Views bug squad.

I thus think (= hope) that it’s a mistake to extrapolate from today’s crappy input systems on tablets to a future of tablet-based couch potatoes still watching Hollywood crap. We’re one innovation away from lowering the creativity hurdle on tablets. Maybe it’ll be a truly responsive keyboard. Or something that translates sub-vocalizations into text (because I’m too embarrassed to dictate into my table while in public places). Or, well, something. via Joho the Blog » Will tablets always make us non-social consumers?. I suspect that the idea that input systems on tablets are crappy will rapidly become a tell-tale sign of age. Feature phones have “crappy” input systems and yet with learning, and some rather unfortunate adaptations of language, people quite happily chat away via SMS. The same process will happen with tablets – people will simply learn to type on on-screen keyboards fast enough that they don’t consider them crappy. It’s also important to remember that the vast majority of people can’t type particularly fast on a full size desktop keyboard so the bar for satisfaction is significantly lower than geeks would expect.

Hello everyone, how are you doing? Things have been more than a little hectic for me in the last week or so. I’ve had 3 gigs with my band 20lb Sounds, a load of live recordings to process for Rathole Radio, plenty of other podcasts to produce and on top of that a last minute call to join the Liverpool Sound City music conference. That’s what I’d like to tell to you about today. I’ll recap my personal experience of the event from Thursday to Saturday. Although I had other stuff going on – with a couple of meetings and a gig on the Friday – I still found time to get involved with the conference and see a few bands along the way.

Day 1 – Thursday May 17th – At the beginning of Thursday I was feeling pretty exhausted if I’m honest. I’d been up late getting the live recordings from the Rathole Radio gig out as a podcast. I’d also been contacted the day before by a friend to ask if I wanted to come to Sound City in the first place. I was very fortunate to get the free ticket and I wasn’t going to waste it. Having said that I did sleep a bit late to recover some energy. So I arrived at the swanky Hilton hotel on the Liverpool waterfront at midday, just as others were breaking for lunch. I’d only missed one session but I did feel very slightly guilty for being late. I saw some friends in the lobby and headed up to join the signing in queue. There were a lot of music industry people milling about as you’d expect. Some looked very rock and roll, some not so much. As I was stood there behind a guy who looked to be in his 60′s one of the organisers ran over and I could hear him saying to the man “your VIP table is ready for 4pm sir, there’s meetings till 6 and then….blah blah blah”. “I wonder who that guy is and why he’s so important?” I thought to myself. I even contemplated tapping him on the shoulder and asking “should I know you?”, but in the end I never did find out. Perhaps it was Elton John in disguise, this disguise only consisting of a hat really though hehe. I got to the front of the queue eventually and after a little bit of convincing and checking the girl behind the desk found me on the register, then tried to print out my pass. I say “tried” because she did this for about 15 minutes and even asked a colleague to help. Fair play to her she wasn’t giving up. The label printer wasn’t working and my inner IT support technician was bursting to get out. I kept telling myself “you’re not here for that Dan, just let someone else fix it” but in the end I couldn’t. I said “do you mind if I have a look?” explaining that I did know a bit about computers. She obliged my interfering and pretty soon I had my delegate badge.

The Random Panel I Ended Up At

My friends were having lunch downstairs in the Hilton restaurant by the time I got back to the lobby, but when I tried to walk in and join them at the table I was told I was “too late for lunch”. It was 12:30, what time can lunch finish?! I was dressed quite smartly I thought but I suspect my slightly shabby trainers had “too later for lunch whatever time it is” written all over them to the staff. I waited in the lobby watching my friends eat from across the way. They’d pretty much finished anyway to be honest and I got a pasty from Gregg’s round the corner later. So put that in your pipe and smoke it Hilton restaurant! Not literally, it’s an English figure of speech. You’d need quite a big pipe for that sort of this I imagine. Anyway, I digress. After lunch I attended my first session of the day and it turned out to be a cracker. It was titled the LSC Digital Marketing Panel and featured Darren Hemmings and David Emery. A 3rd panel member had dropped out but nevertheless it was a fascinating discussion about using the web to promote your music as an artist or label. It was interesting to hear from people inside the industry just how effective Youtube pre-roll ads can be, and also just how ineffective paid Facebook ads are. It satisfied the geek and the musician in me all at once I think. Darren and David were both funny and informative. I was glad I went. Later I popped into another random session quite near the end but I don’t think I even worked out what it was about. I’m sure that information is really useful to you.

Following this I headed to the LSC Tech Keynote, which turned out to be an interview really and not a speech. Journalist Eamonn Forde interviewed Andrew Mains the head of a company called Mobile Roadie. I’d never heard of Mobile Roadie but they must be a pretty big deal as they’ve built mobile apps for Madonna and many others. The room was pretty empty when I arrived and sadly after 20mins I could see why. The discussion was boring and pretty hard to concentrate on. Out of British politeness I stayed another 10mins not wanting to obviously walk out on the speakers. There’s only so many levels of Angry Birds you can complete on your phone though before you think “I should just leave and see what else is on”.

The last session of the day I attended was titled the LSC Social Media Surgery. The discussion was a bit slow and I think I already knew most of what was mentioned but had hoped to pick up some tips. It came as something of a relief though when I had to leave before the end anyway. I was making a habit of it. I’m no social media expert but I wasn’t convinced any of these people were either really. It’s quite an esoteric subject as a whole and anyone who claims to be a social media guru usually turns out to be a buffoon. I headed home to meet a friend and unfortunately didn’t catch any of the gigs on Thursday night. I was too busy. All the people who tried to get LSC tickets and couldn’t may now boo and hiss at me.

At this point dear reader I should warn you that I’ve decided to serialise this recap. I’ve rambled on so much about day 1 that it seems a good time to stop and have a break. Let’s have a cup of tea and come back to this hey. I’m putting the kettle on, and like Mrs Doyle I won’t take now for an answer. “Will ya have a cup of tea? Ah go on. Go on, go on, go on.”

….TO BE CONTINUED….

Insert your own dramatic music here, dun dun dun!

Apper has finally a new version!!

Last release is from February, not much changed in PackageKit since that time this is why you haven’t seen a new release, but this release doesn’t include last PackageKit feature (repair system). The focus I’ve put on this was fixing most of the important bugs we had in Apper, so it should be a rock solid release, it is surely not bug free as there is one bug that I won’t fix (I can’t reproduce nor have an idea how to fix, and the QML port will kill it).

The next release will include repair system, showing which packages are untrusted before installing them, and maybe some initial QML usage. Yes, I’m very excited with QML, it allows for a really cool flow especially for this kind of app, though we don’t have Desktop components yet, we can already do lots of cool stuff with it, but first I want to see Matthias work on Appstream and Software Center with openSUSE, that way we can have application data to browse without worry about the distribution!

What this new version includes:

• Automatic Refresh Cache properly fixed
• Initial Listaller support (optional)
• Supported filter added (depends on the backend)
• KDED plugin runs on a separate thread to avoid
  desktop freezes
• Fixed updating packages that were on untrusted repos
• Some man pages (thanks to ximion)
• Many other bug and Krazy fixes

Download (now from KDE ftp’s), mirrors might be still syncing: http://download.kde.org/stable/apper/0.7.2/src/apper-0.7.2.tar.bz2.mirrorlist

Enjoy

Every now and then I’m visting my best friend’s mom (generation > 50a) to update her old Pentium 4 system with the last opensuse software. Each time, I have to restore her Desktop to provide the basic features like managing opened windows, add the clock again, etc. Each time, I pay carefully attention to lock the screen afterwards. Sometimes I get doubts, that they are just fooling me, but my friend declined this, of course.

This time they disarranged the screen in a very extreme way1. Take it for amusement or for considering a clearer warning of unlocking the screen. I vote for:

You might end with a coruppted system!
Please copy this into the form field below:
I asked my son’s friend and got his permission.

;)

1. Did you recognize these empty plasma panels on each edge of the screen which prevents all application to get maximized properly?↩