FLOSS Project Planets

PyCharm: PyCharm 5.1 EAP is Open

Planet Python - Thu, 2016-02-11 09:04

Today we’re extremely glad to announce that the first PyCharm 5.1 Early Access Preview (EAP) build 144.3891.17 is finally available for download! Head on over to the PyCharm Early Access Preview (EAP) page and have a look at the download section where you can get it for your platform right now.

What is the Early Access Program (EAP)?
We at JetBrains believe that making tools for developers should greatly involve listening to developers. Early Access Programs allow you to try pre-release versions of our software to evaluate features that will be added to the next release, closely participate in discussions about PyCharm, and influence the release from early stages onwards.

Once you’ve downloaded the latest EAP build (from its official page), you’ll start getting updates with new features and improvements every week (to receive notifications about updates, make sure to set “Early Access Program” in your update settings). Your feedback is welcome and very much appreciated in our issue tracker or in the comments section of this blog post.

PyCharm 5.1 EAP build # 144.3891.17 introduces a lot of new features, bug-fixes and improvements. There are a number of recently added features that are unique to PyCharm, as well as additional features and improvements from the Web and IntelliJ Platform sides.

Python Related Improvements

Tox support

We’re pleased to announce that PyCharm 5.1 EAP now supports Tox, the virtualenv management and testing tool. Simply right-click the tox.ini file in the project view and choose Run “Tox” or press Ctrl (Cmd) + Shift + F10. See the tests running and results nicely represented in the graphical test runner:

Devpi and Optional PyPI repositories support

Take full control over package repositories and PyPI mirroring solutions with support for optional repositories in PyCharm. To specify a custom repository (including devpi repos):

  1. Go to Settings (Preferences) | Project | Project Interpreter and click the “+” icon:

2. In the Available Packages dialog box, click “Manage Repositories”.
3. In the Manage Repositories dialog box, add, delete or edit the URLs of any repositories:

Folding for Python code blocks, collection literals, and multi-line comments

In addition to the existing code folding functionality in PyCharm’s code editor, we’ve implemented code folding for other code structure elements such as multi-line dictionaries, lists and comments, and if-else, for/while and try-except-finally code blocks in case a code block contains more than one statement:

You can find additional options in Settings (Preferences) | Editor | General | Code folding (“Long string literals”, “Long collection literals”, “Sequential comments”). Read more about code folding options in PyCharm help.

Stepping over yield from and await

A small but neat improvement for developers who use Python 3, PyCharm’s debugger is now able to step over yield from and await expressions. Put a breakpoint before a yield from or an await expression and press the Step over button (F8). The debugger will step to the next line after the expression:

Debugger performance improvements

In addition to stepping over yield from and await expressions as described above, PyCharm’s debugger receives a number of performance improvements that we hope you’ll enjoy. One year before we released PyCharm 4, we joined efforts with Fabio Zadrozny, a PyDev maintainer, and merged our debugger code with that from the original PyDev repository. Now the work on the debugger continues from both sides (PyCharm and PyDev) in one place. As a result of the unification and the work done since then, the debugger in PyCharm has grown more powerful.

The very recent improvement that you can now see in PyCharm 5.1 EAP is that the debugger became much faster and now has optional Cython modules for even additional speedups. The debugger is reported to be now overall 40% faster without Cython and 138% faster with Cython in the benchmarks created. In real world cases, even better gains are expected as the benchmarks were done for the cases where the debugger has more overhead than usual. You can read more about the performance improvements and see some detailed benchmark statistics in the original blog post written by Fabio.

Web Development Improvements

As usual, PyCharm 5.1 will have all features of the upcoming WebStorm 12 release. WebStorm 12 preview builds are already available (read more in the WebStorm blog), so here is just a short summary of what’s new and available in the PyCharm 5.1 preview build 144.3891.17 from the Web side:

  • Better coding assistance for JavaScript, ECMAScript and TypeScript
  • Unused imports warning and сode folding for ES6 imports
  • Remote run and debug for Node.js apps
  • Code assistance in tsconfig.json, .babelrc and .eslintrc
  • Inline rename for TypeScript
  • Debugging JS async code
  • Improved support for React, Angular 1.5 and 2
  • npm support improvements and Webpack support
  • and more…

Some of these features are already bundled with PyCharm 5.1 EAP, while others are available for installation from the JetBrains plugin repository.

Platform Improvements

In addition to the new features and improvements from the Web side, PyCharm 5.1 receives a lot of changes from the underlying IntelliJ Platform. The most noteworthy changes include:

  • Support for RTL languages (Arabic and Hebrew).
  • Easier merging with Git and other VCS thanks to the added by-word difference highlighting.
  • Two new actions to help you work with Git branches, available via the Branches popup (VCS | Git | Branches): Checkout with Rebase and Rename.
  • Git worktree support and updated look and feel of Git Log.
  • Improved speed-search in the show usages popup (Ctrl(Cmd)+Alt+7).

PyCharm 5.1 runs on Java 8

Another important announcement is that the whole IntelliJ Platform has migrated to Java 8. That means that now you can not launch PyCharm under a JDK older than Java 8. This change affects all the EAP builds (144.*) and further major releases this upcoming spring.

Download PyCharm 5.1 preview build 144.3891.17 for your platform right from the project EAP page, and please report any bugs and feature request to our Issue Tracker.

Note: The PyCharm 5.1 preview build is available in two editions – free/open source PyCharm Community Edition and full-fledged PyCharm Professional Edition. While both of them are available for download and use at no charge, PyCharm Professional Edition has a 30-day time-limited evaluation period.

PyCharm Team
The Drive to Develop

Categories: FLOSS Project Planets

Vote for Presentations - OpenStack Summit Austin 2016

Planet KDE - Thu, 2016-02-11 08:50
The first OpenStack Summit this year will take place in Austin (TX, US) from April 25-29, 2016. The "Call for Speakers" period ended some days ago and now the community voting for presentation started and will end 17th February, 11:59 PST (18th February 7:59 UTC / 08:59 CEST). 
I've submitted this time one talk this time:
  • "From Hardware to Application - NFV@OpenStack and Ceph" - This talk will provide insight into NFV cloud specific hardware and data center design, preferred setup and requirements to OpenStack and Ceph and also important design implications and requirements for applications and developers on this platform. 

You can vote, if you are interested to see my talk at the summit, every vote is highly welcome. The full abstract can be found at the voting page.
There is a long list of interesting Ceph related talks, simply use the search field on the voting page. I may provide a list in an additional post later as last time.
Categories: FLOSS Project Planets

PythonClub - A Brazilian collaborative blog about Python: Salvando gráfico de contribuições do Github com Python e Selenium

Planet Python - Thu, 2016-02-11 08:47

Como alguns sabem, sou apaixonado por Python. Atualmente, minha linguagem favorita por conta de sua simplicade e poder (além de ficar LINDJA toda indentada, hahahaha).

Uma das coisas mais legais da linguagem é a enorme quantidade de bibliotecas disponíveis. Cada dia que abro um grupo de discussão acabo conhecendo alguma funcionalidade interessante. Se você faz parte de algum desses grupos, provavelmente já viu o post do Alex Recker "Using Selenium to Buy a Bus Pass", em que ele mostra como automatizou a compra de passagens de ônibus com Selenium e Python.

Eu já havia ouvido falar do Selenium, mas nunca tinha experimentado na prática e o post do Alex foi o empurrão que faltava.

Obviamente, meu projetinho é bem mais simples, mas foi algo divertido de se fazer como forma de aprendizado. Batizei-o de GHSS(Github Screenshot). Como o próprio nome sugere, ele entra no seu perfil do Github e tira um screenshot do gráfico de contribuições, salvando com a data atual.

Abaixo, irei mostrar como fazer. Visto que há muita gente que usa Python sem ser programador por profissão, tentarei explicar de forma mais simples possível. O código completo pode ser encontrado no meu Github.

Neste código, utilizaremos o Python2.

Primeiramente, temos que importar todas as bibliotecas necessárias.

Na linha 1, importamos o "OS", que será utilizado para "acharmos" o arquivo secrets.yml. Explicarei daqui a pouco.

Na linha 2, importamos do Selenium o Webdriver, responsável pela automatização (abertura das páginas e preenchimento dos campos).

Nas próximas duas linhas, importamos as bibliotecas restantes que são responsáveis pelo nosso arquivo secrets.yml, no qual o username e password serão guardados, e pela data que será salva no nome do arquivo final.

Na última linha, importamos o responsável por tirar o screenshot.

import os from selenium import webdriver import yaml from datetime import date import pyscreenshot as ImageGrab

Neste bloco de código, mostramos ao nosso programa onde está nosso arquivo secrets.yml e o carregamos.

cur_dir = os.path.dirname(os.path.realpath(__file__)) secret_path = os.path.join(cur_dir, 'secrets.yml') with open(secret_path, 'r') as stream: data = yaml.load(stream) USERNAME = data.get('user','') PASSWORD = data.get('password')

O arquivo secrets.yml é composto por apenas dois campos, "password" e "user", que, PASMEM, são para inserir sua senha e seu usuário.

password: senha_do_zezinho_hacker user: zezinhohacker123

Nestas três linhas abrimos o Firefox, passamos para ele qual o endereço desejamos acessar e maximizamos a janela, respectivamente.

driver = webdriver.Firefox() driver.get("https://github.com/login") driver.maximize_window()

Aqui é onde a "mágica" acontece.

Na primeira linha, a propriedade "find_element_by_id" busca o campo "login_field", onde devemos inserir o nome de usuário. Na linha posterior, enviamos aquele username informado lá no secrets, lembra?

Nas próximas duas linhas, é feito o mesmo procedimento, mas, desta vez, com a senha.

Na última, clicamos o botão para logarmos.

email = driver.find_element_by_id("login_field") email.send_keys(USERNAME) senha = driver.find_element_by_id("password") senha.send_keys(PASSWORD) driver.find_element_by_name('commit').click()

Nesta linha, nós entramos no nosso perfil do Github.

Quando utilizamos {0}, "guardamos" o espaço, para o que informarmos adiante. Ou seja, no espaço reservado, será inserido o username.

driver.get("https://github.com/{0}" .format(USERNAME))

Por exemplo, se fizermos o seguinte código:

print("Meus esportes preferidos são: {0}, {1} e {2}" .format("futebol", "basquete", "corrida"))

O resultado será:

Meus esportes preferidos são: futebol, basquete e corrida.

Deu para entender?

Na última linha do programa, salvamos a imagem.

No campo bbox, informamos qual área da tela queremos dar o screenshot, na ordem: X1, Y1, X2, Y2. Você pode alterá-lo de acordo com seu navegador.

No save, utilizamos o que ensinei acima para gerar o arquivo da seguinte maneira: "dataatual_gitshot_nomedousuario".

img = ImageGrab.grab(bbox=(460,540,770,208)).save("{0}_gitshot_{1}.png" .format(date.today(), USERNAME))

Este será o resultado. O nome do arquivo, no meu caso, ficou "2016-01-24_gitshot_othonalberto.png".

Código completo:

#!/usr/bin/env python # -*- coding: utf-8 -*- import os from selenium import webdriver import yaml from datetime import date import pyscreenshot as ImageGrab cur_dir = os.path.dirname(os.path.realpath(__file__)) secret_path = os.path.join(cur_dir, 'secrets.yml') with open(secret_path, 'r') as stream: data = yaml.load(stream) USERNAME = data.get('user','') PASSWORD = data.get('password') driver = webdriver.Firefox() driver.get("https://github.com/login") driver.maximize_window() email = driver.find_element_by_id("login_field") email.send_keys(USERNAME) senha = driver.find_element_by_id("password") senha.send_keys(PASSWORD) driver.find_element_by_name('commit').click() driver.get("https://github.com/{0}" .format(USERNAME)) img = ImageGrab.grab(bbox=(460,540,770,208)).save("{0}_gitshot_{1}.png" .format(date.today(), USERNAME)) # bbox=(X1, Y1, X2, Y2)

É isso! Espero ter contribuído com o conhecimento de vocês com este post e gerado curiosidade para que experimentem o Selenium.

Quem quiser contribuir, seja com código ou sugestões, sinta-se à vontade.


Categories: FLOSS Project Planets

Bits from Debian: Tails installer is now in Debian

Planet Debian - Thu, 2016-02-11 08:30

Tails (The amnesic incognito live system) is a live OS based on Debian GNU/Linux which aims at preserving the user's privacy and anonymity by using the Internet anonymously and circumventing censorship. Installed on a USB device, it is configured to leave no trace on the computer you are using unless asked explicitly.

As of today, the people the most needy for digital security are not computer experts. Being able to get started easily with a new tool is critical to its adoption, and even more in high-risk and stressful environments. That's why we wanted to make it faster, simpler, and more secure to install Tails for new users.

One of the components of Tails, the Tails Installer is now in Debian thanks to the Debian Privacy Tools Maintainers Team.

Tails Installer is a graphical tool to install or upgrade Tails on a USB stick from an ISO image. It aims at making it easier and faster to get Tails up and running.

The previous process for getting started with Tails was very complex and was problematic for less tech-savvy users. It required starting Tails three times, and copying the full ISO image onto a USB stick twice before having a fully functional Tails USB stick with persistence enabled.

This can now be done simply by installing Tails Installer in your existing Debian system, using sid, stretch or jessie-backports, plugging a USB stick and choosing if one wants to update the USB stick or to install Tails using a previously downloaded ISO image.

Tails Installer also helps Tails users to create an encrypted persistent storage for personal files and settings in the rest of the available space.

Categories: FLOSS Project Planets

drunken monkey: Create Git diffs with proper function context

Planet Drupal - Thu, 2016-02-11 07:37

For years I have been annoyed (slightly, but still) that Git diffs for PHP classes always just contained the class header instead of the method header as the function context. I finally got round to doing a bit of research and it turns out that the solution is astonishingly easy: just one small and simple config file and it will magically work.

The problem

You might have noticed it, and maybe been annoyed by it, too: when you create a diff file with Git and have changes inside PHP class files, you get hunks that look like the following:

@@ -40,6 +40,7 @@ class SearchApiViewsHandlerFilter extends views_handler_filter {
       '<=' => t('Is less than or equal to'),
       '=' => t('Is equal to'),
       '<>' => t('Is not equal to'),
+      '!=' => t('Is REALLY not equal to'),
       '>=' => t('Is greater than or equal to'),
       '>' => t('Is greater than'),
       'empty' => t('Is empty'),

So, where you would normally expect the function context, to quickly get an idea what that change means, the diff instead just contains the class name, which is much less helpful (especially if the file name already tells you the class).

Well, after years of being regularly (slightly) annoyed by this (more so in recent years, with the OOP shift in Drupal 8), I finally searched this new place called "the interwebs" and arrived at the very simple solution.

The solution

It turns out Git's diff functionality already has support for creating the right function context information for various file types – it just doesn't know which files correspond to which file type. (It seems not even the standard .php extension is recognized by default.)

To remedy this, simply create a file with the following contents:

*.engine   diff=php
*.inc      diff=php
*.install  diff=php
*.module   diff=php
*.php      diff=php
*.profile  diff=php
*.test     diff=php
*.theme    diff=php

Save the file either in .git/info/attributes or .gitattributes (for just the local project), or (to change this globally) in $HOME/.config/git/attributes (or $XDG_CONFIG_HOME/git/attributes, if that variable is set). That's it – testing again, we now see the proper function context in the diff:

@@ -40,6 +40,7 @@ public function operator_options() {
       '<=' => t('Is less than or equal to'),
       '=' => t('Is equal to'),
       '<>' => t('Is not equal to'),
+      '!=' => t('Is REALLY not equal to'),
       '>=' => t('Is greater than or equal to'),
       '>' => t('Is greater than'),
       'empty' => t('Is empty'),

Much better, right?

Note: I haven't yet found out where to put the global attributes file for Windows. If you know, please post a comment and tell me, and I'll include it here.

Background information

For more information on Git attributes, please see the documentation. I didn't know about this at all before, but it seems there's actually a lot you can configure per file type, so there might be other very practical tips for customization.
If you find or know of any, please post them so others can profit from them, too – just as I hope this post was useful to at least a few of you!

As an afterthought, I've also added this tip to the Git documentation in Drupal.org's handbook. Hopefully, more people will find it that way.

Categories: FLOSS Project Planets

Robin Wilson: Hacking the Worcester Wave thermostat in Python – Part 2

Planet Python - Thu, 2016-02-11 07:02

In the previous part we had established that the Worcester Wave thermostat app communicates with a remote server (run by Worcester Bosch) using the XMPP protocol, with TLS encryption. However, because of the encryption we haven’t yet managed to see the actual content of any of these messages!

To decrypt the messages we need to do a man in the middle attack. In this sort of attack we put insert ourselves between the two ends of the conversation and intercept messages, decrypt them and then forward them on to the original destination. Although it is called an ‘attack’, here I am basically attacking myself – because I’m basically monitoring communications going from a phone I own through a network I own. Beware that doing this to networks that you do not own or have permission to use in this way is almost certainly illegal.

There is a good guide to setting all of this up using a tool called sslsplit, although I had to do things slightly differently as I couldn’t get sslsplit to work with the STARTTLS method used by the Worcester Wave (as you may remember from the previous part, STARTTLS is a way of starting the communication in an unencrypted manner, and then ‘turning on’ the encryption part-way through the communication).

The summary of the approach that I used is:

  1. I configured a Linux server on my network to use Network Address Translation (NAT) – so it works almost like a router. This means that I can then set up another device on the network to use that server as a ‘gateway’, which means it will send all traffic to that server, which will then forward it on to the correct place.
  2. I created a self-signed root certificate on the server. A root certificate is a ‘fully trusted’ certificate that can be used to trust any other certificates or keys derived from it (that explanation is probably technically wrong, but it’s conceptually right).
  3. I installed this root certificate on a spare Android phone, connected the phone to my home wifi and configured the Linux server as the gateway. I then tested, and could access the internet fine from the phone, with all of the communications going through the server.

Now, if I use the Worcester Wave app on the phone, all of the communications will go through the server – and the phone will believe the server when it says that it is the Bosch server at the other end, because of the root certificate we installed.

Now we’ve got all of the certificates and networking stuff configured, we just need to actually decrypt the messages. As I said above, I tried using SSLSplit, but it couldn’t seem to cope with STARTTLS. I found the same with Wireshark itself, so looked for another option.

Luckily, I found a great tool called starttls-mitm which does work with STARTTLS. Even more impressively it’s barely 80 lines of Python, and so it’s very easy to understand the code. This does mean that it is less configurable than a more complex tool like SSLSplit – but that’s not a problem for me, as the tool does exactly what I want . It is even configured for XMPP by default too! (Of course, as it is written in Python I could always modify the code myself if I needed to).

So, running starttls-mitm with the appropriate command-line parameters (basically your keys, certificates etc) will print out all communications: both the unencrypted ones before the STARTTLS call, and the decrypted version of the encrypted ones after the STARTTLS call. If we then start doing something with the app while this is running, what do we get?

Well, first we get the opening logging information starttls-mitm telling us what it is doing:

LISTENER ready on port 8443
CLIENT CONNECT from: ('', 57913)

We then start getting the beginnings of the communication:

C->S 129 '<stream:stream to="wa2-mz36-qrmzh6.bosch.de" xmlns="jabber:client" xmlns:stream="http://etherx.jabber.org/streams" version="1.0">'
S->C 442 '<?xml version=\'1.0\' encoding=\'UTF-8\'?><stream:stream xmlns:stream="http://etherx.jabber.org/streams" xmlns="jabber:client" from="wa2-mz36-qrmzh6.bosch.de" id="260d2859" xml:lang="en" version="1.0"><stream:features><starttls xmlns="urn:ietf:params:xml:ns:xmpp-tls"></starttls><mechanisms xmlns="urn:ietf:params:xml:ns:xmpp-sasl"><mechanism>DIGEST-MD5</mechanism></mechanisms><auth xmlns="http://jabber.org/features/iq-auth"/></stream:features>'

Fairly obviously, C->S are messages from the client to the server, and S->C are messages back from the server (the numbers directly afterwards are just the length of the message). These are just the initial handshaking communications for the start of XMPP communication, and aren’t particularly exciting as we saw this in Wireshark too.

However, now we get to the interesting bit:

C->S 51 '<starttls xmlns="urn:ietf:params:xml:ns:xmpp-tls"/>'
S->C 50 '<proceed xmlns="urn:ietf:params:xml:ns:xmpp-tls"/>'
Wrapping sockets.

The STARTTLS message is sent, and the server says PROCEED, and – crucially – starttls-mitm notices this and announces that it is ‘wrapping sockets’ (basically enabling the decryption of the communication from this point onwards).

I’ll skip the boring TLS handshaking messages now, and skip to the initialisation of the XMPP protocol itself. I’m no huge XMPP expert, but basically the iq messages are ‘info/query’ messages which are part of the handshaking process with each side saying who they are, what they support etc. This part of the communication finishes with each side announcing its ‘presence’ (remember, XMPP is originally a chat protocol, so this is the equivalent of saying you are ‘Online’ or ‘Active’ on Skype, Facebook Messenger or whatever).

C->S 110 '<iq id="lj8Vq-1" type="set"><bind xmlns="urn:ietf:params:xml:ns:xmpp-bind"><resource>70</resource></bind></iq>'
S->C 188 '<iq type="result" id="lj8Vq-1" to="wa2-mz36-qrmzh6.bosch.de/260d2859"><bind xmlns="urn:ietf:params:xml:ns:xmpp-bind"><jid>rrccontact_458921440@wa2-mz36-qrmzh6.bosch.de/70</jid></bind></iq>'
C->S 87 '<iq id="lj8Vq-2" type="set"><session xmlns="urn:ietf:params:xml:ns:xmpp-session"/></iq>'
S->C 86 '<iq type="result" id="lj8Vq-2" to="rrccontact_458921440@wa2-mz36-qrmzh6.bosch.de/70"/>'
C->S 74 '<iq id="lj8Vq-3" type="get"><query xmlns="jabber:iq:roster" ></query></iq>'
S->C 123 '<iq type="result" id="lj8Vq-3" to="rrccontact_458921440@wa2-mz36-qrmzh6.bosch.de/70"><query xmlns="jabber:iq:roster"/></iq>'
C->S 34 '<presence id="lj8Vq-4"></presence>'
C->S 34 '<presence id="lj8Vq-5"></presence>'

Now all of the preliminary messages are dealt with, we get to the good bit. The message below is sent from the client (the phone app) to the server:

C->S 162 '<message id="lj8Vq-6" to="rrcgateway_458921440@wa2-mz36-qrmzh6.bosch.de" type="chat"><body>GET /ecus/rrc/uiStatus HTTP /1.0\nUser-Agent: NefitEasy</body></message>'

It basically seems to be a HTTP GET request embedded within an XMPP message. That seems a bit strange to me – why not just use HTTP directly? – but at least it is easy to understand. The URL that is being requested also makes sense – I was on the ‘home screen’ of the app at that point, so it was grabbing the status for displaying in the user-interface (things like current temperature, set-point temperature, whether the boiler is on or not, etc).

Now we can see the response from the server:

S->C 904 '<message to="rrccontact_458921440@wa2-mz36-qrmzh6.bosch.de/70" type="chat" xml:lang="en" from="rrcgateway_458921440@wa2-mz36-qrmzh6.bosch.de/RRC-RestApi"><body>HTTP/1.0 200 OK\nContent-Length: 640\nContent-Type: application/json\nconnection: close\n\n5EBW5RuFo7QojD4F1Uv0kOde1MbeVA46P3RDX6ZEYKaKkbLxanqVR2I8ceuQNbxkgkfzeLgg6D5ypF9jo7yGVRbR/ydf4L4MMTHxvdxBubG5HhiVqJgSc2+7iPvhcWvRZrRKBEMiz8vAsd5JleS4CoTmbN0vV7kHgO2uVeuxtN5ZDsk3/cZpxiTvvaXWlCQGOavCLe55yQqmm3zpGoNFolGPTNC1MVuk00wpf6nbS7sFaRXSmpGQeGAfGNxSxfVPhWZtWRP3/ETi1Z+ozspBO8JZRAzeP8j0fJrBe9u+kDQJNXiMkgzyWb6Il6roSBWWgwYuepGYf/dSR9YygF6lrV+iQdZdyF08ZIgcNY5g5XWtm4LdH8SO+TZpP9aocLUVR1pmFM6m19MKP+spMg8gwPm6L9YuWSvd62KA8ASIQMtWbzFB6XjanGBQpVeMLI1Uzx4wWRaRaAG5qLTda9PpGk8K6LWOxHwtsuW/CDST/hE5jXvWqfVmrceUVqHz5Qcb0sjKRU5TOYA+JNigSf0Z4CIh7xD1t7bjJf9m6Wcyys/NkwZYryoQm99J2yH2khWXyd2DRETbsynr1AWrSRlStZ5H9ghPoYTqvKvgWsyMVTxbMOht86CzoufceI2W+Rr9</body></message>'

Oh. This looks a bit more complicated, and not very easily interpretable. Lets format the main body of the message formatted a bit more nicely:

HTTP/1.0 200 OK Content-Length: 640 Content-Type: application/json connection: close 5EBW5RuFo7QojD4F1Uv0kOde1MbeVA46P3RDX6ZEYKaKkbLxanqVR2I8ceuQNbxkgkfzeLgg6D5ypF9jo7yGVRbR/ydf4L4MMTHxvdxBubG5HhiVqJgSc2+7iPvhcWvRZrRKBEMiz8vAsd5JleS4CoTmbN0vV7kHgO2uVeuxtN5ZDsk3/cZpxiTvvaXWlCQGOavCLe55yQqmm3zpGoNFolGPTNC1MVuk00wpf6nbS7sFaRXSmpGQeGAfGNxSxfVPhWZtWRP3/ETi1Z+ozspBO8JZRAzeP8j0fJrBe9u+kDQJNXiMkgzyWb6Il6roSBWWgwYuepGYf/dSR9YygF6lrV+iQdZdyF08ZIgcNY5g5XWtm4LdH8SO+TZpP9aocLUVR1pmFM6m19MKP+spMg8gwPm6L9YuWSvd62KA8ASIQMtWbzFB6XjanGBQpVeMLI1Uzx4wWRaRaAG5qLTda9PpGk8K6LWOxHwtsuW/CDST/hE5jXvWqfVmrceUVqHz5Qcb0sjKRU5TOYA+JNigSf0Z4CIh7xD1t7bjJf9m6Wcyys/NkwZYryoQm99J2yH2khWXyd2DRETbsynr1AWrSRlStZ5H9ghPoYTqvKvgWsyMVTxbMOht86CzoufceI2W+Rr9

So, it seems to be a standard HTTP response (200 OK), but the body looks like it is encoded somehow. I assume that the decoded body would be something like JSON or XML or something containing the various status values – but how do we decode it to get that?

I tried all sorts of things like Base64, MD5 and so on but nothing seemed to work. I gave up on this for a few days, while gently pondering it in the back of my mind. When I came back to it, I realised that the data here was probably actually encrypted, using the Access Code that comes with the Wave and the password that you set up when you first connect the Wave. Of course, to decrypt it we need to know how it was encrypted…so time to break out the next tool: a decompiler.

Yes, that’s right: to fully understand exactly what the Wave app is doing, I needed to decompile the Android app’s APK file and look at the code. I did this using the aptly named Android APK Decompiler, and got surprisingly readable Java code out of it! (I mean, it had a lot of goto statements, but at least the variables had sensible names!)

It’s difficult to explain the full details of the encryption/decryption algorithm in prose – so I’ve included the Python code I implemented to do this below. However, a brief summary is that: the main encryption is AES using ECB, with keys generated from the MD5 sums of combinations of the Access Code, the password and a ‘secret’ (a value hard-coded into the app).

def encode(s): abyte1 = get_md5(access + secret) abyte2 = get_md5(secret + password) key = abyte1 + abyte2 a = AES.new(key) a = AES.new(key, AES.MODE_ECB) res = a.encrypt(s) encoded = base64.b64encode(res) return encoded def decode(data): decoded = base64.b64decode(data) abyte1 = get_md5(access + secret) abyte2 = get_md5(secret + password) key = abyte1 + abyte2 a = AES.new(key) a = AES.new(key, AES.MODE_ECB) res = a.decrypt(decoded) return res

Using these functions we can decrypt the response to the GET /ecus/rrc/uiStatus message that we saw earlier, and we get this:

{'id': '/ecus/rrc/uiStatus', 'recordable': 0, 'type': 'uiUpdate', 'value': {'ARS': 'init', 'BAI': 'CH', 'BBE': 'false', 'BLE': 'false', 'BMR': 'false', 'CPM': 'auto', 'CSP': '31', 'CTD': '2014-12-26T12:34:27+00:00 Fr', 'CTR': 'room', 'DAS': 'off', 'DHW': 'on', 'ESI': 'off', 'FPA': 'off', 'HED_DB': '', 'HED_DEV': 'false', 'HED_EN': 'false', 'HMD': 'off', 'IHS': 'ok', 'IHT': '16.70', 'MMT': '15.5', 'PMR': 'false', 'RS': 'off', 'TAS': 'off', 'TOD': '0', 'TOR': 'on', 'TOT': '17.0', 'TSP': '17.0', 'UMD': 'clock'}, 'writeable': 0}

This makes far more sense!

It may not be immediately apparent what each field is (three character variable names – great!), but some of them are fairly obvious (CTD presumably stands for something like Current Time/Date), or can be established by decoding a number of messages with the boiler in different states (showing that DHW stands for Domestic Hot Water and BAI for Burner Active Indicator).

We’ve made a lot of progress in the second part of this guide: we’ve now decrypted the communications, and worked out how to get all of the status information that is shown on the app home screen. At this point I set up a simple temperature monitoring system to produce nice graphs of temperature over time – but I’ll leave the description of that to later in the series. In the next part we’re going to look at sending messages to actually change the state of the thermostat (such as setting a new temperature, or switching to manual mode), and then have a look at the Python library I’ve written to control the thermostat.

Categories: FLOSS Project Planets

Season of KDE (3) – KDE Action Restrictions

Planet KDE - Thu, 2016-02-11 06:58

To have a working KIOSK tool again, Confine needs to support another key feature of the KIOSK framework: KDE Action Restrictions.

A KDE Application can check if a certain Action is allowed. For example the action logout could be forbidden. So every time the logout action is performed, an application checks, if this action is allowed. Thus a user can still use an application, but certain actions are restricted to him.

These settings are stored in the kdeglobals file, in your KDE profile directory.

For my season of KDE project I had to compile a list of all currently used KDE Action Restrictions. I did a quick search throughout all KDE projects for the use of the KAuthorized framework and published my results in the confine repository. Although Confine is not finished yet, you might found this list beneficial, if you want to configure KIOSK restrictions manually.

Still a lot of KDE applications don’t support KDE Action Restrictions (anymore). For a revival of the whole KIOSK concept it would be highly beneficial, if more application are starting to use KDE Action Restrictions.

Categories: FLOSS Project Planets

Wellnet Blog: Logging in Drupal 8

Planet Drupal - Thu, 2016-02-11 05:48

Many Drupal websites are not just simple websites but real applications, with many functionalities, integrations and logic. It’s often necessary to keep track of what is happening in the application to understand if something went wrong or even just to monitor the user activities.
It is...

Categories: FLOSS Project Planets

eGenix.com: Python Meeting Düsseldorf - 22 New Videos Online

Planet Python - Thu, 2016-02-11 04:00

The following text is in German, since we're announcing videos available from a regional user group meeting in Düsseldorf, Germany.

Was ist das Python Meeting Düsseldorf ?

Das Python Meeting Düsseldorf ist eine Veranstaltung, die alle drei Monate in Düsseldorf stattfindet und sich an Python Begeisterte aus der Region wendet.

Bei jedem Treffen werden Vorträge gehalten und anschließend in Diskussionen vertieft. Die Meetings dauern üblicherweise ca. 2 Stunden und münden anschließend in eine Restaurant-Session.

Teilnehmer kommen aus ganz Nordrhein-Westfalen, hauptsächlich allerdings aus der näheren Umgebung.

Neue Videos

Um die Vorträge auch für andere Python Enthusiasten zugänglich zu machen, nehmen wir die Vorträge auf, produzieren daraus Videos und laden diese auf unseren PyDDF YouTube Channel hoch.

In den letzten Wochen haben wir die Videos der letzten Treffen aufgearbeitet. Insgesamt sind 22 neue Videos dazugekommen. Viel Spaß damit:

Python Meeting Düsseldorf 2016-01-19 Python Meeting Düsseldorf 2015-10-21 Python Meeting Düsseldorf 2015-07-29 Python Meeting Düsseldorf 2015-04-29
Die vollständige Liste aller mehr als 90 Python Meeting Videos ist über unsere Video Liste verfügbar.
Weitere Informationen

Weitere Informationen und Termine rund um das Python Meeting Düsseldorf stehen auf unserer Webseite:


Viel Spaß !

Marc-Andre Lemburg, eGenix.com

Categories: FLOSS Project Planets

Senthil Kumaran: Access android devices from LXC

Planet Apache - Thu, 2016-02-11 01:35

LXC aka Linux Containers are a convenient way to run a light weight Virtual Machine. LXC provides a complete operating system with access to devices attached to host machine. Let us see how we can access an Android device from a LXC instance via adb or fastboot. I assume you have a working LXC with networking setup properly. I am using a LXC named 'test-lxc' which is a Debian sid based container (root@test-lxc:/#) and a Google Nexus 4 as android device with debug mode enabled. My host machine (stylesen@harshu:~$) is a Debian sid based Thinkpad.When I plug in the USB cable from the android device to my host machine I could see the following in the lsusb output:

stylesen@harshu:~$ lsusb
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 007: ID 04f2:b217 Chicony Electronics Co., Ltd Lenovo Integrated Camera (0.3MP)
Bus 001 Device 005: ID 147e:2016 Upek Biometric Touchchip/Touchstrip Fingerprint Sensor
Bus 001 Device 021: ID 18d1:4ee0 Google Inc.
Bus 001 Device 008: ID 0835:1601 Action Star Enterprise Co., Ltd
Bus 001 Device 003: ID 0424:2514 Standard Microsystems Corp. USB 2.0 Hub
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

From the above we can see my Nexus 4 (Google Inc.) is connected in USB bus 001 as device 021. The actual path of the Nexus 4 device translates to the following:


Within my LXC, though the Nexus 4 appears in lsusb output as follows, adb or fastboot does not have access to this device yet:

root@test-lxc:/# lsusb
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 007: ID 04f2:b217 Chicony Electronics Co., Ltd Lenovo Integrated Camera (0.3MP)
Bus 001 Device 005: ID 147e:2016 Upek Biometric Touchchip/Touchstrip Fingerprint Sensor
Bus 001 Device 021: ID 18d1:4ee0 Google Inc.
Bus 001 Device 008: ID 0835:1601 Action Star Enterprise Co., Ltd
Bus 001 Device 003: ID 0424:2514 Standard Microsystems Corp. USB 2.0 Hub
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Both fastboot and adb cannot see the device as shown below:

root@test-lxc:/# fastboot devices
root@test-lxc:/# adb devices
List of devices attached


In order to make this device accessible from within the container, use the following command on the host machine:

stylesen@harshu:~$ sudo lxc-device -n test-lxc add /dev/bus/usb/001/021

Once the above command is run, we can access the Nexus 4 via fastboot or adb as follows:

root@test-lxc:/# fastboot devices
04f228d1d9c76f39    fastboot
root@test-lxc:/# fastboot reboot

finished. total time: 3.011s

Every time the Nexus 4 is disconnected from USB port and reconnected which also includes a 'reboot' or 'reboot-bootloader', the device number within the USB bus changes, though the bus number remains the same. For example, for every reboot or disconnection the device path will become something like the following:

after reboot:  /dev/bus/usb/001/022

after reboot:  /dev/bus/usb/001/023


after reboot: /dev/bus/usb/001/0NN

With the above it is difficult to automate things and also every time you must see output of lsusb to identify the device number and add it to the container with lxc-device command. To make things simple and definite, I have the following udev rule in /etc/udev/rules.d/51-android.rules

SUBSYSTEM=="usb", ATTR{idVendor}=="18d1", ATTR{idProduct}=="4ee2", ATTRS{serial}=="04f228d1d9c76f39", MODE="0666", GROUP="plugdev", SYMLINK+="android-nexus4"

Note ATTRS{serial} and SYMLINK+="android-nexus4" which can help us to identify and create a symlink to the Nexus 4 device consitently without worrying about the USB device number on the bus. With the above in place we get a device as follows:

stylesen@harshu:~$ ls -alh /dev/android-nexus4
lrwxrwxrwx 1 root root 15 Feb 11 11:36 /dev/android-nexus4 -> bus/usb/001/022

Now things should be simple to add the above android device to the container with the following command:

stylesen@harshu:~$ sudo lxc-device -n test-lxc add $(sudo readlink -f /dev/android-nexus4)

Within the the container we can access the Nexus 4 via adb as follows:

root@test-lxc:/# adb devices
List of devices attached
04f228d1d9c76f39    device

NOTE1: lsusb is a command which is available via usbutils package in Debian.

Categories: FLOSS Project Planets

DrupalCon News: User Experience Design with Drupal and Beyond

Planet Drupal - Wed, 2016-02-10 23:04

Did you know you, yes YOU, could potentially provide the most valuable information at all of DrupalCon New Orleans?

Categories: FLOSS Project Planets

GCompris: Patreon and New Logo

Planet KDE - Wed, 2016-02-10 21:10

Hello everyone,

A few days ago, I created a page on Patreon to support my work on making new graphics on GCompris. As you may know, last year I started this project, and could make a good start thanks to a little crowd-funding campaign. However there’s a lot of remaining work to finish the task. A lot of activities need to be updated, and new activities will always need some new design and graphics.

So if you want to support GCompris, you can become my patron for this project.
Before resuming my work on the activities, I took the hard and delicate task to update the logo and the main icon of the application.

Now is a good time to have a new icon, for several reasons.
-The old icon had no real meaning, only legacy (which, for a kid that sees GCompris for the first time, doesn’t mean anything)
-Tux is already the mascot of a completely different kind of software. Having him along with other FLOSS mascots inside some activities is cool, but he doesn’t represent enough GCompris to be in the icon.
-The Qt port is still in progress, and it makes sense to have a new icon for it.
-With the new graphics in the application, GCompris needs a good branding that looks good and makes sense.

Also, as some people said they would like to keep the legacy biplane+tux, I tried. I spent countless hours trying to make something looking good, looked at it from every angles. I really couldn’t find a way, and at some point I was feeling like loosing my time.

Full of energy from these failures, I started a new icon from scratch. We had a brainstorm topic on the mailing list recently for a new icon, so I had some indications to begin with. It should mean things like education and gaming, be colorful and cute.

I spare you all the iterations, but after pages of sketches, several proposal and lot of constructive discussions on IRC, here is the final result, along with some explanations:

This is the new icon.
The globe is a symbol for the educational part of GCompris. Also luckily, it is still linked in a way to the idea of the plane from the previous icon. Also it is the same G and orange circle that is used as about button in the main menu.
The dice is a symbol for the gaming part of GCompris, and it also represents counting and maths.
I chose the orange color for the globe for several reasons, probably the most important is because it still contains some yellow from the previous icon, but it is warmer. The blue for the dice adds some contrast.

I tweaked it to follow the main guidelines of Breeze-icon-design, I like the look it gives to it.

This is the new logo with the full name.
It started as a clean-up of the previous one, changing the style and colors of the letters to something soft and colored. Then after making the icon, I added the globe to it, thanks to a suggestion on IRC.

This is a “light” version of the logo, without the globe so it fits better inside a line.

I hope everyone will be happy with this new logo and icon. I know lot of old-timers had some affection for the plane and tux logo, but if you read what I said above, you can see that it was a well considered and discussed change, with lot of good reasons to happen.

Again, if you like my work on GCompris, check this link to see how you can support it. Expect a new activity update next month.

Categories: FLOSS Project Planets

Neon and Plasma Relationship

Planet KDE - Wed, 2016-02-10 19:21

As we saw neon, a new and fresh Linux distribution was launched last week. This project is incubated by the KDE Community, sharing KDE's hosting and community. Hopefully we'll see neon flourish into an awesome distribution over time.

However, I have seen some potential confusion in an article reaching a conclusion that this might be in some way problematic for other distributions to deploy KDE software. To make sure we're all on the same page I wanted to give a clarifying statement from the Plasma mantainer.

Plasma is and remains distro-agnostic. It's in our interest to help all of our distribution channels. As long as distributions continue to keep up with the dependencies we need and work well with us, we support everyone as best as we can.

Categories: FLOSS Project Planets

Justin Mason: Links for 2016-02-10

Planet Apache - Wed, 2016-02-10 18:58
Categories: FLOSS Project Planets

KDE Applications 16.04 Schedule finalized

Planet KDE - Wed, 2016-02-10 16:59

It is available at the usual place https://techbase.kde.org/Schedules/Applications/16.04_Release_Schedule.

Dependency freeze is in 4 weeks and Feature Freeze in 6 weeks, so hurry up!

Categories: FLOSS Project Planets

Acquia Developer Center Blog: What We're Up To at DrupalCon Asia in Mumbai

Planet Drupal - Wed, 2016-02-10 15:07
Jeffrey A. "jam" McGuire

Acquia is once again a sponsor of DrupalCon, and those of us who have the chance to take part in DrupalCon Asia in Mumbai, India are getting excited. This post covers the sessions we'll be part of in Mumbai.

Tags: acquia drupal planetdrupalconMumbai
Categories: FLOSS Project Planets

J-P Stacey: Building a Drupal site with Behaviour-Driven Development

Planet Drupal - Wed, 2016-02-10 15:03

(This article first appeared on the Agile Collective blog.)

The Global Canopy Programme (GCP) needed to retrieve news syndicated from many public sources, manage it via an internal application, then re-syndicate it reliably to several public-facing websites. This application—called Forest Desk—needed to be described and built “just in time”, both to fit the clear initial requirements but also to adapt to any discoveries made along the way.

Read more of "Building a Drupal site with Behaviour-Driven Development"

Categories: FLOSS Project Planets

Launching docs.krita.org: the Krita Learning Place!

Planet KDE - Wed, 2016-02-10 15:03

For months, we have been working on something awesome: docs.krita.org! And now it’s the time to share our work with you. Over the past year, we created a comprehensive manual for Krita on KDE’s Userbase website, and now we’ve got a beautiful new home for our work. All the content has been ported to the new learning area — and we want to extend the content further as well as add more Krita tutorials!

The new and updated docs.krita.org is the place for everyone who wants to use Krita. Artists who need a good introduction, painters who want to browse brush packs, or curious sketchers looking for information on what all of the features in Krita do. The perfect place to start when learning anything about Krita. And digital painting in general.

Here are some of the things we’re sure you’ll appreciate:

Better Search Capabilities

The docs site has its own search functionality now! The search will pick up not just page titles, but also content. This makes it much easier to find what you are looking for! And the live search bar also will give suggestions as you type.

Improved Navigation

All the content is now organized in a page tree display. You can drill down into the specific areas that you are interested in. The navigation turns into a fixed layout to make it easy to reference where you are. And pages include a previous and next page function to help you move around. Breadcrumbs exist above the title as well. Click on them to go up a directory, as usual!

Combined Educational Resources

No more bouncing between different websites for learning. We have moved the User Manual and the FAQ to the learning area. Combined with the live search, it means finding answers to your questions has never been easier! And there’s so much content here already, most common questions are answered, and quite a few esoteric ones as well!

Updated & New Content

We are always trying to update content, but we spent a little more time while working on these updates.

We have a new Unstable section with new features that being worked on right now, like Animation. Then, when a feature is released in a stable version, we will move the documentation out of the Unstable section.

If you start out using Krita, you might have questions like “how to save an image for the web”, or would like to see examples of how to use Krita. There are lots of tutorials spread all over world, created by Krita developers and users. So many that it’s getting difficult to find, and even more, to find them again! For this we created the tutorials section.

And if you’ve used Krita for a while, you’ll have seen that Krita has plenty features that are unusual, or even unique! Photoshop tutorials won’t help you here! So we created a dedicated area where we can tell you how to use Krita’s advanced features, and where they go beyond to what you might have been expecting.

Of course, updating the documentation and education for Krita, and keeping it up-to-date is a work-in-progress. It’ll always be a work in progress! But we are really proud of all these improvements! Learning Krita, or getting the most out of Krita, just got a whole lot easier!

Categories: FLOSS Project Planets

Drupal Association News: Membership Drive 2015 recap

Planet Drupal - Wed, 2016-02-10 14:44

From October 13 to December 30, 2015, we ran our biggest membership drive campaign ever. We did a lot of experimenting and I want to share some results and learnings with you.

But first, I'll tell you about our program. Drupal Association membership is for anyone who uses Drupal, and anyone who wants to support our community and the project through a donation of membership dues. Membership is one way to contribute to Drupal, but it is by no means the only way.

The biggest accomplishment in this campaign: 10% growth

During the campaign period, we saw 10% overall growth in our membership (3,266 to 3,590 members). For the last two years over this same period of time, we had seen 1% growth in membership. Last year’s growth is a huge win.

Goal vs actual

Our campaign goals were to raise $100,000 and to have 1,000 new members sign up.

The results looked more like this: $50,896 and 480 new members.

We also had 148 lapsed members return and our overall membership grew. Despite not hitting our goals, we are happy to have new and renewing members and some new knowledge about campaign content.

Testing content

When the campaign launched on October 13, we put our first banner ever on drupal.org to test whether an increased presence on the site would make a difference in membership sales. The answer is a big Yes. There were spikes in membership sales around the times we launched and changed the content of the banners. The biggest day of sales took place on the day after the launch, Oct. 14, when 74 members paid dues.

On December 14, we refreshed the banner to include a photo and quote from a member who gave us a testimonial. We rotated the banners and a separate block on the drupal.org homepage until the end of the campaign to highlight a few members.

This graph shows the results during our December banner rotation period. There was a trough around the Christmas holiday, so testing this at a non-holiday time of year will be beneficial in a future campaign.

What we learned

A greater drupal.org presence helps sell memberships, and being mindful about what content to show our active members is important for inspiring people to share the campaign. If members see a banner, they should easily be able to share it or hide it.

We need to keep telling your stories. I saw some of the members who gave testimonials were given kudos in the community. It feels good to see the faces of the community and to see just why you care. We have continued sharing stories, starting with DrupalCon Asia.

On days we launched banners or refreshed content, we saw the biggest sales spikes and flurries of social activity. The time of year when many people step away from their computers could impact campaign performance, so testing at another time of year should be done.

Thanks to everyone who participated in this campaign. From the 27 members who gave testimonials, to everyone who joined or renewed membership or encouraged the community to be a part of the drive, we appreciate your help.

See the full report at https://docs.google.com/presentation/d/1h0r32Tfw7TZ6osQbgtfMMJCZ3gjHMmwYlqOF8Xnmc80/edit#slide=id.g34532c755_069

Personal blog tags: Membership
Categories: FLOSS Project Planets
Syndicate content