FLOSS Project Planets

FSF Events: Richard Stallman - «El Software Libre y Tu Libertad» (Bogota, Colombia)

GNU Planet! - Fri, 2014-09-26 05:55
Richard Stallman hablará sobre las metas y la filosofía del movimiento del Software Libre, y el estado y la historia del sistema operativo GNU, el cual junto con el núcleo Linux, es actualmente utilizado por decenas de millones de personas en todo el mundo.

Esa charla de Richard Stallman formará parte de la Semana Distrital de la Cultura Libre (2014-10-06--10). No será técnica y será abierta al público; todos están invitados a asistir.

Favor de rellenar este formulario, para que podamos contactarle acerca de eventos futuros en la región de Bogotá.

Categories: FLOSS Project Planets

Overcoming fear

Planet KDE - Fri, 2014-09-26 05:21

In the last few posts, I've been exploring ideas expressed by Ed Catmull in Creativity, Inc. Everyone likes good ideas! But putting them into practice can be both difficult, and frightening. Change is work, and creating something which has never existed before, is creating the future. The unknown is daunting.

In meetings with the Braintrust, where new film ideas are viewed and judged, Catmull says,
It is natural for people to fear that such an inherently critical environment will feel threatening and unpleasant, like a trip to the dentist. The key is to look at the viewpoints being offered, in any successful feedback group, as additive, not competitive. A competitive approach measures other ideas against your own, turning the discussion into a debate to be won or lost. An additive approach, on the other hand, starts with the understanding that each participant contributes something (even if it's only an idea that fuels the discussion--and ultimately doesn't work). The Braintrust is valuable because it broadens your perspective, allowing you to peer--at least briefly--through other eyes.[101]Catmull presents an example where the Braintrust found a problem in The Incredibles film. In this case, they knew something was wrong, but failed to correctly diagnose it. Even so, the director was able, with the help of his peers, to ultimately fix the scene. The problem turned out not to be the voices, but the physical scale of the characters on the screen!

This could happen because the director and the team let go of fear and defensiveness, and trust that everyone is working for the greater good. I often see us doing this in KDE, but in the Community Working Group cases which come before us, I see this breaking down sometimes. It is human nature to be defensive. It takes healthy community to build trust so we can overcome that fear.

Categories: FLOSS Project Planets

Acquia: Ultimate Guide to Drupal 8: Episode 8 - Answers to Your Burning Questions

Planet Drupal - Fri, 2014-09-26 05:08

Welcome to the 8th and FINAL installment of an 8-part blog series we're calling "The Ultimate Guide to Drupal 8." Whether you're a site builder, module or theme developer, or simply an end-user of a Drupal website, Drupal 8 has tons in store for you! This blog series will attempt to enumerate the major changes in Drupal 8.

Categories: FLOSS Project Planets

Tim Golden: PyCon UK 2014: The Happening

Planet Python - Fri, 2014-09-26 03:57

Someone I meet only in Coventry mentioned in the hallway in this year’s PyCon UK that I’d not written about anything this year which he could comment on when he met me there. And indeed I was surprised to realise that my last post on this blog was about last year’s PyCon UK! I know I’d had several ideas for what to blog about, but clearly those ideas never became a reality.

As I pointed out in that year-ago post, every PyCon UK charts its own course for me each year. This year I was heavily involved in the Education & Kids’ tracks across the road in the Simulation Centre. In fact, apart from quick visits to the dining hall for food, I hardly interacted with the main conference at all throughout Friday and Saturday. As I tweeted at the time, the first talk I attended at the conference proper was the one I was giving first thing on Sunday morning!

Others have already done so, but I’d like to give tons of credit to Nicholas Tollervey who organised and ran the Education and Kids’ track (as well as giving a talk at the conference proper on turning footfall data into music). This involved liaising with schools and teachers for over 30 teachers to be able to attend PyCon UK as part of their Professional Development, in some cases assisted by money (generously provided by the Bank of America) to help pay for classroom cover. The Bank of America also paid for the venue, while the Python Software Foundation provided money to cover travel & accommodation. And then there’s the job of organising for 70 kids to come and have some experience of programming in a supportive environment. Those who only know Nicholas as a developer and organiser-in-chief of the monthly London Python Code Dojo may not be aware that he was a professional tuba player and, significantly here, a Secondary School teacher. This (I assume) gives him an insight into what will give the most benefit to the teachers and youngsters attending.

I hope to write a separate blog post on the Education slice of the conference on Friday. (If only to ensure that I have at least two blog posts to my name this year!). The Saturday event for children was certainly well-attended. We were over there not long after 7am setting up chairs, tables, RPis, keyboards, mice, screens and a *lot* of power cables to keep 70 youngsters occupied. Because of the numbers, activities were split over two large rooms plus two smaller rooms for specific activities. There were guided sessions on Minecraft, using the Pi camera module, and using PyGame plus a fair amount of freeflow action, mostly involving Minecraft. At the end was a show-and-tell where at least some of the various projects were showcased.

The Raspberry Pi Foundation Educational Outreach Team (or whatever they’re called) were there on both days, and in fact throughout the conference including the sprints on Monday. As well as running workshops in the Education track, they also provided a Model B+ Raspberry Pi for each of the youngsters attending, and gave talks and keynotes speeches at the main conference.

I attended relatively few of the talks at the main conference. I did like the gov.uk team’s presentation on their approach to using Flask to implement wizard-style forms (a project codenamed “Gandalf” apparently!) and which they hope to be able to open-source. The lightning talks I did get to see (ably compered as usual by Lightning Talk Man) were interesting; and one of them was even using my active_directory module. Another blog post from me in the making there, I hope.

The one very obvious aspect of the PyCon UK this year was the numbers. And, as a result, the queues, especially in the dining hall. Even my own talk – on the fairly niche subject of Core Python Development on Windows – was fully attended, presumably as a result of refugees who couldn’t fit into other talks and were desperate for somewhere they could at least sit down. It’s nice problem to have: to be too popular; but I imagine the organisers are looking hard at arrangements for next year.

I stayed for the sprints on Monday and so was looking for somewhere to eat on Sunday evening, traditionally the least organised of the PyCon UK evenings. I went looking with Conor & Ben for somewhere to eat in Coventry on a Sunday evening. With limited success. (We did eventually find somewhere open). I spent a pleasantly quiet rest of the evening with a pot of tea in the hotel bar, and chatting to Giacomo about this and that.

Monday was sprints, and although I was available to help people get started on core dev work, I was actually sprinting on my own Screenful of Python idea, of which more in a later post.

Categories: FLOSS Project Planets

Greg Casamento: Recent Article About Swift Confirms Apple's Position

GNU Planet! - Fri, 2014-09-26 02:46
I really do hate being right sometimes.  I believe that's enough said on the subject, don't you?  All I know now is that action must be taken.   The era of closed source languages is over and has been for some time.
Categories: FLOSS Project Planets

Dirk Eddelbuettel: R and Docker

Planet Debian - Thu, 2014-09-25 22:57

Earlier this evening I gave a short talk about R and Docker at the September Meetup of the Docker Chicago group.

Thanks to Karl Grzeszczak for setting the meeting, and for providing a pretty thorough intro talk regarding CoreOS and Docker.

My slides are now up on my presentations page.

Categories: FLOSS Project Planets

Mediacurrent: Exploring the Picture Element Module (Part 1)

Planet Drupal - Thu, 2014-09-25 22:01

Responsive Web Design or RWD, has come a long way since it was first introduced in 2010, and you would think that by now, given the popularity of the subject, all things have been sorted out and all questions have been answered. Not quite.  RWD is a moving target that continues to evolve, but for the most part the majority of the techniques used to accomplish the goal of an adaptable website are unquestionable except one, images. 

Categories: FLOSS Project Planets

Adrian Sutton: Safely Encoding Any String Into JavaScript Code Using JavaScript

Planet Apache - Thu, 2014-09-25 21:21

When generating a JavaScript file dynamically it’s not uncommon to have to embed an arbitrary string into the resulting code so it can be operated on. For example:

function createCode(inputValue) {
return "function getValue() { return '" + inputValue + "'; }"
}

This simplistic version works great for simple strings:

createCode("Hello world!");
// Gives: function getValue() { return 'Hello world!'; }

But breaks as soon as inputValue contains a special character, e.g.

createCode("Hello 'quotes'!");
// Gives: function getValue() { return 'Hello 'quotes' !'; }

You can escape single quotes but it still breaks if the input contains a \ character. The easiest way to fully escape the string is to use JSON.stringify:

function createCode(inputValue) {
return "function getValue() { return " +
JSON.stringify(String(inputValue)) +
"; }"
}

Note that JSON.stringify even adds the quotes for us. This works because a JSON string is a JavaScript string, so if you pass a string to JSON.stringify it will return a perfectly valid JavaScript string complete with quotes that is guaranteed to evaluate back to the original string.

The one catch is that JSON.stringify will happily JavaScript objects and numbers, not just strings, so we need to force the value to be a string first – hence, String(inputValue).

 

Categories: FLOSS Project Planets

CMS Quick Start: Drupal 7 Login Methods and Module Roundup: Part 1

Planet Drupal - Thu, 2014-09-25 17:45
If your site relies on user engagement, chances are you are using Drupal's powerful built in user modules. Sometimes, however, it can be difficult to understand what tweaks you can make to the default login process to make it a better experience all around. Should you present the login form on every page? Where should you put it? What methods can you use to display the login form in unobtrusive ways? What action does your site take after someone logs in? We're going to be presenting an array of options to hopefully point you in the right direction.

read more

Categories: FLOSS Project Planets

FSF News: Free Software Foundation statement on the GNU Bash "shellshock" vulnerability

GNU Planet! - Thu, 2014-09-25 17:35

Bash is the GNU Project's shell; it is part of the suite of software that makes up the GNU operating system. The GNU programs plus the kernel Linux form a commonly used complete free software operating system, called GNU/Linux. The bug, which is being referred to as "shellshock," can allow, in some circumstances, attackers to remotely access and control systems using Bash (and programs that call Bash) as an attack vector, regardless of what kernel they are running. The bug probably affects many GNU/Linux users, along with those using Bash on proprietary operating systems like Apple's OS X and Microsoft Windows. Additional technical details about the issue can be found at CVE-2014-6271 and CVE-2014-7169.

GNU Bash has been widely adopted because it is a free (as in freedom), reliable, and featureful shell. This popularity means the serious bug that was published yesterday is just as widespread. Fortunately, GNU Bash's license, the GNU General Public License version 3, has facilitated a rapid response. It allowed Red Hat to develop and share patches in conjunction with Bash upstream developers efforts to fix the bug, which anyone can download and apply themselves. Everyone using Bash has the freedom to download, inspect, and modify the code -- unlike with Microsoft, Apple, or other proprietary software.

Software freedom is a precondition for secure computing; it guarantees everyone the ability to examine the code to detect vulnerabilities, and to create new and safe versions if a vulnerability is discovered. Your software freedom does not guarantee bug-free code, and neither does proprietary software: bugs happen no matter how the software is licensed. But when a bug is discovered in free software, everyone has the permission, rights, and source code to expose and fix the problem. That fix can then be immediately freely distributed to everyone who needs it. Thus, these freedoms are crucial for ethical, secure computing.

Proprietary, (aka nonfree) software relies on an unjust development model that denies users the basic freedom to control their computers. When software's code is kept hidden, it is vulnerable not only to bugs that go undetected, but to the easier deliberate addition and maintenance of malicious features. Companies can use the obscurity of their code to hide serious problems, and it has been documented that Microsoft provides intelligence agencies with information about security vulnerabilities before fixing them.

Free software cannot guarantee your security, and in certain situations may appear less secure on specific vectors than some proprietary programs. As was widely agreed in the aftermath of the OpenSSL "Heartbleed" bug, the solution is not to trade one security bug for the very deep insecurity inherently created by proprietary software -- the solution is to put energy and resources into auditing and improving free programs.

Development of Bash, and GNU in general, is almost exclusively a volunteer effort, and you can contribute. We are reviewing Bash development, to see if increased funding can help prevent future problems. If you or your organization use Bash and are potentially interested in supporting its development, please contact us.

The patches to fix this issue can be obtained directly at http://ftp.gnu.org/gnu/bash/.

Media Contacts

John Sullivan
Executive Director
Free Software Foundation
+1 (617) 542 5942
campaigns@fsf.org

Categories: FLOSS Project Planets

Steve Kemp: Today I mostly removed python

Planet Debian - Thu, 2014-09-25 15:11

Much has already been written about the recent bash security problem, allocated the CVE identifier CVE-2014-6271, so I'm not even going to touch it.

It did remind me to double-check my systems to make sure that I didn't have any packages installed that I didn't need though, because obviously having fewer packages installed and fewer services running reduces the potential attack surface.

I had noticed in the past I had python installed and just though "Oh, yeah, I must have python utilities running". It turns out though that on 16 out of 19 servers I control I had python installed solely for the lsb_release script!

So I hacked up a horrible replacement for `lsb_release in pure shell, and then became cruel:

~ # dpkg --purge python python-minimal python2.7 python2.7-minimal lsb-release

That horrible replacement is horrible because it defers detection of all the names/numbers to the /etc/os-release which wasn't present in earlier versions of Debian. Happily all my Debian GNU/Linux hosts run Wheezy or later, so it all works out.

So that left three hosts that had a legitimate use for Python:

  • My mail-host runs offlineimap
    • So I purged it.
    • I replaced it with isync.
  • My host-machine runs KVM guests, via qemu-kvm.
    • qemu-kvm depends on Python solely for the script /usr/bin/kvm_stat.
    • I'm not pleased about that but will tolerate it for now.
  • The final host was my ex-mercurial host.
    • Since I've switched to git I just removed tha package.

So now 1/19 hosts has Python installed. I'm not averse to the language, but given that I don't personally develop in it very often (read "once or twice in the past year") and by accident I had no python-scripts installed I see no reason to keep it on the off-chance.

My biggest surprise of the day was that now that we can use dash as our default shell we still can't purge bash. Since it is marked as Essential. Perhaps in the future.

Categories: FLOSS Project Planets

Aigars Mahinovs: Distributing third party applications via Docker?

Planet Debian - Thu, 2014-09-25 14:54

Recently the discussions around how to distribute third party applications for "Linux" has become a new topic of the hour and for a good reason - Linux is becoming mainstream outside of free software world. While having each distribution have a perfectly packaged, version-controlled and natively compiled version of each application installable from a per-distribution repository in a simple and fully secured manner is a great solution for popular free software applications, this model is slightly less ideal for less popular apps and for non-free software applications. In these scenarios the developers of the software would want to do the packaging into some form, distribute that to end-users (either directly or trough some other channels, such as app stores) and have just one version that would work on any Linux distribution and keep working for a long while.

For me the topic really hit at Debconf 14 where Linus voiced his frustrations with app distribution problems and also some of that was touched by Valve. Looking back we can see passionate discussions and interesting ideas on the subject from systemd developers (another) and Gnome developers (part2 and part3).

After reading/watching all that I came away with the impression that I love many of the ideas expressed, but I am not as thrilled about the proposed solutions. The systemd managed zoo of btrfs volumes is something that I actually had a nightmare about.

There are far simpler solutions with existing code that you can start working on right now. I would prefer basing Linux applications on Docker. Docker is a convenience layer on top of Linux cgroups and namespaces. Docker stores its images in a datastore that can be based on AUFS or btrfs or devicemapper or even plain files. It already has a semantic for defining images, creating them, running them, explicitly linking resources and controlling processes.

Lets play a simple scenario on how third party applications should work on Linux.

Third party application developer writes a new game for Linux. As his target he chooses one of the "application runtime" Docker images on Docker Hub. Let's say he chooses the latest Debian stable release. In that case he writes a simple Dockerfile that installs his build-dependencies and compiles his game in "debian-app-dev:wheezy" container. The output of that is a new folder containing all the compiled game resources and another Dockerfile - this one describes the runtime dependencies of the game. Now when a docker image is built from this compiled folder, it is based on "debian-app:wheezy" container that no longer has any development tools and is optimized for speed and size. After this build is complete the developer exports the Docker image into a file. This file can contain either the full system needed to run the new game or (after #8214 is implemented) just the filesystem layers with the actual game files and enough meta-data to reconstruct the full environment from public Docker repos. The developer can then distribute this file to the end user in the way that is comfortable for them.

The end user would download the game file (either trough an app store app, app store website or in any other way) and import it into local Docker instance. For user convenience we would need to come with an file extension and create some GUIs to launch for double click, similar to GDebi. Here the user would be able to review what permissions the app needs to run (like GL access, PulseAudio, webcam, storage for save files, ...). Enough metainfo and cooperation would have to exist to allow desktop menu to detect installed "apps" in Docker and show shortcuts to launch them. When the user does so, a new Docker container is launched running the command provided by the developer inside the container. Other metadata would determine other docker run options, such as whether to link over a socket for talking to PulseAudio or whether to mount in a folder into the container to where the game would be able to save its save files. Or even if the application would be able to access X (or Wayland) at all.

Behind the scenes the application is running from the contained and stable libraries, but talking to a limited and restricted set of system level services. Those would need to be kept backwards compatible once we start this process.

On the sandboxing part, not only our third party application is running in a very limited environment, but also we can enhance our system services to recognize requests from such applications via cgroups. This can, for example, allow a window manager to mark all windows spawned by an application even if the are from a bunch of different processes. Also the window manager can now track all processes of a logical application from any of its windows.

For updates the developer can simply create a new image and distribute the same size file as before, or, if the purchase is going via some kind of app-store application, the layers that actually changed can be rsynced over individually thus creating a much faster update experience. Images with the same base can share data, this would encourage creation of higher level base images, such as "debian-app-gamegl:wheezy" that all GL game developers could use thus getting a smaller installation package.

After a while the question of updating abandonware will come up. Say there is is this cool game built on top of "debian-app-gamegl:wheezy", but now there was a security bug or some other issue that requires the base image to be updated, but that would not require a recompile or a change to the game itself. If this Docker proposal is realized, then either the end user or a redistributor can easily re-base the old Docker image of the game on a new base. Using this mechanism it would also be possible to handle incompatible changes to system services - ten years down the line AwesomeAudio replaces PulseAudio, so we create a new "debian-app-gamegl:wheezy.14" version that contains a replacement libpulse that actually talks to AwesomeAudio system service instead.

There is no need to re-invent everything or push everything and now package management too into systemd or push non-distribution application management into distribution tools. Separating things into logical blocks does not hurt their interoperability, but it allows to recombine them in a different way for a different purpose or to replace some part to create a system with a radically different functionality.

Or am I crazy and we should just go and sacrifice Docker, apt, dpkg, FHS and non-btrfs filesystems on the altar of systemd?

P.S. You might get the impression that I dislike systemd. I love it! Like an init system. And I love the ideas and talent of the systemd developers. But I think that systemd should have nothing to do with application distribution or processes started by users. I am sometimes getting an uncomfortable feeling that systemd is morphing towards replacing the whole of System V jumping all the way to System D and rewriting, obsoleting or absorbing everything between the kernel and Gnome. In my opinion it would be far healthier for the community if all of these projects would developed and be usable separately from systemd, so that other solutions can compete on a level playing field. Or, maybe, we could just confess that what systemd is doing is creating a new Linux meta-distribution.

Categories: FLOSS Project Planets

Fabio Zadrozny: Attaching debugger to running process (PyDev 3.8.0)

Planet Python - Thu, 2014-09-25 14:08
The latest PyDev 3.8.0 has just been released... along with a bunch of bugfixes, the major feature added is the possibility of attaching the debugger to a running process.

So, I thought about explaining a bit how to use it (and later a bit on how it was done).

The first thing is that the Debug perspective must be activated (as the attach to process menu is only shown by default in the Debug Perspective). Then, when on the debug perspective, select PyDev > Attach to Process (as the image below shows).



When that action is activated, a list with the active processes is shown so that the process we should attach to can be selected (as a note, by default only Python processes are filtered, but if you have an executable running Python under a different name, it's also possible to select it).


After selecting the executable, the user must provide a Python interpreter that's compatible with the interpreter we'll attach to (i.e.: if we're attaching to a 64 bit interpreter, the interpreter selected must also be a 64 bit process -- and likewise for 32 bits).


And if everything went right, we'll be connected with the process after that step (you can note in the screenshot below that in the Debug View we have a process running regularly and we connected to the process through the remote debugger) -- after it's connected, it's possible to pause the running process (through right-click process > suspend)


So, this was how to use it within PyDev... now, I'll explain a bit on the internals as I had a great time implementing that feature :)

The first thing to note is that we currently support (only) Windows and Linux... although the basic idea is the same on both cases (we get a dll loaded in the target process and then execute our attach code through it), the implementations are actually pretty different (as it's Open Source, it can be seen at: https://github.com/fabioz/PyDev.Debugger/tree/development/pydevd_attach_to_process)

So, let me start with the Windows implementation...

In the Windows world, I must thank a bunch of people that came before me in order to make it work:
First, Mario Vilas for Winappdbg: https://github.com/MarioVilas/winappdbg -- mostly, this is a Windows debugger written in Python. We use it to attach a dll to a target process. Then, after the dll is loaded, there's some hand-crafted shellcode created to execute a function from that dll (which is actually different for 32 bits and for 64 bits -- I did spend quite some time here since my assembly knowledge was mostly theoretical, so, it was nice to see it working in practice: that's the GenShellCodeHelper class in add_code_to_python_process.py and it's used in the run_python_code_windows function).

Ok, so, that gives us the basic hackery needed in order to execute some code in a different process (which Winappdbg does through the Windows CreateRemoteThread API) -- so, initially I had a simple version working which did (all in shellcode) an acquire of the Python GIL/run some hand-crafted Python code through PyRun_SimpleString/release GIL... but there are a number of problems with that simple approach: the first one is that although we'll execute some Python code there, we'll do it under a new thread, which under Python 3 meant that this would be no good since we have to initialize the Python threading if it still wasn't initialized. Also, to setup the Debugger we need to call sys.settrace on existing threads (while executing in that thread -- or at least making Python think we're at that thread as there's no API for that)... at that point I decided on having the dll (and not only shellcode).

So, here I must thank the PVTS guys (which actually have an attach to process on Windows which does mixed mode debugging), so, the attach.cpp file which generates our dll is adapted from PVTS to the PyDev use case. It goes through a number of hoops to initialize the threading facility in Python if it still wasn't initialized and does the sys.settrace while having all threads suspended and makes Python think we're actually at the proper thread to make that call... looking at it, I find it really strange that Python itself doesn't have an API for that (it should be easy to do on Python, but it's such a major hack on the debugger because that API is not available).

Now, on to Linux: in Linux the approach is simpler as we reuse a debugger that should be readily available for Linux developers: gdb. Also, because gdb stops threads for us and executes the code in an existing thread, things become much simpler... first, because we're executing in an existing thread, we don't have to start the threading if it's not started -- the only reason this is needed in Windows is because we're executing the code in a new thread in the process, created through CreateRemoteThread -- and also, as gdb has a way to script in Python were we can switch threads and execute something in it while having other threads stopped, we also don't need to do the trick on Python to make it think we're in a thread to execute a sys.settrace as if we were in a thread when in reality we weren't, as we can switch to a thread and really execute code on it.

So, all in all, it should be working properly, although there are a number of caveats... it may fail if we don't have permissions for the CreateRemoteThread on Windows or in Linux it could fail if the ptrace permissions are not set for us to attach with GDB -- and probably a bunch of other things I still didn't think of :)

Still, it's nice to see it working!

Also, the last thanks goes to JetBrains/PyCharm, which helped in sponsoring the work in the debugger -- as I mentioned earlier: http://pydev.blogspot.com.br/2014/08/pydev-370-pydevpycharm-debugger-merge.html the debugger in PyDev/PyCharm is now merged :)
Categories: FLOSS Project Planets

PyTennessee: Python for Ada

Planet Python - Thu, 2014-09-25 14:00

All of us at PyTennessee are huge fans of the diverse community that python is becoming.  We work hard every year to seek out and encourage a diverse group of speakers and attendees.  I wanted to make sure all of you have seen the Python for Ada donation drive.  There are two days left, and we’re past the goal, but we can go further.  So please go checkout the campaign and contribute if you can, but spread the word at least.

Note: We put our money where our mouth is, and we donated as well. Also just like last year, at the conference on Sunday we will be having a fundraiser for PyLadies via a mani/pedi party and silent auction.

Categories: FLOSS Project Planets

Jan Wagner: Redis HA with Redis Sentinel and VIP

Planet Debian - Thu, 2014-09-25 13:56

For an actual project we decided to use Redis for some reasons. As there is availability a critical part, we discovered that Redis Sentinel can monitor Redis and handle an automatic master failover to a available slave.

Setting up the Redis replication was straight forward and even setting up Sentinel. Please keep in mind, if you configure Redis to require an authentication password, you even need to provide that for the replication process (masterauth) and for the Sentinel connection (auth-pass).

The more interesting part is, how to migrate over the clients to the new master in case of a failover process. While Redis Sentinel could also be used as configuration provider, we decided not to use this feature, as the application needs to request the actual master node from Redis Sentinel much often, which will maybe a performance impact.
The first idea was to use some kind of VRRP, implemented into keepalived or something like this. The problem with such a solution is, you need to notify the VRRP process when a redis failover is in progress.
Well, Redis Sentinel has a configuration option called 'sentinel client-reconfig-script':

# When the master changed because of a failover a script can be called in # order to perform application-specific tasks to notify the clients that the # configuration has changed and the master is at a different address. # # The following arguments are passed to the script: # # <master-name> <role> <state> <from-ip> <from-port> <to-ip> <to-port> # # <state> is currently always "failover" # <role> is either "leader" or "observer" # # The arguments from-ip, from-port, to-ip, to-port are used to communicate # the old address of the master and the new address of the elected slave # (now a master). # # This script should be resistant to multiple invocations.

This looks pretty good and as there is provided a <role>, I thought it would be a good idea to just call a script which evaluates this value and based on it's return, it adds the VIP to the local network interface, when we get 'leader' and removes it when we get 'observer'. It turned out that this was not working as <role> didn't returned 'leader' when the local redis instance got master and 'observer' when got slave in any case. This was pretty annoying and I was short before giving up.
Fortunately I stumpled upon a (maybe) chinese post about Redis Sentinal, where was done the same like I did. On the second look I recognized that the decision was made on ${6} which is <to-ip>, nothing more then the new IP of the Redis master instance. So I rewrote my tiny shell script and after some other pitfalls this strategy worked out well.

Some notes about convergence. Actually it takes round about 6-7 seconds to have the VIP migrated over to the new node after Redis Sentinel notifies a broken master. This is not the best performance, but as we expect this happen not so often, we need to design the application using our Redis setup to handle this (hopefully) rare scenario.

Categories: FLOSS Project Planets

Drupal Watchdog: Drupal 7 Form Building

Planet Drupal - Thu, 2014-09-25 13:45
Article

Static websites, comprising web pages that do not respond to any user input, may be adequate for listing information, but little else. Dynamic pages are what make the Web more than just interlinked digital billboards. The primary mechanism that enables this is the humble web form – whether a modest single button or a multi-page form with various controls that allow the user to input text, make choices, upload files, etc.

Anyone developing a new Drupal-based website will usually need to create forms for gathering data from users. There are at least two possible approaches to solving this problem: One approach mostly relies on Drupal core, and the other uses a contributed module dedicated to forms.

The Node Knows

If you understand how to create content types and attach fields to them, then that could be a straightforward way to make a form – not for creating nodes to be used later for other purposes, but solely for gathering user input. To add different types of form fields to a content type, you will need to install and enable the corresponding modules, all of which are available from Drupal.org's modules section.

Some of these modules are part of Drupal's core: File (for uploading files), List (for selection lists), Number (for integers, decimals, and floats), Options (for selection controls, checkboxes, and radio buttons), Taxonomy (for tagging content with terms), and Text (for single- and multi-line entry fields).

Other field-related modules have been contributed by the Drupal community, including:

Categories: FLOSS Project Planets

David Reid: Detective Story

Planet Apache - Thu, 2014-09-25 13:35

A little while ago, someone who knows we are both interested in photography gave us a camera that had been in a lost property box for a while and asked if we could find it’s owners. I wasn’t present when the actual exchange took place, but it ended up at our house and sat on a shelf until the other day when things were explained to me.

So, this is what we were given.

It’s a Sony TX-5. The battery was totally dead when I started looking at it, but we have, by chance, a Sony charger that fitted the battery and so have been able to charge it.

The SD card was 8Gb and contained a lot of pictures, starting in Dec 30th 2010 – was it a christmas present?

There is no name given for copyright in the EXIF data, so I started looking at the pictures to try and find the owner.

I found a car with a legible number plate, but as it looked like a holiday this was probably a rental car.

Discovering that there was a wedding and I could identify the venue, I started to get my hopes up. the venue is still going, had a web page and so I sent an email with some details in the hope they could help. Sadly they couldn’t as the venue has changed hands and no records are available for the date.

Next stop was to look at the recurring pictures of a street. I assume it’s where the owner lives and by looking at street signs and using Google I’ve been able to narrow down the house the pictures were taken from to 2 houses. As the house is in Orgeon and I have no way of getting more information, I sent an email to the local police department – who replied and are looking into what they can do!

Sat, 27th Sept 2014

The property that I identified was in Medford, OR – which is a long way from Scotland! As I wasn’t able to follow up myself I sent an email to the Medford Police Dept hoping that they may be able to assist. The enquiry was passed to Gena Criswell who responded in the best way possible – by offering to help! After a few emails and a few additional pictures being supplied she was able to contact someone from the pictures who confirmed the camera belonged to her brother! Yes, after all this time it’s owner has been identified.

I’m waiting for her brother to get in touch and will then arrange to send the camera to him.

I still can’t really believe that this has had such a good outcome, but a large portion of that is down to the amazing efforts of Gena and the Medford Police Department.

Categories: FLOSS Project Planets

Here’s how to patch Ubuntu 8.04 or anything where you have to build bash from source

LinuxPlanet - Thu, 2014-09-25 13:03

Just a quick post to help those who might be running older/unsupported distributions of linux, mainly Ubuntu 8.04 who need to patch their version of bash due to the recent exploit here:

http://thehackernews.com/2014/09/bash-shell-vulnerability-shellshock.html

I found this post and can confirm it works:

https://news.ycombinator.com/item?id=8364385

Here are the steps(make a backup of /bin/bash just in case):

#assume that your sources are in /src
cd /src
wget http://ftp.gnu.org/gnu/bash/bash-4.3.tar.gz
#download all patches
for i in $(seq -f “%03g” 0 25); do wget http://ftp.gnu.org/gnu/bash/bash-4.3-patches/bash43-$i; done
tar zxvf bash-4.3.tar.gz
cd bash-4.3
#apply all patches
for i in $(seq -f “%03g” 0 25);do patch -p0 < ../bash43-$i; done
#build and install
./configure && make && make install

Categories: FLOSS Project Planets

Kubuntu: Frameworks 5.2.0 Released and Plasma 5.0.2 is ready for testing.

Planet KDE - Thu, 2014-09-25 13:00

Kubuntu 14.10 with KDE Plasma 5.0.2

KDE Frameworks 5.2.0 Has been released to Utopic archive!
(Actually a few days ago, we are playing catch up since Akademy)

Also, I have finished packaging Plasma 5.0.2, it looks and runs great!
We desperately need more testers! If you would like to help us test,
please join us in IRC in #kubuntu-devel thanks!

Categories: FLOSS Project Planets

Gunnar Wolf: #bananapi → On how compressed files should be used

Planet Debian - Thu, 2014-09-25 12:37

I am among the lucky people who got back home from DebConf with a brand new computer: a Banana Pi. Despite the name similarity, it is not affiliated with the very well known Raspberry Pi, although it is a very comparable (although much better) machine: A dual-core ARM A7 system with 1GB RAM, several more on-board connectors, and same form-factor.

I have not yet been able to get it to boot, even from the images distributed on their site (although I cannot complain, I have not devoted more than a hour or so to the process!), but I do have a gripe on how the images are distributed.

I downloaded some images to play with: Bananian, Raspbian, a Scratch distribution, and Lubuntu. I know I have a long way to learn in order to contribute to Debian's ARM port, but if I can learn by doing... ☻

So, what is my gripe? That the three images are downloaded as archive files:

  1. 0 gwolf@mosca『9』~/Download/banana$ ls -hl bananian-latest.zip \
  2. > Lubuntu_For_BananaPi_v3.1.1.tgz Raspbian_For_BananaPi_v3.1.tgz \
  3. > Scratch_For_BananaPi_v1.0.tgz
  4. -rw-r--r-- 1 gwolf gwolf 222M Sep 25 09:52 bananian-latest.zip
  5. -rw-r--r-- 1 gwolf gwolf 823M Sep 25 10:02 Lubuntu_For_BananaPi_v3.1.1.tgz
  6. -rw-r--r-- 1 gwolf gwolf 1.3G Sep 25 10:01 Raspbian_For_BananaPi_v3.1.tgz
  7. -rw-r--r-- 1 gwolf gwolf 1.2G Sep 25 10:05 Scratch_For_BananaPi_v1.0.tgz

Now... that is quite an odd way to distribute image files! Specially when looking at their contents:

  1. 0 gwolf@mosca『14』~/Download/banana$ unzip -l bananian-latest.zip
  2. Archive: bananian-latest.zip
  3. Length Date Time Name
  4. --------- ---------- ----- ----
  5. 2032664576 2014-09-17 15:29 bananian-1409.img
  6. --------- -------
  7. 2032664576 1 file
  8. 0 gwolf@mosca『15』~/Download/banana$ for i in Lubuntu_For_BananaPi_v3.1.1.tgz \
  9. > Raspbian_For_BananaPi_v3.1.tgz Scratch_For_BananaPi_v1.0.tgz
  10. > do tar tzvf $i; done
  11. -rw-rw-r-- bananapi/bananapi 3670016000 2014-08-06 03:45 Lubuntu_1404_For_BananaPi_v3_1_1.img
  12. -rwxrwxr-x bananapi/bananapi 3670016000 2014-08-08 04:30 Raspbian_For_BananaPi_v3_1.img
  13. -rw------- bananapi/bananapi 3980394496 2014-05-27 01:54 Scratch_For_BananaPi_v1_0.img

And what is bad about them? That they force me to either have heaps of disk space available (2GB or 4GB for each image) or to spend valuable time extracting before recording the image each time.

Why not just compressing the image file without archiving it? That is,

  1. 0 gwolf@mosca『7』~/Download/banana$ tar xzf Lubuntu_For_BananaPi_v3.1.1.tgz
  2. 0 gwolf@mosca『8』~/Download/banana$ xz Lubuntu_1404_For_BananaPi_v3_1_1.img
  3. 0 gwolf@mosca『9』~/Download/banana$ ls -hl Lubun*
  4. -rw-r--r-- 1 gwolf gwolf 606M Aug 6 03:45 Lubuntu_1404_For_BananaPi_v3_1_1.img.xz
  5. -rw-r--r-- 1 gwolf gwolf 823M Sep 25 10:02 Lubuntu_For_BananaPi_v3.1.1.tgz

Now, wouldn't we need to decompress said files as well? Yes, but thanks to the magic of shell redirections, we can just do it on the fly. That is, instead of having 3×4GB+1×2GB files sitting on my hard drive, I just need to have several files ranging between 145M and I guess ~1GB. Then, it's as easy as doing:

  1. 0 gwolf@mosca『8』~/Download/banana$ dd if=<(xzcat bananian-1409.img.xz) of=/dev/sdd

And the result should be the same: A fresh new card with Bananian ready to fly. Right, right, people using these files need to have xz installed on their systems, but... As it stands now, I can suppose current prospective users of a Banana Pi won't fret about facing a standard Unix tool!

(Yes, I'll forward this rant to the Banana people, it's not just bashing on my blog :-P )

[update] Several people (thanks!) have contacted me stating that I use a bashism: The <(…) construct is specific to Bash. If you want to do this with any other shell, it can be done with a simple pipe:

  1. $ xzcat bananian-1409.img.xz | dd of=/dev/sdd

That allows for less piping to be done on the kernel, and is portable between different shells. Also, a possibility would be:

  1. $ xzcat bananian-1409.img.xz > /dev/sdd

Although that might not be desirable, as it avoids the block-by-block nature of dd. I'm not sure if it makes a realdifference, but it's worth saying :)

And yes, some alternatives for not unarchiving the file — Here in the blog, an anon commenter suggests (respectively, for zip and .tar.gz files):

  1. $ dd if=<(unzip -p bananian-latest.zip) of=/dev/sdd
  2. $ dd if=<(tar -xOf Lubuntu_For_BananaPi_v3.1.1.tgz) of=/dev/sdd

And a commenter by IRC suggests:

  1. $ paxtar -xOaf Raspbian_For_BananaPi_v3.1.tgz Raspbian_For_BananaPi_v3_1.img | sudo dd bs=262144 of=/dev/…

Thanks!

Categories: FLOSS Project Planets
Syndicate content