Planet Debian

Syndicate content
Planet Debian - http://planet.debian.org/
Updated: 9 hours 18 min ago

Petter Reinholdtsen: I spent last weekend recording MakerCon Nordic

Thu, 2014-10-23 17:00

I spent last weekend at Makercon Nordic, a great conference and workshop for makers in Norway and the surrounding countries. I had volunteered on behalf of the Norwegian Unix Users Group (NUUG) to video record the talks, and we had a great and exhausting time recording the entire day, two days in a row. There were only two of us, Hans-Petter and me, and we used the regular video equipment for NUUG, with a dvswitch, a camera and a VGA to DV convert box, and mixed video and slides live.

Hans-Petter did the post-processing, consisting of uploading the around 180 GiB of raw video to Youtube, and the result is now becoming public on the MakerConNordic account. The videos have the license NUUG always use on our recordings, which is Creative Commons Navngivelse-Del på samme vilkår 3.0 Norge. Many great talks available. Check it out! :)

Categories: FLOSS Project Planets

Enrico Zini: systemd-default-rescue

Thu, 2014-10-23 16:06
Alternate rescue boot entry with systemd

Since systemd version 215, adding systemd.debug-shell to the kernel command line activates the debug shell on tty9 alongside the normal boot. I like the idea of that, and I'd like to have it in my standard 'rescue' entry in my grub menu.

Unfortunately, by default update-grub does not allow to customize the rescue menu entry options. I have just filed #766530 hoping for that to change.

After testing the patch I proposed for /etc/grub.d/10_linux, I now have this in my /etc/default/grub, with some satisfaction:

GRUB_CMDLINE_LINUX_RECOVERY="systemd.log_target=kmsg systemd.log_level=debug systemd.debug-shell"

Further information:

Thanks to sjoerd and uau on #debian-systemd for their help.

Categories: FLOSS Project Planets

Gunnar Wolf: Listadmin — *YES*

Thu, 2014-10-23 14:05

Petter posted yesterday about Listadmin, the quick way to moderate mailman lists.

Petter: THANKS.

I am a fan of automatization. But, yes, I had never thouguht of doing this. Why? Don't know. But this is way easier than using the Web interface for Mailman:

$ listadmin fetching data for conoc_des@my.example.org ... nothing in queue fetching data for des_polit_pub@my.example.org ... nothing in queue fetching data for econ_apl@my.example.org ... nothing in queue fetching data for educ_ciencia_tec@my.example.org ... nothing in queue fetching data for est_hacend_sec_pub@my.example.org ... [1/1] ============== est_hacend_sec_pub@my.example.org ====== From: sender@example.org Subject: Invitación al Taller Insumo Producto Reason: El cuerpo del mensaje es demasiado grande: 777499 Spam? 0 Approve/Reject/Discard/Skip/view Body/Full/jump #/Undo/Help/Quit ? a Submit changes? [yes] fetching data for fiscal_fin@my.example.org ... nothing in queue fetching data for historia@my.example.org ... nothing in queue fetching data for industrial@my.example.org ... nothing in queue fetching data for medio_amb@my.example.org ... nothing in queue fetching data for mundial@my.example.org ... nothing in queue fetching data for pol_des@my.example.org ... nothing in queue fetching data for sec_ener@my.example.org ... nothing in queue fetching data for sec_prim@my.example.org ... nothing in queue fetching data for trab_tec@my.example.org ... nothing in queue fetching data for urb_reg@my.example.org ... nothing in queue fetching data for global@my.example.org ... nothing in queue

I don't know how in many years of managing several mailing lists I never thought about this! I'm echoing this, as I know several of my readers run mailman as well, and might not be following Planet Debian.

Categories: FLOSS Project Planets

Dirk Eddelbuettel: Introducing Rocker: Docker for R

Thu, 2014-10-23 12:39

You only know two things about Docker. First, it uses Linux
containers. Second, the Internet won't shut up about it.

-- attributed to Solomon Hykes, Docker CEO

So what is Docker?

Docker is a relatively new open source application and service, which is seeing interest across a number of areas. It uses recent Linux kernel features (containers, namespaces) to shield processes. While its use (superficially) resembles that of virtual machines, it is much more lightweight as it operates at the level of a single process (rather than an emulation of an entire OS layer). This also allows it to start almost instantly, require very little resources and hence permits an order of magnitude more deployments per host than a virtual machine.

Docker offers a standard interface to creation, distribution and deployment. The shipping container analogy is apt: just how shipping containers (via their standard size and "interface") allow global trade to prosper, Docker is aiming for nothing less for deployment. A Dockerfile provides a concise, extensible, and executable description of the computational environment. Docker software then builds a Docker image from the Dockerfile. Docker images are analogous to virtual machine images, but smaller and built in discrete, extensible and reuseable layers. Images can be distributed and run on any machine that has Docker software installed---including Windows, OS X and of course Linux. Running instances are called Docker containers. A single machine can run hundreds of such containers, including multiple containers running the same image.

There are many good tutorials and introductory materials on Docker on the web. The official online tutorial is a good place to start; this post can not go into more detail in order to remain short and introductory.

So what is Rocker?

At its core, Rocker is a project for running R using Docker containers. We provide a collection of Dockerfiles and pre-built Docker images that can be used and extended for many purposes.

Rocker is the the name of our GitHub repository contained with the Rocker-Org GitHub organization.

Rocker is also the name the account under which the automated builds at Docker provide containers ready for download.

Current Rocker Status Core Rocker Containers

The Rocker project develops the following containers in the core Rocker repository

  • r-base provides a base R container to build from
  • r-devel provides the basic R container, as well as a complete R-devel build based on current SVN sources of R
  • rstudio provides the base R container as well an RStudio Server instance

We have settled on these three core images after earlier work in repositories such as docker-debian-r and docker-ubuntu-r.

Rocker Use Case Containers

Within the Rocker-org organization on GitHub, we are also working on

  • Hadleyverse which extends the rstudio container with a number of Hadley packages
  • rOpenSci which extends hadleyverse with a number of rOpenSci packages
  • r-devel-san provides an R-devel build for "Sanitizer" run-time diagnostics via a properly instrumented version of R-devel via a recent compiler build
  • rocker-versioned aims to provided containers with 'versioned' previous R releases and matching packages

Other repositories will probably be added as new needs and opportunities are identified.

Deprecation

The Rocker effort supersedes and replaces earlier work by Dirk (in the docker-debian-r and docker-ubuntu-r GitHub repositories) and Carl. Please use the Rocker GitHub repo and Rocker Containers from Docker.com going forward.

Next Steps

We intend to follow-up with more posts detailing usage of both the source Dockerfiles and binary containers on different platforms.

Rocker containers are fully functional. We invite you to take them for a spin. Bug reports, comments, and suggestions are welcome; we suggest you use the GitHub issue tracker.

Acknowledgments

We are very appreciative of all comments received by early adopters and testers. We also would like to thank RStudio for allowing us the redistribution of their RStudio Server binary.

Published concurrently at rOpenSci blog and Dirk's blog.

Authors

Dirk Eddelbuettel and Carl Boettiger

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Categories: FLOSS Project Planets

Alessio Treglia: Bits from the Debian Multimedia Maintainers

Thu, 2014-10-23 07:30

This brief announcement was released yesterday to the debian-devel-announce mailing list.

 

Ciao!

The Debian Multimedia Maintainers have been quite active since the Wheezy release, and have some interesting news to share for the Jessie release. Here we give you a brief update on what work has been done and work that is still ongoing.

Let’s see what’s cooking for Jessie then.

 

Frameworks and libraries Support for many new media formats and codecs.

The codec library libavcodec, which is used by popular media playback applications including vlc, mpv, totem (using gstreamer1.0-libav), xine, and many more, has been updated to the latest upstream release version 11 provided by Libav. This provides Debian users with HEVC playback, a native Opus decoder, Matroska 3D support, Apple ProRes, and much more. Please see libav’s changelog for a full list of functionality additions and updates.

libebur128

libebur128 is a free implementation of the European Broadcasting Union Loudness Recommendation (EBU R128), which is essentially an alternative to ReplayGain. The library can be used to analyze audio perceived loudness and subsequentially normalize the volume during playback.

libltc

libltc provides functionalities to encode and decode Linear (or Longitudinal) Timecode (LTC) from/to SMPTE data timecode.

libva

libva and the driver for Intel GPUs has been updated to the 1.4.0 release. Support for new GPUs has been added. libva now also supports Wayland.

Pure Data

A number of new additional libraries (externals) will appear in Jessie, including (among others) Eric Lyon’s fftease and lyonpotpourrie, Thomas Musil’s iemlib, the pdstring library for string manipulation and pd-lua that allows to write Pd-objects in the popular lua scripting language.

 

JACK and LADI

LASH Audio Session Handler was abandoned upstream a long time ago in favor of the new session management system, called ladish (LADI Session Handler). ladish allows users to run many JACK applications at once and save/restore their configuration with few mouse clicks.

The current status of the integration between the session handler and JACK may be summarized as follows:

  • ladish provides the backend;
  • laditools contains a number of useful graphical tools to tune the session management system’s whole configuration (including JACK);
  • gladish provides a easy-to-use graphical interface for the session handler.

Note that ladish uses the D-Bus interface to the jack daemon, therefore only Jessie’s jackd2 provides support for and also cooperates fine with it.

 

Plugins: LV2 and LADSPA

Debian Jessie will bring the newest 1.10.0 version of the LV2 technology. Most changes affect the packaging of new plugins and extensions, a brief list of packaging guidelines is now available.
A number of new plugins and development tools too have been made available during the Jessie development cycle:

LV2 Toolkit

LVTK provides libraries that wrap the LV2 C API and extensions into easy to use C++ classes. The original work for this was mostly done by Lars Luthman in lv2-c++-tools.

Vee One Suite

The whole suite by Rui Nuno Capela is now available in Jessie, and consists of three components:

  • drumkv1: old-school drum-kit sampler synthesizer
  • samplv1: polyphonic sampler
  • synthv1: analog-style 4-oscillator substractive synthesizer

All three are provided in both forms of LV2 plugins and stand-alone JACK client. JACK session, JACK MIDI, and ALSA MIDI are supported too.

x42-plugins and zam-plugins

LV2 bundles containing many audio plugins for high quality processing.

Fomp

Fomp is an LV2 port of the MCP, VCO, FIL, and WAH plugins by Fons Adriaensen.

Some other components have been upgraded to more recent upstream versions:

  • ab2gate: 1.1.7
  • calf: 0.0.19+git20140915+5de5da28
  • eq10q: 2.0~beta5.1
  • NASPRO: 0.5.1

We’ve packaged ste-plugins, Fons Adriaensen’s new stereo LADSPA plugins bundle.

A major upgrade of frei0r, namely the standard collection for the minimalistic plugin API for video effects, will be available in Jessie.

 

New multimedia applications Advene

Advene (Annotate Digital Video, Exchange on the NEt) is a flexible video
annotation application.

Ardour3

The new generation of the popular digital audio workstation will make its very first appearance in Debian Jessie.

Cantata

Qt4 front-end for the MPD daemon.

Csound

Csound for jessie will feature the new major series 6, with the improved IDE CsoundQT. This new csound supports improved array data type handling, multi-core rendering and debugging features.

din

DIN Is Noise is a musical instrument and audio synthesizer that supports JACK audio output, MIDI, OSC, and IRC bot as input sources. It could be extended and customized with Tcl scripts too.

dvd-slideshow

dvd-slideshow consists of a suite of command line tools which come in handy to make slideshows from collections of pictures. Documentation is provided and available in `/usr/share/doc/dvd-slideshow/’.

dvdwizard

DVDwizard can fully automate the creation of DVD-Video filesystem. It supports graphical menus, chapters, multiple titlesets and multi-language streams. It supports both PAL and NTSC video modes too.

flowblade

Flowblade is a video editor – like the popular KDenlive based on the MLT engine, but more lightweight and with some difference in editing concepts.

forked-daapd

Forked-daapd switched to a new, active upstream again dropping Grand Central Dispatch in favor of libevent. The switch fixed several bugs and made forked-daapd available on all release architectures instead of shipping only on amd64 and i386. Now nothing prevents you from setting up a music streaming (DAAP/DACP) server on your favorite home server no matter if it is based on mips, arm or x86!

harvid

HTTP Ardour Video Daemon decodes still images from movie files and serves them via HTTP. It provides frame-accurate decoding and is main use-case is to act as backend and second level cache for rendering the
videotimeline in Ardour.

Groove Basin

Groove Basin is a music player server with a web-based user interface inspired by Amarok 1.4. It runs on a server optionally connected to speakers. Guests can control the music player by connecting with a laptop, tablet, or smart phone. Further, users can stream their music libraries remotely.
It comes with a fast, responsive web interface that supports keyboard shortcuts and drag drop. It also provides the ability to upload songs, download songs, and import songs by URL, including YouTube URLs. Groove Basin supports Dynamic Mode which automatically queues random songs, favoring songs that have not been queued recently.
It automatically performs ReplayGain scanning on every song using the EBU R128 loudness standard, and automatically switches between track and album mode. Groove Basin supports the MPD protocol, which means it is compatible with MPD clients. There is also a more powerful Groove Basin protocol which you can use if the MPD protocol does not meet your needs.

HandBrake

HandBrake, a versatile video transcoder, is now available for Jessie. It could convert video from nearly any format to a wide range of commonly supported codecs.

jack-midi-clock

New jackd midiclock utility made by Robin Gareus.

laborejo

Laborejo, Esperanto for “Workshop”, is used to craft music through notation. It is a LilyPond GUI frontend, a MIDI creator and a tool collection to inspire and help music composers.

mpv

mpv is a movie player based on MPlayer and mplayer2. It supports a wide variety of video file formats, audio and video codecs, and subtitle types. The project focuses mainly on modern systems and encourages developer activity. As such, large portions of outdated code originating from MPlayer have been removed, and many new features and improvements have been added. Note that, although there are still some similarities to its predecessors, mpv should be considered a completely different program (e.g. lacking compatibility with both mplayer and mplayer2 in terms of command-line arguments and configuration).

smtube

SMTube is a stand-alone graphical video browser and player, which makes YouTube’s videos browsing, playing, and download such a piece of cake.
It has so many features that, we are sure, will make YouTube lovers very, very happy.

sonic-visualiser

Sonic Visualiser Application for viewing and analysing the contents of music audio files.

SoundScapeRenderer

SoundScapeRenderer (aka SSR) is a (rather) easy to use render engine for spatial audio, that provides a number of different rendering algorithms, ranging from binaural (headphone) playback via wave field synthesis to higher-order ambisonics.

Videotrans

videotrans is a set of scripts that allow its user to reformat existing movies into the VOB format that is used on DVDs.

XBMC

XBMC has been partially rebranded as XBMC from Debian to make it clear that it is changed to conform to Debian’s Policy. The latest stable release, 13.2 Gotham will be part of Jessie making Debian a good choice for HTPC-s.

zita-bls1

Binaural stereo signals converter made by Fons Adriaensen

zita-mu1

Stereo monitoring organiser for jackd made by Fons Adriaensen

zita-njbridge

Jack clients to transmit multichannel audio over a local IP network made by Fons Adriaensen

radium-compressor

Radium Compressor is the system compressor of the Radium suite. It is provided in the form of stand-alone JACK application.

 

Multimedia Tasks

With Jessie we are shipping a set of multimedia related tasks.
They include package lists for doing several multimedia related tasks. If you are interested in defining new tasks, or tweaking the current, existing ones, we are very much interested in hearing from you.

 

Upgraded applications and libraries
  • Aeolus: 0.9.0
  • Aliki: 0.3.0
  • Ams: 2.1.1
  • amsynth: 1.4.2
  • Audacious: 3.5.2
  • Audacity: 2.0.5
  • Audio File Library: 0.3.6
  • Blender: 2.72b
  • Bristol: 0.60.11f
  • C* Audio Plugin Suite: 0.9.23
  • Cecilia: 5.0.9
  • cmus: 2.5.0
  • DeVeDe: 3.23.0-13-gbfd73f3
  • DRC: 3.2.1
  • EasyTag: 2.2.2
  • ebumeter: 0.2.0
  • faustworks: 0.5
  • ffDiaporama: 1.5
  • ffms: 2.20
  • gmusicbrowser: 1.1.13
  • Hydrogen: 0.9.6.1
  • IDJC: 0.8.14
  • jack-tools: 20131226
  • LiVES: 2.2.6
  • mhWaveEdit: 1.4.23
  • Mixxx: 1.11.0
  • mp3fs: 0.91
  • MusE: 2.1.2
  • Petri-Foo: 0.1.87
  • PHASEX: 0.14.97
  • QjackCtl: 0.3.12
  • Qtractor: 0.6.3
  • rtaudio: 4.1.1
  • Rosegarden: 14.02
  • rtmidi: 2.1.0
  • SoundTouch: 1.8.0
  • stk: 4.4.4
  • streamtuner2: 2.1.3
  • SuperCollider: 3.6.6
  • Synfig Studio: 0.64.1
  • TerminatorX: 3.90
  • tsdecrypt: 10.0
  • Vamp Plugins SDK: 2.5
  • VLC: Jessie will release with the 2.2.x series of VLC
  • XCFA: 4.3.8
  • xwax: 1.5
  • xjadeo: 0.8.0
  • x264: 0.142.2431+gita5831aa
  • zynaddsubfx: 2.4.3

 

What’s not going to be in Jessie

With the aim to improve the overall quality of the multimedia software available in Debian, we have dropped a number of packages which were abandoned upstream:

  • beast
  • flumotion
  • jack-rack
  • jokosher
  • lv2fil (suggested replacement for users is eq10q or calf eq)
  • phat
  • plotmm
  • specimen (suggested replacement for users is petri-foo – fork of specimen)
  • zynjacku (suggested replacement for users is jalv)

We’ve also dropped mplayer, presently nobody seems interested in maintaining it.
The suggested replacements for users are mplayer2 or mpv. Whilst the former is mostly compatible with mplayer in terms of command-line arguments and configuration (and adds a few new features too), the latter adds a lot of new features and improvements, and it is actively maintained upstream.

Please note that although the mencoder package is no longer available anymore, avconv and mpv do provide encoding functionality. For more information see avconv’s manual page and documentation, and mpv’s encoding documentation.

 

Broken functionalities

rtkit under systemd is broken at the moment.

 

Activity statistics

More information about team’s activity are available.

 

Where to reach us

The Debian Multimedia Maintainers can be reached at pkg-multimedia-maintainers AT lists.alioth.debian.org for packaging related topics, or at debian-multimedia AT lists.debian.org for user and more general discussion.
We would like to invite everyone interested in multimedia to join us there. Some of the team members are also in the #debian-multimedia channel on OFTC.

Cheers!

Alessio Treglia
on behalf of the Debian Multimedia Maintainers

 

Categories: FLOSS Project Planets

Erich Schubert: Clustering 23 mio Tweet locations

Thu, 2014-10-23 05:01
To test scalability of ELKI, I've clustered 23 million Tweet locations from the Twitter Statuses Sample API obtained over 8.5 months (due to licensing restrictions by Twitter, I cannot make this data available to you, sorry. 23 million points is a challenge for advanced algorithms. It's quite feasible by k-means; in particular if you choose a small k and limit the number of iterations. But k-means does not make a whole lot of sense on this data set - it is a forced quantization algorithm, but does not discover actual hotspots. Density-based clustering such as DBSCAN and OPTICS are much more appropriate. DBSCAN is a bit tricky to parameterize - you need to find the right combination of radius and density for the whole world. Given that Twitter adoption and usage is quite different it is very likely that you won't find a single parameter that is appropriate everywhere. OPTICS is much nicer here. We only need to specify a minimum object count - I chose 1000, as this is a fairly large data set. For performance reasons (and this is where ELKI really shines) I chose a bulk-loaded R*-tree index for acceleration. To benefit from the index, the epsilon radius of OPTICS was set to 5000m. Also, ELKI allows using geodetic distance, so I can specify this value in meters and do not get much artifacts from coordinate projection. To extract clusters from OPTICS, I used the Xi method, with xi set to 0.01 - a rather low value, also due to the fact of having a large data set. The results are pretty neat - here is a screenshot (using KDE Marble and OpenStreetMap data, since Google Earth segfaults for me right now):
Some observations: unsurprisingly, many cities turn up as clusters. Also regional differences are apparent as seen in the screenshot: plenty of Twitter clusters in England, and low acceptance rate in Germany (Germans do seem to have objections about using Twitter; maybe they still prefer texting, which was quite big in Germany - France and Spain uses Twitter a lot more than Germany). Spam - some of the high usage in Turkey and Indonesia may be due to spammers using a lot of bots there. There also is a spam cluster in the ocean south of Lagos - some spammer uses random coordinates [0;1]; there are 36000 tweets there, so this is a valid cluster... A benefit of OPTICS and DBSCAN is that they do not cluster every object - low density areas are considered as noise. Also, they support clusters of different shape (which may be lost in this visualiation, which uses convex hulls!) and different size. OPTICS can also produce a hierarchical result. Note that for these experiments, the actual Tweet text was not used. This has a rough correspondence to Twitter popularity "heatmaps", except that the clustering algorithms will actually provide a formalized data representation of activity hotspots, not only a visualization. You can also explore the clustering result in your browser - the Google Drive visualization functionality seems to work much better than Google Earth. If you go to Istanbul or Los Angeles, you will see some artifacts - odd shaped clusters with a clearly visible spike. This is caused by the Xi extraction of clusters, which is far from perfect. At the end of a valley in the OPTICS plot, it is hard to decide whether a point should be included or not. These errors are usually the last element in such a valley, and should be removed via postprocessing. But our OpticsXi implementation is meant to be as close as possible to the published method, so we do not intend to "fix" this. Certain areas - such as Washington, DC, New York City, and the silicon valley - do not show up as clusters. The reason is probably again the Xi extraction - these region do not exhibit the steep density increase expected by Xi, but are too blurred in their surroundings to be a cluster. Hierarchical results can be found e.g. in Brasilia and Los Angeles. Compare the OPTICS results above to k-means results (below) - see why I consider k-means results to be a meaningless quantization?

Sure, k-means is fast (30 iterations; not converged yet. Took 138 minutes on a single core, with k=1000. The parallel k-means implementation in ELKI took 38 minutes on a single node, Hadoop/Mahout on 8 nodes took 131 minutes, as slow as a single CPU core!). But you can see how sensitive it is to misplaced coordinates (outliers, but mostly spam), how many "clusters" are somewhere in the ocean, and that there is no resolution on the cities? The UK is covered by 4 clusters, with little meaning; and three of these clusters stretch all the way into Bretagne - k-means clusters clearly aren't of high quality here. If you want to reproduce these results, you need to get the upcoming ELKI version (0.6.5~201410xx - the output of cluster convex hulls was just recently added to the default codebase), and of course data. The settings I used are: -dbc.in coords.tsv.gz -db.index tree.spatial.rstarvariants.rstar.RStarTreeFactory -pagefile.pagesize 500 -spatial.bulkstrategy SortTileRecursiveBulkSplit -time -algorithm clustering.optics.OPTICSXi -opticsxi.xi 0.01 -algorithm.distancefunction geo.LngLatDistanceFunction -optics.epsilon 5000.0 -optics.minpts 1000 -resulthandler KMLOutputHandler -out /tmp/out.kmz and the total runtime for 23 million points on a single core was about 29 hours. The indexes helped a lot: less than 10000 distances were computed per point, instead of 23 million - the expected speedup over a non-indexed approach is 2400. Don't try this with R or Matlab. Your average R clustering algorithm will try to build a full distance matrix, and you probably don't have an exabyte of memory to store this matrix. Maybe start with a smaller data set first, then see how long you can afford to increase the data size.
Categories: FLOSS Project Planets

Matthew Garrett: Linux Container Security

Thu, 2014-10-23 03:47
First, read these slides. Done? Good.

Hypervisors present a smaller attack surface than containers. This is somewhat mitigated in containers by using seccomp, selinux and restricting capabilities in order to reduce the number of kernel entry points that untrusted code can touch, but even so there is simply a greater quantity of privileged code available to untrusted apps in a container environment when compared to a hypervisor environment[1].

Does this mean containers provide reduced security? That's an arguable point. In the event of a new kernel vulnerability, container-based deployments merely need to upgrade the kernel on the host and restart all the containers. Full VMs need to upgrade the kernel in each individual image, which takes longer and may be delayed due to the additional disruption. In the event of a flaw in some remotely accessible code running in your image, an attacker's ability to cause further damage may be restricted by the existing seccomp and capabilities configuration in a container. They may be able to escalate to a more privileged user in a full VM.

I'm not really compelled by either of these arguments. Both argue that the security of your container is improved, but in almost all cases exploiting these vulnerabilities would require that an attacker already be able to run arbitrary code in your container. Many container deployments are task-specific rather than running a full system, and in that case your attacker is already able to compromise pretty much everything within the container. The argument's stronger in the Virtual Private Server case, but there you're trading that off against losing some other security features - sure, you're deploying seccomp, but you can't use selinux inside your container, because the policy isn't per-namespace[2].

So that seems like kind of a wash - there's maybe marginal increases in practical security for certain kinds of deployment, and perhaps marginal decreases for others. We end up coming back to the attack surface, and it seems inevitable that that's always going to be larger in container environments. The question is, does it matter? If the larger attack surface still only results in one more vulnerability per thousand years, you probably don't care. The aim isn't to get containers to the same level of security as hypervisors, it's to get them close enough that the difference doesn't matter.

I don't think we're there yet. Searching the kernel for bugs triggered by Trinity shows plenty of cases where the kernel screws up from unprivileged input[3]. A sufficiently strong seccomp policy plus tight restrictions on the ability of a container to touch /proc, /sys and /dev helps a lot here, but it's not full coverage. The presentation I linked to at the top of this post suggests using the grsec patches - these will tend to mitigate several (but not all) kernel vulnerabilities, but there's tradeoffs in (a) ease of management (having to build your own kernels) and (b) performance (several of the grsec options reduce performance).

But this isn't intended as a complaint. Or, rather, it is, just not about security. I suspect containers can be made sufficiently secure that the attack surface size doesn't matter. But who's going to do that work? As mentioned, modern container deployment tools make use of a number of kernel security features. But there's been something of a dearth of contributions from the companies who sell container-based services. Meaningful work here would include things like:

  • Strong auditing and aggressive fuzzing of containers under realistic configurations
  • Support for meaningful nesting of Linux Security Modules in namespaces
  • Introspection of container state and (more difficult) the host OS itself in order to identify compromises

These aren't easy jobs, but they're important, and I'm hoping that the lack of obvious development in areas like this is merely a symptom of the youth of the technology rather than a lack of meaningful desire to make things better. But until things improve, it's going to be far too easy to write containers off as a "convenient, cheap, secure: choose two" tradeoff. That's not a winning strategy.

[1] Companies using hypervisors! Audit your qemu setup to ensure that you're not providing more emulated hardware than necessary to your guests. If you're using KVM, ensure that you're using sVirt (either selinux or apparmor backed) in order to restrict qemu's privileges.
[2] There's apparently some support for loading per-namespace Apparmor policies, but that means that the process is no longer confined by the sVirt policy
[3] To be fair, last time I ran Trinity under Docker under a VM, it ended up killing my host. Glass houses, etc.

comments
Categories: FLOSS Project Planets

Sylvain Le Gall: Release of OASIS 0.4.5

Wed, 2014-10-22 18:42

On behalf of Jacques-Pascal Deplaix

I am happy to announce the release of OASIS v0.4.5.

OASIS is a tool to help OCaml developers to integrate configure, build and install systems in their projects. It should help to create standard entry points in the source code build system, allowing external tools to analyse projects easily.

This tool is freely inspired by Cabal which is the same kind of tool for Haskell.

You can find the new release here and the changelog here. More information about OASIS in general on the OASIS website.

Here is a quick summary of the important changes:

  • Build and install annotation files.
  • Use builtin bin_annot and annot tags.
  • Tag .mly files on the same basis as .ml and .mli files (required by menhir).
  • Remove 'program' constraint from C-dependencies. Currently, when a library has C-sources and e.g. an executable depends on that library, then changing the C-sources and running '-build' does not yield a rebuild of the library. By adding these dependencies (rather removing the constraint), it seems to work fine.
  • Some bug fixes

Features:

  • no_automatic_syntax (alpha): Disable the automatic inclusion of -syntax camlp4o for packages that matches the internal heuristic (if a dependency ends with a .syntax or is a well known syntax).
  • compiled_setup_ml (alpha): Fix a bug using multiple arguments to the configure script.

This new version is a small release to catch up with all the fixes/pull requests present in the VCS that have not yet been published. This should made the life of my dear contributors easier -- thanks again for being patient.

I would like to thanks again the contributor for this release: Christopher Zimmermann, Jerome Vouillon, Tomohiro Matsuyama and Christoph Höger. Their help is greatly appreciated.

Categories: FLOSS Project Planets

Petter Reinholdtsen: listadmin, the quick way to moderate mailman lists - nice free software

Wed, 2014-10-22 14:00

If you ever had to moderate a mailman list, like the ones on alioth.debian.org, you know the web interface is fairly slow to operate. First you visit one web page, enter the moderation password and get a new page shown with a list of all the messages to moderate and various options for each email address. This take a while for every list you moderate, and you need to do it regularly to do a good job as a list moderator. But there is a quick alternative, the listadmin program. It allow you to check lists for new messages to moderate in a fraction of a second. Here is a test run on two lists I recently took over:

% time listadmin xiph fetching data for pkg-xiph-commits@lists.alioth.debian.org ... nothing in queue fetching data for pkg-xiph-maint@lists.alioth.debian.org ... nothing in queue real 0m1.709s user 0m0.232s sys 0m0.012s %

In 1.7 seconds I had checked two mailing lists and confirmed that there are no message in the moderation queue. Every morning I currently moderate 68 mailman lists, and it normally take around two minutes. When I took over the two pkg-xiph lists above a few days ago, there were 400 emails waiting in the moderator queue. It took me less than 15 minutes to process them all using the listadmin program.

If you install the listadmin package from Debian and create a file ~/.listadmin.ini with content like this, the moderation task is a breeze:

username@example.org spamlevel 23 default discard discard_if_reason "Posting restricted to members only. Remove us from your mail list." password secret adminurl https://{domain}/mailman/admindb/{list} mailman-list@lists.example.com password hidden other-list@otherserver.example.org

There are other options to set as well. Check the manual page to learn the details.

If you are forced to moderate lists on a mailman installation where the SSL certificate is self signed or not properly signed by a generally accepted signing authority, you can set a environment variable when calling listadmin to disable SSL verification:

PERL_LWP_SSL_VERIFY_HOSTNAME=0 listadmin

If you want to moderate a subset of the lists you take care of, you can provide an argument to the listadmin script like I do in the initial screen dump (the xiph argument). Using an argument, only lists matching the argument string will be processed. This make it quick to accept messages if you notice the moderation request in your email.

Without the listadmin program, I would never be the moderator of 68 mailing lists, as I simply do not have time to spend on that if the process was any slower. The listadmin program have saved me hours of time I could spend elsewhere over the years. It truly is nice free software.

As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

Categories: FLOSS Project Planets

Konstantinos Margaritis: Eigen NEON port extended to ARMv8!

Wed, 2014-10-22 06:44

Soon after the VSX port, and as promised I have completed the ARMv8 NEON (a.k.a. Advanced SIMD) port. Basically this extends support to 64-bit doubles and also provides faster alternatives to division as ARMv8 has builtin instructions for division both for 32-bit floats and 64-bit doubles. Preliminary benchmarks (bench_gemm):

Categories: FLOSS Project Planets

Steve Kemp: On writing test-cases and testsuites.

Wed, 2014-10-22 05:21

Last night I mostly patched my local copy of less to build and link against the PCRE regular expression library.

I've wanted to do that for a while, and reading Raymond Chen's blog post last night made me try it out.

The patch was small and pretty neat, and I'm familiar with GNU less having patched it in the past. But it doesn't contain tests.

Test cases are hard. Many programs, such as less, are used interactively which makes writing a scaffold hard. Other programs suffer from a similar fate - I'm not sure how you'd even test a web browser such as Firefox these days - mangleme would catch some things, eventually, but the interactive stuff? No clue.

In the past MySQL had a free set of test cases, but my memory is that Oracle locked them up. SQLite is famous for its decent test coverage. But off the top of my head I can't think of other things.

As a topical example there don't seem to be decent test-cases for either bash or openssl. If it compiles it works, more or less.

I did start writing some HTTP-server test cases a while back, but that was just to automate security attacks. e.g. Firing requests like:

GET /../../../etc/passwd HTTP/1.0 GET //....//....//....//etc/passwd HTTP/1.0 etc

(It's amazing how many toy HTTP server components included in projects and products don't have decent HTTP-servers.)

I could imagine that being vaguely useful, especially because it is testing the protocol-handling rather than a project-specific codebase.

Anyway, I'm thinking writing test cases for things is good, but struggling to think of a decent place to start. The project has to be:

  • Non-interactive.
  • Open source.
  • Widely used - to make it a useful contribution.
  • Not written in some fancy language.
  • Open to receiving submissions.

Comments welcome; but better yet why not think about the test-coverage of any of your own packages and projects...?

Categories: FLOSS Project Planets

Russ Allbery: Another haul post

Tue, 2014-10-21 23:44

I know I've been very quiet here lately. That's due to a variety of reasons, but mostly because settling in to a new job is taking nearly all of my attention and time. When that's combined with getting obsessed with watching the League of Legends world championships, it means no real time for writing things.

I've had lots of time for reading things, though, and have a lot of book reviews that I need to write. So, of course, I felt like buying more books.

Elizabeth Bear — One-Eyed Jack (sff)
Steven Brust — Hawk (sff)
Kenneth T. Jackson — Crabgrass Frontier (non-fiction)
Ann Leckie — Ancillary Sword (sff)
Scott Lynch — Republic of Thieves (sff)
Randall Munroe — What If? (non-fiction)
Sarah Tolmie — The Stone Boatmen (sff)
Jeffrey Toobin — The Oath (non-fiction)

I'm pretty excited about everything in this shipment, but particularly the new Vlad Taltos novel from Brust and the sequel to Ancillary Justice (probably the best novel that I've read so far this year). And of course there's What If?.

Categories: FLOSS Project Planets

Junichi Uekawa: Migrating my diary system to some new server.

Tue, 2014-10-21 20:31
Migrating my diary system to some new server. I took the chance to migrate my system from CVS-based system to Git-based system. It no longer relies on a chain of CVS commit hooks, and now I have a makefile to publish. I also took the chance to rewrite my 15 year old elisp so that I can use UTF-8 instead of a mix of ISO-2022-JP and EUC-JP. Dusting off some old code. No test exists, what could go wrong!

Categories: FLOSS Project Planets

DebConf team: DebConf15 dates are set, come and join us! (Posted by DebConf15 team)

Tue, 2014-10-21 10:30

At DebConf14 in Portland, Oregon, USA, next year’s DebConf team presented their conference plans and announced the conference dates: DebConf15 will take place from 15 to 22 August 2015 in Heidelberg, Germany. On the Open Weekend on 15/16 August, we invite members of the public to participate in our wide offering of content and events, before we dive into the more technical part of the conference during following week. DebConf15 will also be preceeded by DebCamp, a time and place for teams to gather for intensive collaboration.

A set of slides from a quick show-case during the DebConf14 closing ceremony provide a quick overview of what you can expect next year. For more in-depth information, we invite you to watch the video recording of the full session, in which the team provides detailed information on the preparations so far, location and transportation to the venue at Heidelberg, the different rooms and areas at the Youth Hostel (for accommodation, hacking, talks, and social activities), details about the infrastructure that are being worked on, and the plans around the conference schedule.

We invite everyone to join us in organising this conference. There are different areas where your help could be very valuable, and we are always looking forward to your ideas. Have a look at our wiki page, join our IRC channels and subscribe to our mailing lists.

We are also contacting potential sponsors from all around the globe. If you know any organisation that could be interested, please consider handing them our sponsorship brochure or contact the fundraising team with any leads.

Let’s work together, as every year, on making the best DebConf ever!

Categories: FLOSS Project Planets

Lucas Nussbaum: Tentative summary of the amendments of the init system coupling GR

Tue, 2014-10-21 09:07

This is an update of my previous attempt at summarizing this discussion. As I proposed one of the amendments, you should not blindly trust me, of course. :-)

First, let’s address two FAQ:

What is the impact on jessie?
On the technical level, none. The current state of jessie already matches what is expected by all proposals. It’s a different story on the social level.

Why are we voting now, then?
Ian Jackson, who submitted the original proposal, explained his motivation in this mail.

We now have four different proposals: (summaries are mine)

  • [iwj] Original proposal (Ian Jackson): Packages may not (in general) require one specific init system (Choice 1 on this page)
  • [lucas] Amendment A (Lucas Nussbaum): support for alternative init systems is desirable but not mandatory (Choice 2 on this page)
  • [dktrkranz] Amendment B (Luca Falavigna): Packages may require a specific init system (Choice 3 on this page)
  • [plessy] Amendment C (Charles Plessy): No GR, please: no GR required (Choice 4 on this page)

[plessy] is the simplest, and does not discuss the questions that the other proposals are answering, given it considers that the normal Debian decision-making processes have not been exhausted.

In order to understand the three other proposals, it’s useful to break them down into several questions.

Q1: support for the default init system on Linux
A1.1: packages MUST work with the default init system on Linux as PID 1.
(That is the case in both [iwj] and [lucas])

A1.2: packages SHOULD work with the default init system on Linux as PID 1.
With [dktrkranz], it would no longer be required to support the default init system, as maintainers could choose to require another init system than the default, if they consider this a prerequisite for its proper operation; and no patches or other derived works exist in order to support other init systems. That would not be a policy violation. (see this mail and its reply for details). Theoretically, it could also create fragmentation among Debian packages requiring different init systems: you would not be able to run pkgA and pkgB at the same time, because they would require different init systems.

Q2: support for alternative init systems as PID 1
A2.1: packages MUST work with one alternative init system (in [iwj])
(Initially, I thought that “one” here should be understood as “sysvinit”, as this mail, Ian detailed why he chose to be unspecific about the target init system. However, in that mail, he later clarified that a package requiring systemd or uselessd would be fine as well, given that in practice there aren’t going to be many packages that would want to couple specifically to systemd _or_ uselessd, but where support for other init systems is hard to provide.)
To the user, that brings the freedom to switch init systems (assuming that the package will not just support two init systems with specific interfaces, but rather a generic interface common to many init systems).
However, it might require the maintainer to do the required work to support additional init systems, possibly without upstream cooperation.
Lack of support is a policy violation (severity >= serious, RC).
Bugs about degraded operation on some init systems follow the normal bug severity rules.

A2.2: packages SHOULD work with alternative init systems as PID 1. (in [lucas])
This is a recommendation. Lack of support is not a policy violation (bug severity < serious, not RC). A2.3: nothing is said about alternative init systems (in [dktrkranz]). Lack of support would likely be a wishlist bug.

Q3: special rule for sysvinit to ease wheezy->jessie upgrades
(this question is implicitly dealt with in [iwj], assuming that one of the supported init systems is sysvinit)

A3.1: continue support for sysvinit (in [lucas])
For the jessie release, all software available in Debian ‘wheezy’ that supports being run under sysvinit should continue to support sysvinit unless there is no technically feasible way to do so.

A3.2: no requirement to support sysvinit (in [dktrkranz])
Theoretically, this could require two-step upgrades: first reboot with systemd, then upgrade other packages

Q4: non-binding recommendation to maintainers
A4.1: recommend that maintainers accept patches that add or improve
support for alternative init systems. (in both [iwj] and [lucas], with a different wording)

A4.2: say nothing (in [dktrkranz])

Q5: support for init systems with are the default on non-Linux ports
A5.1: non-binding recommendation to add/improve support with a high priority (in [lucas])

A5.2: say nothing (in [iwj] and [dktrkranz])

 

Comments are closed: please discuss by replying to that mail.

Categories: FLOSS Project Planets

Erich Schubert: Avoiding systemd isn't hard

Tue, 2014-10-21 08:17
Don't listen to trolls. They lie. Debian was and continues to be about choice. Previously, you could configure Debian to use other init systems, and you can continue to do so in the future. In fact, with wheezy, sysvinit was essential. In the words of trolls, Debian "forced" you to install SysV init! With jessie, it will become easier to choose the init system, because neither init system is essential now. Instead, there is an essential meta-package "init", which requires you to install one of systemd-sysv | sysvinit-core | upstart. In other words, you have more choice than ever before. Again: don't listen to trolls. However, notice that there are some programs such as login managers (e.g. gdm3) which have an upstream dependency on systemd. gdm3 links against libsystemd0 and depends on libpam-systemd; and the latter depends on systemd-sysv | systemd-shim so it is in fact a software such as GNOME that is pulling systemd onto your computer. IMHO you should give systemd a try. There are some broken (SysV-) init scripts that cause problems with systemd; but many of these cases have now been fixed - not in systemd, but in the broken init script. However, here is a clean way to prevent systemd from being installed when you upgrade to jessie. (No need to "fork" Debian for this, which just demonstrates how uninformed some trolls are ... - apart from Debian being very open to custom debian distributions, which can easily be made without "forking".) As you should know, apt allows version pinning. This is the proper way to prevent a package from being installed. All you need to do is create a file named e.g. /etc/apt/preferences.d/no-systemd with the contents: Package: systemd-sysv Pin: release o=Debian Pin-Priority: -1 from the documentation, a priority less than 0 disallows the package from being installed. systemd-sysv is the package that would enable systemd as your default init (/sbin/init). This change will make it much harder for aptitude to solve dependencies. A good way to help it to solve the dependencies is to install the systemd-shim package explicitly first: aptitude install systemd-shim After this, I could upgrade a Debian system from wheezy to jessie without being "forced" to use systemd... In fact, I could also do an aptitude remove systemd systemd-shim. But that would have required the uninstallation of GNOME, gdm3 and network-manager - you may or may not be willing to do this. On a server, there shouldn't be any component actually depending on systemd at all. systemd is mostly a GNOME-desktop thing as of now. As you can see, the trolls are totally blaming the wrong people, for the wrong reasons... and in fact, the trolls make up false claims (as a fact, systemd-shim was updated on Oct 14). Stop listening to trolls, please. If you find a bug - a package that needlessly depends on systemd, or a good way to remove some dependency e.g. via dynamic linking, please contribute a patch upstream and file a bug. Solve problems at the package/bug level, instead of wasting time doing hate speeches.
Categories: FLOSS Project Planets

Thomas Goirand: OpenStack Juno is out, Debian (and Ubuntu Trusty ports) packages ready

Tue, 2014-10-21 01:45

This is just a quick announce: Debian packages for Juno are out. In fact, they were ready the day of the release, on the 16th of October. I uploaded it all (to Experimental) the same day, literally a few hours after the final released was git tagged. But I had no time to announce it.

This week-end, I took the time to do an Ubuntu Trusty port, which I also publish (it’s just a mater of rebuilding all, and it should work out of the box). Here are the backports repositories. For Wheezy:

deb http://archive.gplhost.com/debian juno-backports main

deb http://archive.gplhost.com/debian juno main

For trusty:

deb http://archive.gplhost.com/debian trusty-juno-backports main

But of course, everything is also available directly in Debian. Since Sid/Jessie contains OpenStack Icehouse (which has more chance to receive long enough security support), and it will be like this until Jessie is released. So I have uploaded all of Juno into Debian Experimental. This shows on the OpenStack qa page (you may also notice that the team is nearly reaching 200 packages… though am planning to off-load some of that to the Python module team, when the migration to Git will be finished). On the QA page, you may also see that I uploaded all of the last Icehouse point release to Sid, and that all packages migrated to Jessie. There’s only a few minor issues with some Python modules which I fixed, that haven’t migrated to Jessie yet.

I can already tell that all packages can be installed without an issue, and that I know Horizon at least works as expected. But I didn’t have time to test it all just yet. I’m currently working on doing even more installation automation at the package level (by providing some OVS bridging init script and such, to make it more easy to run Tempest functional testing). I’ll post more about this when it’s ready.

Categories: FLOSS Project Planets

Michal &#268;iha&#345;: Hosted Weblate has new UI

Mon, 2014-10-20 09:00

The biggest part of this HackWeek will be spent on Weblate. The major task is to complete new UI for it. There have been already some blog posts about that here, so regular readers of my blog already know it is using Twitter Bootstrap.

Today it has reached point where I think it's good enough for wider testing and I've deployed it at Hosted Weblate (see Weblate website for conditions for getting hosting there).

I expect there will be some rough edges, so don't hesitate to report any issues, so that I can quickly fix them.

Filed under: English phpMyAdmin SUSE Weblate | 0 comments | Flattr this!

Categories: FLOSS Project Planets

Michal &#268;iha&#345;: Enca 1.16

Mon, 2014-10-20 04:00

As a first tiny project in this HackWeek, Enca 1.16 has been just released. It mostly brings small code cleanups and missing aliases for languages, but fixes also some minor bugs found by Coverity Scan.

If you don't know Enca, it is an Extremely Naive Charset Analyser. It detects character set and encoding of text files and can also convert them to other encodings using either a built-in converter or external libraries and tools like libiconv, librecode, or cstocs.

Full list of changes for 1.16 release:

  • Fixed typo in Belarusian language name
  • Added aliases for Chinese and Yugoslavian languages

Still enca is in maintenance mode only and I have no intentions to write new features. However there is no limitation to other contributors :-).

You can download from http://cihar.com/software/enca/.

Filed under: Enca English SUSE | 0 comments | Flattr this!

Categories: FLOSS Project Planets

Francois Marier: LXC setup on Debian jessie

Sun, 2014-10-19 22:00

Here's how to setup LXC-based "chroots" on Debian jessie. While this is documented on the Debian wiki, I had to tweak a few things to get the networking to work on my machine.

Start by installing (as root) the necessary packages:

apt-get install lxc libvirt-bin debootstrap Network setup

I decided to use the default /etc/lxc/default.conf configuration (no change needed here):

lxc.network.type = veth lxc.network.flags = up lxc.network.link = virbr0 lxc.network.hwaddr = 00:FF:AA:xx:xx:xx lxc.network.ipv4 = 0.0.0.0/24

but I had to make sure that the "guests" could connect to the outside world through the "host":

  1. Enable IPv4 forwarding by putting this in /etc/sysctl.conf:

    net.ipv4.ip_forward=1
  2. and then applying it using:

    sysctl -p
  3. Ensure that the network bridge is automatically started on boot:

    virsh -c lxc:/// net-start default virsh -c lxc:/// net-autostart default
  4. and that it's not blocked by the host firewall, by putting this in /etc/network/iptables.up.rules:

    -A INPUT -d 224.0.0.251 -s 192.168.122.1 -j ACCEPT -A INPUT -d 192.168.122.255 -s 192.168.122.1 -j ACCEPT -A INPUT -d 192.168.122.1 -s 192.168.122.0/24 -j ACCEPT
  5. and applying the rules using:

    iptables-apply
Creating a container

Creating a new container (in /var/lib/lxc/) is simple:

sudo MIRROR=http://http.debian.net/debian lxc-create -n sid64 -t debian -- -r sid -a amd64

You can start or stop it like this:

sudo lxc-start -n sid64 -d sudo lxc-stop -n sid64 Connecting to a guest using ssh

The ssh server is configured to require pubkey-based authentication for root logins, so you'll need to log into the console:

sudo lxc-stop -n sid64 sudo lxc-start -n sid64

then install a text editor inside the container because the root image doesn't have one by default:

apt-get install vim

then paste your public key in /root/.ssh/authorized_keys.

Then you can exit the console (using Ctrl+a q) and ssh into the container. You can find out what IP address the container received from DHCP by typing this command:

sudo lxc-ls --fancy Fixing Perl locale errors

If you see a bunch of errors like these when you start your container:

perl: warning: Setting locale failed. perl: warning: Please check that your locale settings: LANGUAGE = (unset), LC_ALL = (unset), LANG = "fr_CA.utf8" are supported and installed on your system. perl: warning: Falling back to the standard locale ("C").

then log into the container as root and use:

dpkg-reconfigure locales

to enable the same locales as the ones you have configured in the host.

Categories: FLOSS Project Planets