FLOSS Project Planets

Aten Design Group: Automating Drupal Configuration

Planet Drupal - Tue, 2014-10-21 17:42

Last month at the Central Denver Drupal meeting, Nick Switzer from Elevated Third showed how they are using a structured spreadsheet format for describing their Drupal configuration in a way that makes it easy to build. They based their spreadsheet format on a template Palantir published a while ago, and someone mentioned Lullabot has been using something similar. This looked to me a lot like what we were doing at Aten, even though we had missed the de facto standard that was developing. We are now using that de facto standard.

This was particularly interesting to me because I've been doing a lot of work lately around declarative interfaces and standardized Drupal configuration. Spreadsheets are declarative and CINC has a working YAML import, so when we got to the question and answer portion of the presentation, I knew exactly what I wanted to ask: "Why are we still building Drupal sites manually when these spreadsheets contain everything we would need to automate it?"

No one offered a reason not to automate this process, so I volunteered to present at this month's meeting and show an automated process that did not yet exist. I have since built that process. It still needs a lot more testing and bug fixes, but it's already a compelling alternative to the traditional Drupal site building process.

Sheet2Module

Sheet2Module takes a Google spreadsheet and produces a Drupal module that will create the configuration described therein. The exported modules use YAML files for configuration, which works natively in Drupal 8, and works in Drupal 7 with the CINC YAML submodule. With a standard spreadsheet format, Sheet2Module, and CINC YAML, you can build a reasonably complex Drupal site configuration in a few minutes. The process looks like this:

  1. Describe your Drupal configuration in a Google spreadsheet.
  2. Use Sheet2Module to auto-generate a module from that spreadsheet.
  3. Enable that module to auto-generate your Drupal configuration.
  4. (Optional) Spend the hours you would otherwise spend on Drupal configuration helping improve this process.

Both Sheet2Module and CINC YAML almost certainly have bugs, as they've had very limited testing. Both are open source (CINC on Drupal.org, Sheet2Module on GitHub), and patches and pull requests will be met with enthusiastic appreciation. Beyond my appreciation, I'm convinced custom-tailored interfaces like this are the future of Drupal configuration, and you have a lot to gain from helping shape that future.

Outside code contributions, simply trying out the process and giving feedback is very useful, and a good way to make sure this works for your own workflow. Even the incomplete current solution will likely save you hours on your next Drupal build, and you can still manually add any configuration that doesn't work automatically. So you have nothing to lose and hours to gain by trying it out.

Drupal Spreadsheet Standard

I suspect there are more than a few shops already using a similar spreadsheet format to describe Drupal configuration, so before we go too far down the path of building tools around this format, we should turn this into a real, documented community standard. To that end, I've started creating a Drupal Configuration Spreadsheet Standard on GitHub. If you're already using spreadsheets to describe your Drupal configuration, take a look at the documentation and contribute your own format improvements to the wider community. If you're just getting started using spreadsheets to describe your Drupal configuration, this is a good place to start.

Own Your Process

Even if you're not using spreadsheets to describe Drupal configuration, it's worth taking a look at this automation for ideas on how you can improve your own process. I've mentioned before that the declarative format for Drupal configuration adopted in Drupal 8 (and available Drupal 7 with CINC) allows us all to customize our workflows. I'm going to keep mentioning it until this becomes common enough in the Drupal community that it's boring to mention. But for now, this is still a new and exciting space to be working in, and you should join the fun.

Categories: FLOSS Project Planets

gnuzilla @ Savannah: GNU IceCat 31.2.0 released

GNU Planet! - Tue, 2014-10-21 16:06

GNUzilla is the GNU version of the Mozilla suite, and GNU IceCat is the
GNU version of the Firefox browser. Its main advantage is an ethical
one: it is entirely free software. While the Firefox source code from
the Mozilla project is free software, they distribute and recommend
non-free software as plug-ins and addons. Also their trademark license
restricts distribution in several ways incompatible with freedom 0.
https://www.gnu.org/software/gnuzilla/

Source tarballs, binaries for generic GNU/Linux systems and translations
are available at http://ftp.gnu.org/gnu/gnuzilla/31.2.0/
New gpg key ID:D7E04784 GNU IceCat releases
Fingerprint: A573 69A8 BABC 2542 B5A0 368C 3C76 EED7 D7E0 4784

This is a new iteration of the IceCat project, based on new build
scripts and with an extra focus on privacy.
The new maintainer is Ruben Rodriguez.

IceCat will continue to stick to the ESR (Extended Support Release)
cycle (https://www.mozilla.org/en-US/firefox/organizations/faq/) because
it provides security updates over a stable base. That will also allow to
port privacy features from TorBrowser, which is now following v31ESR.

== Changes since v24 ==

  • Javascript can be disabled through the configuration interface.
  • Third party cookies are disabled.
  • Referrers are spoofed (to the same server where the file lives).
  • The user is not asked to install plugins (such as flash or java).
  • Only free software gets offered by IceCat.
  • Installed plugins (flash, java) require per-site activation.
  • DuckDuckGO as default search engine, through https and without JS.
  • DoNotTrack header enabled.
  • Reporting features disabled (Avoids send data to mozilla's partners

about crashes or security related events).

  • Disabled "Social API" that brings integration with Facebook.
  • Disabled "Safe browsing", which asks Google if websites are safe

before browsing them.

  • Disabled access to the clipboard from JS.
  • Don't recommend online services for IRC.

Preinstalled add-ons:

  • LibreJS 6.0.1 checks for the freedom of the javascript you run
  • HttpsEverywhere 4.0.2 redirects requests through https when possible.
  • Spyblock, custom made and based on AdblockPlus, provides:

- A blacklist of trackers that is used in any browsing mode.
Self-served, privacy-friendly advertising is preserved.
- A filter for all third-party requests while in private browsing.
- A filter for javascript data retrieval while in private browsing.
- Autoupdate for filter lists is optional.

  • A custom homepage lists this and other features with links to

documentation and the possibility to disable them quickly if needed.

Fingerprinting:

  • Spoofing the useragent to:

- Mozilla/5.0 (Windows NT 6.1; rv:31.0) Gecko/20100101 Firefox/31.0

  • Fonts can be listed with this methods:

- Plugins like java or flash: these are disabled by default in
IceCat, requiring the user to enable them in a per-site basis. Also
Gnash doesn't work for fingerprinting.
- JS probing: the custom homepage allows to disable custom fonts.

  • Plugins: IceCat no longer discloses the list of installed plugins.
  • Extra spoofing: appname, appversion, buildID, oscpu and platform.
  • Request pages in english by default.

To Do:

  • Add the needed documentation at libreplanet (volunteers welcome!):

- http://libreplanet.org/wiki/Group:IceCat/
- http://libreplanet.org/wiki/Group:IceCat/icecat-help
- http://libreplanet.org/wiki/Group:IceCat/Tour
- http://libreplanet.org/wiki/Group:IceCat/keyboard-shortcuts

  • Incorporate patches from TorBrowser 4.0
  • Build binaries for Windows and MacOS
Categories: FLOSS Project Planets

Creative Juices: 27 Questions (and Answers) from My First Drupal 8 Site Build

Planet Drupal - Tue, 2014-10-21 13:10
27 Questions (and Answers) from My First Drupal 8 Site Build I recently built my first site with Drupal 8, off of the public beta. It was a great experience. I kept a list of questions as I worked, and wrote down the answers when I found them. matt Tue, 10/21/2014 - 13:10
Categories: FLOSS Project Planets

Code Karate: Drush Cheat Sheet

Planet Drupal - Tue, 2014-10-21 11:42

As developers we always are looking for ways to become more efficient. After all, time is money.

Categories: FLOSS Project Planets

blog.studio.gd: Inline Entity Display

Planet Drupal - Tue, 2014-10-21 11:31

At Studio.gd we love the Drupal ecosystem and it became very important to us to give back and participate.
Today we're proud to announce a new module that we hope will help you !

Inline Entity Display module will help you handle the display of referenced entity fields directly in the parent entity.
For exemple if you reference a taxomony "Tags" to an Article node, you will be able directly in the manage display of the article to display tags' fields. It can become very usefull with more complex referenced entity like field collection for exemple.

VOIR LE MODULE : https://www.drupal.org/project/inline_entity_display



Features

- You can control, for each compatible reference field instances, if the fields from the referenced entities would be available as extra fields. Disabled by default.

- You can manage the visibility of the referenced entities fields on the manage display form. Hidden by default.

- View modes are added to represent this context and manage custom display settings for the referenced entities fields in this context {entity_type}_{view_mode} Example: "Node: Teaser" is used to render referenced entities fields, when you reference an entity into a node, and you view this node as a teaser if there are no custom settings for this view mode, fields are rendered using the default view mode settings.

- Extra data attributes are added on the default fields markup, so the field of the same entity can be identified.

Compatible with Field group on manage display form.

Compatible with Display Suite layouts on manage display form.


Requirements

- Entity API
- One of the compatible reference fields module.


Tutorials

simplytest.me/project/inline_entity_display/7.x-1.x
The simplytest.me install of this module will come automatically with these modules: entity_reference, field_collection, field_group, display suite.


VOIR LE MODULE : https://www.drupal.org/project/inline_entity_display


We are currently developping a similar module for Drupal 8 but more powerful and more flexible, Stay tuned !

Categories: FLOSS Project Planets

Blink Reaction: Blog and ebook Series; Responsive Content and Design

Planet Drupal - Tue, 2014-10-21 11:23

Blink Reaction's Director of IT, Kenny Silanskas takes a look at why content is crucial when it comes to creating responsive design. Here Kenny breaks it down with a few easy sports analogies.

Categories: FLOSS Project Planets

Adrian Sutton: So you want to write a bash script…

Planet Apache - Tue, 2014-10-21 10:43

Before writing any even half serious bash script, stop and read:

Any other particularly good articles on writing reliable bash scripts that should be added to this list?

Categories: FLOSS Project Planets

DebConf team: DebConf15 dates are set, come and join us! (Posted by DebConf15 team)

Planet Debian - Tue, 2014-10-21 10:30

At DebConf14 in Portland, Oregon, USA, next year’s DebConf team presented their conference plans and announced the conference dates: DebConf15 will take place from 15 to 22 August 2015 in Heidelberg, Germany. On the Open Weekend on 15/16 August, we invite members of the public to participate in our wide offering of content and events, before we dive into the more technical part of the conference during following week. DebConf15 will also be preceeded by DebCamp, a time and place for teams to gather for intensive collaboration.

A set of slides from a quick show-case during the DebConf14 closing ceremony provide a quick overview of what you can expect next year. For more in-depth information, we invite you to watch the video recording of the full session, in which the team provides detailed information on the preparations so far, location and transportation to the venue at Heidelberg, the different rooms and areas at the Youth Hostel (for accommodation, hacking, talks, and social activities), details about the infrastructure that are being worked on, and the plans around the conference schedule.

We invite everyone to join us in organising this conference. There are different areas where your help could be very valuable, and we are always looking forward to your ideas. Have a look at our wiki page, join our IRC channels and subscribe to our mailing lists.

We are also contacting potential sponsors from all around the globe. If you know any organisation that could be interested, please consider handing them our sponsorship brochure or contact the fundraising team with any leads.

Let’s work together, as every year, on making the best DebConf ever!

Categories: FLOSS Project Planets

Colm O hEigeartaigh: Kerberos Credential Delegation support in Apache CXF

Planet Apache - Tue, 2014-10-21 09:25
Apache CXF provides full support for integrating Kerberos with JAX-WS and JAX-RS services. A previous tutorial (here and here) described how to set up Kerberos with WS-Security in CXF, where the client obtains a Kerberos service ticket and encodes it in the security header of the request, and where it is validated in turn by the service. In this post we will discuss support for kerberos credential delegation for JAX-WS clients and services in Apache CXF. For more information on using kerberos with JAX-RS please consult the CXF documentation.

1) Kerberos Client Configuration

CXF provides a number of JAX-WS properties that can be used to configure Kerberos on the client side (documented here under "Kerberos Configuration Tags"). Essentially there are two different ways of doing it. The client must explicitly allow kerberos credential delegation by setting a property.

1.1) Create and configure a KerberosClient Object directly

The KerberosClient in the CXF WS-Security runtime module is used to retrieve a kerberos ticket. It can be configured by setting various properties and then referenced via the JAX-WS property:
  • ws-security.kerberos.client - A reference to the KerberosClient class used to obtain a service ticket.
The "requestCredentialDelegation" property of the KerberosClient must be set to "true" to allow credential delegation. Here is an example in Spring:

<bean class="org.apache.cxf.ws.security.kerberos.KerberosClient" id="kerberosClient">
        <constructor-arg ref="cxf"/>
        <property name="contextName" value="bob"/>
        <property name="serviceName" value="bob@service.ws.apache.org"/>
        <property name="requestCredentialDelegation" value="true"/>
</bean>

<jaxws:client name="{service}port" createdFromAPI="true">
        <jaxws:properties>
            <entry key="ws-security.kerberos.client" value-ref="kerberosClient"/>
        </jaxws:properties>
</jaxws:client>

1.2) Use JAX-WS properties to configure Kerberos

Rather than use the KerberosClient above, it is possible to configure Kerberos via JAX-WS properties:
  • ws-security.kerberos.jaas.context - The JAAS Context name to use for Kerberos.
  • ws-security.kerberos.spn - The Kerberos Service Provider Name (spn) to use.
  • ws-security.kerberos.is.username.in.servicename.form - Whether the Kerberos username is in servicename form or not.
  • ws-security.kerberos.use.credential.delegation - Whether to use credential delegation or not in the KerberosClient.
  • ws-security.kerberos.request.credential.delegation - Whether to request credential delegation or not in the KerberosClient.
The latter property must be set to "true" on the client side to allow kerberos credential delegation.

2) Kerberos Service Configuration

A JAX-WS service validates a kerberos ticket received in the security header of a request via a KerberosTokenValidator. Here is an example:

<bean id="kerberosValidator"
           class="org.apache.wss4j.dom.validate.KerberosTokenValidator">
        <property name="contextName" value="bob"/>
        <property name="serviceName" value="bob@service.ws.apache.org"/>
</bean>
<jaxws:endpoint ...>
        <jaxws:properties>
            <entry key="ws-security.bst.validator" value-ref="kerberosValidator"/>
        </jaxws:properties>
</jaxws:endpoint>
   
3) Using Kerberos Credential Delegation

After a service has validated a kerberos token sent by the client, it can obtain another kerberos token "on behalf of" the client, assuming the client enabled credential delegation in the first place. To use the client credential for delegation the "useDelegatedCredential" property of the KerberosClient must be set to "true" (see here), or else the JAX-WS property "ws-security.kerberos.use.credential.delegation" must be set to "true" if not configuring Kerberos via the KerberosClient Object.

To see how a concrete use-case for this functionality, take a look at the KerberosDelegationTokenTest in the CXF STS advanced systests. Here we have a backend service which requires a SAML Token issued by an STS. However, the clients only know how to obtain a Kerberos token. So we have an intermediary service which requires a Kerberos token. The clients enable credential delegation + send a ticket to the Intermediary. The Intermediary validates the ticket, then uses it to obtain a Kerberos token "OnBehalfOf" the client, which in turn is used to authenticate to the STS + retrieve a SAML Token, which is then forwarded on to the backend service.
Categories: FLOSS Project Planets

Drupalize.Me: Including Image Styles With Your Drupal 8 Theme

Planet Drupal - Tue, 2014-10-21 09:21

One of many new features in Drupal 8, made possible by the configuration management system, is the ability to add a default image style to your theme, instead of needing to use a module in tandem with your theme, or creating the image style by hand. Here's a look at working with this new feature in Drupal 8.

Categories: FLOSS Project Planets

Lucas Nussbaum: Tentative summary of the amendments of the init system coupling GR

Planet Debian - Tue, 2014-10-21 09:07

This is an update of my previous attempt at summarizing this discussion. As I proposed one of the amendments, you should not blindly trust me, of course. :-)

First, let’s address two FAQ:

What is the impact on jessie?
On the technical level, none. The current state of jessie already matches what is expected by all proposals. It’s a different story on the social level.

Why are we voting now, then?
Ian Jackson, who submitted the original proposal, explained his motivation in this mail.

We now have four different proposals: (summaries are mine)

  • [iwj] Original proposal (Ian Jackson): Packages may not (in general) require one specific init system (Choice 1 on this page)
  • [lucas] Amendment A (Lucas Nussbaum): support for alternative init systems is desirable but not mandatory (Choice 2 on this page)
  • [dktrkranz] Amendment B (Luca Falavigna): Packages may require a specific init system (Choice 3 on this page)
  • [plessy] Amendment C (Charles Plessy): No GR, please: no GR required (Choice 4 on this page)

[plessy] is the simplest, and does not discuss the questions that the other proposals are answering, given it considers that the normal Debian decision-making processes have not been exhausted.

In order to understand the three other proposals, it’s useful to break them down into several questions.

Q1: support for the default init system on Linux
A1.1: packages MUST work with the default init system on Linux as PID 1.
(That is the case in both [iwj] and [lucas])

A1.2: packages SHOULD work with the default init system on Linux as PID 1.
With [dktrkranz], it would no longer be required to support the default init system, as maintainers could choose to require another init system than the default, if they consider this a prerequisite for its proper operation; and no patches or other derived works exist in order to support other init systems. That would not be a policy violation. (see this mail and its reply for details). Theoretically, it could also create fragmentation among Debian packages requiring different init systems: you would not be able to run pkgA and pkgB at the same time, because they would require different init systems.

Q2: support for alternative init systems as PID 1
A2.1: packages MUST work with one alternative init system (in [iwj])
(Initially, I thought that “one” here should be understood as “sysvinit”, as this mail, Ian detailed why he chose to be unspecific about the target init system. However, in that mail, he later clarified that a package requiring systemd or uselessd would be fine as well, given that in practice there aren’t going to be many packages that would want to couple specifically to systemd _or_ uselessd, but where support for other init systems is hard to provide.)
To the user, that brings the freedom to switch init systems (assuming that the package will not just support two init systems with specific interfaces, but rather a generic interface common to many init systems).
However, it might require the maintainer to do the required work to support additional init systems, possibly without upstream cooperation.
Lack of support is a policy violation (severity >= serious, RC).
Bugs about degraded operation on some init systems follow the normal bug severity rules.

A2.2: packages SHOULD work with alternative init systems as PID 1. (in [lucas])
This is a recommendation. Lack of support is not a policy violation (bug severity < serious, not RC). A2.3: nothing is said about alternative init systems (in [dktrkranz]). Lack of support would likely be a wishlist bug.

Q3: special rule for sysvinit to ease wheezy->jessie upgrades
(this question is implicitly dealt with in [iwj], assuming that one of the supported init systems is sysvinit)

A3.1: continue support for sysvinit (in [lucas])
For the jessie release, all software available in Debian ‘wheezy’ that supports being run under sysvinit should continue to support sysvinit unless there is no technically feasible way to do so.

A3.2: no requirement to support sysvinit (in [dktrkranz])
Theoretically, this could require two-step upgrades: first reboot with systemd, then upgrade other packages

Q4: non-binding recommendation to maintainers
A4.1: recommend that maintainers accept patches that add or improve
support for alternative init systems. (in both [iwj] and [lucas], with a different wording)

A4.2: say nothing (in [dktrkranz])

Q5: support for init systems with are the default on non-Linux ports
A5.1: non-binding recommendation to add/improve support with a high priority (in [lucas])

A5.2: say nothing (in [iwj] and [dktrkranz])

 

Comments are closed: please discuss by replying to that mail.

Categories: FLOSS Project Planets

Erich Schubert: Avoiding systemd isn't hard

Planet Debian - Tue, 2014-10-21 08:17
Don't listen to trolls. They lie. Debian was and continues to be about choice. Previously, you could configure Debian to use other init systems, and you can continue to do so in the future. In fact, with wheezy, sysvinit was essential. In the words of trolls, Debian "forced" you to install SysV init! With jessie, it will become easier to choose the init system, because neither init system is essential now. Instead, there is an essential meta-package "init", which requires you to install one of systemd-sysv | sysvinit-core | upstart. In other words, you have more choice than ever before. Again: don't listen to trolls. However, notice that there are some programs such as login managers (e.g. gdm3) which have an upstream dependency on systemd. gdm3 links against libsystemd0 and depends on libpam-systemd; and the latter depends on systemd-sysv | systemd-shim so it is in fact a software such as GNOME that is pulling systemd onto your computer. IMHO you should give systemd a try. There are some broken (SysV-) init scripts that cause problems with systemd; but many of these cases have now been fixed - not in systemd, but in the broken init script. However, here is a clean way to prevent systemd from being installed when you upgrade to jessie. (No need to "fork" Debian for this, which just demonstrates how uninformed some trolls are ... - apart from Debian being very open to custom debian distributions, which can easily be made without "forking".) As you should know, apt allows version pinning. This is the proper way to prevent a package from being installed. All you need to do is create a file named e.g. /etc/apt/preferences.d/no-systemd with the contents: Package: systemd-sysv Pin: release o=Debian Pin-Priority: -1 from the documentation, a priority less than 0 disallows the package from being installed. systemd-sysv is the package that would enable systemd as your default init (/sbin/init). This change will make it much harder for aptitude to solve dependencies. A good way to help it to solve the dependencies is to install the systemd-shim package explicitly first: aptitude install systemd-shim After this, I could upgrade a Debian system from wheezy to jessie without being "forced" to use systemd... In fact, I could also do an aptitude remove systemd systemd-shim. But that would have required the uninstallation of GNOME, gdm3 and network-manager - you may or may not be willing to do this. On a server, there shouldn't be any component actually depending on systemd at all. systemd is mostly a GNOME-desktop thing as of now. As you can see, the trolls are totally blaming the wrong people, for the wrong reasons... and in fact, the trolls make up false claims (as a fact, systemd-shim was updated on Oct 14). Stop listening to trolls, please. If you find a bug - a package that needlessly depends on systemd, or a good way to remove some dependency e.g. via dynamic linking, please contribute a patch upstream and file a bug. Solve problems at the package/bug level, instead of wasting time doing hate speeches.
Categories: FLOSS Project Planets

Joining Kolab Systems

Planet KDE - Tue, 2014-10-21 07:45

I've been a long time fan of Kolab, the free software collaboration and groupware system. I have recommended it, and even helped deploy it a few times, since it launched some ten years ago. I used it back then with KDE's Kontact, and still do to this day.

Kolab interested me because it had the opportunity to join such key free software products as LibreOffice (then Open Office) and Firefox in terms of importance and usage. Think about it: in a professional setting (business, government or educational) what key software tools are universally required? Certainly among them are tools to read and edit office documents; a world-class web browser; and collaboration software (email, calendaring, contacts, resource booking, notes, task lists, file sharing ...). The first two were increasingly well covered, but that last one? Not so much.

And then Kolab walked on to the stage and held out the promise of completing the trifecta.
However, there were years in between then and now when it was less obvious to me that Kolab had a glowing future. It was an amazing early-stage product that filled a huge gap in the free software stack, but development seemed to slow up and promotion was extremely limited. This felt like a small tragedy.

So when I heard that Kolab Systems was launching back in 2010 as a company centered around Kolab, I was excited: Could this be a vehicle which tows Kolab forward towards success? Could this new company propel Kolab effectively into the market which is currently the domain of proprietary products? Only time would tell ... I knew the founders personally, and figured that if anyone could pull this off it would be them. I also knew that they would work with freedom and upstream communities as priorities.

Four years later and Kolab Systems has indeed been successful in bringing Kolab significantly forward technologically and in adoption. Today Kolab is more reliable and has a spectacular set of features, thanks to the solid engineering team that has come together with the help and support of Kolab Systems.

Their efforts have also resulted in Kolab being used more: Fortune 100 companies are using Kolab, the city of Munich is currently migrating to it, there are educational systems using it and, of course, there is My Kolab which is a hosted instance of Kolab hat is being used by an ever growing number of people.

Kolab Systems has also helped the free software it promotes and relies on flourish by investing in it: developers are paid to work on upstream free software such as Roundcube and Kontact in addition to the Kolab serer; community facilitation and public promotion are in focus ... there's a rather nice balance between company and community at play.

There is still a lot to do, however. This is not the end of a success story, perhaps only the end of the beginning. So when the opportunity arose to join Kolab Systems I didn't have to think twice. Starting this month I am joining the Kolab Systems team where I will be engaged in technical efforts (more so in the near term) as well as business and community development. I'm really excited to be joining what is a pretty stellar team of people working on technology I believe in.

Before I wrapping up, I'd like to share something that helped convince me about Kolab Systems. I've known Georg Greve, Kolab Systems' CEO and Free Software Foundation Europe founder, for a good number of years. One afternoon during a friendly walk-and-chat in the countryside near his house, he noted that we should not be satisfied with just making software that is free-as-in-freedom; it should also be awesome software, presented as something worth wanting. It is unrealistic to expect everyone to use free software solely because it is ethically the right thing to do (which it is), but we might expect people to choose free software because it is the most desirable option they know of. To phrase it as an aspiration:

Through excellence we can spread freedom.
I'll probably write more about this philosophy another time, as there are a number of interesting facets to it. I'll also write from time to time about the the interesting things going on in the Kolab world .. but that's all for another time. Right now I need to get back to making notes-on-emails-sync'd-with-a-kolab-server work well. :)

Categories: FLOSS Project Planets

undpaul: Make your styleguide a living styleguide!

Planet Drupal - Tue, 2014-10-21 02:09

Don't you know that, too? You or your team is building a site and during this process all implemented parts are styled through templates and CSS. The CSS files (at best you are using a CSS preprocessor like SASS) are getting bigger, more sophisticated and even more confusing - not to mention that these files are getting almost unmaintainable and more and more error-prone.

drupal planet englishCSS
Categories: FLOSS Project Planets

Thomas Goirand: OpenStack Juno is out, Debian (and Ubuntu Trusty ports) packages ready

Planet Debian - Tue, 2014-10-21 01:45

This is just a quick announce: Debian packages for Juno are out. In fact, they were ready the day of the release, on the 16th of October. I uploaded it all (to Experimental) the same day, literally a few hours after the final released was git tagged. But I had no time to announce it.

This week-end, I took the time to do an Ubuntu Trusty port, which I also publish (it’s just a mater of rebuilding all, and it should work out of the box). Here are the backports repositories. For Wheezy:

deb http://archive.gplhost.com/debian juno-backports main

deb http://archive.gplhost.com/debian juno main

For trusty:

deb http://archive.gplhost.com/debian trusty-juno-backports main

But of course, everything is also available directly in Debian. Since Sid/Jessie contains OpenStack Icehouse (which has more chance to receive long enough security support), and it will be like this until Jessie is released. So I have uploaded all of Juno into Debian Experimental. This shows on the OpenStack qa page (you may also notice that the team is nearly reaching 200 packages… though am planning to off-load some of that to the Python module team, when the migration to Git will be finished). On the QA page, you may also see that I uploaded all of the last Icehouse point release to Sid, and that all packages migrated to Jessie. There’s only a few minor issues with some Python modules which I fixed, that haven’t migrated to Jessie yet.

I can already tell that all packages can be installed without an issue, and that I know Horizon at least works as expected. But I didn’t have time to test it all just yet. I’m currently working on doing even more installation automation at the package level (by providing some OVS bridging init script and such, to make it more easy to run Tempest functional testing). I’ll post more about this when it’s ready.

Categories: FLOSS Project Planets

Jasha Joachimsthal: iSwitched

Planet Apache - Tue, 2014-10-21 01:41

After 5 years of using different Android phones I switched to an iPhone. In 2009 I bought an HTC Hero, which was replaced by the HTC Desire a year later. Then my former employer gave me a Samsung Galaxy S2 which was replaced by the S3 after I switched jobs. I was about to switch to the Sony Xperia Z3 but I decided to go for an iDevice: the iPhone 6.

What do I miss in iOS so far? Going back

I miss the back button on the phone. In iOS it’s up to the application to provide a back link somewhere in the screen. Usually in the top left corner, but sometimes it’s not there. That top left corner is hard to reach with one hand and Apple has created a workaround for that: tap (not click) the home button twice and the screen goes down so you can reach this button. Adding that extra physical button would have been easier. Another difference is that the back button is only within the context of the current application while the Android back button can bring you back to the previous application if that had opened another application.

No widgets on the home screen

When I opened the S3 I saw the latest weather information on my home screen. On the next screen I had an monthly overview of my calendar. On the iPhone there are only app icons. The weather and upcoming appointments for today are in the Notification Centre. It’s not in my system yet to open the Notification Centre or the Calendar app to see what’s on my schedule for the next week/month (and yes I already forgot an appointment).

Swype

It was one of the first apps I bought, but unfortunately there’s no dictionary available for Dutch yet. Let’s hope that’s just a matter of time, because Swype supports many more languages in the Android version.

Notification LED

The S3 had a blinking LED to notify me of unread messages: dark blue for mail and SMS, light blue for WhatsApp and green for Telegram and purple for MeetUp. It kept flashing until I took action to read the message or remove the notification. The iPhone blinks shortly when a message arrives, but that’s it. I don’t have my phone on me all of the time and sometimes I don’t hear or feel it, so that blinking LED was handy.

What is better? Integration with iTunes

Samsung, I like your devices but Kies sucks. Most of the time it starts, but not always. When it has started it may or may not recognise the phone. After sacrificing a goat to the gods of Samsung it may also finish a back up successfully. More often it failed in the back up (luckily I never needed a restore) or didn’t recognise the phone. The KiesViaWifiAgent was turning my MacBook into a heating fan. iTunes just works. It recognises the phone, makes back ups and installs iOS updates.

Touch ID

The US department of Homeland Security took prints of all my fingers when I wanted to enter the US. The Dutch government wanted 2 finger prints for my new passport.  Now Apple also has a finger print of me to unlock my phone or authorise purchases (but I have no clue what else they do with it).

Permissions

I didn’t switch because of the Apple logo. Although I’ve been a fan of Apple’s desktop OS since System 6, I’ve never been attracted to the iOS. The main reason I switched from Android to iOS are the permissions apps get. In Android it’s an all or nothing decision. For instance the Facebook app wants access to my contacts, calendar, SMS and call history (and a lot more). If you don’t want to give permission to let the app access your SMS history, then you can’t install the app. This is different in iOS, where you as user can control whether the app gets access to your contacts or calendar (SMS and call history are normally inaccessible by apps). More information is in the article iOS Has App Permissions, Too: And They’re Arguably Better Than Android’s.

Categories: FLOSS Project Planets

Justin Mason: Links for 2014-10-20

Planet Apache - Mon, 2014-10-20 19:58
  • Load testing Apache Kafka on AWS

    This is a very solid benchmarking post, examining Kafka in good detail. Nicely done. Bottom line:

    I basically spend 2/3 of my work time torture testing and operationalizing distributed systems in production. There’s some that I’m not so pleased with (posts pending in draft forever) and some that have attributes that I really love. Kafka is one of those systems that I pretty much enjoy every bit of, and the fact that it performs predictably well is only a symptom of the reason and not the reason itself: the authors really know what they’re doing. Nothing about this software is an accident. Performance, everything in this post, is only a fraction of what’s important to me and what matters when you run these systems for real. Kafka represents everything I think good distributed systems are about: that thorough and explicit design decisions win.

    (tags: testing aws kafka ec2 load-testing benchmarks performance)

Categories: FLOSS Project Planets

Mike Driscoll: PyWin32 – How to Bring a Window to Front

Planet Python - Mon, 2014-10-20 18:15

I recently saw someone asking how to bring a window to the front in Windows and I realized I had had some old unreleased code that might help someone with this task. A long time ago, Tim Golden (and possibly some other fellows on the PyWin32 mailing list) showed me how to make windows come to the front on Windows XP, although it should be noted that it also works on Windows 7. If you’d like to follow along, you will need to download and install your own copy of PyWin32.

We will need to choose something to bring to the front. I like to use Notepad for testing as I know it will be on every Windows desktop in existence. Open up Notepad and then put some other application’s window in front of it.

Now we’re ready to look at some code:

import win32gui   def windowEnumerationHandler(hwnd, top_windows): top_windows.append((hwnd, win32gui.GetWindowText(hwnd)))   if __name__ == "__main__": results = [] top_windows = [] win32gui.EnumWindows(windowEnumerationHandler, top_windows) for i in top_windows: if "notepad" in i[1].lower(): print i win32gui.ShowWindow(i[0],5) win32gui.SetForegroundWindow(i[0]) break

We only need PyWin32’s win32gui module for this little script. We write a little function that takes a window handle and a Python list. Then we call win32gui’s EnumWindows method, which takes a callback and an extra argument that is a Python object. According to the documentation, the EnumWindows method “Enumerates all top-level windows on the screen by passing the handle to each window, in turn, to an application-defined callback function”. So we pass it our method and it enumerates the windows, passing a handle of each window plus our Python list to our function. It works kind of like a messed up decorator.

Once that’s done, your top_windows list will be full of lots of items, most of which you didn’t even know were running. You can print that our and inspect your results if you like. It’s really quite intereting. But for our purposes, we will skip that and just loop over the list, looking for the word “Notepad”. Once we find it, we use win32gui’s ShowWindow and SetForegroundWindow methods to bring the application to the foreground.

Note that really need to look for a unique string so that you bring up the right window. What would happen if you had multiple Notepad instance running with different files open? With the current code, you would bring the first Notepad instance that it found forward, which might not be what you want.

You may be wondering why anyone would even want to go to the trouble of doing this in the first place. In my case, I once had a project where I had to bring a certain window to the foreground and enter automate it using SendKeys. It was an ugly piece of brittle code that I wouldn’t wish on anyone. Fortunately, there are better tools for that sort of thing nowadays such as pywinauto, but you still might find this code helpful in something esoteric that is thrown your way. Have fun!

Note: This code was tested using Python 2.7.8 and PyWin32 219 on Windows 7.

Categories: FLOSS Project Planets

Carl Trachte: subprocess.Popen() or Abusing a Home-grown Windows Executable

Planet Python - Mon, 2014-10-20 17:11
Each month I redo 3D block model interpolations for a series of open pits at a distant mine.  Those of you who follow my twitter feed often see me tweet, "The 3D geologic block model interpolation chuggeth . . ."  What's going on is that I've got all the processing power maxed out dealing with millions of model blocks and thousands of data points.  The machine heats up and with the fan sounds like a DC-9 warming up before flight.

All that said, running everything roughly in parallel is more efficient time-wise than running it sequentially.  An hour of chugging is better than four.  The way I've been doing this is using the Python (2.7) subprocess module's Popen method, running my five interpolated values in parallel.  Our Python programmer Lori originally wrote this to run in sequence for a different set of problems.  I bastardized it for my own.

The subprocess part of the code is relatively straightforward.  Function startprocess() in my code covers that.

What makes this problem a little more challenging:

1) it's a vendor supplied executable we're dealing with . . . without an API or source . . . that's interactive (you can't feed it the config file path; it asks for it).  This results in a number of time.sleep() and <process>.stdin.write() calls that can be brittle.

2) getting the processes started, as I just mentioned, is easy.  Finding out when to stop, or kill them, requires knowledge of the app and how it generates output.  I've gone for an ugly, but effective check of report file contents.

3) while waiting for the processes to finish their work, I need to know things are working and what's going on.  I've accomplished this by reporting the data files' sizes in MB.

4) the executable isn't designed for a centralized code base (typically all scripts are kept in a folder for the specific project or pit), so it only allows about 100 character columns in the file paths sent to it.  I've omitted this from my sanitized version of the code, but it made things even messier than they are below.  Also, I don't know if all Windows programs do this, but the paths need to be inside quotes - the path kept breaking on the colon (:) when not quoted.

Basically, this is a fairly ugly problem and a script that requires babysitting while it runs.  That's OK; it beats the alternative (running it sequentially while watching each run).  I've tried to adhere to DRY (don't repeat yourself) as much as possible, but I suspect this could be improved upon.

The reason why I blog it is that I suspect there are other people out there who have to do the same sort of thing with their data.  It doesn't have to be a mining problem.  It can be anything that requires intensive computation across voluminous data with an executable not designed with a Python API.

Notes: 

1) I've omitted the file multirunparameters.py that's in an import statement.  It has a bunch of paths and names that are relevant to my project, but not to the reader's programming needs.

2) python 2.7 is listed at the top of the file as "mpython."  This is the Python that our mine planning vendor ships that ties into their quite capable Python API.  The executable I call with subprocess.Popen() is a Windows executable provided by a consultant independent of the mine planning vendor.  It just makes sense to package this interpolation inside the mine planning vendor's multirun (~ batch file) framework as part of an overall working of the 3D geologic block model.  The script exits as soon as this part of the batch is complete.  I've inserted a 10 second pause at the end just to allow a quick look before it disappears.

#!C:/MineSight/x64/mpython

"""
Interpolate grades with <consultant> program
from text files.
"""

import argparse
import subprocess as subx
import os
import collections as colx
import time
from datetime import datetime as dt

# Lookup file of constants, pit names, assay names, paths, etc.
import multirunparameters as paramsx

parser = argparse.ArgumentParser()
# 4 letter argument like 'kwat'
# Feed in at command line.
parser.add_argument('pit', help='four letter, lower case pit abbreviation (kwat)', type=str)
args = parser.parse_args()
PIT = args.pit

pitdir = paramsx.PATHS[PIT]
pathx = paramsx.BASEPATH.format(pitdir)
controlfilepathx = paramsx.CONTROLFILEPATH.format(pitdir)

timestart = dt.now()
print(timestart)

PROGRAM = 'C:/MSPROJECTS/EOMReconciliation/2014/Multirun/AllPits/consultantprogram.exe'

ENDTEXT = 'END <consultant> REPORT'

# These names are the only real difference between pits.
# Double quote is for subprocess.Popen object's stdin.write method
# - Windows path breaks on colon without quotes.
ASSAY1DRIVER = 'KDriverASSAY1{:s}CBT.csv"'.format(PIT)
ASSAY2DRIVER = 'KDriverASSAY2{:s}CBT.csv"'.format(PIT)
ASSAY3DRIVER = 'KDriverASSAY3_{:s}CBT.csv"'.format(PIT)
ASSAY4DRIVER = 'KDriverASSAY4_{:s}CBT.csv"'.format(PIT)
ASSAY5DRIVER = 'KDriverASSAY5_{:s}CBT.csv"'.format(PIT)

RETCHAR = '\n'

ASSAY1 = 'ASSAY1'
ASSAY2 = 'ASSAY2'
ASSAY3 = 'ASSAY3'
ASSAY4 = 'ASSAY4'
ASSAY5 = 'ASSAY5'

NAME = 'name'
DRFILE = 'driver file'
OUTPUT = 'output'
DATFILE = 'data file'
RPTFILE = 'report file'

# data, report files
ASSAY1K = 'ASSAY1K.csv'
ASSAY1RPT = 'ASSAY1.RPT'
ASSAY2K = 'ASSAY2K.csv'
ASSAY2RPT = 'ASSAY2.RPT'
ASSAY3K = 'ASSAY3K.csv'
ASSAY3RPT = 'ASSAY3.RPT'
ASSAY4K = 'ASSAY4K.csv'
ASSAY4RPT = 'ASSAY4.RPT'
ASSAY5K = 'ASSAY5K.csv'
ASSAY5RPT = 'ASSAY5.RPT'

OUTPUTFMT = '{:s}output.txt'

ASSAYS = {1:{NAME:ASSAY1,
             DRFILE:controlfilepathx + ASSAY1DRIVER,
             OUTPUT:pathx + OUTPUTFMT.format(ASSAY1),
             DATFILE:pathx + ASSAY1K,
             RPTFILE:pathx + ASSAY1RPT},
          2:{NAME:ASSAY2,
             DRFILE:controlfilepathx + ASSAY2DRIVER,
             OUTPUT:pathx + OUTPUTFMT.format(ASSAY2),
             DATFILE:pathx + ASSAY2K,
             RPTFILE:pathx + ASSAY2RPT},
          3:{NAME:ASSAY3,
             DRFILE:controlfilepathx + ASSAY3DRIVER,
             OUTPUT:pathx + OUTPUTFMT.format(ASSAY3),
             DATFILE:pathx + ASSAY3K,
             RPTFILE:pathx + ASSAY3RPT},
          4:{NAME:ASSAY4,
             DRFILE:controlfilepathx + ASSAY4DRIVER,
             OUTPUT:pathx + OUTPUTFMT.format(ASSAY4),
             DATFILE:pathx + ASSAY4K,
             RPTFILE:pathx + ASSAY4RPT},
          5:{NAME:ASSAY5,
             DRFILE:controlfilepathx + ASSAY5DRIVER,
             OUTPUT:pathx + OUTPUTFMT.format(ASSAY5),
             DATFILE:pathx + ASSAY5K,
             RPTFILE:pathx + ASSAY5RPT}}

DELFILE = 'delete file'
INTERP = 'interp'
SLEEP = 'sleep'
MSGDRIVER = 'message driver'
MSGRETCHAR = 'message return character'
FINISHED1 = 'finished one assay'
FINISHEDALL = 'finished all interpolations'
TIMEELAPSED = 'time elapsed'
FILEEXISTS = 'report file exists'
DATSIZE = 'data file size'
DONE = 'number interpolations finished'
DATFILEEXIST = 'data file not yet there'
SIZECHANGE = 'report file changed size'

# for converting to megabyte file size from os.stat()
BITSHIFT = 20
# sleeptime - 5 seconds
SLEEPTIME = 5
FINISHED = 'finished'
RPTFILECHSIZE = """
        
Report file for {:s}
changed size; killing process . . .
"""

MESGS = {DELFILE:'\n\nDeleting {} . . .\n\n',
         INTERP:'\n\nInterpolating {:s} . . .\n\n',
         SLEEP:'\nSleeping 2 seconds . . .\n\n',
         MSGDRIVER:'\n\nWriting driver file name to stdin . . .\n\n',
         MSGRETCHAR:'\n\nWriting retchar to stdin for {:s} . . .\n\n',
         FINISHED1:'\n\nFinished {:s}\n\n',
         FINISHEDALL:'\n\nFinished interpolation.\n\n',
         TIMEELAPSED:'\n\n{:d} elapsed seconds\n\n',
         FILEEXISTS:'\n\nReport file for {:s} exists . . .\n\n',
         DATSIZE:'\n\nData file size for {:s} is now {:d}MB . . .\n\n',
         DONE:'\n\n{:d} out of {:d} assays are finished . . .\n\n',
         DATFILEEXIST:"\n\n{:s} doesn't exist yet . . .\n\n",
         SIZECHANGE:RPTFILECHSIZE}

def cleanslate():
    """
    Delete all output files prior to interpolation
    so that their existence can be tracked.
    """
    for key in ASSAYS:
        files = (ASSAYS[key][DATFILE],
                 ASSAYS[key][RPTFILE],
                 ASSAYS[key][OUTPUT])
        for filex in files:
            print(MESGS[DELFILE].format(filex))
            if os.path.exists(filex) and os.path.isfile(filex):
                os.remove(filex)
    return 0

def startprocess(assay):
    """
    Start <consultant program> run for given interpolation.
    Return subprocess.Popen object,
    file object (output file).
    """
    print(MESGS[INTERP].format(ASSAYS[assay][NAME]))
    # XXX - I hate time.sleep - hack
    # XXX - try to re-route standard output so that
    #       it's not all jumbled together.
    print(MESGS[SLEEP])
    time.sleep(2)
    # output file for stdout
    f = open(ASSAYS[assay][OUTPUT], 'w')
    procx = subx.Popen('{0}'.format(PROGRAM), stdin=subx.PIPE, stdout=f)
    print(MESGS[SLEEP])
    time.sleep(2)
    # XXX - problem, starting up Excel CBT 22JUN2014
    #       Ah - this is what happens when the <software usb licence>
    #            key is not attached :-(
    print(MESGS[MSGDRIVER])
    print('\ndriver file = {:s}\n'.format(ASSAYS[assay][DRFILE]))
    procx.stdin.write(ASSAYS[assay][DRFILE])
    print(MESGS[SLEEP])
    time.sleep(2)
    # XXX - this is so jacked up -
    #       no idea what is happening when
    print(MESGS[MSGRETCHAR].format(ASSAYS[assay][NAME]))
    procx.stdin.write(RETCHAR)
    print(MESGS[SLEEP])
    time.sleep(2)
    print(MESGS[MSGRETCHAR].format(ASSAYS[assay][NAME]))
    procx.stdin.write(RETCHAR)
    print(MESGS[SLEEP])
    time.sleep(2)
    return procx, f

def crosslookup(assay):
    """
    From assay string, get numeric
    key for ASSAYS dictionary.
    Returns integer.
    """
    for key in ASSAYS:
        if assay == ASSAYS[key][NAME]:
            return key
    return 0

def checkprocess(assay, assaydict):
    """
    Check to see if assay
    interpolation is finished.
    assay is the item in question
    (ASSAY1, ASSAY2, etc.).
    assaydict is the operating dictionary
    for the assay in question.
    Returns True if finished.
    """
    # Report file indicates process finished.
    assaykey = crosslookup(assay)
    rptfile = ASSAYS[assaykey][RPTFILE]
    datfile = ASSAYS[assaykey][DATFILE]
    if os.path.exists(datfile) and os.path.isfile(datfile):
        # Report size of file in MB.
        datfilesize = os.stat(datfile).st_size >> BITSHIFT
        print(MESGS[DATSIZE].format(assay, datfilesize))
    else:
        # Doesn't exist yet.
        print(MESGS[DATFILEEXIST].format(datfile))
    if os.path.exists(rptfile) and os.path.isfile(rptfile):
        # XXX - not the most efficient way,
        #       but this checking the file appears
        #       to work best.
        f = open(rptfile, 'r')
        txt = f.read()
        f.close()
        # XXX - hack - gah.
        if txt.find(ENDTEXT) > -1:
            # looking for change in reportfile size
            # or big report file
            print(MESGS[SIZECHANGE].format(assay))
            print(MESGS[SLEEP])
            time.sleep(2)
            return True
    return False

PROCX = 'process'
OUTPUTFILE = 'output file'

# Keeps track of files and progress of <consultant program>.
opdict = colx.OrderedDict()

# get rid of preexisting files
cleanslate()

# start all five roughly in parallel
# ASSAYS keys are numbers
for key in ASSAYS:
    # opdict - ordered with assay names as keys
    namex = ASSAYS[key][NAME]
    opdict[namex] = {}
    assaydict = opdict[namex]
    assaydict[PROCX], assaydict[OUTPUTFILE] = startprocess(key)
    # Initialize active status of process.
    assaydict[FINISHED] = False

# For count.
numassays = len(ASSAYS)
# Loop until all finished.
while True:
    # Cycle until done then break.
    # Sleep SLEEPTIME seconds at a time and check between.
    time.sleep(SLEEPTIME)
    # Count.
    i = 0
    for key in opdict:
        assaydict = opdict[key]
        if not assaydict[FINISHED]:
            status = checkprocess(key, assaydict)
            if status:
                # kill process when report file changes
                opdict[key][PROCX].kill()
                assaydict[FINISHED] = True
                i += 1
        else:
            i += 1
    print(MESGS[DONE].format(i, numassays))
    # all done
    if i == numassays:
        break

print('\n\nFinished interpolation.\n\n')
timeend = dt.now()
elapsed = timeend - timestart

print(MESGS[TIMEELAPSED].format(elapsed.seconds))
print('\n\n{:d} elapsed minutes\n\n'.format(elapsed.seconds/60))

# Allow quick look at screen.
time.sleep(10)


Categories: FLOSS Project Planets
Syndicate content