Feeds

Shane Curcuru: Goodbye OSCON! See you next year!

Planet Apache - Thu, 2014-07-24 20:25

It was an amazing week at OSCON – so many great people to meet and share ideas with, both people I’d never met (some who knew me!) and plenty of old friends. Unfortunately I missed traveling out early to attend the Community Leadership Summit, a mistake I will not repeat next year – I’ll see folks there!

While my OSCON talk submissions on brand didn’t quite make the cut (they had an amazing number of great CFP submissions), I was lucky enough to present an OSCON Ignite 5-minute talk on Why your project’s brand is more important than the code, or BRAND > CODE. Since the slides for the 5 minute talk (they auto-advance every 15 seconds) are mostly graphics, I’ve posted my BRAND > CODE script as well.

Oh, hey, OSCON Ignite videos are already up – yay!

Giving the talk was great, but what was even better was the number of people who have come up during the conference who mentioned they liked it and/or that it opened their eyes to thinking about the bigger story that your project tells to the world.

Hope to see some more folks when I get to speak at ApacheCon Europe this November in Budapest – the conference schedule is now posted! And I’m hoping to be speaking at OSCON again next year as well!

Categories: FLOSS Project Planets

Justin Mason: Links for 2014-07-24

Planet Apache - Thu, 2014-07-24 19:58
Categories: FLOSS Project Planets

Cutelyst 0.3.0 is now C10K ready!

Planet KDE - Thu, 2014-07-24 19:18

Release early, release often has never been my strength especially since I don’t do a fair scheduling of all the projects I’m involved…

So since I was in need to better polish Cutelyst I took more time on it, and the result is great, around 100% speed up and a few new features added.

Cutelyst uWSGI plugin now has support for –thread, which will create a QThread to process a request, however I strongly discourage its usage in Cutelyst, the performance is ~7% inferior and a crash in your code will break other requests, and as of now ASYNC mode is not supported in threaded mode due to a limitation in uWSGI request queue.

Thanks to valgrind I managed to make a hello world application from 5K request per second on 0.2.0 to 10K req/s on 0.3.0 on an Intel Core2Duo 2.4Ghz (or 44K req/s on an AMD Phenom II 965 x4 3.4Ghz), however if you enable Grantlee templating you get around 600 req/s so if I happen to have time I will be looking into improving its performance.

Response::body() is now a QIODevice so you can set a QFile* of something else and have Cutelyst to send it back.

Now http://cutelyst.org points to a gitorious Wiki which is slowly getting populated, and API is available as http://api.cutelyst.org.

The 0.3.0 tarball can be downloaded here

Have fun :)


Categories: FLOSS Project Planets

Gael Varoquaux: The 2014 international scikit-learn sprint

Planet Python - Thu, 2014-07-24 18:59

A week ago, the 2014 edition of the scikit-learn sprint was held in Paris. This was the third time that we held an internation sprint and it was hugely productive, and great fun, as always.

Great people and great venues

We had a mix of core contributors and newcomers, which is a great combination, as it enables us to be productive, but also to foster the new generation of core developers. Were present:

  • Laurent Direr
  • Michael Eickenberg
  • Loic Esteve
  • Alexandre Gramfort
  • Olivier Grisel
  • Arnaud Joly
  • Kyle Kastner
  • Manoj Kumar
  • Balazs Kegl
  • Nicolas Le Roux
  • Andreas Mueller
  • Vlad Niculae
  • Fabian Pedregosa
  • Amir Sani
  • Danny Sullivan
  • Gabriel Synnaeve
  • Roland Thiolliere
  • Gael Varoquaux

As the sprint extended through a French bank holiday and the week end, we were hosted in a variety of venues:

  • La paillasse, a Paris bio-hacker space
  • INRIA, the French computer-science national research, and the place where I work
  • Criteo, a French company doing word-wide add-banner placement. The venue there was absolutely gorgeous, with a beautiful terrace on the roofs of Paris.
    And they even had a social event with free drinks one evening.
  • Tinyclues, a French startup mining e-commerce data.

I must say that we were treated like kings during the whole stay; each host welcoming us as well they could. Thank you to all of our hosts!

Sponsored by the Paris-Saclay Center for Data Science

Beyond our hosts, we need to thank the Paris-Saclay Center for Data Science. The CDS gave us funding that covered some of the lunches, acomodations, and travel expenses to bring in our
contributors from abroad.

Achievements during the sprint

The first day of the sprint was dedicated to polishing the 0.15 release, which was finally released on the morning of the second day, after 10 months of development.

A large part of the efforts of the sprint were dedicated to improving the coding base, rather than directly adding new features. Some files were reorganized. The input validation code was cleaned up (opening the way for better support of pandas structures in scikit-learn). We hunted dead code, deprecation warnings, numerical instabilities and tests randomly failing. We made the test suite faster, and refactored our common tests that scan all the model.

Some work of our GSOC student, Manoj Kumar, was merged, making some linear models faster.

Our online documentation was improved with the API documentation pointing to examples and source code.

Still work in progress:

  • Faster stochastic gradient descent (with AdaGrad, ASGD, and one day SAG)
  • Calibration of probabilities for models that do not have a ‘predict_proba’ method
  • Warm restart in random forests to add more estimators to an existing ensemble.
  • Infomax ICA algorithm.

Categories: FLOSS Project Planets

Drupal.org frontpage posts for the Drupal planet: Drupal 7.30 released

Planet Drupal - Thu, 2014-07-24 18:12

Drupal 7.30, a maintenance release with several bug fixes (no security fixes), including a fix for regressions introduced in Drupal 7.29, is now available for download. See the Drupal 7.30 release notes for a full listing.

Download Drupal 7.30

Upgrading your existing Drupal 7 sites is recommended. There are no new features in this release. For more information about the Drupal 7.x release series, consult the Drupal 7.0 release announcement.

Security information

We have a security announcement mailing list and a history of all security advisories, as well as an RSS feed with the most recent security advisories. We strongly advise Drupal administrators to sign up for the list.

Drupal 7 includes the built-in Update Manager module, which informs you about important updates to your modules and themes.

There are no security fixes in this release of Drupal core.

Bug reports

Drupal 7.x is being maintained, so given enough bug fixes (not just bug reports), more maintenance releases will be made available, according to our monthly release cycle.

Changelog

Drupal 7.30 is a bug fix only release. The full list of changes between the 7.29 and 7.30 releases can be found by reading the 7.30 release notes. A complete list of all bug fixes in the stable 7.x branch can be found in the git commit log.

Update notes

See the 7.30 release notes for details on important changes in this release.

Known issues

None.

Front page news: Planet DrupalDrupal version: Drupal 7.x
Categories: FLOSS Project Planets

Node.js support in KDevelop

Planet KDE - Thu, 2014-07-24 18:00

This is a quick blog post to say that I’m working on support for the Node.js framework by the KDevelop QML/JS language plugin. The first part of my work is already finished: “require” is properly recognized and Node.js modules are properly looked-up and included. The “exports” and “module.exports” identifiers are also recognized and allow Javascript files to export symbols and to be used as modules. This means that you can now edit Javascript files using Node.js and have most of the modules recognized, included and parsed.

The second part is writing huge Javascript files for the built-in Node.js modules. “http”, “buffer”, “fs”, “url” and many other modules exist only as part of Node.js. There are Javascript files for them in the Node.js Git repository, but these files don’t allow the QML/JS plugin to infer the types of the methods. They are also fairly big and would need a big cleanup in order to be fast enough to parse.

As with ECMAScript, I solve the problem by shipping with QML/JS stripped down versions of those files. These files describe the classes, methods and attributes of each module, and follow a very simple format. This code is not designed to be run, only to be parsed by the QML/JS plugin, that will discover the types of the function parameters and their return type.

1 2exports.spawn = function (command, args, options) { return new exports.ChildProcess(); }; exports.spawn("", [], new Object());

Writing all these files is tiring and will take some time (in fact, I’m writing them for a bit more than one day, now). So, there may be fewer commit than usual in the QML/JS repository, but I hope that this feature will serve many users. Here is a teaser of what is coming (note that functions are now listed in a way more human-friendly form):

Once this feature is finished, I will concentrate my efforts on the QML code-completion. I’ll also experiment with the KDE Frameworks 5 (and the KF5 version of KDevelop), because I’d like to offer a KF5 version of the QML/JS plugin. The reason is twofold: Qt5 offers the wonderful QtQuick.Controls module (it will allow me to implement many navigation widget, that will assist the users in choosing a font family, configuring and previewing animations, etc), and early KF5/Plasma5 adopters may be well pleased to be able to use the latest and greatest KDevelop on their latest and greatest environment, while still being able to work on QML and Javascript files.

UPDATE: Here is an additional screenshot that shows that child_process.ChildProcess is correctly recognized, and that it correctly inherits from event.EventEmitter (you see that addListener is in the list), which in turn inherits from Object (there is an “assign” method at the very bottom of the list). Having all of this working was a bit more difficult than expected, but now function prototypes and class inheritance seem to work.

Categories: FLOSS Project Planets

Mediacurrent: Code Review for Non-Technical Folks

Planet Drupal - Thu, 2014-07-24 17:07

Andrew Riley leads us through a high level walkthrough of what Code Reviews are and why we need them. In this talk he covers what we check for at Mediacurrent (syntax, security, efficiency etc) and why code reviews are important for our customers and any company that writes their own code.

Additional Resources

Drupal 8 In Pictures (for Users) | Mediacurrent Blog

Categories: FLOSS Project Planets

A wild "monday" report appears …

Planet KDE - Thu, 2014-07-24 15:32
Arguably our insanely-coolest member, Philipp Stefan contributed this "monday" report.Yes, it has been some time since the last entry, but with the Plasma 5 release and Jens’ busy work schedule there hasn’t been much time for him to write one, unfortunately. I hope I can keep you up to date more regularly from now on. So let’s dive right in:

What’s being worked onThe work on revisiting and expanding the Human Interface Guideline on tooltips has begun. If there’s something that has always bothered you about how tooltips in KDE Applications and Plasma look and feel consider to join in. The work is still in its early stages, so now would be the best time to voice your concerns. [https://forum.kde.org/viewtopic.php?f=285&t=121892]

Martin Gräßlin has started to work on a new window decoration API for KWin and is also working to port the Breeze window decoration to C++. This should bring the necessary performance boost to make Breeze the default window theme in future releases. We can also expect one feature many love in Oxygen to be present in the new Breeze implementation: window tabs. The awesome community member EraX has stepped up and made some mockups of how window tabs could look like in Breeze.

If you have other ideas and feedback, consider to chime in!Additionally Martin has created some tasks you can take up to help us out porting Breeze to C++. Take a look at Martin’s blog post for more information [http://blog.martin-graesslin.com/blog/2014/07/kdecoration2-the-road-ahead/]

Behind the scenes Jens Reuteberg and Heiko Tietze has started to collect information for animation guidelines. If you have an expertise in this area or are just generally interested and want to help us out try to contact them. [https://forum.kde.org/viewtopic.php?f=285&t=121993]

We migrated to KDE’s new todo-board infrastructure, todo.kde.org. It’s really nice having everything consolidated in one place and accessible with one’s KDE-Identity account. In future we also hope to migrate away from Google Docs and Hangouts. We have been looking into something like an OwnCloud installation to create, share and store documents. Nothing is written in stone though and frankly we’ve got more important things to do right now.  

What has been overlookedWe have noticed that the participation rate in the VDG forums has plummeted and only a small core has remained, which are doing awesome work! It has to be expected that participation rates decline at the end of every release cycle, when all the work gets finalized there’s not much to be discussed after all. Nethertheless, we’d like to recreate the same experience we had with the KMyMoney, the window decoration, the widget and the cursor theme.We identified several causes that could have contributed to the decline in participation:
  • Back then the VDG forum was new – people simply got disinterested over time
  • The VDG hasn’t communicated as much as in the beginning – the process became less transparent
  • There has not been as much guidance in where to start helping out as in the beginning
We try to address these issue better from now on. We have reorganized ourselves a bit, so you can expect more blog entries from more VDG members in future. We also intend to open new “challenges” (not in the sense of a contest) at the beginning of each release cycle where the community can participate in.Please tell us if you have ideas how we can foster a more vibrant design community in KDE, critique happily appreciated. We also have a thread in the forums about a related issue. If you feel the same way consider to leave a comment.
Overall I can only encourage you to browse the VDG forum. There are many interesting threads that could need some community love.
Categories: FLOSS Project Planets

Forum One: Drupal 8 Code Sprint at the Jersey Shore

Planet Drupal - Thu, 2014-07-24 14:52

On the heels of our own Drupal 8 code sprint in DC, I spent the last weekend with the New Jersey Drupal group who organized a Drupal 8 code sprint in Asbury Park, NJ – and although I was never greeted by The Boss, I was pleased to participate thanks to the generosity of the event organizers.

Issue-Focused

I worked on the MenuLink NG part 1 issue extensively with Peter Wolanin and YesCT. This issue is part of the [Meta] New plan, Phase 2 issue which proposed performance improvement and UI improvement in Drupal 8. This issue originally had a 600KB patch, but to make it more reviewable/committable the issue was split into five child issues.

Three of us spent a solid two days and more than 30 hours addressing every single point that had been raised by reviewers – and which had been holding up the process of adding this to Core.

Image courtesy of Blink Reaction

About this Issue (a high level overview)

A site builder or developer can create menu links in Drupal via configuration by changing the weight, position, creating links in menu, or in code. All these different types of menu links need to be rendered together in menus, so that they present the same in the API to developers. The developer experience on this issue needs to be easy, as almost everything depends on menu items.

While we toiled on this issue, other sprinters worked on DrupalCon Austin’s Consensus Banana, testing the migration path from Drupal 6 to Drupal 7, along with some other issues.

Results and Commits

The sprint was a very productive one, and resulted in Menu part 1 and Menu part 2 being committed to Core, which was a beta-blocker issue. As of this sprint there were seven beta-blocker issues, but solving the Menu issue helps to put us one step closer to the Drupal 8 Beta release!

For those interested, here are the commits for part 1 and part 2. And for those needing a chuckle, Alex “The Situation” Pott – who thankfully preferred DCC (Drupal Core Commits) over GTL (Gym, Tan, Laundry) – drew this goaty looking lama to celebrate his commit

Image courtesy of Blink Reaction

It was very rewarding to work for a weekend with this group of talented developers, and I think we all left the shore content in the knowledge that we’d made strides toward bringing Drupal 8 that much closer to completion.

Check out these other pictures taken by Peter Wolanin and DrupalCamp NJ at the event »

Categories: FLOSS Project Planets

Kate “master” branch now KF5 based!

Planet KDE - Thu, 2014-07-24 14:18

Hi,

from today on, the master branch of kate.git is KF5 based.

That means, for the next KDE applications release after 4.14, Kate will use the awesome KF5 stuff!

The KTextEditor framework is already in a good shape and most active KatePart development is since months pure KF5 based.

The same should now be true for Kate itself (and KWrite).

Pâté will need a lot of love, as PyQt5 is there, but PyKDE5/PyKF5 still in the works.

Happy hacking, help us to make KF5 based Kate awesome, too.

For 4.14: keep fixing bugs, but all new shiny stuff that might regress things really should only go into our KF5 based master branch!

Categories: FLOSS Project Planets

Data Community DC: Building Data Apps with Python on August 23rd

Planet Python - Thu, 2014-07-24 14:16

Data Community DC and District Data Labs are excited to be hosting another Building Data Apps with Python workshop on August 23rd.  For more info and to sign up, go to http://bit.ly/V4used.  There’s even an early bird discount if you register before the end of this month!

Overview

Data products are usually software applications that derive their value from data by leveraging the data science pipeline and generate data through their operation. They aren’t apps with data, nor are they one time analyses that produce insights – they are operational and interactive. The rise of these types of applications has directly contributed to the rise of the data scientist and the idea that data scientists are professionals “who are better at statistics than any software engineer and better at software engineering than any statistician.”

These applications have been largely built with Python. Python is flexible enough to develop extremely quickly on many different types of servers and has a rich tradition in web applications. Python contributes to every stage of the data science pipeline including real time ingestion and the production of APIs, and it is powerful enough to perform machine learning computations. In this class we’ll produce a data product with Python, leveraging every stage of the data science pipeline to produce a book recommender.

What You Will Learn

Python is one of the most popular programming languages for data analysis.  Therefore, it is important to have a basic working knowledge of the language in order to access more complex topics in data science and natural language processing.  The purpose of this one-day course is to introduce the development process in Python using a project-based, hands-on approach. In particular you will learn how to structure a data product using every stage of the data science pipeline including ingesting data from the web, wrangling data into a structured database, computing a non-negative matrix factorization with Python, then producing a web based report.

Course Outline

 The workshop will cover the following topics:

  • Basic project structure of a Python application

  • virtualenv & virtualenvwrapper

  • Managing requirements outside the stdlib

  • Creating a testing framework with nose

  • Ingesting data with requests.py

  • Wrangling data into SQLite Databases using SQLAlchemy

  • Building a recommender system with Python

  • Computing a matrix factorization with Numpy

  • Storing computational models using pickles

  • Reporting data with JSON

  • Data visualization with Jinja2

After this course you should understand how to build a data product using Python and will have built a recommender system that implements the entire data science pipeline.

Instructor: Benjamin Bengfort

Benjamin is an experienced Data Scientist and Python developer who has worked in military, industry, and academia for the past eight years. He is currently pursuing his PhD in Computer Science at The University of Maryland, College Park, doing research in Metacognition and Active Logic. He is also a Data Scientist at Cobrain Company in Bethesda, MD where he builds data products including recommender systems and classifier models. He holds a Masters degree from North Dakota State University where he taught undergraduate Computer Science courses. He is also adjunct faculty at Georgetown University where he teaches Data Science and Analytics.

The post Building Data Apps with Python on August 23rd appeared first on Data Community DC.

Categories: FLOSS Project Planets

Sagara Gunathunga: WebSocket security patterns

Planet Apache - Thu, 2014-07-24 14:02
WebSocket protocol introduced "wss" prefix to define secure web socket connections that is for transport level security (TLS).  But it does not define any authentication or Authorization  mechanism, instead it is possible to reuse existing HTTP based authentication/Authorization mechanisms during the handshake phase.  
Here I discuss two security patterns that can be used to connect secure WebSocket endpoint from a client. Assume WebSocket endpoint is secured with HTTP BasicAuth while HTTP/SSL used for transport level security.  

1. ) Browser based clients 
For web browser based clients most popular choice is to use JavaScript API for WebSocket but this API does not provide any approach to send "Authorization" or any other headers along with the handshake request. Following pattern can be used  to overcome above issue. 


The technique we use here is secure the web page where we run JS WebSocket client through BasicAuth security. Please refer the message flow. 

1. User access the secure page through a web browser through HTTPS. 
2. Since the page is secured web server return 401 status code.
3. Browser challenges user to enter valid user name and password then send them as a encoded value with  "Authorization" header. 
4. If credentials are correct server returns secure page. 
5. JS WebSocket client on secured page send handshake request to the secured remote WebSocket endpoint through WSS protocol. Due to previous interaction with the same page browser persist and send authorization details along with the handshake request. 

6. Since handshake request transmit through HTTPS it fulfil the both requirements, BasicAuth and TLS. Server returns handshake response back to the client.   
7. Now it's possible to establish WebSocket connection among above two parties. 



2.) Agent based client (Non browser based)
For agent based clients you could use a WebSocket framework which facilitate  to add authentication headers and also to configure SSL configuration such as key store.  Following diagram illustrate a pattern which we be can used with agents based clients. 


1. Using the client side API of the WebScoket framework create a handshake request. 
2. Set Authorization header and other required key store information for TLS. 
3. Send  handshake request through WebSocket framework API. 
4. Server receive handshake and Authorization header  through HTTPS . Validate the header and if valid  send the handshake response back to the client. 
5. Now it's possible to establish WebSocket connection among above two parties. 
As an example Java API for WebSocket  allows to send custom headers  along with handshake request by writing a custom Configurator which extend from  ClientEndpointConfig.Configurator
Here is such example.

public class ClientEndpointConfigurator extends
ClientEndpointConfig.Configurator {
@Override
public void beforeRequest(Map<String, List<String>> headers) {
String auth = "Basic " +
Base64Utils.encodeToString("user:pass".getBytes(Charset.forName("UTF-8")), false);
headers.put("Authorization", Arrays.asList(new String[]{auth }));
super.beforeRequest(headers);
}
}


Once you wrote this ClientEndpointConfigurator you can refer it from Client endpoint using  'ClientEndpoint' annotation as follows. 
 @ClientEndpoint(configurator=ClientEndpointConfigurator.class)
public class EchoClientEndpoint { ........................ }



By the way there is no portable API to define SSL configurations required for TLS but some framework such as  Tyrus provides proprietary APIs . As an example refer how this facilitated in Tyrus through ClientManager API.




Categories: FLOSS Project Planets

Let’s party!

Planet KDE - Thu, 2014-07-24 13:10

Yesterday, we released ownCloud 7. You might have read that somewhere on the internet – it was widely announced and broadly picked up. If you do not have ownCloud yet, you really should try it now, and if you are one of the people happily using ownCloud for a while, update soon!

In my opinion, ownCloud 7 is a real step ahead. So much work went into it from the brave server guys, and the result is simply impressive: It is fast, beautiful and fully focused on what it is supposed to be and do: File sync and share, even though you still have of course all the other useful stuff like calendaring and such.

Apart from the wellknown features ownCloud 7 also brings some things that you do not see on the first look. Some of these things relate to syncing and as a guy working mainly on the sync client I am especially happy about that: Already with the current released clients you will see a performance win in syncing, because ownCloud 7 handles parallel HTTP requests way better. Moreover, ownCloud 7 sends some interesting information about it’s files, and the next generation of sync clients will for example know which files are shared and their permissions. We are currently working on some exciting stuff on the client side, stay tuned.

That all should be reason enough to celebrate together, because ownCloud is done by all of us in the community.

On Tuesday, august 5th, we will do a Release Party in Nuremberg, Germany. As last time, the Coworkingspace Nürnberg is hosting us. We will start at 18:00.

Everybody who is interested in ownCloud for whatever reason is very welcome to show up. We will probably have a short informal demonstration or two, but of course plenty room for questions, discussions, meet and greet with all kind of FOSS people, some core ownCloud people, nice beer and big fun.

You really should come! And don’t forget to tell your friends, thanks!


Categories: FLOSS Project Planets

Logilab: EP14 Pylint sprint Day 1 report

Planet Python - Thu, 2014-07-24 10:39

We've had a fairly enjoyable and productive first day in our little hidden room at EuroPython in Berlin ! Below are some noticeable things we've worked on and discussed about.

First, we discussed and agreed that while we should at some point cut the cord to the logilab.common package, it will take some time notably because of the usage logilab.common.configuration which would be somewhat costly to replace (and is working pretty well). There are some small steps we should do but basically we should mostly get back some pylint/astroid specific things from logilab.common to astroid or pylint. This should be partly done during the sprint, and remaining work will go to tickets in the tracker.

We also discussed about release management. The point is that we should release more often, so every pylint maintainers should be able to do that easily. Sylvain will write some document about the release procedure and ensure access are granted to the pylint and astroid projects on pypi. We shall release pylint 1.3 / astroid 1.2 soon, and those releases branches will be the last one supporting python < 2.7.

During this first day, we also had the opportunity to meet Carl Crowder, the guy behind http://landscape.io, as well as David Halter which is building the Jedi completion library (https://github.com/davidhalter/jedi). Landscape.io runs pylint on thousands of projects, and it would be nice if we could test beta release on some part of this panel. On the other hand, there are probably many code to share with the Jedi library like the parser and ast generation, as well as a static inference engine. That deserves a sprint on his own though, so we agreed that a nice first step would be to build a common library for import resolution without relying on the python interpreter for that, while handling most of the python dark import features like zip/egg import, .pth files and so one. Indeed that may be two nice future collaborations!

Last but not least, we got some actual work done:

  • Michal Novikowsky from Intel in Poland joined us to work on the ability to run pylint in different processes so it may drastically improve performance on multiple cores box.
  • Torsten did continue some work on various improvements of the functionnal test framework.
  • Sylvain did merge logilab.common.modutils module into astroid as it's mostly driven by astroid and pylint needs. Also fixed the annoying namespace package crash.
  • Claudiu keep up the good work he does daily at improving and fixing pylint :)
Categories: FLOSS Project Planets

Dries Buytaert: The business behind Open Source

Planet Drupal - Thu, 2014-07-24 10:38
Topic: DrupalAcquiaBusinessThe future

A few days ago, I sat down with Quentin Hardy of The New York Times to talk Open Source. We spoke mostly about the Drupal ecosystem and how Acquia makes money. As someone who spent almost his entire career in Open Source, I'm a firm believer in the fact that you can build a high-growth, high-margin business and help the community flourish. It's not an either-or proposition, and Acquia and Drupal are proof of that.

Rather than an utopian alternate reality as Quentin outlines, I believe Open Source is both a better way to build software, and a good foundation for an ecosystem of for-profit companies. Open Source software itself is very successful, and is capable of running some of the most complex enterprise systems. But failure to commercialize Open Source doesn't necessarily make it bad.

I mentioned to Quentin that I thought Open Source was Darwinian; a proprietary software company can't afford to experiment with creating 10 different implementations of an online photo album, only to pick the best one. In Open Source we can, and do. We often have competing implementations and eventually the best implementation(s) will win. One could say that Open Source is a more "wasteful" way of software development. In a pure capitalist read of On the Origin of Species, there is only one winner, but business and Darwin's theory itself is far more complex. Beyond "only the strongest survive", Darwin tells a story of interconnectedness, or the way an ecosystem can dictate how an entire species chooses to adapt.

While it's true that the Open Source "business model" has produced few large businesses (Red Hat being one notable example), we're also evolving the different Open Source business models. In the case of Acquia, we're selling a number of "as-a-service" products for Drupal, which is vastly different than just selling support like the first generation of Open Source companies did.

As a private company, Acquia doesn't disclose financial information, but I can say that we've been very busy operating a high-growth business. Acquia is North America's fastest growing private company on the Deloitte Fast 500 list. Our Q1 2014 bookings increased 55 percent year-over-year, and the majority of that is recurring subscription revenue. We've experienced 21 consecutive quarters of revenue growth, with no signs of slowing down. Acquia's business model has been both disruptive and transformative in our industry. Other Open Source companies like Hortonworks, Cloudera and MongoDB seem to be building thriving businesses too.

Society is undergoing tremendous change right now -- the sharing and collaboration practices of the internet are extending to transportation (Uber), hotels (Airbnb), financing (Kickstarter, LendingClub) and music services (Spotify). The rise of the collaborative economy, of which the Open Source community is a part of, should be a powerful message for the business community. It are the established, proprietary vendors whose business models are at risk, and not the other way around.

Hundreds of other companies, including several venture backed startups, have been born out of the Drupal community. Like Acquia, they have grown their businesses while supporting the ecosystem from which they came. That is more than a feel-good story, it's just good business.

Categories: FLOSS Project Planets

No Gmail integration in 4.14 after all :(

Planet KDE - Thu, 2014-07-24 10:32

Hi folks,

I’m sorry to bring bad news, but after trying to fight some last minute bugs in the new Gmail resource today, I realized that pushing the resource into KDE Applications 4.14 was too hurried, and so I decided not to ship it in KDE Applications 4.14. I know many of you are really excited about the Gmail integration, but there are far too many issues that cannot be solved this late in 4.14 cycle. And since this will probably be the last 4.x release, shipping something that does not perform as expected and cannot be fixed properly would only be disappointing and discouraging to users. In my original post I explained that I was working on the Gmail integration to provide user experience as close as possible to native Gmail web interface so that people are not tempted to switch away from KMail to Gmail. But with the current state of the resource, the effect would be exactly the opposite. And if the resource cannot fulfil it’s purpose, then there’s no point in offering it to users.

Instead I will focus on implementing the new native Gmail API and merging together the existing Google resources to create a single groupware solution that will provide integration will all Google’s PIM services – contacts, calendars, tasks and emails.

 

Categories: FLOSS Project Planets

drunken monkey: Updating the Search API to D8 – Part 3: Creating your own service

Planet Drupal - Thu, 2014-07-24 08:28

Even though there was somewhat of a delay since my last post in this series, it seems no-one else has really covered any of the advanced use cases of Drupal 8 in tutorials yet. So, here is the next installment in my series. I initially wanted to cover creating a new plugin type, but since that already requires creating a new servive, I thought I'd cover that smaller part first and then move on to plugin types in the next post.
I realize that now already a lot more people have started on their Drupal 8 modules, but perhaps this will make this series all the more useful.

Services in Drupal 8

First, a short overview of what a service even is. Basically it is a component (represented as a class) providing a certain, limited range of functionality. The database is a service, the entity manager (which is what you now use for loading entities) is a service, translation, configuration – everything handled by services. Getting the current user – also a service now, ridding us of the highly unclean global variable.
In general, a lot of what was previously a file in includes/ containing some functions with a common prefix is now a service (or split into multiple services).

The upsides of this is that the implementation and logic is cleanly bundled and properly encapsulated, that all these components can easily be switched out by contrib or later core updates, and that these systems can also be very well tested with unit tests. Even more, since services can be used with dependency injection, it also makes it much easier to test all other classes that use any of these services (if they can use dependency injection and do it properly).

(For reference, here is the official documentation on services.)

Dependency injection

This has been covered already in a lot of other blog posts, probably since it is both a rather central concept in Drupal 8, and a bit complicated when you first encounter it. However, before using it, I should still at least skim over the topic. Feel free to skip to the next heading if you feel you already know what dependency injection is and how it roughly works in Drupal 8.

Dependency injection is a programming technique where a class with external dependencies (e.g., a mechanism for translating) explicitly defines these dependencies (in some form) and makes the class which constructs it responsible for supplying those dependencies. That way, the class itself can be self-contained and doesn't need to know about where it can get those dependencies, or use any global functions or anything to achieve that.

Consider for example the following class:

<?php
class ExampleClass {

  public function getDefinition() {
    return array(
      'label' => t('example class'),
      'type' => 'foo',
    );
  }

}
?>

For translating the definition label, this explicitly uses the global t() function. Now, what's bad about this I, hear you ask, it worked well enough in Drupal 7, right?
The problem is that it becomes almost impossible to properly unit-test that method without bootstrapping Drupal to the point where the t() function becomes available and functional. It's also more or less impossible to switch out Drupal's translation mechanism without hacking core, since there is no way to redirect the call to t().

But if translation is done by a class with a defined interface (in other words, a service), it 's possible to do this much cleaner:

<?php
class ExampleClass {

  public function __construct(TranslationServiceInterface $translation) {
    $this->translation = $translation;
  }

  public function getDefinition() {
    return array(
      'label' => $this->translation->translate('example class'),
      'type' => 'foo',
    );
  }

}
?>

Then our example class just has to make it easily possible for code that wants to instantiate it to know how to pass its dependencies to it. In Drupal, there are two ways to do this, depending on what you are creating:

  • Services, which themselves use dependency injection to get their dependencies (as you will see in a minute) have a definition in a YAML file that exactly states which services need to be passed to the service's constructor.
  • Almost anything else (I think) uses a static create() method which just receives a container of all available services and is then responsible for passing the correct ones to the constructor.

In either case, the idea is that subclasses/replacements of ExampleClass can easily use other dependencies without any changes being necessary to code elsewhere instantiating the class.

Creating a custom service

So, when would you want to create your own service in a module? Generally, the .module file should more or less only contain hook implementations now, any general helper functions for the module should live in classes (so they can be easily grouped by functionality, and the code can be lazy-loaded when needed). The decision to make that class into a service then depends on the following questions:

  • Is there any possibility someone would want to swap out the implementation of the class?
  • Do you want to unit-test the class?
  • Relatedly, do you want dependency injection in the class?

I'm not completely sure myself about how to make these decisions, though. We're still thinking about what should and shouldn't be a service in the Search API, currently there is (apart from the ones for plugins) only one service there:

The "Server task manager" service

The "server tasks" system, which already existed in D7, basically just ensures that when any operations on a server (e.g., removing or adding an index, deleting items, …) fails for some reason (e.g., Solr is temporarily unreachable) it is regularly retried to always ensure a consistent server state. While in D7 the system consisted of just a few functions, in D8 it was decided to encapsulate the functionality in a dedicated service, the "Server task manager".

Defining an interface and a class for the service

The first thing you need, so the service can be properly swapped out later, is an interface specifying exactly what the service should be able to do. This completely depends on your use case for the service, nothing to keep in mind here (and also no special namespace or anything). In our case, for server tasks:

<?php
namespace Drupal\search_api\Task;

interface ServerTaskManagerInterface {

  public function execute(ServerInterface $server = NULL);

  public function add(ServerInterface $server, $type, IndexInterface $index = NULL, $data = NULL);

  public function delete(array $ids = NULL, ServerInterface $server = NULL, $index = NULL);

}
?>

(Of course, proper PhpDocs are essential here, I just skipped them for brevity's sake.)

Then, just create a class implementing the interface. Again, namespace and everything else is completely up to you. In the Search API, we opted to put interface and class (they usually should be in the same namespace) into the namespace \Drupal\search_api\Task. See here for their complete code.
For this post, the only relevant part of the class code is the constructor (the rest just implements the interface's methods):

<?php
class ServerTaskManager implements ServerTaskManagerInterface {

  public function __construct(Connection $database, EntityManagerInterface $entity_manager) {
    $this->database = $database;
    $this->entity_manager = $entity_manager;
  }

}
?>

As you can see, we require the database connection and the entity manager as dependencies, and just included them in the constructor. We then save them to properties to be able to use them later in the other methods.

Now we just need to tell Drupal about our service and its dependencies.

The services.yml file

As mentioned earlier, services need a YAML definition to work, where they also specify their dependencies. For this, each module can have a MODULE.services.yml file listing services it wants to publish.

In our case, search_api.services.yml looks like this (with the plugin services removed):

services:
  search_api.server_task_manager:
    class: Drupal\search_api\Task\ServerTaskManager
    arguments: ['@database', '@entity.manager']

As you see, it's pretty simple: we assign some ID for the service (search_api.server_task_manager – properly namespaced by having the module name as the first part), specify which class the service uses by default (which, like the other definition keys, can then be altered by other modules) and specify the arguments for its constructor (i.e., its dependencies). database and entity.manager in this example are just IDs of other services defined elsewhere (in Drupal core's core.services.yml, in this case).

There are more definition keys available here, and also more features that services support, but that's more or less the gist of it. Once you have its definition in the MODULE.services.yml file, you are ready to use your new service.

Using a service

You already know one way of using a service: you can specify it as an argument for another service (or any other dependency injection-enabled component). But what if you want to use it in a hook, or any other place where dependency injection is not available (like entities, annoyingly)?

You simply do this:

<?php
/** @var \Drupal\search_api\Task\ServerTaskManagerInterface $server_task_manager */
$server_task_manager = \Drupal::service('search_api.server_task_manager');
$server_task_manager->execute();
?>

That's it, now all our code needing server tasks functionality benefits from dependency injection and all the other Drupal 8 service goodness.

Categories: FLOSS Project Planets

Craig Small: PHP uniqid() not always a unique ID

Planet Debian - Thu, 2014-07-24 08:17

For quite some time modern versions of JFFNMS have had a problem. In large installations hosts would randomly appear as down with the reachability interface going red. All other interface types worked, just this one.

Reachability interfaces are odd, because they call fping or fping6 do to the work. The reason is because to run a ping program you need to have root access to a socket and to do that is far too difficult and scary in PHP which is what JFFNMS is written in.

To capture the output of fping, the program is executed and the output captured to a temporary file. For my tiny setup this worked fine, for a lot of small setups this was also fine. For larger setups, it was not fine at all. Random failed interfaces and, most bizzarely of all, even though a file disappearing. The program checked for a file to exist and then ran stat in a loop to see if data was there. The file exist check worked but the stat said file not found.

At first I thought it was some odd load related problem, perhaps the filesystem not being happy and having a file there but not really there. That was, until someone said “Are these numbers supposed to be the same?”

The numbers he was referring to was the filename id of the temporary file. They were most DEFINITELY not supposed to be the same. They were supposed to be unique. Why were they always unique for me and not for large setups?

The problem is with the uniqid() function. It is basically a hex representation of the time.  Large setups often have large numbers of child processes for polling devices. As the number of poller children increases, the chance that two child processes start the reachability poll at the same time and have the same uniqid increases. It’s why the problem happened, but not all the time.

The stat error was another symptom of this bug, what would happen was:

  • Child 1 starts the poll, temp filename abc123
  • Child 2 starts the poll in the same microsecond, temp filename is also abc123
  • Child 1 and 2 wait poller starts, sees that the temp file exists and goes into a loop of stat and wait until there is a result
  • Child 1 finishes, grabs the details, deletes the temporary file
  • Child 2 loops, tries to run stat but finds no file

Who finishes first is entirely dependent on how quickly the fping returns and that is dependent on how quicky the remote host responds to pings, so its kind of random.

A minor patch to use tempnam() instead of uniqid() and adding the interface ID in the mix for good measure (no two children will poll the same interface, the parent’s scheduler makes sure of that.) The initial responses is that it is looking good.

 

Categories: FLOSS Project Planets

Andrew Dalke: Lausanne Cheminformatics workshop and contest

Planet Python - Thu, 2014-07-24 08:00

Dietrich Rordorf from MDPI sent an announcement to the CHMINF mailing list about the upcoming 9th Workshop in Chemical Information. It will be on 12 September 2014 in Lausanne, Switzerland. It seems like it will be a nice meeting, so I thought to forward information about it here. They also have a software contest, with a 2,000 CHF prize, which I think will interest some of my readers.

The workshop has been around for 10 years, so I was a bit suprised that I hadn't heard of it before. Typically between 30 and 50 people attend, which I think is a nice size. The preliminary program is structured around 20 minute presentations, including:

  • Peter Ertl - Database of bioactive ring systems with calculated properties and its use in bioisosteric design and scaffold hopping
  • Michaël Zasso - Slice based electronic laboratory notebook
  • Dragos Horvath - Chemigenomics - is it more than inductive transfer?
  • Guillaume Godin - GC/MS identification in a Browser
  • Modest Korff - Sub-pharmacophore models as seeds in drug discovery
  • Jean-Louis Reymond - Interactive tools for visualisation and virtual screening of large compound databases
  • Michael J E Sternberg - INDDEx - logic-based ligand screening for scaffold hopping
  • Thomas Sander - 2D scaling with rubber bands and descriptors
If you know the authors, you might recognize that one is from Strasbourg and another London, and the rest from Switzerland. I can understand. From where I live in Sweden it will cost over US $300 in order to get there, and Lausanne doesn't have its own commercial airport so I would need to fly into Geneva or Bern, while my local air hub doesn't fly there directly.

But I live in a corner of Europe, and my constraints aren't yours.

Source code contest

I had an email conversation with Luc Patiny about an interesting aspect of the workshop. They are running a contest to identify the best open source cheminformatics tool of the year, with a prize of 2000 CHF. That's 1650 EUR, 2200 USD, 1300 GBP, or 15000 SEK, which is plenty enough for someone in Europe or even the US to be able to travel there! They have a time slot set aside for the winner of the contest to present the work. The main requirement is that contest participants are selected from submissions (1-2 pages long) to the open access journal Challenges. (And no, there are no journal page fees for this contest, so it doesn't seem like a sneaky revenue generating trick.)

The other requirement is that the submission be "open source". I put that in quotes because much of my conversation with Luc was to understand what they mean. They want people to be able to download the (unobsfucated) software source code for no cost and be able to read and review it to gain insight.

I think this is very much in line with classical peer review thought, even though it can include software which are neither open source nor free software. For example, software submissions for this contest could be "for research purposes only" or "not for use in nuclear reactors", or "redistributions of modified versions are not permitted."

Instead, I think their definition is more in line with Microsoft terms shared source.

In my case, my latest version of chemfp is 'commercial open source', meaning that those who pay me money for it get a copy of it under the MIT open source license. It's both "free software" and "open source", but it's not eligible for this workshop because it costs money to download it.

But I live in a corner of open source, and my constraints aren't yours. ;) If you have a free software project, open source software project, or shared source software project, then you might be interested in submitting it to this workshop and journal. If you win, think of it as an all-expenses paid trip to Switzerland. If you don't win, think of it as a free publication.

Categories: FLOSS Project Planets

Martin Pitt: vim config for Markdown+LaTeX pandoc editing

Planet Debian - Thu, 2014-07-24 05:38

I have used LaTeX and latex-beamer for pretty much my entire life of document and presentation production, i. e. since about my 9th school grade. I’ve always found the LaTeX syntax a bit clumsy, but with good enough editor shortcuts to insert e. g. \begin{itemize} \item...\end{itemize} with just two keystrokes, it has been good enough for me.

A few months ago a friend of mine pointed out pandoc to me, which is just simply awesome. It can convert between a million document formats, but most importantly take Markdown and spit out LaTeX, or directly PDF (through an intermediate step of building a LaTeX document and calling pdftex). It also has a template for beamer. Documents now look soo much more readable and are easier to write! And you can always directly write LaTeX commands without any fuss, so that you can use markdown for the structure/headings/enumerations/etc., and LaTeX for formulax, XYTex and the other goodies. That’s how it should always should have been! ☺

So last night I finally sat down and created a vim config for it:

"-- pandoc Markdown+LaTeX ------------------------------------------- function s:MDSettings() inoremap <buffer> <Leader>n \note[item]{}<Esc>i noremap <buffer> <Leader>b :! pandoc -t beamer % -o %<.pdf<CR><CR> noremap <buffer> <Leader>l :! pandoc -t latex % -o %<.pdf<CR> noremap <buffer> <Leader>v :! evince %<.pdf 2>&1 >/dev/null &<CR><CR> " adjust syntax highlighting for LaTeX parts " inline formulas: syntax region Statement oneline matchgroup=Delimiter start="\$" end="\$" " environments: syntax region Statement matchgroup=Delimiter start="\\begin{.*}" end="\\end{.*}" contains=Statement " commands: syntax region Statement matchgroup=Delimiter start="{" end="}" contains=Statement endfunction autocmd BufRead,BufNewFile *.md setfiletype markdown autocmd FileType markdown :call <SID>MDSettings()

That gives me “good enough” (with some quirks) highlighting without trying to interpret TeX stuff as Markdown, and shortcuts for calling pandoc and evince. Improvements appreciated!

Categories: FLOSS Project Planets
Syndicate content