FLOSS Project Planets

Dries Buytaert: Mike Sullivan joins Acquia as CEO

Planet Drupal - Mon, 2017-11-13 10:59

Today, I am excited to announce that Michael Sullivan will be joining Acquia as its CEO.

The search for a new CEO

Last spring, Tom Erickson announced that he was stepping down as Acquia's CEO. For over eight years, Tom and I have been working side-by-side to build and run Acquia. I've been lucky to have Tom as my partner as he is one of the most talented leaders I know. When Tom announced he'd be stepping down as Acquia's CEO, finding a new CEO became my top priority for Acquia. For six months, the search consumed a good deal of my time. I was supported by a search committee drawn from Acquia's board of directors, including Rich D'Amore, Tom Bogan, and Michael Skok. Together, we screened over 140 candidates and interviewed 10 in-depth. Finding the right candidate was hard work and time consuming, but we kept the bar high at all times. As much as I enjoyed meeting so many great candidates and hearing their perspective on our business, I'm glad that the search is finally behind me.

The right fit for Acquia

Finding a business partner is like dating; you have to get to know each other, build trust, and see if there is a match. Identifying and recruiting the best candidate is difficult because unlike dating, you have to consider how the partnership will also impact your team, customers, partners, and community. Once I got to know Mike, it didn't take me long to realize how he could help scale Acquia and help make our customers and partners successful. I also realized how much I would enjoy working with him. The fit felt right.

With 25 years of senior leadership in SaaS, enterprise content management and content governance, Mike is well prepared to lead our business. Mike will join Acquia from Micro Focus, where he participated in the merger of Micro Focus with Hewlett Packard Enterprise's software business. The combined company became the world's seventh largest pure-play software company and the largest UK technology firm listed on the London Stock Exchange. At Micro Focus and Hewlett Packard Enterprise, Mike was the Senior Vice President and General Manager for Software-as-a-Service and was responsible for managing over 30 SaaS products.

This summer, I shared that Acquia expanded its focus from website management to data-driven customer journeys. We extended the capabilities of the Acquia Platform with journey orchestration, commerce integrations and digital asset management tools. The fact that Mike has so much experience running a diverse portfolio of SaaS products is something I really valued. Mike's expertise can guide us in our transformation from a single product company to a multi-product company.

Creating a partnership

For many years, I have woken up everyday determined to set a vision for the future, formulate a strategy to achieve that vision, and help my fellow Acquians figure out how to achieve that vision.

One of the most important things in finding a partner and CEO for Acquia was having a shared vision for the future and an understanding of the importance of cloud, Open Source, data-driven experiences, customer success and more. This was very important to me as I could not imagine working with a partner who isn't passionate about these same things. It is clear that Mike shares this vision and is excited about Acquia's future.

Furthermore, Mike's operational strength and enterprise experience will be a natural complement to my focus on vision and product strategy. His expertise will allow Acquia to accelerate its mission to "build the universal platform for the world's greatest digital experiences."

Formalizing my own role

In addition to Mike joining Acquia as CEO, my role will be elevated to Chairman. I will also continue in my position as Acquia CTO. My role has always extended beyond what is traditionally expected of a CTO; my responsibilities have bridged products and engineering, fundraising, investor relations, sales and marketing, resource allocation, and more. Serving as Chairman will formalize the various responsibilities I've taken on over the past decade. I'm also excited to work with Mike because it is an opportunity for me to learn from him and grow as a leader.

Acquia's next decade

The web has the power to change lives, educate the masses, create new economies, disrupt business models and make the world smaller in the best of ways. Digital will continue to change every industry, every company and every life on the planet. The next decade holds enormous promise for Acquia and Drupal because of what the power of digital holds for business and society at large. We are uniquely positioned to deliver the benefits of open source, cloud and data-driven experiences to help organizations succeed in an increasingly complex digital world.

I'm excited to welcome Mike to Acquia as its CEO because I believe he is the right fit for Acquia, has the experience it takes to be our CEO and will be a great business partner to bring Acquia's vision to life. Welcome to the team, Mike!

Categories: FLOSS Project Planets

Doug Hellmann: readline — The GNU readline Library — PyMOTW 3

Planet Python - Mon, 2017-11-13 09:00
The readline module can be used to enhance interactive command line programs to make them easier to use. It is primarily used to provide command line text completion, or “tab completion”. Read more… This post is part of the Python Module of the Week series for Python 3. See PyMOTW.com for more articles from the …
Categories: FLOSS Project Planets

Mike Driscoll: PyDev of the Week: Bert JW Regeer

Planet Python - Mon, 2017-11-13 08:30

This week we welcome Bert JW Regeer as our PyDev of the Week! Bert is a core developer of the Pyramid web framework. You can check our his experience over on his website or go check out his Github profile to see what projects he’s been working on lately. Let’s take a few moments to get to know Bert better!

Can you tell us a little about yourself? (Hobbies, education, etc?):

Oh, I have no idea where to start, but let’s give it a shot. I am first and foremost a geek, I love electronics, which in and of itself is an expensive hobby. Always new toys to play with and buy. I studied Computer Science at the University of Advancing Technology, and have been known to spend a lot of time building cool hardware based projects too. Spent a lot of time on FIRST Robotics, first in HS and then mentoring HS students while in college. Lately the only hardware I get to play with though is home automation, installing new switches and sensors to make my laziness even more lazy!

My other major hobbies all have something in common, they are expensive ones: photography and cars. I am a bit of an amateur photographer and am always looking to get new lenses or find new ideas on how to shoot something new and exciting. My next goal is to do some astrophotography and I am looking to get a nice wide lens with a nice large aperture. I live in Colorado so there are plenty of gorgeous places to go photograph. I drive a Subaru WRX, and I absolutely love going for rides. Been eyeing some upgrades to my car, but so far she is still stock. I enjoy going out and driving around the mountains though, which goes hand in hand with the photography!

Last but not least, lately I have gotten into making my own sourdough bread. There is nothing better than a freshly baked sourdough bread with a little butter. It’s my newest and most recent hobby, and it is also the one that costs almost nothing. I get to make healthier bread, share it with friends and family, and it costs pennies! I work at a small company named Crunch and a few of my colleagues are bread bakers too, which allows us to share tips and ideas on how to improve our breads.

I really should update my website to include this, but on a different note, you can find some of my older projects there!

Why did you start using Python?

I was introduced to Python while I was at university, at the time (2008 time frame) I didn’t really think much of it other than just a quick way to prototype some ideas before translating them into C/C++. At the time I was a pretty big proponent of C++, and working mostly in the backend it was a pretty natural fit. Write your applications in C++ and run on FreeBSD.

It wasn’t until I was at my first programming job where we had to quickly write a new server component that I started reaching for Python first and foremost to deliver the project. After quickly prototyping I was able to prove that Python would provide us with the speed and throughput required while alleviating some of the development concerns we were worried about. As time went on there were still components I wrote in C++, but a large part of our service ended up being written in Python due to the speed of development.

For personal use, I had always written my websites in PHP since it was always available and easy to use, but I never really did enjoy using any of the frameworks built for PHP. All of them were brand new at the time and felt incredibly heavy weight, and because I was using more Python at work it was getting confusing, when do I need to use tabs and when do I use curly braces? It always took me a minute or two to context switch from one programming language to another, so I started looking at Django, Flask, Pylons, Pyramid, Google App Engine and others. I ended up settling on Pyramid due to it’s simplicity and because it allowed me to pick the components I wanted to use. I ended becoming the maintainer for WebOb and recently have become a full core contributor to the Pylons Project.

What other programming languages do you know and which is your favorite?

This is going to be an incredibly long list, so let’s go with the ones I have extensively and not just used for a toy project here and there. As already mentioned I started out with C/C++, being the first language I learned from a C++ for Dummies book when I was 12, since then I’ve ran the gamut, but PHP was fairly big for me a for a while. In high school and university Java of course was used, although it still is my least favorite language, I have used it extensively on some Android projects. I’ve worked on a project that was Objective C, with a little bit of Swift, mainly doing a security audit so I am not sure it really counts as extensively… currently the two languages I use most are Python and JavaScript (or Typescript, transpiled to JavaScript). ES6/ES7 (yay for Babel) are heavily used in various projects.

Python however has definitely become my favorite programming language. It is incredibly versatile and even though it is not the fastest language by far, it is one of the most flexible and I can appreciate how easy it makes my life. Are there things I’d like to see change in Python? Absolutely. Are there pieces missing? Sure. At the same time I am not sure what other language I would enjoy working in as much as I currently do with Python. I’ve tried Golang, it’s just not for me. Rust comes pretty close, but it feels too much like C/C++ and requires a lot more thinking than I think is necessary for the things I am working on.

What projects are you working on now?

Outside of work, just a bunch of open source currently. As I am writing this I am preparing a talk for PloneConf where I am going to talk about taking over maintenance of well loved projects, specifically WebOb which is a Python WSGI/HTTP request/response library that was written by Ian Bicking and is now maintained by the Pylons Project with me as lead.

The Pylons Project is a collective of people that maintain a bunch of different pieces of software, mostly for the HTTP web. Pyramid the web framework, WebOb the WSGI/HTTP request/response library that underlies it, webtest for testing your WSGI applications, form generation, pure HTTP web server and more. We don’t have a lot of people, so there is a lot of work to be done. Releasing new versions of existing software, accepting/reviewing patches and reducing the issue count faster than the issues continue to be generated!

There are also many unfinished projects, my Github is a venerable graveyard of projects that I’ve started and never finished. Great aspirations, just find that if I am doing things for myself once I have figured out the hard part, once I’ve solved “the problem”, completing a project is not nearly as much fun, so off I go to the next project. I always learn something new, just feel bad that it is mostly half-finished code that no-one else can really benefit from.

Which Python libraries are your favorite (core or 3rd party)

I believe it started as a third-party, but is now baked into every Python installation (unless you are on one of those special Linux distributions that rips it out and makes it a separate installable), and that is the virtual environment tool venv or virtualenv. It makes it much simpler to have different environments with different libraries installed. Being able to separate out all of my different projects and not have to install globally is a amazing! C/C++ make this much more difficult especially if you need to include linker flags and all kinds of fun stuff, and pkg-config and friends only get you so far. Similar systems exist for other languages, but it is by far my favorite part about working with Python.

Is there anything else you’d like to say?

We are always looking for new developers to join us in the Pylons Project, if you are looking for someone to mentor you, please reach out and we will do our best. This year we had an absolutely fantastic Google Summer of Code with Ira and I’d be happy to help introduce more new people to not just the Pylons Project but to open source in general.

Thanks for doing the interview!

Categories: FLOSS Project Planets

François Marier: Test mail server on Ubuntu and Debian

Planet Debian - Mon, 2017-11-13 06:11

I wanted to setup a mail service on a staging server that would send all outgoing emails to a local mailbox. This avoids sending emails out to real users when running the staging server using production data.

First, install the postfix mail server:

apt install postfix

and choose the "Local only" mail server configuration type.

Then change the following in /etc/postfix/main.cf:

default_transport = error


default_transport = local:root

and restart postfix:

systemctl restart postfix.service

Once that's done, you can find all of the emails in /var/mail/root.

So you can install mutt:

apt install mutt

and then view the mailbox like this:

mutt -f /var/mail/root
Categories: FLOSS Project Planets

Agiledrop.com Blog: AGILEDROP: Discovering clients expectations is vital. How do we do it?

Planet Drupal - Mon, 2017-11-13 05:53
When starting to talk with a potential client both parties are to some extent unaware what one has to offer and what the other can expect. It is of vital importance you define and manage expectations in such a manner both parties will be satisfied with results. In the next couple of paragraphs, I will outline our approach to setting expectations right up until the day the onboarding process takes place. If all checkboxes up to that point were marked, the developer won’t have any issues when working with the client.  What do we have to offer In my previous blog posts, I did write about ways… READ MORE
Categories: FLOSS Project Planets

Tryton News: Tryton on Docker

Planet Python - Mon, 2017-11-13 04:00

In order to easy the adoption of Tryton, we are publishing Docker images images on the Docker hub for all series starting from 4.4.

They contain the server, the web client and all modules of the series. They are periodically updated and tagged per series. They work with the postgres images as default storage back-end.

The usage is pretty simple:

$ docker run --name postgres -e POSTGRES_DB=tryton -d postgres $ docker run --link postgres:postgres -it tryton/tryton trytond-admin -d tryton --all $ docker run --name tryton -p 8000:8000 --link postgres:postgres -d tryton/tryton

Then you can connect using: http://localhost:8000/

Categories: FLOSS Project Planets

Interview with Lars Pontoppidan

Planet KDE - Mon, 2017-11-13 03:00
Could you tell us something about yourself?

Yes certainly! I’m Lars Pontoppidan; a 36 year old, self-employed programmer, game developer, musician and artist.

I’ve been drawing and painting since I could put my pen to the paper – so about 35 years.

I made my first recognizable painting when I was around 3 or 4 – my mom still has it framed at her house.

I’ve always wanted to end up at some level where I could combine all my skills and hobbies. Somewhere along the way I found out that game development demand a lot of the skills I possess – so 1.5 years ago, I decided to cancel all my contracts with my clients and go for a new path in life as “indie game developer”. I’ve now found out that it’s probably the worst time I could ever get into indie game development. The bubble has more or less already burst. There’s simply too many game releases for the consumers to cope with at the moment. But hey I’ve tried worse so it doesn’t really bother me – and I get to make art with Krita!

Do you paint professionally, as a hobby artist, or both?

Both I’d say. I’ve always been creating things on a hobby level – but have also delivered a lot of designs, logos and custom graphics as self-employed. I like the hobby work the most – as there are no deadlines or rules for when the project is done.

What genre(s) do you work in?

Cartooning, Digital painting, Animation and Video game art. All these (and maybe more) blend in when producing a game. I also like painting dark and gloomy pictures once in a while.

I think I’ve mostly done cartoon styled work – but with a grain of realism in it. My own little mixture.

I started out with pencil and paper – moved to the Deluxe Paint series, when I got my first Amiga – and ended up with Krita (which is an absolute delight to work with. Thanks to you guys!). I still occasionally do some sketching with pencil and paper – depending on my mood.

Whose work inspires you most — who are your role models as an artist?

* A list too long for me to compile here, of sci-fi fantasy artists. Peter Elson is the first that comes to mind. These artists, in my opinion, lay the very foundation of what’s (supposedly) possible with human technology – and currently, the only possibility to get a glimpse of how life might look like other places in the vast universe that surrounds us. It’s mind blowing how they come up with all the alien designs they do.

* Salvador Dalí – It’s hard to find the right words for his creations – which, I think, is why his works speak to me.

* “vergvoktre” He’s made some really dark, twisted and creepy creations that somehow get under my skin.

How and when did you get to try digital painting for the first time?

The very first digital painting program I’ve ever tried was KoalaPainter for the Commodore 64. I had nothing but a joystick and made, if I recall correctly, a smiley face in black and white.

Thankfully my Amiga 500 came with a copy of Deluxe Paint IV, a two-button mouse and the luxury of a 256+ color palette.

What makes you choose digital over traditional painting?

The glorious “Undo” buffer. I mean… It’s just magic.Especially in the first part of the day (before the first two cups of coffee) where your hand just won’t draw perfect circles, nor any straight lines.

How did you find out about Krita?

I read an article about the Calligra office suite online. It described how Calligra compared to Open Office. I eventually installed it to see how it compared to Open Office and boom there was Krita as part of the package. This was my first encounter – unfortunately it ended up with an uninstall – because of stability issues with the Calligra suite in general.

What was your first impression?

The first impression was actually really good – unfortunately it ended up a bit in the shadows of the Calligra suite’s combined impression. This wasn’t so positive after a few segfaults in the different applications. Luckily I tried Krita later when it entered the Qt5 based versions. I haven’t looked back since.

What do you love about Krita?

The brush engines and the “Layers” docker.

The brushes, and most of the default settings for them, just feel right. Also the many options to tweak the brushes are really awesome.

The layers docker was actually what gave me the best impression of the program – you had working group layers – and you could give any layer the same names! None of the graphic creation applications I used a few years back had these basic, fundamental features done right (Inkscape and GIMP – I’m looking at you). Krita’s layers didn’t feel somewhat broken, hacked-on and had no naming scheme limitations. A small thing that has made a big difference to me.

What do you think needs improvement in Krita? Is there anything that really annoys you?

Uhm… I was going to write ‘speed’ – but everybody is screaming for more of that already. I know how the developers are doing their best to get more juice.

Some great overall stability would be nice. I’ve only ever had 2 or 3 crashes with GIMP over a long period of time – the count is a bit higher with Krita – on a shorter time scale.

My biggest feature request would be: Cut’n’paste functionality through multiple layers, that also paste in separate layers. This would greatly improve my workflow. I’ve always worked with a group layer containing separate layers for outline, color, texture, shadow etc. – on each e.g. movable part in a character rig. So I would really benefit from a (selection based) cut’n’paste that could cut through all the selected layers – and paste all these separate selection+layers elsewhere in the layer tree.

What sets Krita apart from the other tools that you use?

I find that most of Krita’s tools actually do what you expect them to do – without any weird limitations or special cases. Plus the different brushes, brush engines and all the flexibility to tweak them, are real killer features.

The non-destructive masks (Transparency, Filter and Transform) are also on my list of favourite features. I use these layer types a lot when creating game art – to make them blend in better with the game backgrounds.

And maybe the single most important thing: it’s free and open source. So I’m quite certain I will be able to open up my old Krita files many years into the future.

… and speaking of the future; I really look forward to getting my hands dirty with the Python scripting API.

If you had to pick one favourite of all your work done in Krita so far, what would it be, and why?

It would have to be the opening scene of my upcoming 2D game “non”. It’s using a great variety of Krita’s really awesome and powerful features. The scenes in the game features full day and night cycles where all the lighting and shadows change dynamically – this makes it especially hard to get beautiful painted scenes in all the states each scene has between day and night. Krita’s tool set makes it easier and quicker for me to test out a specific feature for an object or sprite – before throwing it into the game engine.

The biggest scene I have so far is 10200×4080 pixels – Krita was actually performing decently up to a certain point where I had to break the scene into smaller projects. I’m not blaming Krita for this

What techniques and brushes did you use in it?

For cartoon styled work I use a Group layer containing:
* background blend (Transparency Mask)
* shadows (Paint layer)
* outlines (Paint layer)
* textures (Group layer)
* solid base color(s) (Paint layer)

For outlines I use the standard Pixel brush ‘Ink_gpen_10’ – it has a really nice sharp edge at small tip sizes. For texturing I mostly use the ‘Splatter_thin’ Pixel brush – with both standard and custom brush tips and settings depending on the project at hand. For shadowing I really like the ‘Airbrush_pressure’ and ‘Airbrush_linear_noisy’ Pixel brushes. I use a selection mask based on the solid base color layer (Layer name -> Right mouse click -> Select Opaque) – and start shadowing the object.

Where can people see more of your work?

In my games: http://games.blackgrain.dk/
On my band album covers: https://barricadecph.bandcamp.com/

Anything else you’d like to share?

I’d like to thank everyone involved with Krita for making this great open source and free software available to the world. I hope to soon get enough time on my hands to help the project grow.

Take care and be nice to each other.

Categories: FLOSS Project Planets

Full Stack Python: DevOps, Thank You Maintainers and Contributing to Open Source

Planet Python - Mon, 2017-11-13 00:00

DevOps, Continuous Delivery... and You is a blog post with the slides and notes based on a class I taught at the University of Virginia this past week. The talk is relevant as a brief introduction to DevOps and Continuous Delivery, especially for junior developers and less-technical managers of software teams. I'm experimenting with the "talk as blog post" style so let me know via email or a tweet if you enjoy it and would want to see future technical talks in that format.

Speaking of feedback on projects, this GitHub issue thread named "thank you" is incredible to read. The issue ticket blew up on the front page of Hacker News as an example of how powerful genuine positive comments can be for project maintainers. Every time I get a thank you tweet (like this one), email or GitHub issue it certainly helps to motivate me to continue working hard on Full Stack Python.

Contributing to open source is a recent Talk Python to Me podcast episode in the same vein as thanking your maintainer. Working on open source projects with your own contributions to documentation or simple bug fixes can be a great way to become a better programmer. I particularly enjoyed the recommendations of the panel to cut your teeth on smaller open source projects rather than trying to jump into a massive codebase like Django or the CPython implementation. Take a listen to that podcast episode if you are new to open source or have been wondering how to get involved.

As always, send me an email or submit an issue ticket on GitHub to let me know how to improve Full Stack Python as I continue to fill in the table of contents with new pages and new tutorials.

Categories: FLOSS Project Planets

Bryan Pendleton: Sandy, we miss you

Planet Apache - Sun, 2017-11-12 20:45

Some nice candids from a lovely event in honor of Sandy Hamilton.

Friend, colleague, inspiration.

Categories: FLOSS Project Planets

Lars Wirzenius: Unit and integration testing: an analogy with cars

Planet Debian - Sun, 2017-11-12 18:12

A unit is a part of your program you can test in isolation. You write unit tests to test all aspects of it that you care about. If all your unit tests pass, you should know that your unit works well.

Integration tests are for testing that when your various well-tested, high quality units are combined, integrated, they work together. Integration tests test the integration, not the individual units.

You could think of building a car. Your units are the ball bearings, axles, wheels, brakes, etc. Your unit tests for the ball bearings might test, for example, that they can handle a billion rotations, at various temperatures, etc. Your integration test would assume the ball bearings work, and should instead test that the ball bearings are installed in the right way so that the car, as whole, can run a kilometers, and accelerate and brake every kilometer, uses only so much fuel, produces only so much pollution, and doesn't kill passengers in case of a crash.

Categories: FLOSS Project Planets

Shane Curcuru: I’m graduating from POLICE ACADEMY!

Planet Apache - Sun, 2017-11-12 14:26
Well, to be precise, the Arlington Citizen’s Police Academy… it’s not learning to be a police officer, it’s for town residents to learn about our local police and their procedures. Our town’s police are serious about community policing and have … Continue reading →
Categories: FLOSS Project Planets

Bryan Pendleton: Five observations about Stranger Things

Planet Apache - Sun, 2017-11-12 14:14

  1. The Netflix series Stranger Things is a parable: it is about adolescence, and about the hopes, fears, and (quite literally) the demons that children wrestle with during those terrifying years when everything seems to be changing, and nobody knows why, or what things will become.

  2. The show is at its very best when it is most firmly rooted in those middle-school years, helping us see and remember what it was like to be that age, and it is most rewarding when it shows how children develop those coping mechanisms that sustain and guide us through our lives.

  3. The portrayal of the differences between those adults that "get it" (Will's mom, Sheriff Hopper, the middle school science teacher) and those that don't (pretty much everyone else) is particularly fine and elegant.

  4. The government research lab, and all the scientists that work there, are evil to the core? Bleah. Weak and cliched. But it is an easy way to stage a show that really wants to focus on the children, and keep nearly everyone else anonymous.

  5. The single worst part about Stranger Things, though, is the notion that El's power derives from her anger. Boo! Hiss! Shame on all the script writers and creatives! This is more than just vastly disappointing, it's so completely unnecessary. If they had just done this one single thing right, Stranger Things would truly be a show for the ages. Instead, it's in that list of stuff I'll watch, but won't really actively recommend to people.

Categories: FLOSS Project Planets

Stefan Bodewig: Talk about Ant at FOSS Backstage Micro-Summit

Planet Apache - Sun, 2017-11-12 14:05

Next Monday I'm going to talk about some lessons learned during the seventeen years I've so far enjoyed being part of the Ant community at the FOSS Backstage Micro-Summit in Berlin.

Categories: FLOSS Project Planets

Sven Hoexter: Offering a Simtec Entropy Key

Planet Debian - Sun, 2017-11-12 12:29

Since I started to lean a bit towards the concept of minimalism I've got rid of stuff, including all stationary computers. So for now I'm left with just my laptop and that's something where I do not want to attach an USB entropy key permanently. That's why I've a spare Simtec Entropy Key I no longer use, and I'm willing to sell.

In case someone is interested, I'm willing to give it away for 20EUR + shipping. If you can convince me it'll be of use for the Debian project (end up on a DSA managed machine for example) I'm willing to give it away for less. If you're located in Cologne, Copenhagen or Barcelona we might be able, depending on the timing, to do a personal handover (with or without keysigning). Otherwise I guess shipping is mainly interesting for someone also located in Europe.

You can use sven at stormbind dot net or hoexter at debian dot org to contact me and use GPG key 0xA6DC24D9DA2493D1.

Categories: FLOSS Project Planets

Ben Armstrong: The Joy of Cat Intelligence

Planet Debian - Sun, 2017-11-12 09:46

As a cat owner, being surprised by cat intelligence delights me. They’re not exactly smart like a human, but they are smart in cattish ways. The more I watch them and try to sort out what they’re thinking, the more it pleases me to discover they can solve problems and adapt in recognizably intelligent ways, sometimes unique to each individual cat. Each time that happens, it evokes in me affectionate wonder.

Today, I had one of those joyful moments.

First, you need to understand that some months ago, I thought I had my male cat all figured out with respect to mealtimes. I had been cleaning up after my oafish boy who made a watery mess on the floor from his mother’s bowl each morning. I was slightly annoyed, but was mostly curious, and had a hunch. A quick search of the web confirmed it: my cat was left-handed. Not only that, but I learned this is typical for males, whereas females tend to be right-handed. Right away, I knew what I had to do: I adjusted the position of their water bowls relative to their food, swapping them from right to left; the messy morning feedings ceased. I congratulated myself for my cleverness.

You see, after the swap, as he hooked the kibbles with his left paw out of the right-hand bowl, they would land immediately on the floor where he could give them chase. The swap caused the messes to cease because before, his left-handed scoops would land the kibbles in the water to the right; he would then have to scoop the kibble out onto the floor, sprinkling water everywhere! Furthermore, the sodden kibble tended to not skitter so much, decreasing his fun. Or so I thought. Clearly, I reasoned, having sated himself on the entire contents of his own bowl, he turned to pilfering his mother’s leftovers for some exciting kittenish play. I had evidence to back it up, too: he and his mother both seem to enjoy this game, a regular fixture of their mealtime routines. She, too, is adept at hooking out the kibbles, though mysteriously, without making a mess in her water, whichever way the bowls are oriented. I chalked this up to his general clumsiness of movement vs. her daintiness and precision, something I had observed many times before.

Come to think of it, lately, I’ve been seeing more mess around his mother’s bowl again. Hmm. I don’t know why I didn’t stop to consider why …

And then my cat surprised me again.

This morning, with Shadow behind my back as I sat at my computer, finishing up his morning meal at his mother’s bowl, I thought I heard something odd. Or rather, I didn’t hear something. The familiar skitter-skitter sound of kibbles evading capture was missing. So I turned and looked. My dear, devious boy had squished his overgrown body behind his mother’s bowls, nudging them ever so slightly askew to fit the small space. Now the bowl orientation was swapped back again. Stunned, I watched him carefully flip out a kibble with his left paw. Plop! Into the water on the right. Concentrating, he fished for it. A miss! He casually licked the water from his paw. Another try. Swoop! Plop, onto the floor. No chase now, just satisfied munching of his somewhat mushy kibble. And then it dawned on me that I had got it somewhat wrong. Yes, he enjoyed Chase the Kibble, like his mom, but I never recognized he had been indulging in a favourite pastime, peculiarly his own …

I had judged his mealtime messes as accidents, a very human way of thinking about my problem. Little did I know, it was deliberate! His private game was Bobbing for Kibbles. I don’t know if it’s the altered texture, or dabbling in the bowl, but whatever the reason, due to my meddling, he had been deprived of this pleasure. No worries, a thwarted cat will find a way. And that is the joy of cat intelligence.

Categories: FLOSS Project Planets

Russ Allbery: Review: Night Moves

Planet Debian - Sun, 2017-11-12 03:05

Review: Night Moves, by Pat Green

Publisher: Aquarius Copyright: 2014 ISBN: 0-9909741-1-1 Format: Kindle Pages: 159

In the fall of 2012, Pat Green was a preacher of a failing church, out of a job, divorced for six months, and feeling like a failure at every part of his life. He was living in a relative's house and desperately needed work and his father had been a taxi driver. So he got a job as a 6pm to 6am taxi driver in his home town of Joliet, Illinois. That job fundamentally changed his understanding of the people who live in the night, how their lives work, and what it means to try to help them.

This is nonfiction: a collection of short anecdotes about life as a cab driver and the people who have gotten a ride in Green's cab. They're mostly five or six pages long, just a short story or window into someone's life. I ran across Pat Green's writing by following a sidebar link from a post on Patheos (probably from Love, Joy, Feminism, although I no longer remember). Green has an ongoing blog on Patheos about raising his transgender son (who appears in this collection as a lesbian daughter; he wasn't out yet as transgender when this was published), which is both a good sample of his writing and occasionally has excerpts from this book.

Green's previous writing experience, as mentioned at several points in this collection, was newspaper columns in the local paper. It shows: these essays have the succinct, focused, and bite-sized property of a good newspaper article (or blog post). The writing is a little rough, particularly the remembered dialogue that occasionally falls into the awkward valley between dramatic, constructed fictional dialogue and realistic, in-the-moment speech. But the stories are honest and heartfelt and have the self-reflective genuineness of good preaching paired with a solid sense of narrative. Green tries to observe and report first, both the other person and his own reactions, and only then try to draw more general conclusions.

This book is also very hard to read. It's not a sugar-coated view of people who live in the night of a city, nor is it constructed to produce happy endings. The people who Green primarily writes about are poor, or alone, or struggling. The story that got me to buy this book, about taking a teenage girl to a secret liaison that turned out to be secret because her liaison was another girl, is heartwarming but also one of the most optimistic stories here. A lot of people die or just disappear after being regular riders for some time. A lot of people are desperate and don't have any realistic way out. Some people, quite memorably, think they have a way out, and that way out closes on them.

The subtitle of this book is "An Ex-Preacher's Journey to Hell in a Taxi" and (if you followed the link above) you'll see that Green is writing in the Patheos nonreligious section. The other theme of this collection is the church and its effect on the lives of people who are trying to make a life on the outskirts of society. That effect is either complete obliviousness or an active attempt to make their lives even worse. Green lays out the optimism that he felt early in the job, the hope that he could help someone the way a pastor would, guide her to resources, and how it went horribly wrong when those resources turned out to not be interested in helping her at all. And those stories repeat, and repeat.

It's a book that makes it very clear that the actual practice of Christianity in the United States is not about helping poor or marginalized people, but there are certainly plenty of Christian resources for judging, hurting people, closing doors, and forcing abused people back into abusive situations, all in the name of God. I do hope some Christians read this and wince very hard. (And lest the progressive Christians get too smug, one of the stories says almost as brutal things about liberal ministries as the stories of conservative ones.)

I came away feeling even more convinced by the merits of charities that just give money directly to poor people. No paternalism, no assuming that rich people know what they need, no well-meaning intermediary organizations with endless rules, just resources delivered directly to the people who most need resources. Ideally done by the government and called universal basic income. Short of constructing a functional government that builds working public infrastructure, and as a supplement even if one has such a government (since infrastructure can't provide everything), it feels like the most moral choice. Individual people may still stay mired in awful situations, but at least that isn't compounded by other people taking their autonomy away and dictating life to them in complete ignorance.

This is a fairly short and inexpensive book. I found it very much worth reading, and may end up following Green's blog as well. There are moments of joy and moments of human connection, and the details of the day-to-day worries and work style of a taxi driver (in this case, one who drives a company car) are pretty interesting. (Green does skip over some parts for various reasons, such as a lot of the routine fares and most of the stories of violence, but does mention what he's skipping over.) But it's also a brutal book, because so many people are hurting and there isn't much Green can do about it except bear witness and respect them as people in a way that religion doesn't.

Recommended, but brace yourself.

Rating: 8 out of 10

Categories: FLOSS Project Planets

Roman Agabekov: How to stay out of SPAM folder? Setting up PTR, SPF, DKIM under Exim

Planet Drupal - Sat, 2017-11-11 18:39
How to stay out of SPAM folder? Setting up PTR, SPF, DKIM under Exim Submitted by admin on Sat, 11/11/2017 - 23:39

In the previous article, we covered teaching your Drupal installation to send mail to users. But that is only half the battle, now we need to make sure the mail we send hits Inbox and not Spam folder. This article describes some options you have that offer relevant solutions. Unfortunately, no one can guarantee 100% inbox hits, but keeping the amount of mail filtered to Spam to a minimum is quite possible.

Categories: FLOSS Project Planets

Drupal core announcements: New Drupal 8 committer: Francesco Placella!

Planet Drupal - Sat, 2017-11-11 17:40

I am pleased to announce that Francesco Placella (plach) has accepted our invitation to be a Drupal 8 provisional framework manager.

Based in Venice, Italy, Francesco has been contributing to core since 2009. During the Drupal 7 lifecycle he was mainly active in the multilingual area, revamping the language negotiation and content translation subsystems. His initial contributions were really well received:

As well as some great progress on the code front, another major success of the virtual sprint was bringing in some significant new contributors. A highlight here would be Francesco Placella, who is playing a lead role in the work on translatable fields. [...] Drupal needs more contributors like Francesco, and if this sprint has helped him find his place in Drupal development that in itself is a major gain for the community.

After being appointed a maintainer for Drupal core's Language system and the Content translation module, Francesco joined the Drupal 8 Multilingual Initiative, soon becoming one of its most active contributors. Mainly focused on the content translation aspects, he gradually became heavily involved in the entity and field systems, with foundational work on the form and storage subsystems.

Francesco was also very active in the Drupal 8 Accelerate program, which had the goal of fixing all the outstanding critical bugs that were blocking the release of Drupal 8.0.0. This gave him the opportunity to expand his activity also in many other areas, with a particular focus on Views performance. Drupal 8.0.0 was eventually released and Francesco took a break to look after his newborn son.

Francesco showed up again at DrupalCon Vienna and we thought it would be a good time to ask him to become a framework manager. He gladly accepted, so please join me in welcoming him to the core committer team.

Categories: FLOSS Project Planets

Yasoob Khalid: Introduction to Machine Learning and its Usage in Remote Sensing

Planet Python - Sat, 2017-11-11 16:34

Hey guys! I recently wrote a review paper regarding the use of Machine Learning in Remote Sensing. I thought that some of you might find it interesting and insightful. It is not strictly a Python focused research paper but is interesting nonetheless.

Introduction to Machine Learning and its Usage in Remote Sensing

1. Introduction

Machines have allowed us to do complex computations in short amounts of time. This has given rise to an entirely different area of research which was not being explored: teaching machines to predict a likely outcome by looking at patterns. Machine Learning is being used to solve almost all kinds of problems ranging from Stock Market predictions to medical formulae synthesis.

There are multiple famous machine learning algorithms in use today and new algorithms are popping up every other day. Some of the widely known algorithms are:

  1. Support Vector Machines
  2. Neural Networks
  3. Random Forests
  4. K-Nearest Neighbors
  5. Decision Trees
  6. K-Means
  7. Principal Component Analysis

Different important steps are involved in getting the machines to predict dependable and reliable data.

2. Machine Learning in Remote Sensing

The roots of machine learning in remote sensing date back to the 1990s. It was initially introduced as a way to automate knowledge-base building for remote sensing. In their paper, Huang and Jensen (1997) talk about how a knowledge-base was built using minimal input from human experts and then decision trees were created to infer the rules from the human input for the expert system. The generated rules were used at a study site on the Savannah River. The conclusion details how the proposed machine-learning assisted expert system approach yielded the highest accuracy compared to conventional methods at that time. After such similar developments machine learning was soon adopted as an important tool by the remote sensing community. Now it is being used in all sorts of projects, from an unsupervised satellite image scene classification (Li, et al. 2016) to the classification of Australian native forests (Shang & Chisholm, 2014). Now we will take a look at the typical machine learning workflow.

3. Machine Learning Workflow

It is often important to acquaint yourself with the workflow involved. Machine Learning, too, has a workflow which is somewhat common to all of the machine learning based projects.

  • Gathering Data
  • Cleaning Data
  • Model building & selecting the right algorithm
  • Gaining Insights from the results
  • Visualizing the data

In remote sensing one gathers data mostly using satellites or aerial drones. Data cleaning comes in when our dataset has incomplete or missing values and algorithm selection involves getting acquainted with the problem which one is trying to solve (more on this later). If one is making a model just for predictions and not specifically for gaining insights then the workflow ends here and one gets started with implementing the trained model in production. However, if one is writing a research paper or wants to gain insights then one can chart the results using a graphing library and draw insights from the charted data. We will counter the data cleaning and the model building part in this paper.

3.1 Data Cleanup

This process involves cleaning up textual and/or image-based data and making the data manageable (which sometimes might involve reducing the number of variables associated with a record).

3.1.1 Textual Data

Oftentimes, one might encounter missing values in one’s dataset. One has to decide whether to try and fill in the missing data by “guessing” the missing values using the neighbouring data or to drop that particular record altogether. Dropping records seems like a viable option but it might not be feasible if the dataset is already quite small. So one has to resort to filling in the incomplete data cells. There are multiple ways to do this but the easiest one is to take the neighbouring values and calculate an average.

3.1.2 Image Data

Data cleanup also involves manipulating images which might contain some artifacts that can interfere with one’s classification algorithms. Nath et al. (2010) in their paper about water-body area extraction tackle this exact problem. The images that they had contained building shadows which can very easily be confused with water-bodies. They partially solved this problem by calculating the entropy of the image and then used it to segment the image. Entropy refers to randomness. A Water-body has less randomness when compared with its surroundings so it is possible to extract the water-body area by segmenting the image based on the difference in the pixel colors. In other instances the image dataset might contain some blurry images which can gravely affect the accuracy of our algorithm in the training stage. One needs to get rid of such images in the data cleanup step.

3.1.3 Multiple Features

Oftentimes when one records data in the field of remote sensing, one is essentially recording multispectral or hyperspectral data (Shang, et al. 2014). This means that each record will have a lot of variables. If one tries to plot the dataset, one might not be able to make any sense of it because one will have a lot of pairwise correlations to think about if one plots a plethora of variables. To interpret the data more meaningfully, one needs some way to reduce the number of variables. This is where the Principal Component Analysis (PCA) comes in –it will reduce the number of variables to a few, interpretable linear combinations of the data. Each linear combination will correspond to a principal component. There are numerous tools available to help one with PCA. If one is utilizing the famous scikit-learn library, there is a PCA function which one can use.

3.2 Types of Machine Learning Algorithms

There are three broad classes of machine learning algorithms. One class is the supervised machine learning, the second is unsupervised machine learning, and the third is reinforced learning. The difference between supervised and unsupervised is that while using supervised algorithms, one has a dataset containing the output column whereas while using the unsupervised algorithms, one only has a huge dataset and it is the duty of the algorithm to cluster the dataset into various different classes based on the relation it has identified between different records. Reinforcement learning is slightly different. In reinforcement learning, one provides the algorithm with an environment and the algorithm takes decisions in that environment. It keeps on improving itself with each decision based on the feedback it gets for its last decision. We will now discuss three famous algorithms being used in remote sensing.

3.2.1 Random Forest

Random forest algorithms are increasing in popularity in the Remote Sensing community (Belgiu, et al. 2016) because of the accuracy of their classifications. These are ensemble classifiers, which basically means that they make use of multiple decision trees underneath. A major reason for the popularity of RF classifiers is that they help in alleviating the high dimensional problem. They provide a variable importance (VI) measurement which allows one to reduce the number of dimensions of hyperspectral data. Variable Importance is essentially the measure of how much change in a specific input affects the output.

3.2.2 Support Vector Machines

SVMs are supervised learning models which can be used for regression as well as classification problems. They are mostly used for classification problems. The way they work is by plotting the points (features) in a n-dimensional space (n features) and then coming up with a hyperplane which best divides those points. SVMs are being used in almost all types of classification problems in remote sensing, from forest categorization (Shang, X & Chisholm, 2014) to segmentation of multispectral remote sensing images (Mitra, et al. 2004). Just like other algorithms, their success depends on the nature of the problem and one will have to test each algorithm separately and then take a decision based on the performance of each algorithm.

3.2.3 Artificial Neural Networks

Neural Networks are a class of machine learning algorithms which try to mimic the way our brains work. The first application of neural networks (NN) in remote sensing was completed in 1988 (Kanellopoulos and Wilkinson 1997). Artificial Neural Networks are a type of Neural Networks. ANNs are the biologically inspired simulations performed on the computer to perform certain specific tasks like pattern recognition, clustering, classification etc. Their popularity has increased a lot recently due to technical advancements which became possible due to ANNs, an example is AlphaGo defeating the world champion of the game Go. This had never been done before, and it was considered a great feat. Accurate land cover classification used to be done mostly by statistical classifiers, but now ANNs have taken their place because ANNs provide an accurate way to classify land cover and geophysical features without having to rely on statistical assumptions or procedures. ANNs “learn” different patterns in images based on their own (by using artificial neurons) with a minimal set of inputs. They are also referred to as black-box algorithms because oftentimes it is hard to figure out how an ANN is figuring out the outputs.

4. Overfitting and Bias

Most of the times when you are developing a model for predicting/classifying images, you have a big dataset for training and testing your algorithm. We split the dataset into roughly a 75:25 ratio where 75% of the data is used for training and 25% is used for evaluating the performance of the model after it has been trained. 75:25 is not a hard ratio; you can use any other dataset division which strikes your fancy. The only problem you have to take care of is that the training segment of the dataset should have an unbiased representation of the whole dataset and that it should not be too small as compared to the testing segment of the dataset. Unbiased means that it should not have only one type of record from the dataset and should have almost every type of record which is a part of the dataset so that the model is trained over every different kind of input. If the training dataset is too small, then you might not get reliable predictions because the model has not been trained over every different type of input.

Overfitting is another problem which you need to take care of. Overfitting the model generally entails making an overly complex model to explain idiosyncrasies and outliers in the data under study. This means that if you use the same type of data (the type of data on which it has been trained) for evaluating the model, you will get a very high prediction/classification accuracy. However, if you modify the input just a little (something which the model has not seen before), then the prediction/classification accuracy takes a dip. You can fix overfitting by using a bigger dataset and segmenting the dataset properly. Additionally, it is beneficial to reduce the complexity of the model definition so that not all of the extreme edge cases are being classified.

5. Which algorithm is the best one?

The answer to this question depends on the problem which one is trying to solve. In some cases when you have multiple dimensions but limited records, SVM might work better. If you have a lot of records but less dimensions (features), Neural Networks (NN) might yield a better prediction/classification accuracy. One often has to test multiple algorithms on your dataset and choose the one which works the best. Oftentimes, it is necessary to tune various parameters for the different algorithms (i.e variable importance for RF, number of hidden layers and neurons for Neural Networks and “decision function shape” for SVMs etc.). A lot of times, a better accuracy may be achieved by combining multiple algorithms together; this is called ensemble. It is also possible to combine SVM and Neural Networks or SVM and RF (possibilities are endless) to improve the prediction accuracy. Again, one will have to test multiple ensembles in order to choose the best one.

It is also important to note that the prediction accuracy might change based on which particular feature one is trying to use for classification/prediction purposes. For instance, Shang and Chisholm (2014) discuss how when they had to classify Australian native forest species, they decided to use state-of-the-art remote sensing algorithms. They decided to classify trees at leaf, canopy, and community level. They tested various algorithms (SVM, AdaBoost and Random Forest) and found that each algorithm outperformed the other at each different level. At the leaf level, Random Forest achieved the best classification accuracy (94.7%), and Support Vector Machine outperformed the other algorithms at both the canopy (84.5%) and community levels (75.5%).

Another factor which can affect one’s algorithm choice is whether the data is linearly separable or not. For instance, linear classification algorithms (SVM, logistic regression etc.) expect that the data can be divided by a straight line in linear space. Assuming that the data is linearly separable might work for most cases but will be correct for some scenarios and will bring down the prediction/classification accuracy. Hence, we need to make sure that the algorithm used is able to handle the kind of available data.

It is not possible to look at an algorithm and decided theoretically whether it will yield the best results for your dataset or not because a lot of the machine-learning algorithms are black box algorithms. This means that it is hard to see how the algorithm is arriving at a specific result. Therefore, it is useful to first narrow down your algorithm choice based on the type of problem and then apply the narrowed down algorithms on a part of your dataset and see which one performs best.

6. Conclusion

In this paper we looked at what machine learning is, how it was first introduced into the world of remote sensing, what a typical workflow is like, and what kind of problems are being solved using machine learning. Machine learning has a bright future because more and more people are learning the basics of machine learning and applying it in their regular jobs and researches. New algorithms are cropping up every other day, and the accuracy rate of classifications are improving along with it. Those problems in remote sensing (mapping land cover) which seemed difficult and sometimes impossible are being solved by new algorithms every single day. It is not far-fetched to say that most analysis work done in the world today will be done by machine learning algorithms in near future.


  • Belgiu, M., & Drăguţ, L. (2016). Random forest in remote sensing: A review of applications and future directions. ISPRS Journal of Photogrammetry and Remote Sensing, 114, 24-31, doi:10.1016/j.isprsjprs.2016.01.011
  • Rodriguez-Galiano, V. F., & Chica-Rivas, M. (2014). Evaluation of different machine learning methods for land cover mapping of a Mediterranean area using multi-seasonal Landsat images and Digital Terrain Models. International Journal of Digital Earth, 7(6), 492-509, doi:10.1080/17538947.2012.748848
  • Shang, X., & Chisholm, L. A. (2014). Classification of Australian native forest species using hyperspectral remote sensing and machine-learning classification algorithms. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(6), 2481-2489, doi:10.1109/JSTARS.2013.2282166
  • Nath, R. K., & Deb, S. K. (2010). Water-body area extraction from high resolution satellite images-an introduction, review, and comparison. International Journal of Image Processing (IJIP), 3(6), 353-372,
  • Li, Y., Tao, C., Tan, Y., Shang, K., & Tian, J. (2016). Unsupervised multilayer feature learning for satellite image scene classification. IEEE Geoscience and Remote Sensing Letters, 13(2), 157-161, doi:10.1109/LGRS.2015.2503142
  • Huang, X., & Jensen, J. R. (1997). A machine-learning approach to automated knowledge-base building for remote sensing image analysis with GIS data. Photogrammetric engineering and remote sensing, 63(10), 1185-1193.
  • Mitra, P., Shankar, B. U., & Pal, S. K. (2004). Segmentation of multispectral remote sensing images using active support vector machines. Pattern recognition letters, 25(9), 1067-1074, doi:10.1016/j.patrec.2004.03.004
  • Melgani, F., & Bruzzone, L. (2004). Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on geoscience and remote sensing, 42(8), 1778-1790, doi:10.1109/TGRS.2004.831865
  • Kubat, M., Holte, R. C., & Matwin, S. (1998). Machine learning for the detection of oil spills in satellite radar images. Machine learning, 30(2-3), 195-215, doi:10.1023/A:1007452223027
  • Maggiori, E., Tarabalka, Y., Charpiat, G., & Alliez, P. (2017). Convolutional neural networks for large-scale remote-sensing image classification. IEEE Transactions on Geoscience and Remote Sensing, 55(2), 645-657, doi:10.1109/TGRS.2016.2612821
  • Pedregosa, F., Varoquaux, G., Gramfort, A. & Michel, V. (2011). Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12, 2825-2830.
  • Jensen, R. R., Hardin, P. J. and Yu, G. (2009), Artificial Neural Networks and Remote Sensing. Geography Compass, 3: 630–646. doi:10.1111/j.1749-8198.2008.00215.x

I hope you guys enjoyed the paper. I am open to any kind of comments and suggestions. Till next time!


Categories: FLOSS Project Planets
Syndicate content