FLOSS Project Planets

Yuriy Gerasimov: Centralize your logs with logstash (getting started guide)

Planet Drupal - Mon, 2015-01-12 08:51

Logstash is a great tool to centralize logs in your environment. For example we have several drupal webheads that write logs into syslog. It would be really nice to see those logs somewhere centrally to find out about your system's health status and debug potential problems.

In this article I would like to show how easy to start using logstash for local development.

First of all in order to run logstash you need to follow instructions http://logstash.net/docs/1.4.2/tutorials/getting-started-with-logstash.

Logstash has following concepts:

  • inputs -- where we grab logs from. This can be files on local files system, records of database table, redis and many more.
  • codecs -- way you can serialize/unserialize you data. Think about it as json decode when you get records or running json encode when you are saving log message.
  • filters -- instruments to filter particular log records we want to process. Example -- syslog has many records but we want to extract only drupal related.
  • outputs -- where we are passing our processed log records. It can be a file (multiple different formats), stdout or what is most interesting elastic search

Tricky part comes when you need to install Elastic Search to store your logs and Kibana to view them. There is very nice shortcut for development purposes -- to use already built docker image for that.

I have found very handy to use https://registry.hub.docker.com/u/sebp/elk/ image.

So you need docker to be installed (http://docs.docker.com/installation/ubuntulinux/). Then you import docker image and run it.

sudo docker pull sebp/elk sudo docker run -p 5601:5601 -p 9200:9200 -p 5000:5000 -it --name elk sebp/elk

Now we have docker image working plus it has port forwarding to our localhost.

In order to send your logstash logs to elastic search you need to use elasticsearch output. Here is logstash configuration file example that can be run for testing.

input { stdin { } } output { stdout { codec => rubydebug } elasticsearch { host => "localhost" port => "9200" protocol => "http" } }

Now when you run logstash and enter couple of messages they will be fed to elasticsearch. Now you can open http://localhost:5601/ to see kibana in action.

Next step would be to set up your own rules of extracting drupal (or any other type) logs and pushing them to elastic search. But this is very individual task that is out of the scope of this guide.

Tags: drupaldrupal planetlogstash
Categories: FLOSS Project Planets

"Menno's Musings": IMAPClient 0.12

Planet Python - Mon, 2015-01-12 08:14

I'm very happy to announce that IMAPClient 0.12 is out!

This is a big release. Some highlights:

  • Unicode handling has been fixed. Some bad decisions were made during the Python 3 port (v0.10) and this release fixes that. Bytes are now returned in most places (instead of unicode strings).
  • MODSEQ parts in SEARCH responses are now handled correctly. A crash has been fixed when MODSEQ queries (part of the CONDSTORE extension) are made with the search method. The returned MODSEQ value is now available via the "modseq" attribute on the returned list of ids.
  • Extra __init__ keyword args are passed through. This allows access to SSL options that the underlying imaplib library might support (Python version dependent).
  • Python 3.4 is now officially supported.
  • More control over OAUTH2 parameters.
  • The deprecated get_folder_delimiter() method has been removed.

See the NEWS.rst file and manual for more details.

Many thanks go to Inbox for sponsoring the significant unicode changes in this release.

IMAPClient can be installed from PyPI (pip install imapclient) or downloaded from the IMAPClient site.

The next major version of IMAPClient will be 1.0.0, and will be primarily focussed on enhancing TLS/SSL support.

Categories: FLOSS Project Planets

FSF Events: Richard Stallman - "El movimiento del software libre" (Guijón, Spain)

GNU Planet! - Mon, 2015-01-12 07:30
Richard Stallman hablará sobre las metas y la filosofía del movimiento del Software Libre, y el estado y la historia del sistema operativo GNU, el cual junto con el núcleo Linux, es actualmente utilizado por decenas de millones de personas en todo el mundo.

Esa charla de Richard Stallman no será técnica y será abierta al público; todos están invitados a asistir.

La hora y el lugar exacto serán determinados.

Favor de rellenar este formulario, para que podamos contactarle acerca de eventos futuros en la región de Asturias.

Categories: FLOSS Project Planets

FSF Events: Richard Stallman - "Por una sociedad digital libre" (Oviedo, Spain)

GNU Planet! - Mon, 2015-01-12 07:30
Existen muchas amenazas a la libertad en la sociedad digital, tales como la vigilancia masiva, la censura, las esposas digitales, el software privativo que controla a los usuarios y la guerra contra la práctica de compartir. El uso de servicios web presenta otras más amenazas a la libertad de los usuarios. Por último, no contamos con ningún derecho concreto para hacer nada en Internet, todas nuestras actividades en línea son precarias y podremos continuar con ellas siempre y cuando las empresas deseen cooperar.

Esa charla de Richard Stallman no será técnica y será abierta al público; todos están invitados a asistir.

Favor de rellenar este formulario, para que podamos contactarle acerca de eventos futuros en la región de Asturias.

Categories: FLOSS Project Planets

FSF Events: Richard Stallman - "El movimiento del software libre" (Gijón, Spain)

GNU Planet! - Mon, 2015-01-12 07:30
Richard Stallman hablará sobre las metas y la filosofía del movimiento del Software Libre, y el estado y la historia del sistema operativo GNU, el cual junto con el núcleo Linux, es actualmente utilizado por decenas de millones de personas en todo el mundo.

Esa charla de Richard Stallman no será técnica y será abierta al público; todos están invitados a asistir.

La hora y el lugar exacto serán determinados.

Favor de rellenar este formulario, para que podamos contactarle acerca de eventos futuros en la región de Asturias.

Categories: FLOSS Project Planets

FSF Events: Richard Stallman - "El software libre y tu libertad" (Gijón, Spain)

GNU Planet! - Mon, 2015-01-12 07:20
Richard Stallman hablará sobre las metas y la filosofía del movimiento del Software Libre, y el estado y la historia del sistema operativo GNU, el cual junto con el núcleo Linux, es actualmente utilizado por decenas de millones de personas en todo el mundo.

Esa charla de Richard Stallman no será técnica y será abierta al público; todos están invitados a asistir.

Favor de rellenar este formulario, para que podamos contactarle acerca de eventos futuros en la región de Asturia.

Categories: FLOSS Project Planets

FSF Events: Richard Stallman - "Por una sociedad digital libre" (Oviedo, Spain)

GNU Planet! - Mon, 2015-01-12 07:19
Existen muchas amenazas a la libertad en la sociedad digital, tales como la vigilancia masiva, la censura, las esposas digitales, el software privativo que controla a los usuarios y la guerra contra la práctica de compartir. El uso de servicios web presenta otras más amenazas a la libertad de los usuarios. Por último, no contamos con ningún derecho concreto para hacer nada en Internet, todas nuestras actividades en línea son precarias y podremos continuar con ellas siempre y cuando las empresas deseen cooperar.

Esa charla de Richard Stallman no será técnica y será abierta al público; todos están invitados a asistir.

El lugar exacto y la hora de la charla serán determinados.

Favor de rellenar este formulario, para que podamos contactarle acerca de eventos futuros en la región de Asturia.

Categories: FLOSS Project Planets

FSF Events: Richard Stallman - "Le logiciel libre et l'éducation" (Jendouba, Tunisia)

GNU Planet! - Mon, 2015-01-12 06:50

This speech by Richard Stallman will be nontechnical, admission is gratis, and the public is encouraged to attend.

Time and detailed location to be determined.

Please fill out our contact form, so that we can contact you about future events in and around Jendouba.

Categories: FLOSS Project Planets

FSF Events: Richard Stallman - "Copyrights vs. Community" (Tunis, Tunisia)

GNU Planet! - Mon, 2015-01-12 06:38
Copyright developed in the age of the printing press, and was designed to fit with the system of centralized copying imposed by the printing press. But the copyright system does not fit well with computer networks, and only draconian punishments can enforce it. The global corporations that profit from copyright are lobbying for draconian punishments, and to increase their copyright powers, while suppressing public access to technology. But if we seriously hope to serve the only legitimate purpose of copyright--to promote progress, for the benefit of the public--then we must make changes in the other direction.

Registration, which can be done anonymously, while not required, is appreciated; it will help us ensure we can accommodate all the people who wish to attend.

Please fill out our contact form, so that we can contact you about future events in and around Sousse.

Categories: FLOSS Project Planets

Chris Lamb: Giant's Causeway puzzle

Planet Debian - Mon, 2015-01-12 06:34

Over Christmas I found the above puzzle at my parent's house. I don't know if it has a name but it seemed apt to name it after The Giant's Causeway.

The idea is that you swap and/or rotate the outer tiles until the colours at the edges match. The center tile is fixed, or at least I assume it is as the wooden edges are deliberately less rounded.

Of course, solving it manually would be boring so here's a dumb brute force solution. I actually admire how inefficient and stupid it is...

import itertools class Tile(list): def __getitem__(self, x): return super(Tile, self).__getitem__(x % len(self)) def rotate(self, x): return Tile(self[x:] + self[:x]) COLOURS = ('YELLOW', 'WHITE', 'BLACK', 'RED', 'GREEN', 'BLUE') YELLOW, WHITE, BLACK, RED, GREEN, BLUE = range(len(COLOURS)) TILES = ( Tile((WHITE, YELLOW, RED, BLUE, BLACK, GREEN)), Tile((RED, BLUE, BLACK, YELLOW, GREEN, WHITE)), Tile((WHITE, BLACK, YELLOW, GREEN, BLUE, RED)), Tile((WHITE, BLUE, GREEN, YELLOW, BLACK, RED)), Tile((GREEN, BLUE, BLACK, YELLOW, RED, WHITE)), Tile((RED, YELLOW, GREEN, BLACK, BLUE, WHITE)), ) CENTER = Tile((WHITE, BLACK, RED, GREEN, BLUE, YELLOW)) def pairwise(it): a, b = itertools.tee(it) next(b, None) return itertools.izip(a, b) def validate(xs): for idx, x in enumerate(xs): if x[idx + 3] != CENTER[idx]: raise ValueError("Tile does not match center") for idx, (a, b) in enumerate(pairwise(xs)): if a[idx + 2] != b[idx + 5]: raise ValueError("Tile does not match previous anticlockwise tile") return xs def find_solution(): # For all possible tile permutations.. for xs in itertools.permutations(TILES): # ... try all possible rotations for ys in itertools.product(range(len(COLOURS)), repeat=len(COLOURS)): try: return validate([x.rotate(y) for x, y in zip(xs, ys)]) except ValueError: pass raise ValueError("Could not find a solution.") for x in find_solution(): print ', '.join(COLOURS[y] for y in x)

This prints (after 5 minutes!):

$ python giants.py GREEN, BLUE, RED, WHITE, BLACK, YELLOW WHITE, BLUE, GREEN, YELLOW, BLACK, RED YELLOW, GREEN, BLACK, BLUE, WHITE, RED GREEN, WHITE, YELLOW, RED, BLUE, BLACK RED, BLUE, BLACK, YELLOW, GREEN, WHITE BLUE, BLACK, YELLOW, RED, WHITE, GREEN python giants.py 269.08s user 0.02s system 99% cpu 4:29.36 total

UPDATE: Seems like this puzzle is called "Circus Seven". Now to dust off my Prolog...

Categories: FLOSS Project Planets

FSF Events: Richard Stallman - "Copyright vs Community" (Sousse, Tunisia)

GNU Planet! - Mon, 2015-01-12 06:15
Copyright developed in the age of the printing press, and was designed to fit with the system of centralized copying imposed by the printing press. But the copyright system does not fit well with computer networks, and only draconian punishments can enforce it. The global corporations that profit from copyright are lobbying for draconian punishments, and to increase their copyright powers, while suppressing public access to technology. But if we seriously hope to serve the only legitimate purpose of copyright--to promote progress, for the benefit of the public--then we must make changes in the other direction.

Richard Stallman's speech will be the main address during Le Trophée du Libre, an event to recognize and celebrate---and show some love to---free software in Tunisia. Afterwards, he will award a prize to the best association, club, community or group that supports free software in Tunisia. The speech and the event will be nontechnical, admission is free of charge, and the public is encouraged to attend.

Registration, which can be done anonymously, while not required, is appreciated; it will help us ensure we can accommodate all the people who wish to attend.

Please fill out our contact form, so that we can contact you about future events in and around Sousse.

Categories: FLOSS Project Planets

digiKam Quick Tip: Group Photos by Format

Planet KDE - Mon, 2015-01-12 05:38

Using the View -> Group Images -> By Format command, you can group photos in albums by format. This feature can come in useful for managing albums containing photos in different formats: JPEG, TIFF, RAW and video files, etc.

Continue reading

Categories: FLOSS Project Planets

Glyph Lefkowitz: The Glyph

Planet Python - Mon, 2015-01-12 05:00

As you may have seen me around the Internet, I am typically represented by an obscure symbol.

I have been asked literally hundreds of times about the meaning of that symbol, and I’ve always been cryptic in response because I felt that a full explanation is too much work. Since the symbol is something I invented, the invention is fairly intricate, and it takes some time to explain, describing it in person requires a degree of sustained narcissism that I’m not really comfortable with.

You all keep asking, though, and I really do appreciate the interest, so thanks to those of you who have asked over and over again: here it is. This is what the glyph means.

Ulterior Motive

I do have one other reason that I’m choosing to publish this particular tidbit now. Over the course of my life I have spent a lot of time imagining things, doing world-building for games that I have yet to make or books that I have yet to write. While I have published fairly voluminously at this point on technical topics (more than once on actual paper), as well as spoken about them at conferences, I haven’t made many of my fictional ideas public.

There are a variety of reasons for this (not the least of which that I have been gainfully employed to write about technology and nobody has ever wanted to do that for fiction) but I think the root cause is because I’m afraid that these ideas will be poorly received. I’m afraid that I’ll be judged according to the standards for the things that I’m now an expert professional at – software development – for something that I am a rank amateur at – writing fiction. So this problem is only going to get worse as I get better at the former and keep not getting practice at the latter by not publishing.

In other words, I’m trying to break free of my little hater.

So this represents the first – that I recall, at least – public sharing of any of the Divunal source material, since the Twisted Reality Demo Server was online 16 years ago. It’s definitely incomplete. Some of it will probably be bad; I know. I ask for your forbearance, and with it, hopefully I will publish more of it and thereby get better at it.

Backstory

I have been working on the same video game, off and on, for more or less my entire life. I am an extremely distractable person, so it hasn’t seen that much progress - at least not directly - in the last decade or so. I’m also relentlessly, almost pathologically committed to long-term execution of every dumb idea I’ve ever had, so any minute now I’m going to finish up with this event-driven networking thing and get back to the game. I’ll try to avoid spoilers, in case I’m lucky enough for any of you ever actually play this thing.

The symbol comes from early iterations of that game, right about the time that it was making the transition from Zork fan-fiction to something more original.

Literally translated from the in-game language, the symbol is simply an ideogram that means “person”, but its structure is considerably more nuanced than that simple description implies.

The world where Divunal takes place, Divuthan, was populated by a civilization that has had digital computers for tens of thousands of years, so their population had effectively co-evolved with automatic computing. They no longer had a concept of static, written language on anything like paper or books. Ubiquitous availability of programmable smart matter meant that the language itself was three dimensional and interactive. Almost any nuance of meaning which we would use body language or tone of voice to convey could be expressed in varying how letters were proportioned relative to each other, what angle they were presented at, and so on.

Literally every Divuthan person’s name is some variation of this ideogram.

So a static ideogram like the one I use would ambiguously reference a person, but additional information would be conveyed by diacritical marks consisting of other words, by the relative proportions of sizes, colors, and adornments of various parts of the symbol, indicating which person it was referencing.

However, the game itself is of the post-apocalyptic variety, albeit one of the more hopeful entries in that genre, since restoring life to the world is one of the player’s goals. One of the things that leads to the player’s entrance into the world is a catastrophe that has mysteriously caused most of the inhabitants to disappear and disabled or destroyed almost all of their technology.

Within the context of the culture that created the “glyph” symbol in the game world, it wasn’t really intended to be displayed in the form that you see it. The player first would first see such a symbol after entering a ruined, uninhabited residential structure. A symbol like this, referring to a person, would typically have adornments and modifications indicating a specific person, and it would generally be animated in some way.

The display technology used by the Divuthan civilization was all retained-mode, because I imagined that a highly advanced display technology would minimize power cost when not in use (much like e-paper avoids bleeding power by constantly updating the screen). When functioning normally, this was an irrelevant technical detail, of course; the displays displayed what you want them to display. But after a catastrophe that has disrupted network connectivity and ruined a lot of computers, this detail is important because many of the displays were still showing static snapshots of a language intended to use motion and interactivity as ways to convey information.

As the player wandered through the environment, they would find some systems that were still active, and my intent was (or “is”, I suppose, since I do still hold out hope that I’ll eventually actually make some version of this...) that the player would come to see the static, dysfunctional environment around them as melancholy, and set about restoring function to as many of these devices as possible in order to bring the environment back to life. Some of this would be represented quite concretely as time-travel puzzles later in the game actually allowed the players to mitigate aspects of the catastrophe that broke everything in the first place, thereby “resurrecting” NPCs by preventing their disappearance or death in the first place.

Coen

Coen refers to the self, the physical body, the notion of “personhood” abstractly. The minified / independent version is an ideogram for just the head, but the full version as it is presented in the “glyph” ideogram is a human body: the crook at the top is the head (facing right); the line through the middle represents the arms, and the line going down represents the legs and feet.

This is the least ambiguous and nuanced of all the symbols. The one nuance is that if used in its full form with no accompanying ideograms, it means “corpse”, since a body which can’t do anything isn’t really a person any more.

Kset

This is the trickiest ideogram to pronounce. The “ks” is meant to be voiced as a “click-hiss” noise, the “e” has a flat tone like a square wave from a synthesizer, and the “t” is very clipped. It is intended to reference the power-on sound that some of the earliest (remember: 10s of thousands of years before the main story, so it’s not like individuals have a memory of the way these things sounded) digital computers in Divuthan society made.

Honestly though if you try to do this properly it ends up sounding a lot like the English word “cassette”, which I assure you is fitting but completely unintentional.

Kset refers to algorithms and computer programs, but more generally, thought and the life of the mind.

This is a reference to the “Ee” spell power rune in the 80s Amiga game, Dungeon Master, which sadly I can’t find any online explanation of how the manual described it. It is an object poised on a sharp edge, ready to roll either left or right - in other words, a symbolic representation of a physical representation of the algorithmic concept of a decision point, or the computational concept of a branch, or a jump instruction.

Edec

Edec refers to connectedness. It is an ideogram reflecting a social graph, with the individual below and their many connections above them. It’s the general term for “social relationship” but it’s also the general term for “network protocol”. When Divuthan kids form relationships, they often begin by configuring a specific protocol for their communication.

This is how boundary-setting within friendships and work environments (and, incidentally, flirting) works; they use meta-protocol messages to request expanded or specialized interactions for use within the context of their dedicated social-communication channels.

Unlike most of these other ideograms, its pronunciation is not etymologically derived from an onomatopoeia, but rather from an acronym identifying one of the first social-communication protocols (long since obsoleted).

Zenk

“Zenk” is the ideogram for creation. It implies physical, concrete creations but denotes all types of creation, including intellectual products.

The ideogram represents the Divuthan version of an anvil, which, due to certain quirks of Divuthan materials science that is beyond the scope of this post, doubles for the generic idea of a “work surface”. So you could also think of it as a desk with two curved legs. This is the only ideogram which represents something still physically present in modern, pre-catastrophe Divuthan society. In fact, workshop surfaces are often stylized to look like a Zenk radical, as are work-oriented computer terminals (which are basically an iPad-like device the size of a dinner table).

The pronunciation, “Zenk”, is an onomatopoeia, most closely resembled in English by “clank”; the sound of a hammer striking an anvil.

Lesh

“Lesh” is the ideogram for communication. It refers to all kinds of communication - written words, telephony, video - but it implies persistence.

The bottom line represents a sheet of paper (or a mark on that sheet of paper), and the diagonal line represents an ink brush making a mark on that paper.

This predates the current co-evolutionary technological environment, because appropriately for a society featured in a text-based adventure game, the dominant cultural groups within this civilization developed a shared obsession for written communication and symbolic manipulation before they had access to devices which could digitally represent all of it.

All Together Now

There is an overarching philosophical concept of “person-ness” that this glyph embodies in Divuthan culture: although individuals vary, the things that make up a person are being (the body, coen), thinking (the mind, kset), belonging (the network, edec), making (tools, zenk) and communicating (paper and pen, lesh).

In summary, if a Divuthan were to see my little unadorned avatar icon next to something I have posted on twitter, or my blog, the overall impression that it would elicit would be something along the lines of:

“I’m just this guy, you know?”

And To Answer Your Second Question

No, I don’t know how it’s pronounced. It’s been 18 years or so and I’m still working that bit out.

Categories: FLOSS Project Planets

Concert for India’s Environment: Chinmaya Dunster

LinuxPlanet - Sat, 2015-01-10 18:49
By Vasudev Ram



A really good music video by Chinmaya Dunster: Concert for India's Environment. I had first seen it a while ago. Happened to see it again today via a chain of links. Evergreen music and message.

"The video of this concert, blended with interviews with environmentalists, Indian school children reading their own poems on nature and stunning footage of the Indian wilderness, is available free on request at http://www.chinmaya-dunster.com/concert-environment-2.php."


- Vasudev Ram - Dancing Bison Enterprises Signup to hear about new products or services from me.

Contact Page


Share | var addthis_config = {"data_track_clickback":true}; Vasudev Ram

Categories: FLOSS Project Planets

Good HN thread on scaling a web app to 10K users

LinuxPlanet - Sat, 2015-01-10 17:27
By Vasudev Ram




Saw this today on Hacker News:

What does it take to run a web app with 5K – 10K users?

I read the thread. Found that it had a number of good comments, on both sides of the equation - the business side, i.e. acquiring, retaining and supporting users, and the technical side, i.e. scaling the hardware and software to manage the load on the app. Many of the people who commented, run their own web apps at the same or higher scale.

Overall, a worthwhile read, IMO.

- Vasudev Ram - Dancing Bison Enterprises

Signup to hear about new products or services from me.

Contact Page

Share | var addthis_config = {"data_track_clickback":true}; Vasudev Ram

Categories: FLOSS Project Planets
Syndicate content