FLOSS Project Planets

Bryan Pendleton: Cop stories

Planet Apache - Wed, 2017-04-19 20:46

I'll read almost everything; I'm pretty voracious that way.

But certainly a good police procedural is always right up my alley.

So, two recommendations, one old, and one new:

  • The Fairy Gunmother

    Pennac's novel is set in a post-imperial Paris of the mid-1980's, rich with the complexities that entails, and benefits from a truly superb translation by Ian Monk. The result is laugh-out-loud funny while still being atmospheric and compelling.

  • Leviathan Wakes

    Although you'll find this on your Science Fiction shelves at the local bookstore (hah! is there such a thing?), it's really a police procedural set in the future, in space, as more-than-haggard Detective Miller is trying to unravel why a simple missing persons case appears to be much, much deeper than it first seemed.

Each of these is "Book 1 of a series".

And I'll be reading more of each series, straightaway.

Categories: FLOSS Project Planets

Daniel Bader: Let’s Program with Python: Statements, Variables, and Loops (Part 1)

Planet Python - Wed, 2017-04-19 20:00
Let’s Program with Python: Statements, Variables, and Loops (Part 1)

In this four-part introduction for new programmers you’ll learn the basics of programming with Python using step-by-step descriptions and graphical examples.

In this guest post series by Doug Farrell you’ll learn the basics of programming with Python from scratch. If you’ve never programmed before or need a fun little class to work through with your kids, you’re welcome to follow along.

Table of Contents – Part 1 What is Python?

Since you’re reading this I’m hoping you’re interested in learning how to program in Python.

Python is a programming language, which means it’a a language both people and computers can understand. A computer language is a formal subset of an natural language, like English. A computer language lets people express what they want a computer to do, and tells a computer how to do it.

A computer program is a set of instructions written in a particular computer language. There are lots of different computer languages in the world, most were created to solve certain kinds of problems in different ways, and most over lap in the kinds of things they can do.

Python was developed by a Dutch software engineer named Guido van Rossum, who created the language to solve some problems he saw in computer languages of the time.

Python draws from a lot of good ideas in other languages and pulls them together in one place. Python is a pretty easy computer language to learn, and yet is very powerful. The name Python comes from Guido’s favorite comedy group, Monty Python’s Flying Circus.

This course uses Python 3.6.1, but the examples should work with any version of Python 3 and greater.

Natural Language vs. Formal Language

English is a natural language that’s evolved over time to help us talk with each other. It has a big vocabulary, lots of multiple meanings and depends a lot on how it’s used to make the meaning clear.

Natural languages work well for people because we fill in the gaps where needed. This kind of language fails completely for computers because they need exact instructions in order to run. Formal languages (all programming languages) have limited vocabularies and almost no multiple meanings.

Let’s take an English example that’s something like a “program” for a person, how to make scrambled eggs:

1. Place a frying pan on the stove burner 2. Turn the burner to medium 3. Melt butter in the pan 4. Crack two eggs into pan 5. Stir the eggs to cook and scramble them 6. When finished, serve the eggs on a plate

If the steps above are followed in order, someone should be able to make scrambled eggs. This simple set of steps describe how to perform a task. A computer program is very much the same, a set of steps telling a computer how to perform a task.

Elements of Programming

As you learn to program you’ll find you need to do certain things to make the program do what you want: how to make the computer do something, remember things, doing things over and over and make decisions. Almost all programming languages provide ways to do these four basic things, and they’re known as:

  • Statements: the things a program can do, like performing calculations, drawing on the screen, etc.
  • Variables: these are the “things” (information) you want your program to work on and remember
  • Loops: doing things over and over again very quickly
  • Conditionals: these are choices a program can make about what to do, this is what makes programs “appear” smart.

We’ll make use of these four things as we go along.

Enough of That, Let’s Write Some Python!

Our goal is to create a Python program that will draw an image on our computer screen. The image we’re going to create looks something like a flower, and we’re going to learn how to use Python to create it. The end results will look like this:

So how do we create a Python program? There are two ways to work with Python; working with it directly, and creating Python program files.

This is where we can use the tool called Idle. Idle is a program that lets you both work with Python directly and create Python program files.

So let’s start Idle. When you installed Python you should also have gotten the Idle program installed, if so, let’s start it up!

Starting Idle should give you a window that looks something like this:

This window provides a Python command prompt (hit return a couple of times if you don’t see it) that allows you to run Python statements line by line.

This is called interactive mode as it allows us to ‘interact’ with Python. The command prompt in interactive mode looks like this:

>>>

Its at this prompt where you enter Python statements to try things out.

Statements in Python

Statements are the program commands that make the computer do something in a Python program. Statements can be as simple or as complicated as we’d like to make them.

Here are some examples:

>>> print("Hello there") Hello there >>> print(12) 12 >>> 12 * 3 36 >>> 12 / 3 4.0

The above statements print out a welcome string, perform some basic math and Python is responding. What we’ve done above is enter some Python statements at our command prompt, and Python ran them.

Creating Python Program Files

Working with interactive mode, is great for trying things out with Python. However, we want to create a Python program we can run over and over again without having to re-type it every time.

This is where creating a Python program and saving it as a file is very handy. Python program files are just like any other text file, but usually have the extension “.py”.

We can create a Python program file in Idle by clicking the File → New Window menu item. This opens up a new, empty window that is a simple text editor.

You’ll notice there is no >>> Python command prompt in the window. This is because in the file window we’re not interacting with Python directly, we’re creating a Python program file.

Let’s use our new Python program file to create our first Python program.

The “Turtle” Graphics Module

Python comes with a large library of modules that let us do some interesting things, and one of those modules is called turtle.

The turtle module is a nice tool for drawing graphics on the screen. You can find the turtle graphics documentation here.

The turtle module is based on the idea of a “turtle” on the screen that draws a line as it moves around, as if it had a marker taped to it’s shell.

In order to use the turtle module we have to “import” it into our Python program. Importing a module adds the features and capabilities of that module to our Python program.

To import the turtle module add this line to our Python program:

import turtle

Drawing With Turtle

Once the turtle module is available to us we can use it to draw things with a turtle. Enter the following lines into our program:

t = turtle.Turtle() t.shape("turtle") t.forward(100) Saving and Running a Python Program

Once you’ve got this entered, let’s run the program. To do that we have to save the file first, which we can do from the File → Save menu selection.

Give our program a name and save it to a directory on the hard disk where you can find it again.

To run the program select Run → Run Module. If your program runs without any errors (which usually means you have a typo in your program), a window will open up with a turtle shape at the end of a short line.

That window should look something like this:

This is what our program told Python to do, use the turtle module to create a turtle we’re calling t, change it’s shape to look like a ‘turtle’ and move it forward 100 pixels.

Our turtle, t, is the first variable we’ve created with Python in our program.

Variables in Python

In Python things like our turtle t are represented by variables. Variables let us give a name to something so you and the program can remember it and use it later.

For instance, here’s a variable assignment:

x = 10

This looks a lot like math, and that’s actually where the idea of assigning variables came from.

This simple Python statement assigns the number 10 to a variable called x. The equal sign (=) in the line above creates the variable x and assigns it a value.

In our program we’ve done this by using the turtle module to create a turtle (the Python statement turtle.Turtle()) and assigned the results, a turtle object, to a variable we called t.

Let’s Get Back to Drawing!

Let’s add some more statements to our program to make it draw some more. Let’s make our Python program look like this:

import turtle t = turtle.Turtle() t.shape("turtle") t.forward(100) t.right(90) t.forward(100) t.right(90) t.forward(100) t.right(90) t.color("red") t.forward(100) t.right(90)

When you save and run our program the screen your turtle is drawing on should look like this:

So what’s going on here? What we’ve done is given our turtle a set of commands (Python program statements), and it has run them. Here’s what the statements we entered were doing:

  • Line 1: import the turtle module so our program can use it
  • Line 3: use the turtle module to create our turtle, t
  • Line 4: change the shape of our turtle to look like a turtle
  • Line 5: from where the turtle is, move forward 100 pixels
  • Line 6: from where the turtle is, turn right 90 degrees, a right angle
  • Line 7: from where the turtle is, move forward 100 pixels
  • Line 8: from where the turtle is, turn right 90 degrees, a right angle
  • Line 9: from where the turtle is, move forward 100 pixels
  • Line 10: from where the turtle is, turn right 90 degrees, a right angle
  • Line 11: change the color used by the turtle to red
  • Line 12: from where the turtle is, move forward 100 pixels
  • Line 13: from where the turtle is, turn right 90 degrees, a right angle. This brings our turtle back to its original starting position.

These statements made the turtle draw a box with the last side of the box drawn in red. You can see something interesting about drawing with our turtle; what it draws is based on where it is on the screen and which way it’s headed.

Let’s learn some more Python statements to draw with our turtles.

Turtle Speed: To make our turtle draw faster we use the turtle speed() method. To use this we’ll add this statement to our program:

t.speed(0)

The number 0 between the parenthesises is called a parameter, which is being given to the turtle’s speed() method, making our turtle draw as fast as it can.

Turtle Line Width: We can make our turtle draw with a thicker line, making it easier to see on screen. We do this with the turtle width() method. We can pass a parameter to the width method, expressing a value in pixels. So for example adding this line to our program makes our turtle draw with a line 3 pixels wide:

t.width(3)

Filling-in Shapes: We can also fill a shape (like our box) with color using two other turtle methods, begin_fill() and end_fill(), and by modifying our t.color() method. If we use these Python statements:

t.color("yellow", "red") t.begin_fill() # Draw shape t.end_fill()

We have told our turtle to draw with “yellow” and fill in any shapes with “red”.

Then we use begin_fill() at the start of drawing a closed shape, draw our shape and then used end_fill() to fill that shape with “red”.

The following line is a Python comment:

# Draw shape

I’m using the comment here to indicate where our shape drawing code should go. Comments in Python are used to tell us, the programmers, what’s going on, but are ignored by Python.

Putting It All Together

Taking the things we’ve learned from our box drawing, let’s draw something a little more complicated by drawing a sort of flower.

We’ll do this by doing two things; draw multiple boxes, filling the box with color and turning the turtle slightly between each one using the new turtle methods we just learned about.

To draw our flower we’re going to draw multiple boxes, turning each box slightly every time. One way to do that would be to just repeat our box code over and over like this:

import turtle t = turtle.Turtle() t.speed(0) t.color("yellow", "red") t.width(3) # Draw our first filled in box t.begin_fill() t.forward(100) t.right(90) t.forward(100) t.right(90) t.forward(100) t.right(90) t.forward(100) t.right(90) t.end_fill() t.right(10) # Draw our second filled in box t.begin_fill() t.forward(100) t.right(90) t.forward(100) t.right(90) t.forward(100) t.right(90) t.forward(100) t.right(90) t.end_fill() t.right(10) # Keep going till you've drawn your flower

This would work fine, but we would have to repeat these statements for as many petals as we want to give our flower.

One thing you should know about being a programmer is we’re very lazy and don’t like to repeat ourselves if we don’t have to.

Can Python help us not repeat ourselves? Yes, it can by letting us use a loop to repeat drawing the box multiple times.

Loops in Python

One of the elements of programming mentioned earlier is being able to create loops. Loops are statements in a programming language that allows us to repeat a set of program statements over and over in a controlled way. In our program we’d like to repeat the statements:

t.begin_fill() t.forward(100) t.right(90) t.forward(100) t.right(90) t.forward(100) t.right(90) t.forward(100) t.right(90) t.end_fill() t.right(10)

This set of statements creates our outlined and filled in box. We’d like to repeat these statements, with a slight turn of 10 degrees each time, in order to create a flower. Creating a loop lets us do this. One of the looping statements in Python is called a “for loop”, and it’s used to create a loop that repeats a fixed number of times.

Let’s do a little math to figure out how many times we need to repeat this if we want to make a full circle when our turtle is turning right by 10 degrees after every filled in box.

In case you didn’t know already, there are 360 degrees in a full circle. That means dividing a circle of 360 degrees by 10 degrees gives us a value of 36. This means we want to repeat our box 36 times in order to create our flower.

How can we do this with Python?

The “for” loop: We want to repeat our filled in box 36 times, so we know beforehand how many times we want to loop. This is where the Python for loop comes in handy.

It is made for repeating things a known number of times. It looks like this as a Python definition statement:

for <thing> in <list of things>: # Everything we want to do on the loop

What does that mean? That’s a kind of formal presentation of what a for loop should look like in Python. What it means is this:

  • Take one <thing> from the <list of things>
  • End the for statement with a : character
  • All the statements indented under the for loop should be run every time through the loop. Indentation is very important to Python
  • Go back to the start of the for loop, get another <thing> from <list_of_things>
  • Keep doing this till there are no more things in <list_of_things>

This sounds a lot harder than it actually is. For our program we’ll use a for loop that looks like this:

for petal in range(36): t.begin_fill() t.forward(100) t.right(90) t.forward(100) t.right(90) t.forward(100) t.right(90) t.forward(100) t.right(90) t.end_fill() t.right(10)

Our box code is in there, under the for loop, but what’s that strange looking range(36) thing at a the end of our for loop where the <list of things> should go?

The range(36) thing is what provides the <list_of_things> from our formal definition of a for loop. Let’s talk about that.

The “range” function: Let’s jump back over to our Idle interactive window for a minute and enter this statement:

>>> range(36) range(0, 36)

Python responds by running this code and prints out what looks like it just told us about itself. What does this mean?

To Python the range(36) function is going to provide 36 things when used inside a for loop. Each time through the for loop Python will take a value from the range defined (0 to 35) and assign it to a variable, in our case that variable is called petal.

It will continue this until there are no values left in the range. The way we’ve set things up it will loop 36 times, which is what we want. In this case we’re not using the petal variable, but it’s required by Python to create a correct for loop.

To create our flower using a for loop make our program look like this:

import turtle t = turtle.Turtle() t.speed(0) t.color("yellow", "red") t.width(3) for petal in range(36): t.begin_fill() t.forward(100) t.right(90) t.forward(100) t.right(90) t.forward(100) t.right(90) t.forward(100) t.right(90) t.end_fill() t.right(10)

Let’s go through this line by line.

  • Line 1: import our turtle module
  • Line 3: create our turtle object and use our variable t to keep track of it
  • Line 4: set our turtle drawing speed to fast
  • Line 5: tell our turtle to draw in “yellow” and fill in shapes with “red”
  • Line 6: set our turtle drawing width to 3 pixels
  • Line 8: begin our for loop and tell it to loop 36 times
  • Line 9-19: draw our box and then turn slightly right 10 degrees.

Notice how lines 9 through 19 are indented under the for loop. This is important as it tells Python all these lines are part of the for loop. In Python the indentation is required to tell the program a set of statements are part of a block like this.

Once you’ve got this entered, save our program and then run it. If your program runs without any syntax errors (which usually means you have a typo in your program), you should get a window like this:

Conclusion

Congratulations, you’ve written your first colorful, interesting Python program! Stay tuned for the next class in this series where you’ll learn how to write reusable “code building blocks” with functions.

Categories: FLOSS Project Planets

Mediacurrent: Debug Drupal PHP in Vim with Vdebug

Planet Drupal - Wed, 2017-04-19 18:51

I know quite a few developers that love Vim but think they have to use an IDE to be able to debug their applications. In this post I will show you how to set up Vim as a Xdebug client.

The Vdebug plugin for Vim provides a way to do debugging inside Vim using the tools that Vim provides to great advantage. As the project page says,

 

Categories: FLOSS Project Planets

Will McGugan: Speeding up Websockets 60X

Planet Python - Wed, 2017-04-19 18:23

I recently I had the opportunity to speed up some Websocket code that was a major bottleneck. The final solution was 60X (!) faster than the first pass, and an interesting exercise in optimizing inner loops.

When you send data via a websocket it is masked with a randomly generated four byte key. This masking provides protection from certain exploits against proxies (details here). The algorithm is super simple; essentially each byte in the payload is XORed with a corresponding byte from the key.

Solution 1

The first version of my XOR mask function looked this:

from itertools import cycle def mask1(mask, data): return bytes(a ^ b for a, b in zip(cycle(mask), data))

That's the kind of unfussy and elegant code you want to write for a job interview. If that's the solution that first came to mind, then you know your Python. Alas, this code is as slow as it is beautiful.

It is slow because every byte runs through the Python interpreter. For a chat server, it won't matter. But if you are sending megabytes of data over a websocket, chances are you won't be able to mask websocket packets as fast as the network can accept them.

Solution 2

One option to speed this up is to use wsaccel to do the masking, which is much faster due to a tight inner loop written in C. Here's the code for that:

from wsaccel.xormask import XorMaskerSimple def mask2(mask, data): return XorMaskerSimple(mask).process(data)

If you can install wsaccel on your system, then use that. There is no reason not to. Unfortunately, the project I'm working on needs to be able to run a websocket client on systems where it isn't always easy to install C extensions. But without some serious optimization, the pure-python solution is going to be painfully slow. Is there a faster solution than mask1 in pure-Python?

Solution 3

Turns out there is. The following is a rather clever solution from the aiohttp project which runs significantly faster:

native_byteorder = sys.byteorder def mask3(mask, data): datalen = len(data) data = int.from_bytes(data, native_byteorder) mask = int.from_bytes(mask * (datalen // 4) + mask[: datalen % 4], native_byteorder) return (data ^ mask).to_bytes(datalen, native_byteorder)

As awkward as it looks, this solution blows mask1 out of the water in terms of speed. Essentially it treats the websocket payload and mask as a very large number, does the XOR as a single operation, then turns the result back in to bytes. It's fast because the work happens in a single operation, rather than per-byte.

Solution 4

I might have been happy with that, if it worked with Python2.7. Unfortunately, int.from_bytes was introduced in Python3.2 and my code is intended to run on Python2.7 and 3.X. I needed another trick that runs as fast, but works on both major versions of Python.

This is what I came up with:

_XOR_TABLE = [bytes(a ^ b for a in range(256)) for b in range(256)] def mask4(mask, data): data_bytes = bytearray(data) a, b, c, d = (_XOR_TABLE[n] for n in mask) data_bytes[::4] = data_bytes[::4].translate(a) data_bytes[1::4] = data_bytes[1::4].translate(b) data_bytes[2::4] = data_bytes[2::4].translate(c) data_bytes[3::4] = data_bytes[3::4].translate(d) return bytes(data_bytes)

A quick profile indicated that mask4 was faster than mask3 and approaching the same speed as the C version. I really didn't expect such a big win.

The trick here is to pre-calculate the results of the XOR operation in a table. This table contains every combination of two bytes XORed together. So _XOR_TABLE[a][b] is equivalent to a ^ b for every possible value of bytes a and b.

We can now use bytearray.translate to replace payload bytes with a corresponding byte from the table . You can be forgiven if you've never seen (or forgotten) the translate method; it's always been on the Python2 string type and is inherited by bytearray objects.

The reason translate is called four times, is that it can only do a replace for a single row in the table, and we need to pick a row in the table for each byte in the mask. Slices are used to extract and replace the appropriate bytes for each byte of mask.

Speed Comparison

I did some profiling to compare the various versions of the masking function. The pretty pure-python version was so slow that it was impossible to make a meaningful comparison, so I've removed it from the following chart:

© 2017 Will McGugan

This graph was generated on my Macbook, which is way more powerful that the devices the code was written for.

The X axis is the number of bytes in the Websocket payload.

The Y axis is the time taken (milliseconds) to mask the payload.

The translate version is only slightly slower than the C version, which is pleasing. Although it is worth pointing out that up until around 2500 bytes, the from_bytes version is slightly faster than translate.

Conclusion?

The takeaway from this is that explicit inner loops in Python can be slow. If there is a built-in method that operates on a number of elements in a single call, then it will likely be faster. Even if that means doing additional work to prepare.

The caveat is that this level of micro-optimization it is rarely worth the effort. The final version would probably earn you a frowny face in a code review (if it wasn't accompanied by a comment explaining the reasoning behind it). In this instance however, the ugliness is probably justifiable.

The code for the masking functions (which also generates the above graph) is available here.

Categories: FLOSS Project Planets

Aten Design Group: Radios, Checkboxes, and Drupal’s Admin Interface

Planet Drupal - Wed, 2017-04-19 18:09

Custom styled form elements are a common thing to see in a design. That’s because default form styles vary visually from browser to browser and OS to OS. It makes sense that we’d want these elements styled consistently. Styling them is pretty straightforward, with the exception of select dropdowns which can be more complex. Recently, I ran into an unexpected problem when working on a site that needed a branded admin experience.

Styling Radio and Checkbox Buttons

There’s a simple method of styling radio buttons and checkboxes I’ve been using for a while. I first saw it from the people at Tuts+, and they provided this Pen demoing the technique. Briefly explained, we visually hide the input for our radios/checkboxes and draw a new one using :before and :after pseudo elements on the label element. CSS’ :checked selector allows us to toggle our styles based on if the input is checked or not. This technique relies on appropriately marked up inputs and labels, for example:

<div class=”form-element”> <input type=”checkbox” id=”click-me”> <label for=”click-me”>Click Me</label> </div>

Clicking the label (containing the fake checkbox styling) will toggle the state of the real checkbox that’s visually hidden.

Drupal’s Admin Interface

One thing I learned while working with some of Drupal’s admin interfaces is that they only supply the input, and not an accompanying label. This seemed especially true in tabled interfaces, where you’d check off rows of content and perform some action on the selected items. Since we’re hiding an input that doesn’t have a label to attach the visuals to, we just end up with a blank space. There were several options we had for how to address this issue.

1. Drop the Custom Styles

The simplest is to just rely on browser defaults for checkboxes and radios. It’s not a great option, but it is an affordable one for tight budgets.

2. Create the Missing Labels

This ended up being my first approach to fixing this, and became more akin to a game of Whack-a-Mole than I anticipated. After going through various preprocess functions, alters, and render functions I was still encountering inputs that were missing labels. Some I was never able to fully track down where the markup was coming from. Manually finding and fixing every missing label might be a viable solution if your website or application has only a handful of places you need to update. However this is not the most scalable solution, and if your product grows this can quickly become a financial black hole.

3. Create the Missing Labels… with Javascript

Instead of trying to find every place that creates a checkbox or radio on the server side, we could use Javascript to target every checkbox or radio input that is not followed by a label. From there, we just create the label element and insert it after the selected inputs. This is how that might look using jQuery, though it can also be done with Vanilla JS.

This is great, as it solves the problem for every input in one fell swoop. One downside here is the Javascript dependency. Should your Javascript not run for any reason, you’re still left with the original problem of missing inputs. Another is page rendering. User’s might be left with a janky experience as Javascript inserts these elements into the DOM.

4. Drop the Custom Styles… for Older Browsers

In the end, this was the solution that won out. Using CSS Feature Queries and CSS’ appearance property, we’re able to provide styled inputs for most modern browsers and then fall back to default styles in browsers that lack the support we need. This gives us our custom styles, without the scaling problem of #2, and the Javascript dependency of #3. The downside to this solution is that all versions of Internet Explorer and Firefox will use their browser defaults.

Firefox was a surprise to me, as the documentation says it supports appearance. However in practice what I got was a less appealing version of the browser default styles. Also surprisingly was by checking for only -webkit-appearance support, Edge still gets our custom styles applied. This all sat well with me for a working solution. Every team and project has it’s own constraints, so your mileage may vary.

Categories: FLOSS Project Planets

Reproducible builds folks: Reproducible Builds: week 103 in Stretch cycle

Planet Debian - Wed, 2017-04-19 17:00

Here's what happened in the Reproducible Builds effort between Sunday April 9 and Saturday April 15 2017:

Upcoming events

On April 26th Chris Lamb will give a talk at foss-north 2017 in Gothenburg, Sweden on Reproducible Builds.

Media coverage

Jake Edge wrote a summary of Vagrant Cascadian's talk on Reproducible Builds at LibrePlanet.

Toolchain development and fixes

Ximin Luo forwarded patches to GCC for BUILD_PATH_PREFIX_MAP support.

With this patch to backported to GCC-6, as well as a patched dpkg to set the environment variable, he scheduled ~3,300 packages that are unreproducible in unstable-amd64 but reproducible in testing-amd64 - because we vary the build path in the former but not latter case. Our infrastructure ran these in just under 3 days, and we reproduced ~1,700 extra packages.

This is about 6.5% of ~26,100 Debian source packages, and about 1/2 of the ones whose irreproducibility is due to build-path issues. Most of the rest are not related to GCC, such as things built by R, OCaml, Erlang, LLVM, PDF IDs, etc.

(The dip afterwards, in the graph linked above, is due to reverting back to an unpatched GCC-6, but we'll be rebasing the patch continually over the next few weeks so the graph should stay up.)

Packages reviewed and fixed, and bugs filed

Chris Lamb:

Chris West:

Reviews of unreproducible packages

38 package reviews have been added, 111 have been updated and 85 have been removed in this week, adding to our knowledge about identified issues.

6 issue types have been updated:

Added:

Updated:

Removed:

diffoscope development

Development continued in git on the experimental branch:

Chris Lamb:

  • Don't crash on invalid archives (#833697)
  • Tidy up some other code
Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

  • Chris Lamb (3)
  • Chris West (1)
Misc.

This week's edition was written by Ximin Luo, Chris Lamb & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Categories: FLOSS Project Planets

Friday Free Software Directory IRC meetup: April 21st starting at 12:00 p.m. EDT/16:00 UTC

FSF Blogs - Wed, 2017-04-19 15:25

Participate in supporting the Directory by adding new entries and updating existing ones. We will be on IRC in the #fsf channel on irc.freenode.org.

Tens of thousands of people visit directory.fsf.org each month to discover free software. Each entry in the Directory contains a wealth of useful information, from basic category and descriptions, to providing detailed info about version control, IRC channels, documentation, and licensing info that has been carefully checked by FSF staff and trained volunteers.

While the Directory has been and continues to be a great resource to the world over the past decade, it has the potential of being a resource of even greater value. But it needs your help!

After last weeks focus on financial software it seemed like a good chance this week to break its monopoly. This week we are going to work on games, and not just any kind of games. This week we'll be taking a big risk by focusing on table top ones. Although there are currently over thirty games listed, this section needs to grow big like a battleship. Come this Friday, we hope you'll deal yourself in and play a round with us.

If you are eager to help and you can't wait or are simply unable to make it onto IRC on Friday, our participation guide will provide you with all the information you need to get started on helping the Directory today! There are also weekly Directory Meetings pages that everyone is welcome to contribute to before, during, and after each meeting.

Categories: FLOSS Project Planets

Aten Design Group: Migrating Wordpress into Drupal 8

Planet Drupal - Wed, 2017-04-19 15:17

Quite a bit has changed for the Migrate module in Drupal 8: the primary module is part of core and some of the tools have been split into their own modules. Recently, we migrated a Wordpress site into Drupal 8 and this article will help guide you in that process. If you’re looking for information about Wordpress to Drupal 7 migrations, check out Joel Steidl’s article on that here.

At the time of writing this post, the migration modules are considered "experimental" so be aware of that as well. The module's location in core also means that all Drupal core modules also have migration-related code to help out with your Drupal upgrades. We used the WP Migrate module (Migrate Wordpress) as a starting point in bringing this content to Drupal.

This module will give you a good basis for migration, but it is missing a few things that you might want to consider:

  • It will create all vocabularies and taxonomies based on what is in Wordpress but you will need to add some code to connect the taxonomies with posts.
  • Also, it will not bring in featured images.
  • WP content might be using the "line break to paragraphs" functionality, which you need to account for either in your text format for posts or in the migration.

And if you are looking for information about Wordpress to Drupal 7 migrations, check out Joel Steidl's article on that here.

Taxonomy

There's code existing to pull in Wordpress's terms and vocabularies, but you will need to do some work to put them into the right fields with your posts. For this, I ended up taking a more efficient route by querying the source database in prepareRow():

<?php   // place in Posts.php prepareRow()   // get terms for this blog post $tags = $this->select('wp_term_relationships', 'r') ->join('wp_term_taxonomy', 't', 't.term_taxonomy_id=r.term_taxonomy_id') ->fields('r') ->condition('t.taxonomy', 'tags') ->condition('object_id', $row->getSourceProperty('id'))->execute(); $tags = $tags->fetchAll(); $tags = array_map(function($tag) { return intval($tag['term_taxonomy_id']); }, $tags); $row->setSourceProperty('tags', $tags);   // get categories for this blog post $category = $this->select('wp_term_relationships', 'r') ->join('wp_term_taxonomy', 't', 't.term_taxonomy_id=r.term_taxonomy_id') ->fields('r') ->condition('t.taxonomy', 'category') ->condition('object_id', $row->getSourceProperty('id'))->execute(); $category = $category->fetchAll(); $category = array_map(function($tag) { return intval($tag['term_taxonomy_id']); }, $category); $row->setSourceProperty('categories', $category);

And then I updated the migration template with those new values:

# add to the process section field_tags: tags field_category: tags Featured Images

Wordpress stores featured images as attachment posts and stores the relationship in the postmeta table. To bring these in as image fields, we need to make file entities in Drupal which means configuring a new migration.

First, create a migration template called wp_feature_images.yml. Note that I stole some of this from Drupal's core file module:

id: wp_feature_images label: Wordpress Feature Images migration_tags: - Wordpress migration_group: wordpress source: plugin: feature_images destination: plugin: entity:file process: filename: filename uri: uri status: plugin: default_value default_value: 1 # migration_dependencies: # required: # - wp_users

And then create a source plugin:

<?php /** * @file * Contains \Drupal\migrate_wordpress\Plugin\migrate\source\FeatureImages. */   namespace Drupal\migrate_wordpress\Plugin\migrate\source;   use Drupal\migrate\Row; use Drupal\migrate\Plugin\migrate\source\SqlBase; use Drupal\Core\File\FileSystemInterface; use Symfony\Component\DependencyInjection\ContainerInterface; use Drupal\migrate\Plugin\MigrationInterface; use Drupal\Core\State\StateInterface;   /** * Extract feature images from Wordpress database. * * @MigrateSource( * id = "feature_images" * ) */ class FeatureImages extends SqlBase {   public function __construct(array $configuration, $plugin_id, $plugin_definition, MigrationInterface $migration, StateInterface $state, FileSystemInterface $file_system) { parent::__construct($configuration, $plugin_id, $plugin_definition, $migration, $state); $this->fileSystem = $file_system; }   /** * {@inheritdoc} */ public static function create(ContainerInterface $container, array $configuration, $plugin_id, $plugin_definition, MigrationInterface $migration = NULL) { return new static( $configuration, $plugin_id, $plugin_definition, $migration, $container->get('state'), $container->get('file_system') ); }   /** * {@inheritdoc} */ public function query() { $query = $this ->select('wp_postmeta', 'm') ->fields('p', ['ID', 'guid']); $query->join('wp_posts', 'p', 'p.ID=m.meta_value'); $query ->condition('m.meta_key', '_thumbnail_id', '=') ->condition('p.post_type', 'attachment', '=') ->condition('p.guid', '', '<>') // this prevents some duplicates to get the count closer to even ->groupBy('ID, guid'); return $query; }   /** * {@inheritdoc} */ public function fields() { $fields = array( 'ID' => $this->t('The file ID.'), 'guid' => $this->t('The file path'), ); return $fields; }   /** * {@inheritdoc} */ public function prepareRow(Row $row) { $url = $row->getSourceProperty('guid'); $parsed_url = parse_url($url); $filename = basename($parsed_url['path']); $row->setSourceProperty('filename', $filename); $public_path = 'public://' . $parsed_url['path']; $row->setSourceProperty('uri', $public_path);   // download the file if it does not exist if (!file_exists($public_path)) { $public_dirname = dirname($public_path);   // create directories if necessary if (!file_exists($public_dirname)) { $this->fileSystem->mkdir($public_dirname, 0775, TRUE); }   // try to download it $copied = @copy($url, $public_path); if (!$copied) { return FALSE; } } return parent::prepareRow($row); }   /** * {@inheritdoc} */ public function bundleMigrationRequired() { return FALSE; }   /** * {@inheritdoc} */ public function getIds() { return array( 'ID' => array( 'type' => 'integer', 'alias' => 'p', ), ); }   }

In Migrate, the template defines what source, processing, and fields are created. The source plugin is used by that migration to allow you to specify what is created. The source plugin above will get the feature images for posts, but also try and download the image into Drupal's files directory.

You can add this as a dependency for the wp_posts migration. A word of warning though: if one migration (Migration A) depends on a different migration (Migration B), all of the content from A must be migrated before B can be run. If there are images that cannot be resolved for some reason (maybe leftover DB references after an image or post is deleted), this might stop the migration because the dependency cannot be resolved.

And finally, you will also need to add "wp_feature_images" to your manifest_wordpress.yml before running the migration.

Converting content

So far we have updated migration source plugins, but there are also process plugins, which can be used to change row values. As mentioned, the WP content often uses the autop filter to create paragraph/line breaks automatically so we need to change those to HTML for Drupal. (You can also just use this functionality in your text format and skip this step if having this on will not cause issues with other content)

First, create a "src/Plugin/migrate/process" directory if one does not exist in the module and add this processor:

<?php   namespace Drupal\migrate_wordpress\Plugin\migrate\process;   use Drupal\migrate\MigrateExecutableInterface; use Drupal\migrate\ProcessPluginBase; use Drupal\migrate\Row;   /** * Apply the automatic paragraph filter to content * * @MigrateProcessPlugin( * id = "wp_content" * ) */ class WpContent extends ProcessPluginBase {   /** * {@inheritdoc} * * Split the 'administer nodes' permission from 'access content overview'. */ public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) { return _filter_autop($value); }   }

Then, update the "process" section of "wp_posts.yml" to include this processor:

'body/value': plugin: wp_content source: post_content

All of this should put you on the road to getting Wordpress content migrated into a Drupal 8 site, although you’ll probably have to adjust code to your specific circumstances along the way.

Categories: FLOSS Project Planets

Peter Bengtsson: Best practice with retries with requests

Planet Python - Wed, 2017-04-19 12:17
I have a lot of code that does response = requests.get(...) in various Python projects. This is nice and simple but the problem is that networks are unreliable. So it's a good idea to wrap these network calls with retries. Here's one such implementation.
Categories: FLOSS Project Planets

Acquia Developer Center Blog: Acquia in action at DrupalCon Baltimore!

Planet Drupal - Wed, 2017-04-19 12:01

If you’re coming to DrupalCon Baltimore and you’re curious about Acquia, there are a couple of ways to meet the company and see what we’re about beyond the marketing and sales efforts that get directed at potential clients. One great way is to come to our sessions!

Tags: acquia drupal planet
Categories: FLOSS Project Planets

ThinkShout: Meet the ThinkShout Team at DrupalCon Baltimore

Planet Drupal - Wed, 2017-04-19 12:00

We’re packing our bags for Baltimore and polishing up our slide decks for DrupalCon! We’re so excited to join the Drupal community for a full week of Drupal-y things. We’ve got some great content planned for this year’s conference, and we’re very excited to share it with you all - here’s what you need to know:

Exhibit Hall

The ThinkShout Headquarters this year is booth 432! We’ll be giving away free t-shirts and raffling off an Amazon Echo. You can enter to win for the low, low price of one business card. If you have any questions about our work, current available job opportunities, or what the weather’s like in Portland (spoiler: it’s probably raining), stop by - we’d love to chat with you!

ThinkShout Sessions

The ThinkShout team has two sessions in the DrupalCon agenda this year. We’re also very excited to be leading a discussion in our first DrupalCon Nonprofit Summit. Take a look at our lineup and mark your calendars

Rapid Response Campaigns & Digital Tools” - Monday (4/24), 12:30 - 1:15pm, Nonprofit Summit

The news cycle doesn’t stop, and your website must help you respond to emergencies, not act as a barrier. Drupal can help you react quickly, in concert with your other channels, to turn current events into opportunities to spread your message and further your mission. In this breakout session, Brett Meyer and Lev Tsypin will talk about the tools you have at your disposal in Drupal, scenarios that call for rapid response solutions and how to implement them, and strategies that will help you turn these situations into lasting engagement with your constituents.

Demystifying Rendered Content in Drupal 8 Twig Files” - Tuesday (4/25), 3:45 - 4:45pm

Amy Vaillancourt-Sals is going to show you the ins and outs of Twig! Twig is a robust and elegant template engine for PHP. It’s lightweight, fairly quick to pick up, very readable, and it grants users ultimate control over the markup, including wrapping elements and rendering exactly the output you need. In this session, you’ll learn about the debugging process of sorting through twig variables, using xdebug in PHPStorm, the other helpful debugging tools at your disposal, plus common patterns Amy found helpful for rendering content in twig files.

Content Strategy in Popular Culture, Part Deux” - Thursday (4/27), 10:45 - 11:45am

Brett Meyer’s got a sequel to his session from DrupalCon New Orleans. Another year, another array of pop culture obsessions to examine and apply to the work we do. By exploring how crucial aspects of content strategy play out in movies, music, comic books, and video games, we’ll continue to expand the palette of language we can use to explain and convince more people about the importance of content strategy online, and ensure they understand that it’s not just vital, but fun as well.

Let’s Chat

If you’d like to schedule some time to chat with us in advance, drop us a line via our contact form. We’d be happy to meet up with you in Baltimore!

Categories: FLOSS Project Planets

DataCamp: New Course: Deep Learning in Python (first Keras 2.0 online course!)

Planet Python - Wed, 2017-04-19 11:49

Hello there! We have a special course released today: Deep Learning in Python by Dan Becker. This happens to be one of the first online interactive course providing instructions in keras 2.0, which now supports Tensorflow integration and this new API will be consistent in the coming years. So you've come to the right deep learning course.

About the course:

Artificial neural networks (ANN) are a biologically-inspired set of models that facilitate computers learning from observed data. Deep learning is a set of algorithms that use especially powerful neural networks. It is one of the hottest fields in data science, and most state-of-the-art results in robotics, image recognition and artificial intelligence (including the famous AlphaGo) use deep learning. In this course, you'll gain hands-on, practical knowledge of how to use neural networks and deep learning with Keras 2.0, the latest version of a cutting edge library for deep learning in Python.

Take me to Chapter 1!

Deep Learning in Python features interactive exercises that combine high-quality video, in-browser coding, and gamification for an engaging learning experience that will make you a master in deep learning in Python!

What you'll learn:

In the first chapter, you'll become familiar with the fundamental concepts and terminology used in deep learning, and understand why deep learning techniques are so powerful today. You'll build simple neural networks yourself and generate predictions with them. You can take this chapter here for free.

In chapter 2, you'll learn how to optimize the predictions generated by your neural networks. You'll do this using a method called backward propagation, which is one of the most important techniques in deep learning. Understanding how it works will give you a strong foundation to build from in the second half of the course.

In the third chapter, you'll use the keras library to build deep learning models for both regression as well as classification! You'll learn about the Specify-Compile-Fit workflow that you can use to make predictions and by the end of this chapter, you'll have all the tools necessary to build deep neural networks!

Finally, you'll learn how to optimize your deep learning models in keras. You'll learn how to validate your models, understand the concept of model capacity, and experiment with wider and deeper networks. Enjoy!

Dive into Deep Learning Today

Categories: FLOSS Project Planets

Colm O hEigeartaigh: Securing Apache Hadoop Distributed File System (HDFS) - part I

Planet Apache - Wed, 2017-04-19 11:49
Last year, I wrote a series of articles on securing Apache Kafka using Apache Ranger and Apache Sentry. In this series of posts I will look at how to secure the Apache Hadoop Distributed File System (HDFS) using Ranger and Sentry, such that only authorized users can access data stored in it. In this post we will look at a very basic way of installing Apache Hadoop and accessing some data stored in HDFS. Then we will look at how to authorize access to the data stored in HDFS using POSIX permissions and ACLs.

1) Installing Apache Hadoop

The first step is to download and extract Apache Hadoop. This tutorial uses version 2.7.3. The next step is to configure Apache Hadoop as a single node cluster so that we can easily get it up and running on a local machine. You will need to follow the steps outlined in the previous link to install ssh + pdsh. If you can't log in to localhost without a password ("ssh localhost") then you need to follow the instructions given in the link about setting up passphraseless ssh.

In addition, we want to run Apache Hadoop in pseudo-distributed mode, where each Hadoop daemon runs as a separate Java process. Edit 'etc/hadoop/core-site.xml' and add:
Next edit 'etc/hadoop/hdfs-site.xml' and add:

Make sure that the JAVA_HOME variable in 'etc/hadoop/hadoop-env.sh' is correct, and then format the filesystem and start Hadoop via:
  • bin/hdfs namenode -format
  • sbin/start-dfs.sh
To confirm that everything is working correctly, you can open "http://localhost:50090" and check on the status of the cluster there. Once Hadoop has started then upload and then access some data to HDFS:
  • bin/hadoop fs -mkdir /data
  • bin/hadoop fs -put LICENSE.txt /data
  • bin/hadoop fs -ls /data
  • bin/hadoop fs -cat /data/*
2) Securing HDFS using POSIX Permissions

We've seen how to access some data stored in HDFS via the command line. Now how can we create some authorization policies to restrict how to access this data? Well the simplest way is to use the standard POSIX Permissions. If we look at the /data directory we see that it has the following permissions "-rw-r--r--", which means other users can read the LICENSE file stored there. Remove access to other users apart from the owner via:
  • bin/hadoop fs -chmod og-r /data
Now create a test user called "alice" on your system and try to access the LICENSE we uploaded above via:
  • sudo -u alice bin/hadoop fs -cat /data/*
You will see an error that says "cat: Permission denied: user=alice, access=READ_EXECUTE".

3) Securing HDFS using ACLs

Securing access to data stored in HDFS via POSIX permissions works fine, however it does not allow you for example to specify fine-grained permissions for users other than the file owner. What if we want to allow "alice" from the previous section to read the file but not "bob"? We can achieve this via Hadoop ACLs. To enable ACLs, we will need to add a property called "dfs.namenode.acls.enabled" with value "true" to 'etc/hadoop/hdfs-site.xml' + re-start HDFS.

We can grant read access to 'alice' via:
  • bin/hadoop fs -setfacl -m user:alice:r-- /data/*
  • bin/hadoop fs -setfacl -m user:alice:r-x /data
To check to see the new ACLs associated with LICENSE.txt do:
  • bin/hadoop fs -getfacl /data/LICENSE.txt
In addition to the owner, we now have the ACL "user:alice:r--". Now we can read the data as "alice". However another user "bob" cannot read the data. To avoid confusion with future blog posts on securing HDFS, we will now remove the ACLs we added via:
  • bin/hadoop fs -setfacl -b /data
  • bin/hadoop fs -setfacl -b /data/LICENSE.txt
Categories: FLOSS Project Planets

Lullabot: Cross-Pollination between Drupal and WordPress

Planet Drupal - Wed, 2017-04-19 11:27

WordPress controls a whopping 27% of the CMS market share on the web. Although it grew out of a blogging platform, it can now can handle advanced functionality similar to Drupal and is a major (yet friendly) competitor to Drupal. Like Drupal, it’s open source and has an amazing community. Both communities learn from each other, but there is still much more to share between the two platforms.

Recently I had the opportunity to speak at WordCamp Miami on the topic of Drupal. WordCamp Miami is one of the larger WordCamps in the world, with a sold-out attendance of approximately 800 people.

undefined What makes Drupal so great?

Drupal commands somewhere in the neighborhood of 2% of the CMS market share of the web. It makes complex data models easy, and much of this can be accomplished through the user interface. It has very robust APIs and enables modules to share one another’s APIs. Taken together, you can develop very complex functionality with little to no custom code.

So, what can WordPress take away from Drupal? Developer Experience: More and better APIs included in WordPress Core

The WordPress plugin ecosystem could dramatically benefit from standardizing API’s in core.

  • Something analogous to Drupal’s Render API and Form API would make it possible for WordPress plugins to standardize and integrate their markup, which in turn would allow plugins to work together without stepping on each other’s toes.
  • WordPress could benefit from a way to create a custom post type in the core UI. Drupal has this functionality out the the box. WordPress has the functionality available, but only to the developer. This results in WordPress site builders searching for very specific plugins that create a specific post type, and hoping it does what they want.
  • WordPress already has plugins similar to Drupal’s Field API. Plugins such as Advanced Custom Fields and CMB2 go along way to allowing WordPress developers to easily create custom fields. Integrating something similar to this into WordPress core would allow plugin developers to count on a stable API and easily extend it.
  • An API for plugins to set dependencies on other plugins is something that Drupal has done since its beginning. It enables mini-ecosystems to develop that extend more complex modules. In Drupal, we see a module ecosystems built around Views, Fields, Commerce, Organic Groups, and more. WordPress would benefit greatly from this.
  • A go-to solution for custom query/list building would be wonderful for WordPress. Drupal has Views, but WordPress does not, so site builders end up using plugins that create very specific queries with output according to a very specific need. When a user needs to make a list of “popular posts,” they end up looking through multiple plugins dedicated to this single task.

A potential issue with including new APIs in WordPress core is that it could possibly break WordPress’ commitment to backwards compatibility, and would also dramatically affect their plugin ecosystem (much of this functionality is for sale right now).

WordPress Security Improvements

WordPress has a much-maligned security reputation. Because it commands a significant portion of the web, it’s a large attack vector. WordPress sites are also frequently set up by non-technical users, who don’t have the experience to keep it (and all of its plugins) updated, and/or lock down the site properly.

That being said, WordPress has some low-hanging fruit that would go a long way to help the platform’s reputation.

  • Brute force password protection (flood control) would prevent bots from repeatedly connecting to wp-login.php. How often do you see attempted connections to wp-login.php in your server logs?.
  • Raise the minimum supported PHP version from 5.2 (which does not receive security updates). Various WordPress plugins are already doing this, and there’s also talk about changing the ‘recommended’ version of PHP to 7.0.
  • An official public mailing list for all WordPress core and plugin vulnerabilities would be an easy way to alert developers to potential security issues. Note that there are third-party vendors that offer mailing lists like this.
Why is WordPress’ market share so large?

Easy: It can be set up and operated by non-developers—and there are a lot more non-developers than developers! Installing both Drupal and WordPress is dead simple, but once you’re up and running, WordPress becomes much easier.

Case in Point: Changing Your Site's Appearance

Changing what your site looks like is often the first thing that a new owner will want to do. With WordPress, you go to Appearance > Themes > Add New, and can easily browse themes from within your admin UI. To enable the theme, click Install, then click Activate.

undefined

With Drupal, you go to Appearance, but you only see core themes that are installed. If you happen to look at the top text, you read in small text that "alternative themes are available." Below that there is a button to “Install a New Theme.”

undefined

Clicking the button takes you to a page where you can either 1) paste in a URL to the tarball/zip, or upload a downloaded tarball/zip. You still have to know how to to download the zip or tarball, and where to extract it, and then browse to appearance, and enable the theme.

So it goes with Drupal. The same process goes with modules and more. Drupal makes things much more difficult. 

So, what can Drupal learn from WordPress?

To continue to grow, Drupal needs to enable non-developers. New non-developers can eventually turn into developers, and will become “new blood” in the community. Here’s how Drupal can do it:

  • A built in theme and module browser would do wonders for enabling users to discover new functionality and ways to change their site’s appearance. A working attempt at this is the Project Browser module (available only for Drupal 7). The catch 22 of this is that you have to download this the old-fashioned way in order to use it.
  • The ability to download vetted install profiles during the Drupal installation process would be amazing. This would go a long way to enable the “casual explorers," and show them the power of Drupal. A discussion of this can be found here.
  • Automatic security updates is a feature that would be used by many smaller sites. Projects have been steered toward WordPress specifically because smaller clients don’t have the budget to pay developers to keep up with updates. This feature has been conceptually signed off on by Drupal’s core committers, but significant work has yet to be done.
Mitigating Security Risks

The downside for this functionality is that Drupal would need to have a writable file-system, which at it’s core, is less secure. Whether that balances out with automatic updates is debatable.

Automatic security updates and theme/module installation would not have to be enabled out of the box. The functionality could be provided in core modules that could be enabled only when needed.

What has Drupal already learned from WordPress?

Cross-pollination has already been happening for a while. Let’s take a look at what the Drupal community has already, or is in the process of, implementing:

  • Semantic versioning is one of the most important changes in Drupal 8. With semantic versioning, bug fixes and new features can be added at a regular cadence. Prior to this, Drupal developers had to wait a few years for the next major version. WordPress has been doing this for a long time.
  • A better authoring experience is something that Drupal has been working on for years (remember when there was no admin theme?). With Drupal 8, the default authoring experience is finally on par with WordPress and even surpasses it in many areas.
  • Media management is the ability to upload images and video, and easily reference them from multiple pieces of content. There’s currently a media initiative to finally put this functionality in core.
  • Easier major version upgrades is something that WordPress has been doing since it’s inception.

Drupal has traditionally required significant development work in between major versions. That however, is changing. In a recent blog post, the lead of the Drupal project, Dries Buytaert said,

Updating from Drupal 8's latest version to Drupal 9.0.0 should be as easy as updating between minor versions of Drupal 8.

This is a very big deal, as it drastically limits the technical debt of Drupal as new versions of Drupal appear.

Conclusion

Drupal and WordPress have extremely intelligent people contributing to their respective platforms. And, because of the GPL, both platforms have the opportunity to use vetted and proven approaches that are shortcuts to increased usability and usage. This, in turn, can lead to a more open (and usable) web.

Special thanks to Jonathan Daggerhart, John TuckerMatthew Tift, and Juampy NR for reviewing and contributing to this article.

undefined
Categories: FLOSS Project Planets

Lior Kaplan: Open source @ Midburn, the Israeli burning man

Planet Debian - Wed, 2017-04-19 11:00

This year I decided to participate in Midburn, the Israeli version of burning man. Whiling thinking of doing something different from my usual habit, I found myself with volunteering in the midburn IT department and getting a task to make it an open source project. Back into my comfort zone, while trying to escape it.

I found a community of volunteers from the Israeli high tech scene who work together for building the infrastructure for Midburn. In many ways, it’s already an open source community by the way it works. One critical and formal fact was lacking, and that’s the license for the code. After some discussion we decided on using Apache License 2.0 and I started the process of changing the license, taking it seriously, making sure it goes “by the rules”.

Our code is available on GitHub at https://github.com/Midburn/. And while it still need to be more tidy, I prefer the release early and often approach. The main idea we want to bring to the Burn infrastructure is using Spark as a database and have already began talking with parallel teams of other burn events. I’ll follow up on our technological agenda / vision. In the mean while, you are more than welcome to comment on the code or join one of the teams (e.g. volunteers module to organize who does which shift during the event).

 

 


Filed under: Israeli Community
Categories: FLOSS Project Planets

Valuebound: How to create custom Form with CRUD(Create, Delete, Update) operations in Drupal 8

Planet Drupal - Wed, 2017-04-19 10:38

Custom form with CRUD Operations is basically building the form with different fields like UID, Name, Mobile Number, Address, Email id etc. The CRUD operations for these fields is Deleting the value of the fields from the database or updating the existing value with new value, and printing the updated value. 

How To Create a Custom form with CRUD Operations?

To create a custom form with CRUD Operations, we need to follow the following folder structure as shown in the image. To perform the same operation of crud in drupal we need to follow the following folder structure.

Categories: FLOSS Project Planets

Chromatic: Chromatic at DrupalCon Baltimore

Planet Drupal - Wed, 2017-04-19 09:00

All the ways Chromatic will be representing at DrupalCon Baltimore next week.

Categories: FLOSS Project Planets

Dave Hall Consulting: Drupal, We Need To Talk

Planet Drupal - Wed, 2017-04-19 08:00

Drupal has a problem. No, not that problem.

We live in a post peak Drupal world. Drupal peaked some time during the Drupal 8 development cycle. I’ve had conversations with quite a few people who feel that we’ve lost momentum. DrupalCon attendances peaked in 2014, Google search impressions haven’t returned to their 2009 level, core downloads have trended down since 2015. We need to accept this and talk about what it means for the future of Drupal.

Technically Drupal 8 is impressive. Unfortunately the uptake has been very slow. A factor in this slow uptake is that from a developer's perspective, Drupal 8 is a new application. The upgrade path from Drupal 7 to 8 is another factor.

In the five years Drupal 8 was being developed there was a fundamental shift in software architecture. During this time we witnessed the rise of microservices. Drupal is a monolithic application that tries to do everything. Don't worry this isn't trying to rekindle the smallcore debate from last decade.

Today it is more common to see an application that is built using a handful of Laravel micro services, a couple of golang services and one built with nodejs. These applications often have multiple frontends; web (react, vuejs etc), mobile apps and an API. This is more effort to build out, but it likely to be less effort maintaining it long term.

I have heard so many excuses for why Drupal 8 adoption is so slow. After a year I think it is safe to say the community is in denial. Drupal 8 won't be as popular as D7.

Why isn't this being talked about publicly? Is it because there is a commercial interest in perpetuating the myth? Are the businesses built on offering Drupal services worried about scaring away customers? Adobe, Sitecore and others would point to such blog posts to attack Drupal. Sure, admitting we have a problem could cause some short term pain. But if we don't have the conversation we will go the way of Joomla; an irrelevant product that continues its slow decline.

Drupal needs to decide what is its future. The community is full of smart people, we should be talking about the future. This needs to be a public conversation, not something that is discussed in small groups in dark corners.

I don't think we will ever see Drupal become a collection of microservices, but I do think we need to become more modular. It is time for Drupal to pivot. I think we need to cut features and decouple the components. I think it is time for us to get back to our roots, but modernise at the same time.

Drupal has always been a content management system. It does not need to be a content delivery system. This goes beyond "Decoupled (Headless) Drupal". Drupal should become a "content hub" with pluggable workflows for creating and managing that content.

We should adopt the unix approach, do one thing and do it well. This approach would allow Drupal to be "just another service" that compliments the application.

What do you think is needed to arrest the decline of Drupal? What should Drupal 9 look like? Let's have the conversation.

Categories: FLOSS Project Planets

Jan Materne: How to teach Sonar to find new bug?

Planet Apache - Wed, 2017-04-19 06:53

How to teach Sonar to find new bug?

Background

A few days ago my Jenkins instance gave me
the regurlar hint for updates. So I checked the changelog for changes which are
interesting to me. One of them hit my eye: Jenkins 2.53 – „GC Performance:
Avoid using FileInputStream and FileOutputStream in the core codebase.
“ I
read the two tickets (for Jenkins and the JDK itself) and was
surprised. I hadn’t knew that.

Some days later I regognized a longer
blog
about that by CloudBees on DZone.
Also interesting.

During thinking about changing the „new FIS/FOS“
to something better in the open source projects I am working on (Apache Ant, Apache Commons) – Stefan
was faster
.

Categories: FLOSS Project Planets

Code Positive: Why Drupal?

Planet Drupal - Wed, 2017-04-19 06:13

Drupal is more than a pretty face - it's safe as houses, lets you fit in, and offers bells, whistles.....and maybe bagpipes!

 

 

Categories: FLOSS Project Planets
Syndicate content