FLOSS Project Planets

Lisandro Damián Nicanor Pérez Meyer: Developing an nrf51822 based embedded device with Qt Creator and Debian

Planet Debian - Wed, 2017-02-22 08:18
I'm currently developing an nRF51822-based embedded device. Being one the Qt/Qt Creator maintainers in Debian I would of course try to use it for the development. Turns out it works pretty good... with some caveats.

There are already two quite interesting blog posts about using Qt Creator on MAC and on Windows, so I will not repeat the basics, as they are there. Both use qbs, but I managed to use CMake.

Instead I'll add some tips on the stuff that I needed to solve in order to make this happen on current Debian Sid.

  • The required toolchain is already in Debian, just install binutils-arm-none-eabi, gcc-arm-none-eabi and gdb-arm-none-eabi.
  • You will not find arm-none-eabi-gdb-py on the gdb-arm-none-eabi package. Fear not, the provided gdb binary is compiled against python so it will work.
  • To enable proper debugging be sure to follow this flag setup. If you are using CMake like in this example be sure to modify CMake/toolchain_gcc.cmake as necessary.
  • In Qt Creator you might find that, while try to run or debug your app, you are greated with a message box that says "Cannot debug: Local executable is not set." Just go to Projects →Run and change "Run configuration" until you get a valid path (ie, a path to the .elf or .out file) in the "Executable" field.

Categories: FLOSS Project Planets

Amazee Labs: This was Drupal Mountain Camp

Planet Drupal - Wed, 2017-02-22 08:14
This was Drupal Mountain Camp

From 16-19 February, the first Drupal Mountain Camp took place in Davos, Switzerland. A very diverse crowd of 135 attendees from, 17 different countries, came together to share the latest and greatest in Drupal 8 development, as well as case studies from Swiss Drupal vendors.

Josef Dabernig Wed, 02/22/2017 - 14:14

When we started organizing Drupal Mountain Camp in the summer of 2016, it was hard to predict how much interest it would attract and how many people would join for the camp. By reaching out to the local and international Drupal ecosystem we were excited to get so many people to attend from all around the world including Australia, India, and the US.

As a team of a dozen organizers; we split up the tasks, like setting up the venue, registration, social media, room monitoring and much more. It was great seeing that we were able to split the workload across the entire team and keep it well balanced.

We are very thankful for 30 different speakers who travelled from afar and worked hard to share their expertise with the crowd. As a program organizer I might be biased, but I truly believe that the schedule was packed with great content :)

In addition to the sessions, we also provided free workshop trainings to help spread some more Drupal love.

We took all the speakers up to the mountain for Switzerland's most popular dish, cheese fondue, to say thank you for their sessions and inputs.

With Drupal Mountain Camp we wanted to set a theme that would not only excite attendees with Swiss quality sessions but also create a welcoming experience for everyone. On top of our Code of Conduct, we organized various social activities that would allow attendees to experience Switzerland, snow and the mountains.  

Sprints are an essential way to get started with contributing to Drupal. At Drupal Mountain Camp, we organized a First-time sprinter workshop and had Sprint rooms from Thursday until Sunday with many sprinters collaborating.

For our hosting company amazee.io, Drupal Mountain Camp was a great opportunity to demonstrate our docker based development environment and scalable cluster stack using a set of raspberry pies.

And of course, we ended the conference with skiing and snowboarding at the Swiss mountains :)

Pictures from the camp: selection and all. Curious about the next Drupal Mountain Camp? Follow us on twitter to stay on top and see you at the next event.

Categories: FLOSS Project Planets

Enrico Zini: staticsite news: github mode and post series

Planet Debian - Wed, 2017-02-22 08:10
GitHub mode

Tobias Gruetzmacher implemented GitHub mode for staticsite.

Although GitHub now has a similar site rendering mode, it doesn't give you a live preview: if you run ssite serve on a GitHub project you will get a live preview of README.md and the project documentation.

Post series

I have added support for post series, that allow you to easily interlink posts with previous/next links.

You can see it in action on links and on An Italian song a day, an ongoing series that is currently each day posting a link to an Italian song.

Categories: FLOSS Project Planets

Caktus Consulting Group: Python type annotations

Planet Python - Wed, 2017-02-22 08:00

When it comes to programming, I have a belt and suspenders philosophy. Anything that can help me avoid errors early is worth looking into.

The type annotation support that's been gradually added to Python is a good example. Here's how it works and how it can be helpful.


The first important point is that the new type annotation support has no effect at runtime. Adding type annotations in your code has no risk of causing new runtime errors: Python is not going to do any additional type-checking while running.

Instead, you'll be running separate tools to type-check your programs statically during development. I say "separate tools" because there's no official Python type checking tool, but there are several third-party tools available.

So, if you chose to use the mypy tool, you might run:

$ mypy my_code.py

and it might warn you that a function that was annotated as expecting string arguments was going to be called with an integer.

Of course, for this to work, you have to be able to add information to your code to let the tools know what types are expected. We do this by adding "annotations" to our code.

One approach is to put the annotations in specially-formatted comments. The obvious advantage is that you can do this in any version of Python, since it doesn't require any changes to the Python syntax. The disadvantages are the difficulties in writing these things correctly, and the coincident difficulties in parsing them for the tools.

To help with this, Python 3.0 added support for adding annotations to functions (PEP-3107), though without specifying any semantics for the annotations. Python 3.6 adds support for annotations on variables (PEP-526).

Two additional PEPs, PEP-483 and PEP-484, define how annotations can be used for type-checking.

Since I try to write all new code in Python 3, I won't say any more about putting annotations in comments.

Getting started

Enough background, let's see what all this looks like.

Python 3.6 was just released, so I’ll be using it. I'll start with a new virtual environment, and install the type-checking tool mypy (whose package name is mypy-lang).:

$ virtualenv -p $(which python3.6) try_types $ . try_types/bin/activate $ pip install mypy-lang

Let's see how we might use this when writing some basic string functions. Suppose we're looking for a substring inside a longer string. We might start with:

def search_for(needle, haystack): offset = haystack.find(needle) return offset

If we were to call this with anything that's not text, we'd consider it an error. To help us avoid that, let's annotate the arguments:

def search_for(needle: str, haystack: str): offset = haystack.find(needle) return offset

Does Python care about this?:

$ python search1.py $

Python is happy with it. There's not much yet for mypy to check, but let's try it:

$ mypy search1.py $

In both cases, no output means everything is okay.

(Aside: mypy uses information from the files and directories on its command line plus all packages they import, but it only does type-checking on the files and directories on its command line.)

So far, so good. Now, let's call our function with a bad argument by adding this at the end:

search_for(12, "my string")

If we tried to run this, it wouldn't work:

$ python search2.py Traceback (most recent call last): File "search2.py", line 4, in <module> search_for(12, "my string") File "search2.py", line 2, in search_for offset = haystack.find(needle) TypeError: must be str, not int

In a more complicated program, we might not have run that line of code until sometime when it would be a real problem, and so wouldn't have known it was going to fail. Instead, let's check the code immediately:

$ mypy search2.py search2.py:4: error: Argument 1 to "search_for" has incompatible type "int"; expected "str"

Mypy spotted the problem for us and explained exactly what was wrong and where.

We can also indicate the return type of our function:

def search_for(needle: str, haystack: str) -> str: offset = haystack.find(needle) return offset

and ask mypy to check it:

$ mypy search3.py search3.py: note: In function "search_for": search3.py:3: error: Incompatible return value type (got "int", expected "str")

Oops, we're actually returning an integer but we said we were going to return a string, and mypy was smart enough to work that out. Let's fix that:

def search_for(needle: str, haystack: str) -> int: offset = haystack.find(needle) return offset

And see if it checks out:

$ mypy search4.py $

Now, maybe later on we forget just how our function works, and try to use the return value as a string:

x = len(search_for('the', 'in the string'))

Mypy will catch this for us:

$ mypy search5.py search5.py:5: error: Argument 1 to "len" has incompatible type "int"; expected "Sized"

We can't call len() on an integer. Mypy wants something of type Sized -- what's that?

More complicated types

The built-in types will only take us so far, so Python 3.5 added the typing module, which both gives us a bunch of new names for types, and tools to build our own types.

In this case, typing.Sized represents anything with a __len__ method, which is the only kind of thing we can call len() on.

Let's write a new function that'll return a list of the offsets of all of the instances of some string in another string. Here it is:

from typing import List def multisearch(needle: str, haystack: str) -> List[int]: # Not necessarily the most efficient implementation offset = haystack.find(needle) if offset == -1: return [] return [offset] + multisearch(needle, haystack[offset+1:])

Look at the return type: List[int]. You can define a new type, a list of a particular type of elements, by saying List and then adding the element type in square brackets.

There are a number of these - e.g. Dict[keytype, valuetype] - but I'll let you read the documentation to find these as you need them.

mypy passed the code above, but suppose we had accidentally had it return None when there were no matches:

def multisearch(needle: str, haystack: str) -> List[int]: # Not necessarily the most efficient implementation offset = haystack.find(needle) if offset == -1: return None return [offset] + multisearch(needle, haystack[offset+1:])

mypy should spot that there's a case where we don't return a list of integers, like this:

$ mypy search6.py $

Uh-oh - why didn't it spot the problem here? It turns out that by default, mypy considers None compatible with everything. To my mind, that's wrong, but luckily there's an option to change that behavior:

$ mypy --strict-optional search6.py search6.py: note: In function "multisearch": search6.py:7: error: Incompatible return value type (got None, expected List[int])

I shouldn't have to remember to add that to the command line every time, though, so let's put it in a configuration file just once. Create mypy.ini in the current directory and put in:

[mypy] strict_optional = True

And now:

$ mypy search6.py search6.py: note: In function "multisearch": search6.py:7: error: Incompatible return value type (got None, expected List[int])

But speaking of None, it's not uncommon to have functions that can either return a value or None. We might change our search_for method to return None if it doesn't find the string, instead of -1:

def search_for(needle: str, haystack: str) -> int: offset = haystack.find(needle) if offset == -1: return None else: return offset

But now we don't always return an int and mypy will rightly complain:

$ mypy search7.py search7.py: note: In function "search_for": search7.py:4: error: Incompatible return value type (got None, expected "int")

When a method can return different types, we can annotate it with a Union type:

from typing import Union def search_for(needle: str, haystack: str) -> Union[int, None]: offset = haystack.find(needle) if offset == -1: return None else: return offset

There's also a shortcut, Optional, for the common case of a value being either some type or None:

from typing import Optional def search_for(needle: str, haystack: str) -> Optional[int]: offset = haystack.find(needle) if offset == -1: return None else: return offset Wrapping up

I've barely touched the surface, but you get the idea.

One nice thing is that the Python libraries are all annotated for us already. You might have noticed above that mypy knew that calling find on a str returns an int - that's because str.find is already annotated. So you can get some benefit just by calling mypy on your code without annotating anything at all -- mypy might spot some misuses of the libraries for you.

For more reading:

Categories: FLOSS Project Planets

DataCamp: Pandas Cheat Sheet for Data Science in Python

Planet Python - Wed, 2017-02-22 07:52

The Pandas library is one of the most preferred tools for data scientists to do data manipulation and analysis, next to matplotlib for data visualization and NumPy, the fundamental library for scientific computing in Python on which Pandas was built. 

The fast, flexible, and expressive Pandas data structures are designed to make real-world data analysis significantly easier, but this might not be immediately the case for those who are just getting started with it. Exactly because there is so much functionality built into this package that the options are overwhelming.

That's where this Pandas cheat sheet might come in handy. 

It's a quick guide through the basics of Pandas that you will need to get started on wrangling your data with Python. 

As such, you can use it as a handy reference if you are just beginning their data science journey with Pandas or, for those of you who already haven't started yet, you can just use it as a guide to make it easier to learn about and use it. 

The Pandas cheat sheet will guide you through the basics of the Pandas library, going from the data structures to I/O, selection, dropping indices or columns, sorting and ranking, retrieving basic information of the data structures you're working with to applying functions and data alignment.

In short, everything that you need to kickstart your data science learning with Python!

Do you want to learn more? Start the Intermediate Python For Data Science course for free now or try out our Pandas DataFrame tutorial

Also, don't miss out on our Bokeh cheat sheet for data visualization in Python and our Python cheat sheet for data science

Categories: FLOSS Project Planets

Jonathan Dowland: Hans Rosling and Steve Hewlett

Planet Debian - Wed, 2017-02-22 06:13

I've begun to listen to BBC4's "More Or Less" Podcast. They recently had an episode covering the life and work of Hans Rosling, the inspirational swedish statistician, who has sadly died of pancreatic cancer. It was very moving. Some of Professor Rosling's videos are available to view online. I've heard that they are very much worth watching.

Over the last few months I have also been listening to regular updates by BBC broadcaster Steve Hewlett on his own journey as a cancer sufferer. These were remarkably frank discussions of the ins and outs of his diagnosis, treatment, and the practical consequences on his everyday life. I was very sad to tune in on Monday evening and hear a series of repeated clips from his previous appearances on the PM show, as the implications were clear. And indeed, Steve Hewlett died from oesophagal cancer on Monday. Here's an obituary in the Guardian.

Categories: FLOSS Project Planets

Junichi Uekawa: Trying to use Termux on chromebook.

Planet Debian - Wed, 2017-02-22 04:42
Trying to use Termux on chromebook. I am exclusively using chromebook for my client side work. Android apps work on this device, and so does Termux. I was pondering how to make things more useful, like using Download directory integration and chrome apps, but not quite got things set up. Then I noticed that it's possible to use sshd on termux. It only accepts public key authentication, but that's enough for me. I can now use my SecureShell chrome app to connect and get things working. Android apps don't support all the keybinds but SecureShell does, which improves my life a bit.

Categories: FLOSS Project Planets

Third & Grove: Seniorlink Drupal Case Study

Planet Drupal - Wed, 2017-02-22 03:00
Seniorlink Drupal Case Study antonella Wed, 02/22/2017 - 03:00
Categories: FLOSS Project Planets

Talk Python to Me: #100 Python past, present, and future with Guido van Rossum

Planet Python - Wed, 2017-02-22 03:00
Welcome to a very special episode. This is the 100th episode of Talk Python To Me. It's the perfect chance to take a moment and look at where we have come from, and where we are going. Not just with regard to the podcast but for Python in general. <br/> <br/> And who better to do this than Python's inventor himself. Guido van Rossum. In this episode, we discuss how Guido go into programming, where Python came from and why, and Python's bright future with Python 3. <br/> <br/> Links from the show: <br/> <div style="font-size: .85em;"> <br/> <b>Guido on Twitter</b>: <a href='https://twitter.com/gvanrossum' target='_blank'>@gvanrossum</a> <br/> <b>What's New In Python 3.6</b>: <a href='https://docs.python.org/3/whatsnew/3.6.html' target='_blank'>docs.python.org/3/whatsnew/3.6.html</a> <br/> <b>mypy</b>: <a href='http://mypy-lang.org/' target='_blank'>mypy-lang.org</a> <br/> <br/> <strong>Sponsored items</strong> <br/> <b>Rollbar</b>: <a href='https://rollbar.com/talkpythontome' target='_blank'>rollbar.com/talkpythontome</a> <br/> <b>Hired</b>: <a href='https://hired.com/?utm_source=podcast&utm_medium=talkpythontome&utm_term=cat-tech-software&utm_content=2k&utm_campaign=q1-16-episodesbanner' target='_blank'>hired.com/talkpythontome</a> <br/> <b>Our courses</b>: <a href='https://training.talkpython.fm/' target='_blank'>training.talkpython.fm</a> <br/> <b>Podcast's Patreon</b>: <a href='https://www.patreon.com/mkennedy' target='_blank'>patreon.com/mkennedy</a> <br/> </div>
Categories: FLOSS Project Planets

Joey Hess: early spring

Planet Debian - Tue, 2017-02-21 23:51

Sun is setting after 7 (in the JEST TZ); it's early spring. Batteries are generally staying above 11 volts, so it's time to work on the porch (on warmer days), running the inverter and spinning up disc drives that have been mostly off since fall. Back to leaving the router on overnight so my laptop can sync up before I wake up.

Not enough power yet to run electric lights all evening, and there's still a risk of a cloudy week interrupting the climb back up to plentiful power. It's happened to me a couple times before.

Also, turned out that both of my laptop DC-DC power supplies developed partial shorts in their cords around the same time. So at first I thought it was some problem with the batteries or laptop, but eventually figured it out and got them replaced. (This may have contributed the the cliff earier; seemed to be worst when house voltage was low.)

Soon, 6 months of more power than I can use..

Previously: battery bank refresh late summer the cliff

Categories: FLOSS Project Planets

Shirish Agarwal: The Indian elections hungama

Planet Debian - Tue, 2017-02-21 18:11

Before I start, I would like to point out #855549 . This is a normal/wishlist bug I have filed against apt, the command-line package manager. I sincerely believe having a history command to know what packages were installed, which were upgraded, which were purged should be easily accessible, easily understood and if the output looks pretty, so much the better. Of particular interest to me is having a list of new packages I have installed in last couple of years after jessie became the stable release. It probably would make for some interesting reading. I dunno how much efforts would be to code something like that, but if it works, it would be the greatest. Apt would have finally arrived. Not that it’s a bad tool, it’s just that it would then make for a heck of a useful tool.

Coming back to the topic on hand, Now for the last couple of weeks we don’t have water or rather pressure of water. Water crisis has been hitting Pune every year since 2014 with no end in sight. This has been reported in newspapers addendum but it seems it has been felling on deaf ears. The end result of it is that I have to bring buckets of water from around 50 odd metres.

It’s not a big thing, it’s not like some women in some villages in Rajasthan who have to walk in between 200 metres to 5 odd kilometres to get potable water or Darfur, Western Sudan where women are often kidnapped and sold as sexual slaves when they get to fetch water. The situation in Darfur has been shown quite vividly in Darfur is Dying . It is possible that I may have mentioned about Darfur before. While unfortunately the game is in flash as a web resource, the most disturbing part is that the game is extremely depressing, there is a no-win scenario.

So knowing and seeing both those scenarios, I can’t complain about 50 metres. BUT….but… when you extrapolate the same data over some more or less 3.3-3.4 million citizens, 3.1 million during 2011 census with a conservative 2.3-2.4 percent population growth rate according to scroll.in.

Fortunately or unfortunately, Pune Municipal Corporation elections were held today. Fortunately or unfortunately, this time all the political parties bought majorly unknown faces in these elections. For e.g. I belong to ward 14 which is spread over quite a bit of area and has around 10k of registered voters.

Now the unfortunate part of having new faces in elections, you don’t know anything about them. Apart from the affidavits filed, the only thing I come to know is whether there are criminal cases filed against them and what they have shown as their wealth.

While I am and should be thankful to ADR which actually is the force behind having the collated data made public. There is a lot of untold story about political push-back by all the major national and regional political parties even when this bit of news were to be made public. It took major part of a decade for such information to come into public domain.

But for my purpose of getting clean air and water supply 24×7 to each household seems a very distant dream. I tried to connect with the corporators about a week before the contest and almost all of the lower party functionaries hid behind their political parties manifestos stating they would do the best without any viable plan.

For those not knowing, India has been blessed with 6 odd national parties and about 36 odd regional parties and every election some 20-25 new parties try their luck every time.

The problem is we, the public, don’t trust them or their manifestos. First of all the political parties themselves engage in mud-slinging as to who’s copying whom with the manifesto.Even if a political party wins the elections, there is no *real* pressure for them to follow their own manifesto. This has been going for many a year. OF course, we the citizens are to also blame as most citizens for one reason or other chose to remain aloof of the process. I scanned/leafed through all the manifestos and all of them have the vague-wording ‘ we will make Pune tanker-free’ without any implementation details. While I was unable to meet the soon-to-be-Corporators, I did manage to meet a few of the assistants but all the meetings were entirely fruitless.

I asked why can’t the city follow the Chennai model. Chennai, not so long ago was at the same place where Pune is, especially in relation to water. What happened next, in 2001 has been beautifully chronicled in Hindustan Times . What has not been shared in that story is that the idea was actually fielded by one of Chennai Mayor’s assistants, an IAS Officer, I have forgotten her name, Thankfully, her advise/idea was taken to heart by the political establishment and they drove RWH.

Saying why we can’t do something similar in Pune, I heard all kinds of excuses. The worst and most used being ‘Marathas can never unite’ which I think is pure bullshit. For people unfamiliar to the term, Marathas was a warrior clan in Shivaji’s army. Shivaji, the king of Marathas were/are an expert tactician and master of guerilla warfare. It is due to the valor of Marathas, that we still have the Maratha Light Infantry a proud member of the Indian army.

Why I said bullshit was the composition of people living in Maharashtra has changed over the decades. While at one time both the Brahmins and the Marathas had considerable political and population numbers, that has changed drastically. Maharashtra and more pointedly, Mumbai, Pune and Nagpur have become immigrant centres. Why just a decade back, Shiv Sena, an ultra right-wing political party used to play the Maratha card at each and every election and heckle people coming from Uttar Pradesh and Bihar, this has been documented as the 2008 immigrants attacks and 9 years later we see Shiv Sena trying to field its candidates in Uttar Pradesh. So, obviously they cannot use the same tactics which they could at one point of time.

One more reason I call it bullshit, is it’s a very lame excuse. When the Prime Minister of the country calls for demonetization which affects 1.25 billion people, people die, people stand in queues and is largely peaceful, I do not see people resisting if they bring a good scheme. I almost forgot, as an added sweetener, the Chennai municipality said that if you do RWH and show photos and certificates of the job, you won’t have to pay as much property tax as otherwise you would, that also boosted people’s participation.

And that is not the only solution, one more solution has been outlined in ‘Aaj Bhi Khade hain talaab’ written by just-deceased Gandhian environmental activist Anupam Mishra. His Book can be downloaded for free at India Water Portal . Unfortunately, the said book doesn’t have a good English translation till date. Interestingly, all of his content is licensed under public domain (CC-0) so people can continue to enjoy and learn from his life-work.

Another lesson or understanding could be taken from Israel, the father of the modern micro-drip irrigation for crops. One of the things on my bucket lists is to visit Israel and if possible learn how they went from a water-deficient country to a water-surplus one.

Which brings me to my second conundrum, most of the people believe that it’s the Government’s job to provide jobs to its people. India has been experiencing jobless growth for around a decade now, since the 2008 meltdown. While India was lucky to escape that, most of its trading partners weren’t hence it slowed down International trade which slowed down creation of new enterprises etc. Laws such as the Bankruptcy law and the upcoming Goods and Services Tax . As everybody else, am a bit excited and a bit apprehensive about how the actual implementation will take place.

Even International businesses has been found wanting. The latest example has been Uber and Ola. There have been protests against the two cab/taxi aggregators operating in India. For the millions of jobless students coming out of schools and Universities, there aren’t simply enough jobs for them, nor are most (okay 50%) of them qualified for the jobs, these 50 percent are also untrainable, so what to do ?

In reality, this is what keeps me awake at night. India is sitting on this ticking bomb-shell. It is really, a miracle that the youths have not rebelled yet.

While all the conditions, proposals and counter-proposals have been shared before, I wanted/needed to highlight it. While the issue seems to be local, I would assert that they are all glocal in nature. The questions we are facing, I’m sure both developing and to some extent even developed countries have probably been affected by it. I look forward to know what I can learn from them.

Update – 23/02/17 – I had wanted to share about Debian’s Voting system a bit, but that got derailed. Hence in order not to do, I’ll just point towards 2015 platforms where 3 people vied for DPL post. I *think* I shared about DPL voting process earlier but if not, would do in detail in some future blog post.

Filed under: Miscellenous Tagged: #Anupam Mishra, #Bankruptcy law, #Chennai model, #clean air, #clean water, #elections, #GST, #immigrant, #immigrants, #Maratha, #Maratha Light Infantry, #migration, #national parties, #Political party manifesto, #regional parties, #ride-sharing, #water availability, Rain Water Harvesting
Categories: FLOSS Project Planets

DSPIllustrations.com: Circular Convolution Example

Planet Python - Tue, 2017-02-21 18:00
Circular Convolution

In a previous post, we have explained the importance of the convolution operation for signal processing and signal analysis. We have described the convolution integral and illustrated the involved functions.

In this post we will focus on an operation called Circular convolution which is strongly related to the conventional convolution (also called linear convolution) we have described before. Let us reconsider the normal, linear convolution in the discrete domain. Given two sequences x[n] and h[n], their convolution is given by

(x*h)[n] = \sum_{n'=-\infty}^{\infty}x[n']\cdot h[n-n'], \quad n=-\infty,\dots,\infty.

The linear convolution lets one one sequence slide over the other and sums the overlapping parts. The circular convolution of two sequences x[n], h[n] is now considering a wrap-around of the sequences after a period of N samples. So, the circular convolution is defined by

(x\otimes h)[n]=\sum_{n'=0}^{N-1}x[n']\cdot h[(n'-n)_N], \quad n=0,\dots,N-1,

where (n'-n)_N gives the remainder of n'-n divided by N. For example, (-1)_N=N-1. This means, if the index for x[(n'-n)_N] would leave the range 0,\dots,N-1 to the left, it would wrap around and come in from the right again. This means that the circular convolution is periodic with length N.

In discrete domain, the convolution theorem actually holds only for the circular ...

Categories: FLOSS Project Planets

Flocon de toile | Freelance Drupal: Introduction to Protected file module on Drupal 8

Planet Drupal - Tue, 2017-02-21 17:30

Drupal 8 has several solutions and methods to manage access rights on each elements included in a content, and this in a very granular way. Enabling view or edit access on some field included in a content type can be achieved very simply, with a few lines of code, or with the Field Permissions module. We can use this module to allow certain roles to view or update a particular field.

Categories: FLOSS Project Planets

Steinar H. Gunderson: 8-bit Y'CbCr ought to be enough for anyone?

Planet Debian - Tue, 2017-02-21 17:07

If you take a random computer today, it's pretty much a given that it runs a 24-bit mode (8 bits of each of R, G and B); as we moved from palettized displays at some point during the 90s, we quickly went past 15- and 16-bit and settled on 24-bit. The reasons are simple; 8 bits per channel is easy to work with on CPUs, and it's on the verge of what human vision can distinguish, at least if you add some dither. As we've been slowly taking the CPU off the pixel path and replacing it with GPUs (which has specialized hardware for more kinds of pixels formats), changing formats have become easier, and there's some push to 10-bit (30-bit) “deep color” for photo pros, but largely, 8-bit per channel is where we are.

Yet, I'm now spending time adding 10-bit input (and eventually also 10-bit output) to Nageru. Why? The reason is simple: Y'CbCr.

Video traditionally isn't done in RGB, but in Y'CbCr; that is, a black-and-white signal (Y) and then two color-difference signals (Cb and Cr, roughly “additional blueness“ and “additional redness”, respectively). We started doing this because it was convenient in analog TV (if you separate the two, black-and-white TVs can just ignore the color signal), but we kept doing it because it's very nice for reducing bandwidth: Human vision is much less sensitive to color than to brightness, so we can transfer the color channels in lower resolution and get away with it. (Also, a typical Bayer sensor can't deliver full color resolution anyway.) So most cameras and video codecs work in Y'CbCr, not RGB.

Let's look at the implications of using 8-bit Y'CbCr, using a highly simplified model for, well, simplicity. Let's define Y = 1/3 (R + G + B), Cr = R - Y and Cb = B - Y. (The reverse transformation becomes R = Y + Cr, B = Y + Cb and G = 3Y - R - B.)

This means that an RGB color such as pure gray ([127, 127, 127]) becomes [127, 0, 0]. All is good, and Y can go from 0 to 255, just like R, G and B can. A pure red ([255, 0, 0]) becomes [85, 170, 0], and a pure blue ([255, 0, 0]) becomes correspondingly [85, 0, 170]. But we can also have negative Cr and Cb values; a pure yellow ([0, 255, 255]) becomes [170, -170, 85], for instance. So we need to squeeze values from -170 to +170 into an 8-bit range, losing accuracy.

Even worse, there are valid Y'CbCr triplets that don't correspond to meaningful RGB colors at all. For instance, Y'CbCr [255, 170, 0] would be RGB [425, 85, 255]; R is out of range! And Y'CbCr [255, -170, 0] would be RGB [85, -85, 255], that is, negative green.

This isn't a problem for compression, as we can just avoid using those illegal “colors” with no loss of efficiency. But it means that the conversion in itself causes a loss; actually, if you do the maths on the real formulas (using the BT.601 standard), it turns out only 17% of the 24-bit Y'CbCr code words are valid!

In other words, we lose about two and a half bits of data, and our 24 bits of accuracy have been reduced to 21.5. Or, to put it another way; 8-bit Y'CbCr is roughly equivalent to 7-bit RGB.

Thus, pretty much all professional video uses 10-bit Y'CbCr. It's much more annoying to deal with (especially when you've got subsampling!), but if you're using SDI, there's not even any 8-bit version defined, so if you insist on 8-bit, you're taking data you're getting on the wire (whether you want it or not) and throwing 20% of it away. UHDTV standards (using HEVC) are also simply not defined for 8-bit; it's 10- and 12-bit only, even on the codec level. Parts of this is because UHDTV also supports HDR, so you have a wider RGB range than usual to begin with, and 8-bit would cause excessive banding.

Using it on the codec level makes a lot of sense for another reason, namely that you reduce internal roundoff errors during processing by a lot; errors equal noise, and noise is bad for compression. I've seen numbers of 15% lower bitrate for H.264 at the same quality, although you also have to take into account that the encoeder also needs more CPU power that you could have used for a higher preset in 8-bit. I don't know how the tradeoff here works out, and you also have to take into account decoder support for 10-bit, especially when it comes to hardware. (When it comes to HEVC, Intel didn't get full fixed-function 10-bit support before Kaby Lake!)

So indeed, 10-bit Y'CbCr makes sense even for quite normal video. It isn't a no-brainer to turn it on, though—even though Nageru uses a compute shader to convert the 4:2:2 10-bit Y'CbCr to something the GPU can sample from quickly (ie., the CPU doesn't need to touch it), and all internal processing is in 16-bit floating point anyway, it still takes a nonzero amount of time to convert compared to just blasting through 8-bit, so my ultraportable probably won't make it anymore. (A discrete GPU has no issues at all, of course. My laptop converts a 720p frame in about 1.4 ms, FWIW.) But it's worth considering when you want to squeeze even more quality out of the system.

And of course, there's still 10-bit output support to be written...

Categories: FLOSS Project Planets

William Minchin: Summary Plugin 1.1.0 for Pelican Released

Planet Python - Tue, 2017-02-21 16:51

Summary is a plugin for Pelican, a static site generator written in Python.

Summary allows easy, variable length summaries directly embedded into the body of your articles.


The easiest way to install Summary is through the use of pip. This will also install the required dependencies automatically (currently none beyond Pelican).

pip install minchin.pelican.plugins.summary

Then, in your pelicanconf.py file, add Summary to your list of plugins:

PLUGINS = [ # ... 'minchin.pelican.plugins.summary', # ... ]

You may also need to configure the summary start and end markers (see Configuration below).

Configuration and Usage

This plugin introduces two new settings: SUMMARY_BEGIN_MARKER and SUMMARY_END_MARKER: strings which can be placed directly into an article to mark the beginning and end of a summary. When found, the standard SUMMARY_MAX_LENGTH setting will be ignored. The markers themselves will also be removed from your articles before they are published. The default values are <!-- PELICAN_BEGIN_SUMMARY --> and <!-- PELICAN_END_SUMMARY -->.

If no beginning or end marker is found, and if SUMMARY_USE_FIRST_PARAGRAPH is enabled in the settings, the summary will be the first paragraph of the post.

The plugin also sets a has_summary attribute on every article. It is True for articles with an explicitly-defined summary, and False otherwise. (It is also False for an article truncated by SUMMARY_MAX_LENGTH.) Your templates can use this e.g. to add a link to the full text at the end of the summary.

Known Issues

An issue, as such, is that there is no formal test suite. Testing is currently limited to my in-use observations. I also run a basic check upon uploaded the package to PyPI that it can be downloaded and loaded into Python.

The package is tested in Python 3.5; compatibility with other version of Python is unknown.

Tests are actaully included and can be run from the root directory:

python minchin/pelican/plugins/summary/test_summary.py Changes

This version is basically just repackaging the plugin and making it available on pip.


The code for this project is available on GitHub. Contributions are welcome!


Original plugin from the Pelican-Plugins repo.


The plugin code is assumed to be under the AGPLv3 license (this is the license of the Pelican-Plugins repo).

Categories: FLOSS Project Planets

OhTheHugeManatee: What Crell Doesn't Want You to Know: How to Automate Letsencrypt on platform.sh

Planet Drupal - Tue, 2017-02-21 16:33

If you believe the docs and the twitters, there is no way to automate letsencrypt certificates updates on platform.sh. You have to create the certificates manually, upload them manually, and maintain them manually.

But as readers of this blog know, the docs are only the start of the story. I’ve really enjoyed working with platform.sh with one of my private clients, and I couldn’t believe that with all the flexibility – all the POWER – letsencrypt was really out of reach. I found a few attempts to script it, and one really great snippet on gitlab. But no one had ever really synthesized this stuff into an easy howto. So here we go.

1) Add some writeable directories where platform.sh CLI and letsencrypt need them.

Normally when Platform deploys your application, it puts it all in a read-only filesystem. We’re going to mount some special directories read-write so all the letsencrypt/platform magic can work.

Edit your application’s .platform.app.yaml file, and find the mounts: section. At the bottom, add these three lines. Make sure to match the indents with everything else under the mounts: section!

1 2 3 "/web/.well-known": "shared:files/.well-known" "/keys": "shared:files/keys" "/.platformsh": "shared:files/.platformsh"

Let’s walk through each of these:

  • /web/.well-known: In order to confirm that you actually control example.com, letsencrypt drops a file somewhere on your website, and then tries to fetch it. This directory is where it’s going to do the drop and fetch. My webroot is web, you should change this to match your own environment. You might use public or www or something.
  • /keys: You have to store your keyfiles SOMEWHERE. This is that place.
  • /.platformsh: Your master environment needs a bit of configuration to be able to login to platform and update the certs on your account. This is where that will go.
2) Expose the .well-known directory to the Internet

I mentioned above that letsencrypt test your control over a domain by creating a file which it tries to fetch over the Internet. We already created the writeable directory where the scripts can drop the file, but platform.sh (wisely) defaults to hide your directories from the Internet. We’re going to add some configuration to the “web” app section to expose this .well-known directory. Find the web: section of your .platform.app.yaml file, and the locations: section under that. At the bottom of that section, add this:

1 2 3 4 5 6 7 8 '/.well-known': # Allow access to all files in the public files directory. allow: true expires: 5m passthru: false root: 'web/.well-known' # Do not execute PHP scripts. scripts: false

Make sure you match the indents of the other location entries! In my (default) .platform.app.yaml file, I have 8 spaces before that '/.well-known': line. Also note that the root: parameter there also uses my webroot directory, so adjust that to fit your environment.

3) Download the binaries you need during the application “build” phase

In order to do this, we’re going to need to have the platform.sh CLI tool, and a let’s encrypt CLI tool called lego. We’ll download them during the “build” phase of your application. Still in the platform.app.yaml file, find the hooks: section, and the build: section under that. Add these steps to the bottom of the build:

1 2 3 cd ~ curl -sL https://github.com/xenolf/lego/releases/download/v0.3.1/lego_linux_amd64.tar.xz | tar -C .global/bin -xJ --strip-components=1 lego/lego curl -sfSL -o .global/bin/platform.phar https://github.com/platformsh/platformsh-cli/releases/download/v3.12.1/platform.phar

We’re just downloading reasonably recent releases of our two tools. If anyone has a better way to get the latest release of either tool, please let me know. Otherwise we’re stuck keeping this up to date manually.

4) Configure the platform.sh CLI

In order to configure the platform.sh CLI on your server, we have to deploy the changes from steps 1-3. Go ahead and do that now. I’ll wait.

Now connect to your platform environment via SSH (platform ssh -e master for most of us). First we’ll add a config file for platform. Edit a file in .platformsh/config.yaml with the editor of choice. You don’t have to use vi, but it will win you some points with me. Here are the contents for that file:

1 2 3 4 updates: check: false api: token_file: token

Pretty straightforward: this tells platform not to bother updating the CLI tool automatically (it can’t – read-only filesystem, remember?). It then tells it to login using an API token, which it can find in the file .platformsh/token. Let’s create that file next.

Log into the platform.sh web UI (you can launch it with platform web if you’re feeling sassy), and navigate to your account settings > api tokens. That’s at https://accounts.platform.sh/user/12345/api-tokens (with your own user ID of course). Add an API token, and copy its value into .platformsh/token on the environment we’re working on. The token should be the only contents of that file.

Now let’s test it by running php /app/.global/bin/platform.phar auth:info. If you see your account information, congratulations! You have a working platform.sh CLI installed.

5) Request your first certificate by hand

Still SSH’ed into that environment, let’s see if everything works.

1 2 3 lego --email="support@example.com" --domains="www.example.com" --webroot=/app/public/ --path=/app/keys/ -a run csplit -f /app/keys/certificates/www.example.com.crt- /app/keys/certificates/www.example.com.crt '/-----BEGIN CERTIFICATE-----/' '{1}' -z -s php /app/.global/bin/platform.phar domain:update -p $PLATFORM_PROJECT --no-wait --yes --cert /app/keys/certificates/www.example.com.crt-00 --chain /app/keys/certificates/www.example.com.crt-01 --key /app/keys/certificates/www.example.com.key example.com

This is three commands: register the cert with letsencrypt, then split the resulting file into it’s components, then register those components with platform.sh. If you didn’t get any errors, go ahead and test your site – it’s got a certificate! (yay)

6) Set up automatic renewals on cron

Back to .platform.app.yaml, look for the crons: section. If you’re running drupal, you probably have a drupal cronjob in there already. Add this one at the bottom, matching indents as always.

1 2 3 letsencrypt: spec: '0 0 1 * *' cmd: '/bin/sh /app/scripts/letsencrypt.sh'

Now let’s create the script. Add the file scripts/letsencrypt.sh to your repo, with this content:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 #!/usr/bin/env bash # Checks and updates the letsencrypt HTTPS cert. set -e if [ "$PLATFORM_ENVIRONMENT" = "master-7rqtwti" ] then # Renew the certificate lego --email="example@example.org" --domains="example.org" --webroot=/app/web/ --path=/app/keys/ -a renew # Split the certificate from any intermediate chain csplit -f /app/keys/certificates/example.org.crt- /app/keys/certificates/example.org.crt '/-----BEGIN CERTIFICATE-----/' '{1}' -z -s # Update the certificates on the domain php /app/.global/bin/platform.phar domain:update -p $PLATFORM_PROJECT --no-wait --yes --cert /app/keys/certificates/example.org.crt-00 --chain /app/keys/certificates/example.org.crt-01 --key /app/keys/certificates/example.org.key example.org fi

Obviously you should replace all those example.orgs and email addresses with your own domain. Make the file executable with chmod u+x scripts/letsencrypt.sh, commit it, and push it up to your platform.sh environment.

7) Send a bragging email to Crell

Technically this isn’t supposed to be possible, but YOU DID IT! Make sure to rub it in.

Good luck!

PS – I’m just gonna link one more time to the guy whose snippet made this all possible: Ariel Barreiro did the hardest part of this. I’m grateful that he made his notes public!

Categories: FLOSS Project Planets

Reproducible builds folks: Reproducible Builds: week 95 in Stretch cycle

Planet Debian - Tue, 2017-02-21 13:25

Here's what happened in the Reproducible Builds effort between Sunday February 12 and Saturday February 18 2017:

Upcoming Events

The Reproducible Build Zoo will be presented by Vagrant Cascadian at the Embedded Linux Conference in Portland, Oregon, February 22nd.

Introduction to Reproducible Builds will be presented by Vagrant Cascadian at Scale15x in Pasadena, California, March 5th.

Toolchain development and fixes

Ximin Luo posted a preliminary spec for BUILD_PATH_PREFIX_MAP, bringing together work and research from previous weeks.

Ximin refactored and consolidated much of our existing documentation on both SOURCE_DATE_EPOCH and BUILD_PATH_PREFIX_MAP into one unified page, Standard Environment Variables, with extended discussion on related solutions and how these all fit into people's ideas of what reproducible builds should look like in the long term. The specific pages for each variable still remain, at Timestamps Proposal and Build Path Proposal, only without content that was previously duplicated on both pages.

Ximin filed #855282 against devscripts for debsign(1) to support buildinfo files, and wrote an initial series of patches for it with some further additions from Guillem Jover.

Packages reviewed and fixed, and bugs filed

Chris Lamb:

Reviews of unreproducible packages

35 package reviews have been added, 1 have been updated and 17 have been removed in this week, adding to our knowledge about identified issues.

1 issue type has been added:

Weekly QA work

During our reproducibility testing, the following FTBFS bugs have been detected and reported by:

  • Chris Lamb (2)
diffoscope development

diffoscope 77 was uploaded to unstable by Mattia Rizzolo. It included contributions from:

  • Chris Lamb:
    • Some fixes to tests and testing config
    • Don't track archive directory locations, a better fix for CVE-2017-0359.
    • Add --exclude option. Closes: #854783
  • Mattia Rizzolo:
    • Add my key to debian/upstream/signing-key.asc
    • Add CVE-2017-0359 to the changelog of v76
  • Ximin Luo:
    • When extracting archives, try to keep directory sizes small
strip-nondeterminism development

strip-nondeterminism 0.031-1 was uploaded to unstable by Chris Lamb. It included contributions from:

  • Chris Lamb:
    • Make the tests less brittle, by not testing for stat(2) blksize and blocks. #854937

strip-nondeterminism 0.031-1~bpo8+1 was uploaded to jessie-backports by Mattia.

  • Vagrant Cascadian and Holger Levsen set up two new armhf nodes, p64b and p64c running on pine64 boards with an arm64 kernel and armhf userland. This introduces kernel variations to armhf. New setup & maintenance jobs were set up too, plus 6 new builder jobs for armhf.

This week's edition was written by Ximin Luo & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Categories: FLOSS Project Planets

Mike Driscoll: Python 3 – Unpacking Generalizations

Planet Python - Tue, 2017-02-21 13:15

Python 3.5 added more support for Unpacking Generalizations in PEP 448. According to the PEP, it added extended usages of the * iterable unpacking operator and ** dictionary unpacking operators to allow unpacking in more positions, an arbitrary number of times, and in additional circumstances. What this means is that we can now make calls to functions with an arbitrary number of unpackings. Let’s take a look at a dict() example:

>>> my_dict = {'1':'one', '2':'two'} >>> dict(**my_dict, w=6) {'1': 'one', '2': 'two', 'w': 6} >>> dict(**my_dict, w='three', **{'4':'four'}) {'1': 'one', '2': 'two', 'w': 'three', '4': 'four'}

Interestingly, if the keys are something other then strings, the unpacking doesn’t work:

>>> my_dict = {1:'one', 2:'two'} >>> dict(**my_dict) Traceback (most recent call last): File "<pyshell#27>", line 1, in <module> dict(**my_dict) TypeError: keyword arguments must be strings

Update: One of my readers was quick to point out that the reason this doesn’t work is because I was trying to unpack into a function call (i.e. dict()). If I had done the unpacking using just dict syntax, the integer keys would have worked fine. Here’s what I’m talking about:

>>> {**{1: 'one', 2:'two'}, 3:'three'} {1: 'one', 2: 'two', 3: 'three'}

One other interesting wrinkle to dict unpacking is that later values will always override earlier ones. There’s a good example in the PEP that demonstrates this:

>>> {**{'x': 2}, 'x': 1} {'x': 1}

I thought that was pretty neat. You can do the same sort of thing with ChainMap from the collections module, but this is quite a bit simpler.

However this new unpacking also works for tuples and lists. Let’s try combining some items of different types into one list:

>>> my_tuple = (11, 12, 45) >>> my_list = ['something', 'or', 'other'] >>> my_range = range(5) >>> combo = [*my_tuple, *my_list, *my_range] >>> combo [11, 12, 45, 'something', 'or', 'other', 0, 1, 2, 3, 4]

Before this unpacking change, you would have had do something like this:

>>> combo = list(my_tuple) + list(my_list) + list(my_range) [11, 12, 45, 'something', 'or', 'other', 0, 1, 2, 3, 4]

I think the new syntax is actually quite handy for these kinds of circumstances. I’ve actually run into this a time or two in Python 2 where this new enhancement would have been quite useful.

Wrapping Up

There are lots of other examples in PEP 448 that are quite interesting to read about and try in Python’s interpreter. I highly recommend checking it out and giving this feature a try. I am hoping to start using some of these features in my new code whenever we finally move to Python 3.

Categories: FLOSS Project Planets

Palantir: The Coolest Camp Ever

Planet Drupal - Tue, 2017-02-21 11:31
The Coolest Camp Ever brandt Tue, 02/21/2017 - 10:31 Alex Brandt Feb 21, 2017

We’re heading to Iceland February 24 - 26!

In this post we will cover...
  • Why we’re excited for this new event

  • Who from Palantir will be speaking

  • Some fun facts about Iceland

Stay connected with the latest news on web strategy, design, and development.

Sign up for our newsletter.

Besides being the recent desired destination for Instagram #wanderlust-ers, Iceland is now home to an exciting new Drupal event: DrupalCamp Northern Lights. With twenty speakers, lots of coffee, and a planned sightseeing trip to see the Golden Circle and Northern Lights, it is sure to be an exciting inaugural event.

A small crew of Palantiri will be proudly representing, so if you are making the trek overseas, keep an eye out and say hi to Allison Manley, Michelle Jackson, and Megh Plunkett while you’re taking in the sessions and sights.

Check out the schedule and make sure to stop by our sessions.


Kickoff Meetings, by Allison Manley

  • Time: Saturday, 10:45 - 11:35
  • Location: Room ÞINGVELLIR

How do you make the most use of your face-to-face time with your client and lay the groundwork for a successful project?

Allison will outline how to get the most out of the kickoff meetings that initiate any project. She'll talk about pre-meeting preparation and how to keep organized, and also give some tips on agenda creation, how to keep meetings productive (and fun), and what steps need to be taken once the meetings adjourn.


Competitive Analysis: Your UX must-have on a budget, by Michelle Jackson

  • Time: Sunday, 14:15-15:00
  • Location: Room ÞINGVELLIR

A tight budget and time constraints can make dedicating time and resources to understanding audience needs challenging. Competitive analysis is an affordable way to evaluate how competitor sites are succeeding or failing to meet the needs of your audience.

Michelle will cover how competitive analysis can help you avoid competitor pitfalls, gain insight into what your users want, and lead to better decision-making before you invest in and implement new designs and technical features.

7 Facts You Might Not Have Known About Iceland
  • Iceland was one of the last places on earth to be settled by humans.
  • They are getting their first Costco in May.
  • 60% of the Icelandic population lives in Reykjavík.
  • Babies in Iceland are routinely left outside to nap.
  • Surprisingly, Iceland is not the birthplace of ice cream.
  • First names not previously used in Iceland must be approved by the Icelandic Naming Committee.
  • Owning a pet turtle is against the law. Sorry Rafael, Franklin, and this kid:


Fact Sources: http://www.portsmouth.co.uk/news/people/31-odd-facts-about-iceland-but-how-many-did-you-know-1-7445785, http://icelandreview.com/

We want to make your project a success.

Let's Chat.
Categories: FLOSS Project Planets

InternetDevels: Collection of some useful Drupal 8 modules in 2017

Planet Drupal - Tue, 2017-02-21 11:01

Our tradition of presenting you short overviews of several modules of the month continues with today’s article. Previously we offered you some great contributed Drupal 8 modules in June 2016 a collection of modules in May 2016. In 2017 we published some modules, with the latest available release for Drupal 8 scheduled for the beginning of this year.

Read more
Categories: FLOSS Project Planets
Syndicate content