DrupalCon News: Join in the discussion about the future of Drupal + Media

Planet Drupal - Thu, 2017-03-23 12:18

It's that time of year again when everyone starts getting excited about DrupalCon.  People are getting geared up to attend sessions, meet up with team members and clients, and let's not forget, load up on as much swag as possible.  But an important piece which often gets overlooked are the Summits that happen the Monday before the conference begins.  These events are happening again in Baltimore, and the Media and Publishing Summit is one you should consider attending.

Categories: FLOSS Project Planets

Meet the LibrePlanet 2017 Speakers: Christian Fernandez

FSF Blogs - Thu, 2017-03-23 12:07

His session, Pentesting loves free software, takes place on Saturday, March 25th in session block 5A (15:40 - 16:25).

Could you tell us a bit about yourself?

I moved to the US in 1997. Since then I been traveling around and moving to different cities. I've been into social justice movements in the past. I started into hacking in the late 80s with BBS’ FidoNet, exploring and trying to sniff out all the information I could.

At the same time I got into hacking, I got into free software after that—. Most people I knew online were also involved in both since they go hand by hand.

How did you first get interested in penetration testing with free sotware?

I've seen a lot of newcomers to the security field from academia. As with any other tech field, they learn commercial, proprietary, non-free tools.

I like to point out and show people that you can get the job done even better using free tools. I'm very passionate about this as a free software activist.

Have you been to LibrePlanet before?

Yes, I am a longtime Free Software Foundation member and have been to a number of LibrePlanets.

How can we follow you on social media?

I have A LOT of handles...some nobody knows. :) I use @rek2fernandez on Twitter. B1naryFreed0m is the one I use for politics.

What is a skill or talent you have that you wish more people knew about?

In order to make things better first you have to break them apart and study them. Learning is not about reading a book and going to school, it is about passion and practice, and a lot of frustration sometimes. :D

Categories: FLOSS Project Planets

Steve Purkiss: Co-operative.club - Part II: To Drupal or not

Planet Drupal - Thu, 2017-03-23 11:10
Co-operative.club - Part II: To Drupal or not Steve Purkiss Thu, 23/03/2017 - 15:10

"You'll write about this one day" she said. "and you'll make it sound like you're the hero. You should go to L.A." she added.

Well, turns out for a number of reasons this happens to be that day, and as for the latter, I'll let you the reader decide, although to be quite honest, I don't much care either way as you weren't there, and I'm not asking for opinions. I'm writing this because I want to conclude the story about what I've been trying to create in this world whilst I am still able to. I believe there is a better, more fairer to all, way of living and that we have all the tools we need in order to achieve this change, leaving the world in a far better place than it currently is. For me, for her, for everyone. This isn't going to be an easy read, there's no pretty pictures, no subheadings and no holds barred. I'm sorry if you wanted that, I'll get back to 'normal' mode after this, but I have to get this out as it's been over ten years now and I need to move on, and this is my therapy.

In Part I of The Co-operative.club story I explained how Free Software had enabled me to connect with others through a social business network based on the Drupal platform after being made redundant and having to leave London and return to my father's house. Thirteen years hence I find myself back here/there again, broke again (and more so this time round), but still with the ability to get myself out of this situation thanks to Free Software, and Drupal. The latter however I am, and not for the first time, too sure about given the events of the past 24h, and that's where I'd value your input - not just in words but also in actions. Inaction is also action, I guess we will see.

First let's go back to where I left off last time, in Toronto, where I'd just set up a 'Test Box' which had ended up being just an expensive party and an art gallery. What I hadn't mentioned in the previous post, and which is why I found it hard to write the second part, was that I was at the time in a relationship with a girl I'd met at the Bovine Sex Club. Not a sex club, but a live music bar near where I was staying in Queen St. W. I'd been talking with this pretty girl and when she and her friends were going to leave I thought I'd ask for her number - something I'd never done before but being in another country knowing I'd probably never see her again if I didn't pluck up the courage I thought I would. I did, and she smiled and wrote her number down. I don't think you could've met a happier guy that night than me after that. Sadly, it didn't carry on that way.

I invited her out on a date - one of my favourite British comedians Dave Gorman was playing a gig in Toronto and I thought it would be a nice thing to go to, have a meal and all that jazz. Nothing too serious - a bit of fun to see if we still liked each other outside the dark sweaty confines of a Toronto music bar. The day came, I waited. And waited. And waited. She didn't turn up, I went to the gig alone. Dave was hilarious as usual, talking about the time when he decided to find all the people in the world with the same name as him then go meet some. Silly comedy, that's what I like.

I decided to phone and find out why she hadn't turned up - perhaps something had happened, perhaps she had just changed her mind. She said she thought I wasn't going to turn up, that I didn't really mean what I'd said, and that she didn't want to get hurt again so had decided it was better to stay at home. Coming from a broken home myself and seeing girls get hurt by blokes at college and uni, I kinda could see a little where she was coming from and assured her my intentions were honest, forgave her for not turning up, asked if she'd like to try again and was delighted when she said yes. What I didn't know at that time was she hadn't been on meds for three years and was classed as having 'Borderline Personality Disorder' and a rapid-cycling Bipolar. Heck, I didn't even know what 'meds' were.

We met up again, and again. We talked and talked, and I had stories of how bad the situation was living with her parents and within a very short space of time - perhaps a week or three, she had moved in with me. She seemed like she was in a really bad place at the time so although it seemed to me too soon, I thought I understood her situation and was willing to give cohabiting a go - this could be 'the one' and I didn't want to look back and regret missing that chance at that long-held dream I'd had until then of meeting someone and spending my life with them.

She moved in but pretty soon things started to get very strange and scary. Within a week or two she was crying uncontrollably and cut her wrists. Not in a blood-gushing out gonna die right now way, but enough to make it bleed. I didn't know what to do - I was in a country I didn't know, and I'd never experienced anything like this before. I actually ended up phoning my mum, who of course ended up worried out of her mind and could only suggest we go to the hospital, which she didn't want to do. I saw other scars and found out in time this is how she 'released the pressure'.

I slowly discovered the truth about her diagnosis and medication or lack thereof, and heard stories about how she became to be on that medication - her parents said it was a 'chemical imbalance', however her story was one of a guy at school who had tried to attack her when she had shunned his advances towards her and who had later been jailed for some predatory activity. I don't know to this day what the actual truth is, on either side of the tale. Being rapid-cycling meant she was fine half the day and, to be brutally honest, batshit crazy the other half. She was highly intelligent, with a degree from one of the top universities and previously a fairly high-powered job until that fell apart when her body started to become immune to the lithium she was taking - or not, I don't really know, that's just what she told me.

We used to argue a lot when she'd accuse me of everything under the sun - from looking at other women to me being a 'dictator' because I wanted to create these open source cafes. I remember one night when she decided to rip up my copy of Lawrence Lessig's 'Free Culture' book one page at a time and shove each page under the door of the lounge where I'd barricaded myself in so that she couldn't physically attack me. The rapid cycling meant she'd calm down in a few hours and all would be sweetness and light again, as if nothing had happened. But it had, and I couldn't forget that, which annoyed her more as I didn't feel at all in the mood for anything unlike herself, which I guess is another part of the being on her high. So the cycle would repeat. She'd punch holes in the wall then go to the DIY store and fix them up after. All things I look back at now and think "why the hell didn't I get out of that situation straight away". But you don't - well I didn't. I can't explain how it feels to be in that situation - all I knew is she said she'd be out on the street if I'd chucked her out, and I didn't know why she had this distrust of me that I'd go off with other women. For a start I hardly had the chance because we never seemed to leave the house and when we did it was a nightmare as she'd have a breakdown in the middle of a store and of course everyone would stare at me with that 'what this nasty bloke had done to this poor girl' look in their eyes.

I tried to go to networking meetings - after all I was only on a visiting visa and was there to see if I could connect and build business - but we only managed to go to a few and I wasn't getting enough business in to survive, and my credit cards by this time were maxxed out. The way I usually get business is I'm connecting with people every day, writing blog posts, and talking about Free Software. I couldn't do that in this situation where everyone we met was a 'bad person' if they were male, or a potential threat if they were female. I used to go for long walks when she was mad at me for apparently looking at another girl or whatever I was supposed to have done - I'm a networker and an essential part of that is talking to other people so yes, I talk with other people and I'm friendly with them - doesn't mean I want to go to bed with them. I used to go to the local library and read up on Borderline Personality Disorder and Bipolar. I read a lot, and the main thing I came away with was that saying you didn't have it was a symptom of having it. She said she didn't have it, and the meds were just to dumb her down, she would be able to cope without them.

I managed to convince her that if we were to have any future, it was going to have to be going back on the meds again - at least for a while as I simply couldn't deal with fighting every day and perhaps a different combination than before might help. She went back on them - a horrible to see process as she had such a high dose she was asleep for most of the day and yes, all they seemed to do was make her inactive so she wouldn't think or do much.

This too didn't last long though, she'd not take her meds some days, and things got progressively worse until one day during another attack she'd bitten my arm. I went to the clinic as it had started to go green and the doc said I'd gone just in time as human bites were worse than snake bites. Then another time she'd knelt on my hand so hard she broke it and I had to go to hospital, with her all the time saying "it's not broken" when it plainly was. I see that every time I look at my hand with the knuckle where it shouldn't be and the bone sticking out also where it shouldn't. I've had guys 'joke' to me suggesting that's where I punched her - if only they knew. I only ever restrained, which I hated having to do but there's pretty much nothing else you can do apart of course from getting the hell out of that situation but I hope others who have experienced similar will understand. I guess that's why I've found it hard to talk about at any length until now, because I fear what people assume. What has helped me finally write this is mostly down to people in the Drupal community who have been brave enough to talk about their situations, they know who they are and I'd like to thank every one of them for doing so, I don't think they realise just how much it helps others like me have the confidence to even consider doing this when I still worry hopefully unnecessarily about potential retribution and consequences.

Now she was back on the meds but not doing too well with me there her parents rented her a flat but that didn't go too well either, when I went to visit she hadn't eaten for days and thought she was hearing screams from down the road and had to go investigate. We decided that we wanted to be together(!) but that would involve being nearer to where her parents had moved to out on the lakes a couple of hundred miles up the lake in Kingston, Ontario. We briefly moved into a bed & board place but moved out soon after as I'd managed to find a local client who of course I couldn't work in the office as I was 'just visiting', but I have clients all over the world so work remotely and they don't employ me. That didn't last long though so I went into their office 'just visiting', but that didn't last long either as she'd be on the phone every five minutes.

I started to make connections locally and garnered quite an interest in my project - they had a 'Think Kingston' campaign who said the local uni wanted closer ties with the town as they hadn't invested much of late and this sort of thing could help - in fact so much they said it would be good to have one central location and one in each of the six suburbs to connect the community. No one person would put up the money though so it was a case of continually networking until I brought enough interested parties together to make something happen. I met some interesting people there too as Sun Microsystems had a big office there and had offered me all the computers I'd needed which was great as they had keycards so students and office workers could plug in and be on the same systems they used. I even met one of the first people to have a Unix business who said my main issue was going to be getting everything up and running at the same time - the events, the tech, the food, etc.

At the same time the half/half life was still going on and although she was managing to go to a few Cognitive-based therapy meetings it wasn't working. One day I got a black eye - I'm not sure if it's the same day I was asking about food regulations to the librarian who happened to be female but I have a feeling it might've been. I decided to call the police but they said there wasn't much they could do, she could only admit herself back in hospital. We went to the doctor, she ended up getting up on his desk waving an umbrella at him and shouting racist comments at him. She went back into hospital, I saw her trying to bend her fingers back, and she ended up talking her way around the weekend staff as they weren't as trained and she was back, attacking me. I left. My friend who I'd met through the networking in Toronto offered to pay for my flight home but when I went back to Toronto I decided, in my stupidity, to go back again.

It didn't last long again, I'd be in the kitchen making food and I'd forget something so the light ended up being switched on and off a couple of times - this apparently was me making secret signals to the next-door neighbours wife who I was apparently having an affair with. I kept myself busy online and it was at this time I rediscovered Drupal as one of the fairly high-profile sites I'd built back in the UK had been rebuilt in Drupal - I'd built it in another Open Source CMS called XOOPS. I had a pretty similar view of Drupal as many still have "Ugh, Drupal", however this time I decided to look under the hood at the code and saw that it was just as - if not more so - capable as the system I'd been involved with back in the dotcom days. I saw an API with business logic infused, and all the hallmarks of a system which anyone could use, whether they could code or not. It was modular, capable, and could build anything. I didn't however realise there was a whole community behind it as every time I went to the forums I'd get questions about who I was talking to and they'd end up with the usual fighting, so I just saw the code.

Things go worse again and I found myself many nights out in the freezing temperatures wandering around until she'd calm down. My friend again paid for my flight home, but this time I went. I had no baggage, just a big winter jacket and they thought they'd caught their terrorist so searched and searched but all they found was my tired soul waiting to get the hell out of there.

There were good times, but there were also many very bad times. I haven't covered everything that happened because it's amazing how much did in such a relatively short period of my life and yes I know it's only my side of the story, however I'm not the one who was on meds who then went off them and one of the first things I did when I arrived home was get checked out as she told me that it takes one to know one. I went through interviews and they said I was 'normal'. To be honest, I don't know what normal is, I don't think there is such a thing as normal. I do know where we were in Canada was around the area where the "pill for every ill" started with placebos and the medical world doesn't know much of how our brain works. I also know the culture there seems to be if you don't fit into the 9-5 lifestyle you're obviously in need of meds, whereas here in the UK it's getting more that way but different.

So why am I writing about this now? Because I find myself again questioning whether Drupal is the right tool I want to use for my project, and this time I feel it has relevance with the experience I went through and the knowledge I gained about the wonders of the brain. A prominent member of the Drupal community whom I've had the pleasure of meeting a few times has been 'ousted' due to his personal beliefs. The founder of Drupal, Dries, says he is doing it to protect the values of the community, however the facts as they are available at the time of writing seem to show that nothing nefarious has actually happened, only that there is the perceived chance that something might happen. I don't follow these beliefs, in fact I hadn't heard of this specific community previously and I, along with many others, do worry about it. I have seen vulnerable people and know how people can manipulate situations, and even though there is consent in the situation it is often a blurry line as to whether people are in a capacity to really know what that is and whether it's a good thing or not to be doing it. My personal view on the situation is though that it is none of our business and it should not be affecting the project, it is however and that's a problem. We now have a situation where another person who has practically dedicated their life to the project is now in a place they don't want - or need - to be, all in the name of 'inclusion'. 

Our project recently ousted another member of the community who has given similar amount of their life to it and who now has a gaping hole where friends, fun, and code used to be. I made an official complaint but was met with the usual brush-off and told that there was not much they could do with "undiagnosed autism". So we are now the medical establishment diagnosing people?

We are all human beings with our quirks and strange ways. Free Software (more specifically Copyleft software which ensures users freedom both up and downstream) gives people the opportunity to be included in society no matter who they are and we need to preserve that. If Drupal is to decide who is and who isn't allowed to be part of this, without them actually doing anything which is breaking the law of their respective residential countries, then where does this leave the platform? A Minority Report way of thinking will just keep it to a very small minority. Yes, we need to look after our community and make it as welcome to everyone - and from what I've seen of the actual work this community member who's just been ousted has done he's been nothing but a boon to this. He hasn't - as far as we've seen - done anything illegal or untoward, yet we are branding him otherwise, causing untold damage to his reputation in the process. And that's not fair.

So why do I even bother with Drupal? Aren't there other things I can use? Sure, but none are as advanced as Drupal is in terms of the ability for non-coders to build what they want in order to be able to communicate via the medium of the internet, for free. Drupal although a much smaller share of the market currently than other certain systems, has freedom built in its DNA. I don't have to buy a plugin to do what I want, and if I don't know how to code something it doesn't stop me from using modules or giving something to the community from my skill set which will help others and perhaps others will then be more likely to help me when I need help with something that needs some code. Drupal is pervasive in government institutions and education, and, if the community is scaled organically then we have the opportunity to change the world with it. We can save tax money by sharing code, local communities can be involved in building and maintaining the technology they need, and people who want to build their own businesses and lives online can do it freely and often without code. On top of that, creatives can publish their videos and art and writings online, and we can build a framework for freedom. But not in this current configuration, not when it is seemingly the business players who are supported. Those same ones who many in the community have had to help rescue projects from and been squeezed out because they don't conform to the 9-5 mentality so don't fit into the current corporate structure which seems to be gradually taking over this project. Those corporates who we hear find it fine to take people to strip clubs to help make a sale but then say they are ousting people because they don't align with their values. Well, I don't value taking people to strip clubs just to win a sale - does that mean I'm not Drupal enough? If so, then I don't really want to be part of that, I find that way more harmful actions than any alleged future actions.

Back to the story, because that's why I do support Drupal as it enabled me to earn and live. I began to build my life back up again. I rented a room in my sister's ex's house who lived just down the road from her - they had a son together so were still in contact regularly. I had my newly found interest in Drupal and I did various things to promote it as I thought it was amazing. There was a site which you could post up short videos of 12 (I think) seconds so I used to do my 'module of the day' and tried to make them entertaining - I think I still have them somewhere. But it was when I put the word 'Drupal' on my LinkedIn profile that things changed as I had a call the next day from a London agency who needed some 'urgent' help. It was for a sponsor of the triathlon who wanted a Facebook app integration so people could support their friends. The agency had promised the app but it was late, however they were under contract to deliver so although I built it, it was only live for a day before the event itself. This was my first experience of the digital agency world, there have been many to follow, most of which haven't been that good, which is why I believe the agency model is dead.

The project gave me enough money to move to Brighton - a place I'd heard was the epicentre for the web in the UK so thought it would be a good place to set up my open source coworking cafe concept, and I could fund through my Drupal work. I'd made a little block for my website which displayed my LinkedIn profile and as Drupal had helped me, I decided to post it to the site so others could use it too. I got no response (well I did six years later after talking about it), so I still didn't see the community side of it much. I networked like hell in Brighton and within a short time had gained a couple of high-profile sites to work on, one of which the developer I worked with decided he was moving to London and gave me a couple of his old projects he didn't want to deal with any more. Not realising the amount of work involved with one of these projects I ended up massively underquoting but I'd promised the work so I did it, ending up moving out of my flat into a small bedsit to save money in the process. When I handed over the project I said "you'll need to work with a designer now as I've only selected rounded corners" but still to today all they've done is put a front-page on the site. It's still one of the projects I'm most proud of - it's got more information on children's books than amazon and is highly respected in the industry. I'm surprised it's still going, but my background has always been making things work well so they don't break - not so good for business I guess but I like to take pride in the work I do. You can check the site out at BooksForKeeps.co.uk.

I finished that project and did more networking, including going to an event called 'Connecting Innovation' where I saw Ken Thompson present his work on 'Virtual Enterprise Networks' in which he details how the organisations of the future aren't the big monolithic corporations but instead networks of smaller organisations, freelancers, and so on. He'd written a book including all the models he'd used in practice to build these Virtual Enterprise Networks around the world, for example when NASA wanted to deal with suppliers from outside the U.S. but had no interface for doing this. To me it was an eye-opener as this was precisely how the Drupal community worked. I had set up a local Drupal Users Group and we all shared code and information. I still hadn't been to an 'official' Drupal event, I simply didn't have the funds at the time so only dreamed of attending.

Business grew and after less than a year in my tiny bedsit with an outlook of a wall I managed to have enough to move out. I'd been checking the property ads every day and this amazing-looking apartment with floor to ceiling windows became available in the centre of Brighton which I thought would be a great base for my business, not only the Drupal side as I worked from home but also to start something up on my vision of the open source coworking cafe side. It was central, I could fit a few people in for lunchtime talks, all was good. So I thought, more on that later, first back to the Drupal.

I also had enough funds to attend my first DrupalCon, this year it was in Copenhagen so I decided to go. It was weird though, there were all these people sitting around long tables working away at their laptops to which just confused me - why would they go to the trouble of paying to get to a conference then not go to the talks and just sit tapping away at their computers? Didn't make sense. 

I was keen to talk to someone 'in power' about my findings about the Virtual Enterprise Networks as I thought I'd be bringing them a potential answer as to how they could spread 'good' Drupal by connecting and helping each other out - just as we did with the code and at our local meetups. I saw someone go past who I recognised from the Drupal Association so asked him if there was anyone I could talk to at the DA about this, to which he replied that they were 'all too busy working for large corporations' and brushed me off. I was later told he was 'autistic'. My friend who I was with at the time just looked at me, both of us in surprise. I resigned myself to finding out what else was going on and decided to stop at one of the tables where people were working away and find out what was going on. That's when I first met Angie/webchick who said they were testing the upgrade script from 6 to 7 so I could try that on a site I'd built. The only one I had was the children's book site however I was in my 'module buffet' phase so there were masses of modules on it (tabs within mini-panels within panels and so on!) and surprisingly it didn't work.

This experience didn't stop my pursuit of what I thought was trying to help the community solve its scaling issues though and I started to go to more events. I went to my first DrupalCamp in Cambridge and that's where I met a team of guys who were building native CRM in Drupal which I thought was amazing. Having come from an integrated platform previously I could see how this made Drupal much more of a full product and how essential to the framework it could be. I ran a couple of DrupalCamps in Brighton at which one of them they did a 36 hour BoF on the CRM which again I thought was amazing as this was how software should be debated and developed, by collaboration not everyone in their own pigeon hole making their own version of some common functionality.

I started to go to the CxO meetups which were happening as I wanted to achieve two things - grow my own business so I could build my vision, and connect the Drupal business as that would support my vision too. A platform where we commoditise common functionality across the world - every business vertical has similar functionality required, it's the people and the products that are the differentiators, unless you're a software company. I tried out one of the exercises from the Virtual Enterprise Network book at one of the CxO events - the synergy discovery - but I'm not a good person at this type of thing and when getting people to put up post-it notes of their skills they all just put Drupal. The idea is people obviously have all different skills, experience and expertise and the goal is to group and cluster those and connect with complementary skills so as to create collaborative products and services. I was told at the time by the organiser that it was the way of the community - if something didn't work then people simply weren't interested and the idea wasn't worth pursuing. I realise now that's just another person's opinion, you have to have belief in your own thoughts in order to make things happen.

So, on we go and a few more CxO events down I find we're all talking about the same issues over and over again. Where the code is shared and discussed online via the drupal.org infrastructure, the business people don't use the complicated site and so don't have anywhere online to share. They do collaborate in 'secret' in their own groups, and as I went to more events they all seemed to get further away from community and more 'business', with restrictions on the size of company you had to be to attend, which basically pushed me out of the picture. At one of the last events I went to though I met another Director of the DA and it was at the time when they first introduced community elections for an 'at large' Director. I asked how many had applied to which I was told three, to which I though did no-body care and decided that night to apply. With my following I had grown on social networks by helping others out with their Drupal issues due to the seeming lack of support channels due to many businesses making money from the support side so no want to fund and/or foster the free ways of support, I was voted in and within a week I found I was now also a Director of the DA.

Obviously having to cut a very long story short before we all fall asleep, I didn't get much in the way of communication or involvement in my time there, I did manage to help push through a Marketing & Branding Committee which wasn't really supported that much, and I found myself realising that the agenda was there to stay and they had their way and that was it, I was just seen as trouble. I remember once sitting around the table when they were talking about revenue and how much the individual memberships didn't bring in (look at how they compare to total revenue) and the founder's remarks were "well why do we even bother having them". Now I understand that could've been taken out of context, but then I look back and wonder - this guy is running the fastest growing private software company in the country and has known pretty much nothing else apart from uni and successful career, I don't think he's ever had to really worry about where his next meal is coming from. He's obviously good at business, and seems to be good at people skills when in public, so I gave him the benefit of doubt that I'd just either heard it wrong or taken it the wrong way. Drupal may have been released by him, but it's his army of contributors who build and maintain it, many of whom work tirelessly and often for free in their spare time, even those who do get paid to work on it.

So skipping along, when my term finished at the DA I decided to set up a co-operative online as I'd met many people from many Drupal businesses who were more than keen to help market the project and product better, and who lacked a place to communicate and gather online. Sure there's the partner networks and the DA itself they can sponsor, and the events, but they're all run by someone else with their own agenda - a Virtual Enterprise Network is where the members own and operate the group, much like a co-operative if you look at something like Mondragon Corporation. This was surely a better way to scale than a top-down infrastructure which seemed to result in many issues - helping some get richer but at the expense of making the product and processes harder for the rest. I set up the site, I set up some skills tags and people connected and did business but I failed to capture the business model myself and at the time had high monthly costs so could not keep it going without revenue. I asked for membership dues, of which about four people paid, of which was the equivalent to about half a day's Drupal development, so didn't last long.

By that time I was quite disheartened so I kind of went back to just doing projects but then in 2014 an opportunity came along to rent an annexe opposite the coworking space in Brighton which I had been a member of on-and-off since its inception and as I'd heard that this space was now full I thought this could be an opportunity to finally get my space up and running. I took a gamble and paid the deposit and rent, and worked on two projects at the same time to ensure it had some time to get up and running whilst the overspill from next door came forth. This didn't happen, and I didn't have the time to do enough promotion. The building wasn't exactly right either, and it ended up being a costly three-month experiment and I lost over ten grand. We did however have another spectacular party, and I was proud to host the first VR Brighton events, something I shall never forget being a huge VR fan. As for getting enough things up and running at the same time - well apart from not having enough time, the space wasn't big enough to do that and so it never would've been the dream. I knew most of that but was hoping at least I would be able to get enough members to cover costs and start to build the rest. 

I then started to do my first commercial collaborations with the CRM guys and decided it was something I should promote more as it was profitable and gave me time to work on my other project as opposed to if I was sitting there building sites all day. I was once again too confident and as DrupalCon was in L.A. that year I decided I should go and spread the word about CRM, and I wanted to see the place in real life too, after all, I'd been told I should go there by you-know-who so I thought what the hell and booked my tickets. I guess I was on a high too as I decided as the hotel was the main cost I would take along my friend as he was looking for a way to change his work and I'd been extolling the virtues of Drupal to him and saying how it helped anyone build a career if they wanted to. He obviously just saw it as a free trip, but I don't regret offering, I just need to choose more carefully whom I offer opportunities like this too if I am ever in a position to again.

So L.A. was fun, and we did a BoF on the CRM where a few people turned up - all from very large corporations, but nothing came of it as I didn't have the team around me to make something like this happen. I returned back to Brighton and carried on the continued search for work and odd project. But times were different now and many agencies had grown up and the available pool of work was less. I didn't do front-end so I worked with others on that, but one kept continually letting me down and mocked me when I worked with them again at the fact I had, and in the process had practically lost me any faith that client had in my judgement by then. 

Then came along winter and my flat where I was day in day out became infested with mice. This had happened a few years back but this time they were back with a vengeance. The landlord flat refused to do anything proper about it, a temporary fix was just that, and I was getting ill with the stress of it all. I managed to get some work with a local digital agency but they were another one who knew little about the product they were building so I had to, for the sake of the client's project, do the opposite of what they asked. I implemented a rock solid architecture of custom entities, purposely making it hard for them to mess around with and as far as I know the site is still based on those. They wanted to play around with different views and lots of front-end stuff, which you can do fine if you have the base to do it on, but if you start by doing that you soon end up in trouble. I couldn't deal with their want of me being in the office all the time as nothing apart from seemingly useless meetings happened all day then at the end of the day they'd asked how it was going and I'm sure they must've thought I had a double working on the stuff whilst I was at their inconsequential meetings they just wanted a face at. I ended up working all over the Xmas period with mice running around my flat making me more ill, and I turned up the day they all came back from hols with their architecture and recommended they find someone now with more front-end skills than myself. I recommended a couple of people, they eventually found someone who is still there today but I hear they don't have the project any more. That was the idea that it would be handed over to the client to maintain, but it was months overdue so I don't know what went on after I left. I do know that when I tested the site out I did a search which resulted in no results, that wouldn't have happened if they'd followed my advice using facets and bottom-up SOLR search from actual data instead of top-down "nice looking" but fundamentally flawed implementation they made. Such is life.

So I'm back in the flat, mice running around, and a landlord who refuses to fix it. Why is this relevant? Well it took me four years to get a replacement boiler - the original one made loud bangs and I ended up boiling water for years as I gave up trying. I think this is relevant because I see a pattern of me getting into interdependent relationships where I'm the submissive and I let people dominate me. Do I do it for kinks? No. Am I vulnerable? Maybe - stupid more like and just need to raise my own self-confidence and put up a few barriers, at least know when to get out of situations and when to say no to stuff. I said no to my landlord, I refused to pay him rent until he sorted out the mice problem, and he then decided to use the law to evict me from the flat as laws to stop this were only introduced in 2015 and my contract with him was from before that.

He finally evicted me in June last year and served me notice that he was taking me to court. I wrote up my defence, including all the info about the boiler and how he'd left me without a gas safety certificate for months when the original one finally failed it. His solicitor sent me copies of all the certificates which I thanked him for as it proved my defence, but he still didn't back down. I decided it wasn't worth trying to stay in Brighton and pay expensive living costs when I couldn't afford to start my other business vision due to the extortionate rents there (higher than Silicon Roundabout in London), the town was full of rubbish and with 1 in 69 people homeless the streets were lined with those I wanted to help but couldn't because there was no funding for that sort of thing down south. Sure, if you want to make an internet of things gadget which will suck up lots of data you can sell to the system you'll be fine, but do something which will benefit many but not maybe those who are currently rich getting richer and you're sod out of luck.

As funny as the world goes, I decided to stay with a friend for a week before returning back to Essex to stay with my folks for a while until the next DrupalCamp visit or going abroad opportunity came up (couldn't see the point in staying in expensive UK), and whilst staying there I found my dream space to set up my vision. It was the White Hart Hotel in Lewes and it was where Thomas Paine used to attend the debating society there. He used to talk about his ideas and vision but it was only when a fellow attendee said he should write about them and publish it that he ended up writing Common Sense, attributed to helping the American and French Revolutions. I believe we are now at this point with the internet - we have all the tools we need to communicate, share our ideas, and co-operate together for a better world, we just don't use them. We have the corporate version of the internet where it isn't their interests to help you but to grow their own business. So instead of having local spaces where people can go make use of things like video studios, rooms for presenting in front of people and streamed online, art on the walls and online, and everything else I cover in the Co-operative.club concept, we have instead a surveillance society and a few corporations at the top with many working to keep their systems going for little or no pay.

The Lewes building is £2m. It could work from crowd-funding, but quite rightly I see you need to build community first and grow from there. So I've thought about starting one up here, but first just building the network online and running a few events locally to get things up and running. I didn't do that before because I thought it would take too long but here I am 13 years in, £60k or more in debt, and a community I'm not even sure I actually should be worrying about as although it's kept me alive for years, I've never wanted to build websites and that's where I fail on the business side because my heart simply isn't in it. On top of that we have all the community issues which people again and again talk about and a community which isn't equipped currently to collaborate at scale as a community, they all seem too happy to take the quick cash, keep in their top-down partner networks and everything's ok. Well, I personally don't think that's going to work in the long run, and I'm more upset about the vision we won't be seeing if it just turns totally corporate. You can't pay people to have passion, and some of us aren't equipped or indeed want to 'have a job'. 

I want to help people, including myself, enjoy the time they have on this planet by exploiting free software. It's the LibrePlanet conference this weekend which I hope some will be streamed as it will give me a boost I need. I was lucky enough to attend when I first went to Toronto back in 2005 before all the crazy stuff hit the fan, that's where I met Larry Lessig and one of my heros Eben Moglen who I believe is one of the best orators in the world. I spoke to them about my concept and they 'got it', and that counts for a lot to me. Or is that just my attachment issues again? Am I still suffering from worry about being abandoned when my parents split up? Should I just "get off my arse and do a job?". Well, on the latter front I've a little money due, but no - no job in the immediate future, once I decide on a direction I'll be out there networking like crazy whether on or offline, I don't have the want for trinkets I used to have as I realise that's not what makes me happy and there's a lot cheaper places than the UK I can go live. I did start a daily vlog on my project but I've focused it on the Drupal community as I thought that would be a constraint good enough because we do already collaborate. But now with all the issues I'm not sure of whether it's worth pursuing without some serious backing, and it seems everyone's hunky dory with the way the business runs, down to the everyone-except-for-me clapping at the recent talk mentioning we should stop all the debate about Free vs Open when it is so obviously not an insignificant thing, Open Source is for business, Free Software is for life.

Anyway, that's the ramble. I was going to write this up much nicer but it's been three months and that hasn't happened until today, I'm just sad it took such a shitty event to make it happen and I hope things will change. I certainly need to as I keep seeing buildings which would be perfect for connected community spaces.

Category Creativity Tags drupal Drupal Association Drupal Planet Add new comment
Categories: FLOSS Project Planets

Reinout van Rees: Fossgis: open source for emergencies - Marco Lechner

Planet Python - Thu, 2017-03-23 10:28

(One of my summaries of a talk at the 2017 fossgis conference).

He works for the Bundesamtes fuer Strahlenschutz, basically the government agency that was started after Chernobil to protect against and to measure radioactivity. The software system they use/build is called IMIS.

IMIS consists of three parts:

  • Measurements (automatic + mobile measurements + laboratory results).
  • Prediction system. Including documentation (managed in Plone, a python CMS system).
  • Decision support. Help support the government layers that have to make the decisions.

They have a simple map at odlinfo.bfs.de.

The current core of the system is proprietary. They are dependent on one single firm. The system is heavily customized for their usage.

They need a new system because geographical analysis keeps getting more important and because there are new requirements coming out of the government. The current program cannot handle that.

What they want is a new system that is as simple as possible; that uses standards for geographical exchange; they don't want to be dependent on a single firm anymore. So:

  • Use open standards, so OGC. But also a specific world-wide nuclear info protocol.
  • Use existing open source software. OSGEO.
  • If we need something special, can we change/extend existing open source software?
  • If not, then it is OK to create our their software. Under an open source license.

They use open source companies to help them, including training their employees. And helping getting these employees used to modern software development (jenkins, docker, etc.)

If you use an open source strategy, what do you need to do to make it fair?

  • Your own developments should also be open source!
  • You need your own test and build infrastructure. (For instance Jenkins)
  • You need to make it easy to start working with what you made: documentation, docker, buildout (!), etc.

(Personal note: I didn't expect to hear 'buildout' at this open source GIS conference. I've helped quite a bit with that particular piece of python software :-) )

Categories: FLOSS Project Planets

.VDMi/Blog: Creating default pages in Drupal 8

Planet Drupal - Thu, 2017-03-23 10:16
Here at .VDMi/ we like to use nodes for everything, even for things where you would normally use a View page, a search page for example. We do this because nodes bring a great deal of advantages. This blog explains how we create these default pages, make sure they don’t get deleted accidentally and how to use different nodes as frontpage for different languages.
Categories: FLOSS Project Planets

Bryan Pendleton: Assessing an estimate

Planet Apache - Thu, 2017-03-23 09:56


Can this really be true?

Addressing Detroit’s Basic Skills Crisis

Various estimates of the scale of need for basic skills services in the region convey a crisis-level order of magnitude.
  • The National Institute for Literacy estimates that 47% of adults (more than 200,000 individuals) in the City of Detroit are functionally illiterate, referring to the inability of an individual to use reading, speaking, writing, and computational skills in everyday life situations.
  • We also know that of the 200,000 adults who are functionally illiterate, approximately half have a high school diploma or GED, so this issue cannot be solely addressed by a focus on adult high-school completion.
  • The remaining 100,000 of these functionally illiterate adults (age 25 and older) lack a high school diploma or GED, another prerequisite for employment success.

I'm not sure how this institute made this estimate.

Later, the report expands somewhat on the topic:

Generally, those adults who score at Level 1 (on a scale of 1 to 5, lowest to highest) have difficulty performing such everyday tasks as locating an intersection on a street map, reading and comprehending a short newspaper article, or calculating total costs on an order form.

It isn't clear whether their estimate was that all 47% were at "Level 1", or whether those were five levels of illiteracy (versus five levels of literacy), but no matter how you slice it, those are some astonishing claims about the literacy problem in the greater Detroit region.

Categories: FLOSS Project Planets

Pronovix: Web APIs in Drupal: success takes more than an endpoint

Planet Drupal - Thu, 2017-03-23 09:45

Web APIs are not just useful when making headless sites in Drupal: large Drupal sites often hold valuable information that could also be useful outside the site's context. Media companies might want to expose historical media content, community sites could show data about their community activities, e-commerce sites tend to open an API for their affiliates and partners.

While it is possible to use Drupal 7 and Drupal 8 as an API backend, a lot of functionalities that describe a mature API service do not come out of the box. In this post we will explain what key concepts you have to keep in mind when designing an API service, why they are important and how APIgee Edge can make it easier to build a full-featured API webservice in Drupal successfully.

Categories: FLOSS Project Planets

Chromatic: Dependency Injection in Drupal 8 Plugins

Planet Drupal - Thu, 2017-03-23 09:15

Dependency Injection in Drupal 8 Plugins can trip you up if you focus on the Dependency Injection part and forget about the Plugin part. This blog post shows key differences to keep in mind when you're working with D8 Plugins.

Categories: FLOSS Project Planets

Boosting performance with shader binary caching in Qt 5.9

Planet KDE - Thu, 2017-03-23 09:02

Now that Qt 5.9 is getting closer, let’s take a look at a minor but immensely useful improvement to the basic OpenGL enablers that form the foundation of Qt Quick and the optional OpenGL-based rendering path of QPainter.

Those looking at the documentation snapshots for 5.9 may have already come across some new functions in the venerable QOpenGLShaderProgram. What is more, most internal usages in Qt have been switched over to the new API. What does this mean in practice?

As explained here, such shader programs will attempt to cache the program binaries on disk using GL_ARB_get_program_binary or the standard equivalents in OpenGL ES 3.0. When no support is provided by the driver, the behavior is equivalent to the non-cached case. The files are stored in the global or per-process cache location, whichever is writable. The result is a nice boost in performance when a program is created with the same shader sources next time.

How big is the improvement? It varies. Some drivers have already been doing some sort of caching for the past couple of years, while some others have similar features in the pipeline. However, the gains turn out to be quite significant in practice on devices that are out in the field right now:

Do not read too much into the actual numbers. What is important is the difference between Qt 5.8 and 5.9. Also, a simple Qt Quick or GL-backed QPainter scene will definitely not use 10 programs, but as complexity grows, with Qt Graphical Effects and custom ShaderEffect items entering the picture, getting similar improvements does not look far fetched anymore.

In fact we gain something even on systems that employ shader caching already. Therefore every application’s startup and view switching times are expected to benefit with Qt 5.9 – without having to change anything.

Applications that use QOpenGLShaderProgram on their own can usually switch to the cacheable function variants by just changing the name in the function call. The change have to be a conscious decision, though, since some of the APIs change semantics when program binaries are used. Most notably, QOpenGLShader, addShader(), and removeShader() are incompatible with the program-level caching since they rely on individual shader compilation.

That’s it for now, stay tuned for more posts about exciting upcoming Qt 5.9 and 5.10 features.

The post Boosting performance with shader binary caching in Qt 5.9 appeared first on Qt Blog.

Categories: FLOSS Project Planets

lakshminp.com: DIY Drupal hosting: DrupalVM

Planet Drupal - Thu, 2017-03-23 07:46

Time for a little confession. I didn't intend to showcase DrupalVM as a DIY Drupal hosting solution when I conceived this series idea. Jeff Geerling, DrupalVM's creator hinted at using DrupalVM as a viable solution for small to medium sites in the first post of the series. It was an idea worth exploring and the result is this post.

Categories: FLOSS Project Planets

Neil McGovern: GNOME ED Update – Week 12

Planet Debian - Thu, 2017-03-23 07:43
New release!

In case you haven’t seen it yet, there’s a new GNOME release – 3.24! The release is the result of 6 months’ work by the GNOME community.

The new release is a major step forward for us, with new features and improvements, and some exciting developments in how we build applications. You can read more about it in the announcement and release notes.

As always, this release was made possible partially thanks to the Friends of GNOME project. In particular, it helped us provide a Core apps hackfest in Berlin last November, which had a direct impact on this release.

Conferences GTK+ hackfest

I’ve just come back from the GTK+ hackfest in London – thanks to RedHat and Endless for sponsoring the venues! It was great to meet a load of people who are involved with GNOME and GTK, and some great discussions were had about Flatpak and the creation of a “FlatHub” – somewhere that people can get all their latest Flatpaks from.


As I’m writing this, I’m sitting on a train going to Heathrow, for my flight to LibrePlanet 2017! If you’re going to be there, come and say hi. I’ve a load of new stickers that have been produced as well so these can brighten up your laptop.

Categories: FLOSS Project Planets

PyBites: Module of the Week - ipaddress

Planet Python - Thu, 2017-03-23 06:30

While playing around with code for our post on generators we discovered the ipaddress module, part of the Standard Library. Such a handy little module!

Categories: FLOSS Project Planets

Reinout van Rees: Fossgis: sewer cadastre with qgis - jörg Höttges

Planet Python - Thu, 2017-03-23 06:24

(One of my summaries of a talk at the 2017 fossgis conference).

With engineer firms from the Aachen region they created qkan. Qkan is:

  • A data structure.
  • Plugins for Qgis.
  • Direct access. Not a specific application with restricted access, but unrestricted access from within Qgis. (He noticed lots of interest among the engineers to learn qgis during the project!)

It has been designed for the needs of the engineers that have to work with the data. You first import the data from the local sewer database. Qkan converts the data to what it needs. Then you can do simulations in a separate package. The results of the simulation will be visualized by Qkan in qgis. Afterwards you probably have to make some corrections to the data and give corrections back to the original database. Often you have to go look at the actual sewers to make sure the database is correct. Output is often a map with the sewer system.

Some functionality: import sewer data (in various formats). Simulate water levels. Draw graphs of the water levels in a sewer. Support database-level check ("an end node cannot occur halfway a sewer").

They took care to make the database schema simple. The source sewer database is always very complex because it has to hold lots of metadata. The engineer that has to work with it needs a much simpler schema in order to be productive. Qkan does this.

They used qgis, spatialite, postgis, python and qt (for forms). An important note: they used as many postgis functionality as possible instead of the geographical functions from qgis: the reason is that postgis (and even spatialite) is often much quicker.

With qgis, python and the "qt designer", you can make lots of handy forms. But you can always go back to the database that's underneath it.

The code is at https://github.com/hoettges

Categories: FLOSS Project Planets

CubicWeb: Introducing cubicweb-jsonschema

Planet Python - Thu, 2017-03-23 05:57

This is the first post of a series introducing the cubicweb-jsonschema project that is currently under development at Logilab. In this post, I'll first introduce the general goals of the project and then present in more details two aspects about data models (the connection between Yams and JSON schema in particular) and the basic features of the API. This post does not always present how things work in the current implementation but rather how they should.

Goals of cubicweb-jsonschema

From a high level point of view, cubicweb-jsonschema addresses mainly two interconnected aspects. One related to modelling for client-side development of user interfaces to CubicWeb applications while the other one concerns the HTTP API.

As far as modelling is concerned, cubicweb-jsonschema essentially aims at providing a transformation mechanism between a Yams schema and JSON Schema that is both automatic and extensible. This means that we can ultimately expect that Yams definitions alone would sufficient to have generated JSON schema definitions that would consistent enough to build an UI, pretty much as it is currently with the automatic web UI in CubicWeb. A corollary of this goal is that we want JSON schema definitions to match their context of usage, meaning that a JSON schema definition would not be the same in the context of viewing, editing or relationships manipulations.

In terms of API, cubicweb-jsonschema essentially aims at providing an HTTP API to manipulate entities based on their JSON Schema definitions.

Finally, the ultimate goal is to expose an hypermedia API for a CubicWeb application in order to be able to ultimately build an intelligent client. For this we'll build upon the JSON Hyper-Schema specification. This aspect will be discussed in a later post.

Basic usage as an HTTP API library

Consider a simple case where one wants to manipulate entities of type Author described by the following Yams schema definition:

class Author(EntityType): name = String(required=True)

With cubicweb-jsonschema one can get JSON Schema for this entity type in at different contexts such: view, creation or edition. For instance:

  • in a view context, the JSON Schema will be:

    { "$ref": "#/definitions/Author", "definitions": { "Author": { "additionalProperties": false, "properties": { "name": { "title": "name", "type": "string" } }, "title": "Author", "type": "object" } } }
  • whereas in creation context, it'll be:

    { "$ref": "#/definitions/Author", "definitions": { "Author": { "additionalProperties": false, "properties": { "name": { "title": "name", "type": "string" } }, "required": [ "name" ], "title": "Author", "type": "object" } } }

    (notice, the required keyword listing name property).

Such JSON Schema definitions are automatically generated from Yams definitions. In addition, cubicweb-jsonschema exposes some endpoints for basic CRUD operations on resources through an HTTP (JSON) API. From the client point of view, requests on these endpoints are of course expected to match JSON Schema definitions. Some examples:

Get an author resource:

GET /author/855 Accept:application/json HTTP/1.1 200 OK Content-Type: application/json {"name": "Ernest Hemingway"}

Update an author:

PATCH /author/855 Accept:application/json Content-Type: application/json {"name": "Ernest Miller Hemingway"} HTTP/1.1 200 OK Location: /author/855/ Content-Type: application/json {"name": "Ernest Miller Hemingway"}

Create an author:

POST /author Accept:application/json Content-Type: application/json {"name": "Victor Hugo"} HTTP/1.1 201 Created Content-Type: application/json Location: /Author/858 {"name": "Victor Hugo"}

Delete an author:

DELETE /author/858 HTTP/1.1 204 No Content

Now if the client sends invalid input with respect to the schema, they'll get an error:

(We provide a wrong born property in request body.)

PATCH /author/855 Accept:application/json Content-Type: application/json {"born": "1899-07-21"} HTTP/1.1 400 Bad Request Content-Type: application/json { "errors": [ { "details": "Additional properties are not allowed ('born' was unexpected)", "status": 422 } ] } From Yams model to JSON Schema definitions

The example above illustrates automatic generation of JSON Schema documents based on Yams schema definitions. These documents are expected to help developping views and forms for a web client. Clearly, we expect that cubicweb-jsonschema serves JSON Schema documents for viewing and editing entities as cubicweb.web serves HTML documents for the same purposes. The underlying logic for JSON Schema generation is currently heavily inspired by the logic of primary view and automatic entity form as they exists in cubicweb.web.views. That is: the Yams schema is introspected to determine how properties should be generated and any additionnal control over this can be performed through uicfg declarations [1].

To illustrate let's consider the following schema definitions which:

class Book(EntityType): title = String(required=True) publication_date = Datetime(required=True) class Illustration(EntityType): data = Bytes(required=True) class illustrates(RelationDefinition): subject = 'Illustration' object = 'Book' cardinality = '1*' composite = 'object' inlined = True class Author(EntityType): name = String(required=True) class author(RelationDefinition): subject = 'Book' object = 'Author' cardinality = '1*' class Topic(EntityType): name = String(required=True) class topics(RelationDefinition): subject = 'Book' object = 'Topic' cardinality = '**'

and consider, as before, JSON Schema documents in different contexts for the the Book entity type:

  • in view context:

    { "$ref": "#/definitions/Book", "definitions": { "Book": { "additionalProperties": false, "properties": { "author": { "items": { "type": "string" }, "title": "author", "type": "array" }, "publication_date": { "format": "date-time", "title": "publication_date", "type": "string" }, "title": { "title": "title", "type": "string" }, "topics": { "items": { "type": "string" }, "title": "topics", "type": "array" } }, "title": "Book", "type": "object" } } }

    We have a single Book definition in this document, in which we find attributes defined in the Yams schema (title and publication_date). We also find the two relations where Book is involved: topics and author, both appearing as a single array of "string" items. The author relationship appears like that because it is mandatory but not composite. On the other hand, the topics relationship has the following uicfg rule:

    uicfg.primaryview_section.tag_subject_of(('Book', 'topics', '*'), 'attributes')

    so that it's definition appears embedded in the document of Book definition.

    A typical JSON representation of a Book entity would be:

    { "author": [ "Ernest Miller Hemingway" ], "title": "The Old Man and the Sea", "topics": [ "sword fish", "cuba" ] }
  • in creation context:

    { "$ref": "#/definitions/Book", "definitions": { "Book": { "additionalProperties": false, "properties": { "author": { "items": { "oneOf": [ { "enum": [ "855" ], "title": "Ernest Miller Hemingway" }, { "enum": [ "857" ], "title": "Victor Hugo" } ], "type": "string" }, "maxItems": 1, "minItems": 1, "title": "author", "type": "array" }, "publication_date": { "format": "date-time", "title": "publication_date", "type": "string" }, "title": { "title": "title", "type": "string" } }, "required": [ "title", "publication_date" ], "title": "Book", "type": "object" } } }

    notice the differences, we now only have attributes and required relationships (author) in this schema and we have the required listing mandatory attributes; the author property is represented as an array which items consist of pre-existing objects of the author relationship (namely Author entities).

    Now assume we add the following uicfg declaration:

    uicfg.autoform_section.tag_object_of(('*', 'illustrates', 'Book'), 'main', 'inlined')

    the JSON Schema for creation context will be:

    { "$ref": "#/definitions/Book", "definitions": { "Book": { "additionalProperties": false, "properties": { "author": { "items": { "oneOf": [ { "enum": [ "855" ], "title": "Ernest Miller Hemingway" }, { "enum": [ "857" ], "title": "Victor Hugo" } ], "type": "string" }, "maxItems": 1, "minItems": 1, "title": "author", "type": "array" }, "illustrates": { "items": { "$ref": "#/definitions/Illustration" }, "title": "illustrates_object", "type": "array" }, "publication_date": { "format": "date-time", "title": "publication_date", "type": "string" }, "title": { "title": "title", "type": "string" } }, "required": [ "title", "publication_date" ], "title": "Book", "type": "object" }, "Illustration": { "additionalProperties": false, "properties": { "data": { "format": "data-url", "title": "data", "type": "string" } }, "required": [ "data" ], "title": "Illustration", "type": "object" } } }

    We now have an additional illustrates property modelled as an array of #/definitions/Illustration, the later also added the the document as an additional definition entry.


This post illustrated how a basic (CRUD) HTTP API based on JSON Schema could be build for a CubicWeb application using cubicweb-jsonschema. We have seen a couple of details on JSON Schema generation and how it can be controlled. Feel free to comment and provide feedback on this feature set as well as open the discussion with more use cases.

Next time, we'll discuss how hypermedia controls can be added the HTTP API that cubicweb-jsonschema provides.

[1]this choice is essentially driven by simplicity and conformance when the existing behavior to help migration of existing applications.
Categories: FLOSS Project Planets

Reinout van Rees: Fossgis: creating maps with open street map in QGis - Axel Heinemann

Planet Python - Thu, 2017-03-23 05:55

(One of my summaries of a talk at the 2017 fossgis conference).

He wanted to make a map for a local run. He wanted a nice map with the route and the infrastructure (start, end, parking, etc). Instead of the usual not-quite-readable city plan with a simple line on top. With qgis and openstreetmap he should be able to make something better!

A quick try with QGis, combined with the standard openstreetmap base map, already looked quite nice, but he wanted to do more customizations on the map colors. So he needed to download the openstreetmap data. That turned into quite a challenge. He tried two plugins:

  • OSMDownloader: easy selection, quick download. Drawback: too many objects as you cannot filter. The attribute table is hard to read.
  • QuickOSM: key/value selection, quick. Drawback: you need a bit of experience with the tool, as it is easy to forget key/values.

He then landed on https://overpass-turbo.eu . The user interface is very friendly. There is a wizard to get common cases done. And you can browse the available tags.

With the data downloaded with overpass-turbo, he could easily adjust colors and get a much nicer map out of it.

You can get it to work, but it takes a lot of custom work.

Some useful links:

https://taginfo.openstreetmap.org http://tagfinder.herokuapp.com https://gis.stackexchange.com

Photo explanation: just a nice unrelated picture from the recent beautiful 'on traxs' model railway exibition (see video )

Categories: FLOSS Project Planets

Reinout van Rees: Fossgis: introduction on some open source software packages

Planet Python - Thu, 2017-03-23 05:55

(One of my summaries of a talk at the 2017 fossgis conference).

The conference started with a quick introduction on several open source programs.

Openlayers 3 - Marc Jansen

Marc works on both openlayers and GeoExt. Openlayers is a javascript library with lots and lots of features.

To see what it can do, look at the 161 examples on the website :-) It works with both vector layers and raster layers.

Openlayers is a quite mature project, the first version is from 2006. It changed a lot to keep up with the state of the art. But they did take care to keep everything backwards compatible. Upgrading from 2.0 to 2.2 should have been relatively easy. The 4.0.0 version came out last month.


  • Allows many different data sources and layer types.
  • Has build-in interaction and controls.
  • Is very actively developed.
  • Is well documented and has lots of examples.

The aim is to be easy to start with, but also to allow full control of your map and all sorts of customization.

Geoserver - Marc Jansen

(Again Marc: someone was sick...)

Geoserver is a java-based server for geographical data. It support lots of OGC standards (WMS, WFS, WPS, etc). Flexible, extensible, well documented. "Geoserver is a glorious example that you can write very performant software in java".

Geoserver can connect to many different data sources and make those sources available as map data.

If you're a government agency, you're required to make INSPIRE metadata available for your maps: geoserver can help you with that.

A big advantage of geoserver: it has a browser-based interface for configuring it. You can do 99% of your configuration work in the browser. For maintaining: there is monitoring to keep an eye on it.

Something to look at: the importer plugin. With it you get a REST API to upload shapes, for instance.

The latest version also supports LDAP groups. LDAP was already supported, but group membership not yet.

Mapproxy - Dominik Helle

Dominik is one of the MapProxy developers. Mapproxy is a WMS cache and tile cache. The original goal was to make maps quicker by caching maps.

Some possible sources: WMS, WMTS, tiles (google/bing/etc), MapServer. The output can be WMS, WMS-C, WMTS, TMS, KML. So the input could be google maps and the output WMS. One of their customers combines the output of five different regional organisations into one WMS layer...

The maps that mapproxy returns can be stored on a local disk in order to improve performance. They way they store it allows mapproxy to support intermediary zoom levels instead of fixed ones.

The cache can be in various formats: MBTiles, sqlite, couchdb, riak, arcgis compact cache, redis, s3. The cache is efficient by combining layers and by omitting unneeded data (empty tiles).

You can pre-fill the cache ("seeding").

Some other possibilities, apart from caching:

  • A nice feature: clipping. You can limit a source map to a specific area.
  • Reprojecting from one coordinate system to another. Very handy if someone else doesn't want to support the coordinate system that you need.
  • WMS feature info: you can just pass it on to the backend system, but you can also intercept and change it.
  • Protection of your layers. Password protection. Protect specific layers. Only allow specific areas. Etcetera.
QGis - Thomas Schüttenberg

QGis is an opern source gis platform. Desktop, server, browser, mobile. And it is a library. It runs on osx, linux, windows, android. The base is the QT ui library, hence the name.

Qgis contains almost everything you'd expect from a GIS packages. You can extend it with plugins.

Qgis is a very, very active project. Almost 1 million lines of code. 30.000+ github commits. 332 developers have worked on it, in the last 12 months 104.

Support via documentation, mailinglists and http://gis.stackexchange.com/ . In case you're wondering about the names of the releases: they come from the towns where the twice-a-year project meeting takes place :-)

Since december 2016, there's an official (legal) association.

QGis 3 will have lots of changes: QT 5 and python 3.

Mapbender 3 - Astrid Emde

Mapbender is library to build webgis applications. Ideally, you don't need to write any code yourself, but you configure it instead in your browser. It also supports mobile usage.

You can try it at http://demo.mapbender3.org/ . Examples are at http://mapbender3.org/?q=en/gallery .

You can choose a layout and fill in and configure the various parts. Layers you want to show: add sources. You can configure security/access with roles.

An example component: a search form for addresses that looks up addresses with sql or a web service. Such a search form can be a popup or you can put it in the sidebar, for instance. CSS can be customized.

PostNAS - Astrid Emde, Jelto Buurman

The postnas project is a solution for importing ALKIS data, a data exchange format for the german cadastre (Deutsch: Kataster).

PostNAS is an extension of the GDAL library for the "NAS" vector data format. (NAS = normalisierte Austausch Schnittstelle, "normalized exchange format"). This way, you can use all of the gdal functionality with the cadastre data. But that's not the only thing: there's also a qgis plugin. There is configuration and conversion scripts for postgis, mapbender, mapserver, etc.

They needed postprocessing/conversion scripts to get useful database tables out of the original data, tables that are usable for showing in QGis, for instance.

So... basically a complete open source environment for working with the cadastre data!

Photo explanation: just a nice unrelated picture from the recent beautiful 'on traxs' model railway exibition (see video )

Categories: FLOSS Project Planets

Tomasz Früboes: Unittesting print statements

Planet Python - Thu, 2017-03-23 05:10

Recently I was refactoring a small package that is supposed to allow execution of arbitrary python code on a remote machine. The first implementation was working nicely but with one serious drawback – function handling the actual code execution was running in a synchronous (blocking) mode. As the result all of the output (both stdout and stderr) was presented only at the end, i.e. when code finished its execution. This was unacceptable since the package should work in a way as transparent to the user as possible. So a wall of text when code completes its task wasn’t acceptable.

The goal of the refactoring was simple – to have the output presented to the user immediately after it was printed on the remote host. As a TDD worshipper I wanted to start this in a kosher way, i.e. with a test. And I got stuck.

For a day or so I had no idea how to progress. How do you unittest the print statements? It’s funny when I think about this now. I have used a similar technique many times in the past for output redirection, yet somehow haven’t managed to make a connection with this problem.

The print statement

So how do you do it? First we should understand what happens when print statement is executed. In python 2.x the print statement does two things – converts provided expressions into strings and writes the result to a file like object handling the stdout. Conveniently it is available as sys.stdout (i.e. as a part of sys module). So all you have to do is to overwrite the sys.stdout with your own object providing a ‘write’ method. Later you may discover, that some other methods may be also needed (e.g. ‘flush’ is quite often used), but for starters, having only the ‘write’ method should be sufficient.

A first try – simple stdout interceptor

The code below does just that. The MyOutput class is designed to replace the original sys.stdout:

import unittest import sys def fn_print(nrepeat): print "ab"*nrepeat class MyTest(unittest.TestCase): def test_stdout(self): class MyOutput(object): def __init__(self): self.data = [] def write(self, s): self.data.append(s) def __str__(self): return "".join(self.data) stdout_org = sys.stdout my_stdout = MyOutput() try: sys.stdout = my_stdout fn_print(2) finally: sys.stdout = stdout_org self.assertEquals( str(my_stdout), "abab\n") if __name__ == "__main__": unittest.main()

The fn_print function provides output to test against. After replacing sys.stdout we call this function and compare the obtained output with the expected one. It is worth noting that in the example above the original sys.stdout is first preserved and then carefully restored inside the ‘finally’ block. If you don’t do this you are likely to loose any output coming from other tests.

Is my code async? Logging time of arrival

In the second example we will address the original problem – is output presented as a wall of text at the end or maybe in real time as we want to. For this we will add time of arrival logging capability to the object replacing sys.stdout:

import unittest import time import sys def fn_print_with_delay(nrepeat): for i in xrange(nrepeat): print # prints a single newline time.sleep(0.5) class TestServer(unittest.TestCase): def test_stdout_time(self): class TimeLoggingOutput(object): def __init__(self): self.data = [] self.timestamps = [] def write(self, s): self.timestamps.append(time.time()) self.data.append(s) stdout_org = sys.stdout my_stdout = TimeLoggingOutput() nrep = 3 # make sure is >1 try: sys.stdout = my_stdout fn_print_with_delay(nrep) finally: sys.stdout = stdout_org for i in xrange(nrep): if i > 0: dt = my_stdout.timestamps[i]-my_stdout.timestamps[i-1] self.assertTrue(0.5<dt<0.52) if __name__ == "__main__": unittest.main()

The code is pretty much self-explanatory – the fn_print_with_delay function prints newlines in half of a second intervals. We override sys.stdout with an instance of a class capable of storing timestamps (obtained with time.time()) of all calls to the write method. At the and we assert the timestamps are spaced half of a second approximately. The code above works as expected:

. ---------------------------------------------------------------------- Ran 1 test in 1.502s OK

If we change the interval inside the fn_print_with_delay function to one second, the test will (fortunately) fail.


As we saw, testing for expected output is in fact trivial – all you have to do is to put an instance of a class with a ‘write’ method in proper place (i.e. sys.stdout). The only ‘gotcha’ is the cleanup – you should remember to restore sys.stdout to its original state. You may apply the exact same technique if you need to test stderr (just target the sys.stderr instead of sys.stdout). It is also worth noting that using a similar technique you could intercept (or completely silence) output coming from external libraries.

Categories: FLOSS Project Planets

DataCamp: PySpark Cheat Sheet: Spark in Python

Planet Python - Thu, 2017-03-23 05:10

Apache Spark is generally known as a fast, general and open-source engine for big data processing, with built-in modules for streaming, SQL, machine learning and graph processing. It allows you to speed analytic applications up to 100 times faster compared to technologies on the market today. You can interface Spark with Python through "PySpark". This is the Spark Python API exposes the Spark programming model to Python. 

Even though working with Spark will remind you in many ways of working with Pandas DataFrames, you'll also see that it can be tough getting familiar with all the functions that you can use to query, transform, inspect, ... your data. What's more, if you've never worked with any other programming language or if you're new to the field, it might be hard to distinguish between RDD operations.

Let's face it, map() and flatMap() are different enough, but it might still come as a challenge to decide which one you really need when you're faced with them in your analysis. Or what about other functions, like reduce() and reduceByKey()? 

Even though the documentation is very elaborate, it never hurts to have a cheat sheet by your side, especially when you're just getting into it.

This PySpark cheat sheet covers the basics, from initializing Spark and loading your data, to retrieving RDD information, sorting, filtering and sampling your data. But that's not all. You'll also see that topics such as repartitioning, iterating, merging, saving your data and stopping the SparkContext are included in the cheat sheet. 

Note that the examples in the document take small data sets to illustrate the effect of specific functions on your data. In real life data analysis, you'll be using Spark to analyze big data.

Are you hungry for more? Don't miss our other Python cheat sheets for data science that cover topics such as Python basicsNumpyPandasPandas Data Wrangling and much more! 

Categories: FLOSS Project Planets

Rene Dudfield: pip is broken

Planet Python - Thu, 2017-03-23 05:00

Since asking people to use pip to install things, I get a lot of feedback on pip not working. Feedback like this.
"Our fun packaging Jargon"
What is a pip? What's it for? It's not built into python?  It's the almost-default and almost-standard tool for installing python code. Pip almost works a lot of the time. You install things from pypi. I should download pypy? No, pee why, pee eye. The cheeseshop. You're weird. Just call it pee why pee eye. But why is it called pip? I don't know.
"Feedback like this."pip is broken on the raspberian

pip3 doesn't exist on windows

People have an old pip. Old pip doesn't support wheels. What are wheels? It's a cute bit of jargon to mean a zip file with python code in it structured in a nice way. I heard about eggs... tell me about eggs? Well, eggs are another zip file with python code in it. Used mainly by easy_install. Easy install? Let's use that, this is all too much.

The pip executable or script is for python 2, and they are using python 3.

pip is for a system python, and they have another python installed. How did they install that python? Which of the several pythons did they install? Maybe if they install another python it will work this time.

It's not working one time and they think that sudo will fix things. And now certain files can't be updated without sudo. However, now they have forgotten that sudo exists.

"pip lets you run it with sudo, without warning."
pip doesn't tell them which python it is installing for. But I installed it! Yes you did. But which version of python, and into which virtualenv? Let's use these cryptic commands to try and find out...

pip doesn't install things atomically, so if there is a failed install, things break. If pip was a database (it is)...

Virtual environments work if you use python -m venv, but not virtualenv. Or some times it's the other way around. If you have the right packages installed on Debian, and Ubuntu... because they don't install virtualenv by default.

What do you mean I can't rename my virtualenv folder? I can't move it to another place on my Desktop?

pip installs things into global places by default.

"Globals by default."
Why are packages still installed globally by default?

"So what works currently most of the time?"
python3 -m venv anenv
. ./anenv/bin/activate
pip install pip --upgrade
pip install pygame

This is not ideal. It doesn't work on windows. It doesn't work on Ubuntu. It makes some text editors crash (because virtualenvs have so many files they get sick). It confuses test discovery (because for some reason they don't know about virtual environments still and try to test random packages you have installed). You have to know about virtualenv, about pip, about running things with modules, about environment variables, and system paths. You have to know that at the beginning. Before you know anything at all.

Is there even one set of instructions where people can have a new environment, and install something? Install something in a way that it might not break their other applications? In a way which won't cause them harm? Please let me know the magic words?

I just tell people `pip install pygame`. Even though I know it doesn't work. And can't work. By design. I tell them to do that, because it's probably the best we got. And pip keeps getting better. And one day it will be even better.

Help? Let's fix this.
Categories: FLOSS Project Planets

KStars 2.7.6 for Windows & OSX released

Planet KDE - Thu, 2017-03-23 04:25
I am glad to announce the release of KStars 2.7.6 release for Windows & OSX. Linux users using the official PPA can install the latest release as well.

In this release, we introduce the Ekos Mount Modelling tool developed by Robert Lancaster. It's currently in Beta now and we would appreciate any feedback. The tool enables you to build a comprehensive mount model if supported by your mount. Any mount that improves its internal pointing model after a SYNC command is applicable. Furthermore, INDI mounts that supports INDI Alignment Subsystem (EQMod, Nexstarevo, Synscan..etc) are also applicable.

Along with the advanced mount modelling tool comes the new Solution Results plot in the Align Module. It displays the quality of your GOTO after each solve and it can help you to identify if there are issues with your mount or the quality of the image..etc.

You can zoom, pan, and drag to explore the plot in details. Annotation for the quality of each GOTO is available on mouse over.

Ekos Polar Alignment Assistant tool also received a few bug fixes from the community feedback. Most users were able to achieve impressive results using the this easy to use Polar Alignment tool.

While Ekos is designed for ease of use, it can be intimidating for new users unfamiliar with the architecture of Ekos/INDI on several operation systems. Therefore, a new Ekos Profile Wizard is now available to guide the users to setting up their equipment for the first time in Ekos across several operating systems and connection topologies.

With INDI v1.4.1+, figuring out which port to use for your mount & focuser is now trivial across Linux & OSX. INDI automatically scans ports on your system and can even automatically connect to all potential available ports as well until a successful connection is established.

Last, but not least, KStars' NEO (Near-Earth-Object) data query from NASA's JPL is now properly working again thanks to our newest KStars developer Valentin Boettcher. Valentin (aka Hiro) is only 18 years old but is quite brilliant and experienced with KDE/Qt development environment. Welcome abroad!

Categories: FLOSS Project Planets
Syndicate content