FLOSS Project Planets

Tim Millwood: The Future of Drupal

Planet Drupal - Fri, 2017-04-21 12:22
The Future of Drupal

Over the past month there has been a lot of focus on Drupal, the community. More recently it seems people are back to thinking about the software. Dave Hall and David Hernandez both posted eye opening posts with thoughts and ideas of what needs doing and how we can more forward.

A one line summary of those posts would be "We should slim down core, make it more modular, and have many distros".

To a degree this makes sense, however it could cause divergence. Core is not great at all following the same pattern, but contrib is even worse. As part of the Workflow Initiative specifically there is a lot of work going on to try and get the Entity API aligned, get many more entity types revisionable and publishable, using common base classes, traits, and interfaces. If we maintained Node, Block Content, Taxonomy, Comment, etc all as separate projects then there's a chance less of this would happen. Also by doing this we are laying out foundations and setting examples to be followed.

One solution to this may be to follow Symfony (yet again), they have a monolithic project but then split this up into the various components, which are "read only" repos. It's be pretty awesome if we could do this with Drupal. From there we could make Drupal downloadable without many of the core modules. People with the right skills can create a composer.json file to pull in exactly what parts of Drupal are needed, others could use a form on d.o to select which parts are wanted, which downloads a compiled zip.

What would be more awesome is if we could abstract more of Drupal out of Drupal. Imagine if the Entity API was a PHP generic library. Imagine if you could create a Laravel or Symfony app with Nodes. This would be really tricky, especially since Dries announced the plans to make Drupal upgrades easy forever, but possible.

Currently most Drupal sites are still on 7, and here we're talking about what would be Drupal 9? Maybe we need to take step back and look at why sites aren't being upgraded. Dave mentions "A factor in this slow uptake is that from a developer's perspective, Drupal 8 is a new application. The upgrade path from Drupal 7 to 8 is another factor." Although another reason is also why would a company spend the tens of thousands upgrading to Drupal 8? It looks at works (from a users point of view) the same as Drupal 7. Drupal is a CMS, a Content Management System, and the management of content is more or less the same. Yes, with initiatives like Workflow and Media this is changing, but even then similar functionality can be achieved in Drupal 7 with contrib modules. Will Drupal 8 be the version to skip? go straight from 7 to 9?

As Drupal is now pretty firmly an enterprise platform we need to look at this from a marketing point of view. What is going to sell Drupal 9? Why are people going to upgrade? What do they really want? Is a slimmed down core and more modular application really the selling feature that's wanted?

Drupal is a CMS, quoting Dave again "do one thing and do it well". We need to focus on making the authoring experience awesome, and the workflows that go along with it awesome too. This should all be done in a consolidated way to make managing Node content, Block content, etc just as intuitive as each other. If during this process we can also make things more modular, and less Drupally, that'd be awesome!

timmillwood Fri, 21/04/2017 - 17:22 Tags drupal-planet drupal 8
Categories: FLOSS Project Planets

Python Software Foundation: Community Service Award Recipient Ian Cordasco

Planet Python - Fri, 2017-04-21 11:59
The Python Software Foundation depends on its board of directors in order to function. Board members are elected every year by PSF voting members in a process run internally by non-board members. Ian Cordasco has been the PSF’s Election Administrator since 2015, volunteering his efforts for this important role. Cordasco is also a valuable member of the Python community, frequently mentoring newer coders and supporting their Python endeavors. For these reasons, the PSF is delighted to award the 2017 QA Community Service award to Ian Cordasco:

RESOLVED that the Python Software Foundation award the 2017 Q1 Community Service Award to Ian Cordasco for his contributions to PSF elections and active mentoring of women in Python community.

PSF Elections

Cordasco began as the PSF’s Election Administrator during a time of turmoil. “The first year I ran the election was something of a nightmare,” he recalls. Due to unforeseen circumstances, the previous Election Administrator stepped down on short notice and was unavailable to relaunch the election efforts. “Many people did not get ballots via email as they should. Some people were accidentally excluded from the voting rolls. Further, there was a lot of confusion because I stepped in at the last minute.” Without the aid of documentation and prior experience, Ian threw himself into the cause. The PSF has since reviewed, solidified, and documented the election procedures.

Since his dramatic start as Elections Administrator, Cordasco’s work with PSF elections has been much smoother. Mark Mangoba, PSF’s IT Manager, works closely with Cordasco during the election process. Mangoba notes, “Ian is a great volunteer. He does an excellent job with the elections, assuring that all votes are accounted for and that there is no fraud or issues of any kind.” Cordasco has also gotten creative with how he manages elections. For example, to reduce bias, he uses Python code to break ties and to randomize the order in which candidates appear to voters. Additionally, those that work with Cordasco describe him as an enjoyable collaborator. Mangoba explains, “Ian is energetic and thoughtful. His passion and enthusiasm for the PSF shows through, he’s always available to help and answer questions.”

Mentoring

Cordasco has a history of going out of his way to support and encourage female developers. When Carol Willing, a developer for the Jupyter project, wanted to work on the Requests library, she got in touch with Cordasco. “We worked together on the project and my first commit to the Requests library got accepted!” Cordasco later wrote a fantastic post about it on his blog.

Cordasco has also found newer coders to mentor at Python events, such as Anna Ossowski. “I met Ian at PyTennessee 2015, a day before I was scheduled to give my very first ever conference talk. Ian’s encouragement and support helped me a lot and it’s thanks to him and Carol [Willing] that I had the confidence to go up on stage and deliver my talk.” But his support didn’t stop there, Ossowski goes on to say, “every week he would reserve an hour for me where we would program together, he would answer questions, and just generally help me with any programming issues I experienced. Ian helped me get the PyLadies Remote website up and running, something I would have never managed without his help.”

Adrienne Lowe, a developer at Emma, has also enjoyed Cordasco’s support and encouragement. She recalls, “He models the kind of developer that we all want to be in terms of being encouraging and open.” She continues, “he sets himself apart by being genuine, welcoming, and happy to explain anything from simple things to more complex concepts, all in an ego-less way.”

The Python community as a whole is very lucky to count Cordasco as its member, and we hope he continues to help others contribute and achieve their goals.

CSA 2017 Q1 Winner Ian Cordasco
In his free time, you can find Cordasco blogging on his website, riding his bike, or reading books.
Categories: FLOSS Project Planets

Joachim Breitner: veggies: Haskell code generation from scratch

Planet Debian - Fri, 2017-04-21 11:30

How hard it is to write a compiler for Haskell Core? Not too hard, actually!

I wish we had a formally verified compiler for Haskell, or at least for GHC’s intermediate language Core. Now formalizing that part of GHC itself seems to be far out of reach, with the many phases the code goes through (Core to STG to CMM to Assembly or LLVM) and optimizations happening at all of these phases and the many complicated details to the highly tuned GHC runtime (pointer tagging, support for concurrency and garbage collection).

Introducing Veggies

So to make that goal of a formally verified compiler more feasible, I set out and implemented code generation from GHC’s intermediate language Core to LLVM IR, with simplicity as the main design driving factor.

You can find the result in the GitHub repository of veggies (the name derives from “verifiable GHC”). If you clone that and run ./boot.sh some-directory, you will find that you can use the program some-directory/bin/veggies just like like you would use ghc. It comes with the full base library, so your favorite variant of HelloWorld might just compile and run.

As of now, the code generation handles all the Core constructs (which is easy when you simply ignore all the types). It supports a good number of primitive operations, including pointers and arrays – I implement these as need – and has support for FFI calls into C.

Why you don't want to use Veggies

Since the code generator was written with simplicity in mind, performance of the resulting code is abysmal: Everything is boxed, i.e. represented as pointer to some heap-allocated data, including “unboxed” integer values and “unboxed” tuples. This is very uniform and simplifies the code, but it is also slow, and because there is no garbage collection (and probably never will be for this project), will fill up your memory quickly.

Also, the code is currently only supports 64bit architectures, and this is hard-coded in many places.

There is no support for concurrency.

Why it might be interesting to you nevertheless

So if it is not really usable to run programs with, should you care about it? Probably not, but maybe you do for one of these reasons:

  • You always wondered how a compiler for Haskell actually works, and reading through a little over a thousands lines of code is less daunting than reading through the 34k lines of code that is GHC’s backend.
  • You have wacky ideas about Code generation for Haskell that you want to experiment with.
  • You have wacky ideas about Haskell that require special support in the backend, and want to prototype that.
  • You want to see how I use the GHC API to provide a ghc-like experience. (I copied GHC’s Main.hs and inserted a few hooks, an approach I copied from GHCJS).
  • You want to learn about running Haskell programs efficiently, and starting from veggies, you can implement all the trick of the trade yourself and enjoy observing the speed-ups you get.
  • You want to compile Haskell code to some weird platform that is supported by LLVM, but where you for some reason cannot run GHC’s runtime. (Because there are no threads and no garbage collection, the code generated by veggies does not require a runtime system.)
  • You want to formally verify Haskell code generation. Note that the code generator targets the same AST for LLVM IR that the vellvm2 project uses, so eventually, veggies can become a verified arrow in the top right corner map of the DeepSpec project.

So feel free to play around with veggies, and report any issues you have on the GitHub repository.

Categories: FLOSS Project Planets

Mediacurrent: ngconf 2017 - Build with Us

Planet Drupal - Fri, 2017-04-21 11:03

The first week of April I was among the attendees at the year’s largest US Angular conference, ngconf, in Salt Lake City, Utah. As one of the keynote speakers last year, I was excited at the opportunity to attend again, and curious about how different the atmosphere would be this time around. Last year the community was anxiously awaiting the release of Angular 2.0, and this year 4.0 was released just before the conference.

Categories: FLOSS Project Planets

Dave Hall Consulting: Many People Want To Talk

Planet Drupal - Fri, 2017-04-21 10:52

WOW! The response to my blog post on the future of Drupal earlier this week has been phenomenal. My blog saw more traffic in 24 hours than it normally sees in a 2 to 3 week period. Around 30 comments have been left by readers. My tweet announcing the post was the top Drupal tweet for a day. Some 50 hours later it is still number 4.

It seems to really connected with many people in the community. I am still reflecting on everyone's contributions. There is a lot to take in. Rather than rush a follow up that responds to the issues raised, I will take some time to gather my thoughts.

One thing that is clear is that many people want to use DrupalCon Baltimore next week to discuss this issue. I encourage people to turn up with an open mind and engage in the conversation there.

A few people have suggested a BoF. Unfortunately all of the official BoF slots are full. Rather than that be a blocker, I've decided to run an unofficial BoF on the first day. I hope this helps facilitate the conversation.

Unofficial BoF: The Future of Drupal

When: Tuesday 25 April 2017 @ 12:30-1:30pm
Where: Exhibit Hall - meet at the Digital Echidna booth (#402) to be directed to the group
What: High level discussion about the direction people think Drupal should take.
UPDATE: An earlier version of this post had this scheduled for Monday. It is definitely happening on Tuesday.

I hope to see you in Baltimore.

Categories: FLOSS Project Planets

Coding Diet: Conditional refactorings

Planet Python - Fri, 2017-04-21 09:41

I have been reading through some of the source code from the requests library. This is one of the more well-known, well-used, well-loved, and well-praised Python libraries I know of. I came across some conditional code which I personally would refactor. A first refactor I'm pretty confident about whilst a second I'm less certain is an improvement. I'll detail them both and welcome any comments anyone has.

A specific example of this kind of conditional comes in the adapters module specifically the cert_verify method. Here is the code in question, slightly snipped for brevity:

def cert_verify(self, conn, url, verify, cert): """Verify a SSL certificate. This method should not be called from user ... :param verify: Either a boolean, in which case it controls whether we verify """ if url.lower().startswith('https') and verify: cert_loc = None # Allow self-specified cert location. if verify is not True: cert_loc = verify if not cert_loc: cert_loc = DEFAULT_CA_BUNDLE_PATH if not cert_loc or not os.path.exists(cert_loc): raise IOError("Could not find a suitable TLS CA certificate bundle, " "invalid path: {0}".format(cert_loc)) conn.cert_reqs = 'CERT_REQUIRED' if not os.path.isdir(cert_loc): conn.ca_certs = cert_loc else: conn.ca_cert_dir = cert_loc else: conn.cert_reqs = 'CERT_NONE' conn.ca_certs = None conn.ca_cert_dir = None

The two refactors I would consider is the setting of cert_loc and the order of the two conditional branches, ie. the if and else branches.

Setting cert_loc

This kind of pattern is quite common where a variable is set to a particular value and then a condition is evaluated which may result in the variable being updated. In this particular example we have:

cert_loc = None # Allow self-specified cert location. if verify is not True: cert_loc = verify if not cert_loc: cert_loc = DEFAULT_CA_BUNDLE_PATH

First of all note that verify cannot be Falsey. So at the end of these conditionals cert_loc will either be DEFAULT_CA_BUNDLE_PATH if verify is exactly True or it will be set to whatever verify is. So we can write this more succinctly as:

cert_loc = DEFAULT_CA_BUNDLE_PATH if verify is True else verify

If you don't like the conditional expression you can do this as:

if verify is True: cert_loc = DEFAULT_CA_BUNDLE_PATH else: cert_loc = verify

All this was only really possible because we knew that verify could not be Falsey. Still I consider the setting of a variable to a default before then considering whether or not to update it to be a very minor anti-pattern. So imagine that we could not be sure that verify is not Falsey, we could still re-write our conditional without the default setting as:

if verify is True or not verify: cert_loc = DEFAULT_CA_BUNDLE_PATH else: cert_loc = verify

More generally where you see the pattern:

x = None if cond1: x = val1 if cond2: x = val2 ... if not x: x = default_x

Then you can change this to be:

if cond1 and val1: x = val1 elif cond2 and val2: x = val2 else: x = default_x

Sometimes this is awkward if one or more of the val_? values are interesting expressions.

One defence of the original style is that you know your variable will be set to something so you will never end up with an undefined variable error. However, if this were ever the case then you have not fully written down your logic, and hence the early error is likely a good thing.

Switch the if and else branches

A suggestion I often make when you have two branches, such that one is significantly shorter than the other, is to have the shorter branch first. The reason for this is to give the reader less of a "stack" to recall as they are reading through the code. Often when reading code you get to the end of the long branch, to find a shorter second branch but you've forgotten what the original condition was. So, it's tempting to suggest to modify the original condition as:

if not (url.lower().startswith('https') and verify): conn.cert_reqs = 'CERT_NONE' conn.ca_certs = None conn.ca_cert_dir = None cert_loc = None else: # Allow self-specified cert location. if verify is not True: cert_loc = verify if not cert_loc: cert_loc = DEFAULT_CA_BUNDLE_PATH if not cert_loc or not os.path.exists(cert_loc): raise IOError("Could not find a suitable TLS CA certificate bundle, " "invalid path: {0}".format(cert_loc)) conn.cert_reqs = 'CERT_REQUIRED' if not os.path.isdir(cert_loc): conn.ca_certs = cert_loc else: conn.ca_cert_dir = cert_loc

I hesitate to suggest it in this case because the original if branch has code which depends upon the condition, namely that verify is not Falsey. However, more generally this can be something of a win for the person reading the code.

So, two pretty simple refactors, thoughts?

Categories: FLOSS Project Planets

Vardot: How to Choose the Right Open Source CMS for Your Website (Infographic)

Planet Drupal - Fri, 2017-04-21 09:37
News Read time: 2 minutes

Today’s top CMS platforms all offer their own unique flavor for users trying to get their website up and running. All of them have their own strengths and weaknesses, but for complete and total flexibility, Drupal is the best choice. While it requires a strong grasp of web development, the level of customization it offers is far beyond its competition.

Join us as we look at some of the top reasons to use Drupal, followed by an infographic that breaks down today’s top three platforms.

 

The Top 3 Reasons to Choose Drupal For Your Website

While there are plenty of articles about Drupal, and numerous reasons to use it, let’s take a look at the top three picks:

1. Agile Web 2.0 Development

Drupal embraces the best parts of agile web development. From the moment you install it, you have clean markup code, page and blog publishing, content management, search, polls, forums, user login modules, and plenty more. It cuts your development timeline down to mere days, as opposed to weeks of getting everything ready.

 

2. Thousands of Modules

Modules allow you to quickly add functionality to your website. With over 16,000 to choose from, this number is only growing. From additional security, to social media, to SEO, and backups, you can do just about anything with these additional tools, and provide your users with an excellent experience.

 

3. Ultimate Scalability

Drupal is insanely flexible. You can start with as little as 10 pages on your site, and move up to 10,000 posts without ever changing a thing. Performance and security can be easily modified through the Admin console. Perhaps this is why some of the world's biggest websites like the White House, Nascar, and the Grammys all run on Drupal.

If you are still unsure why Drupal is the best, read this article with more reasons that will convince you.

 

Breaking Down Today’s CMS Platforms (Infographic)

Below you’ll find an infographic that compares Drupal to the other platforms you’ll find in your search. For the reasons above, however, our team of OnBlastBlog recommends Drupal wholeheartedly.

 

And which platform do you use? Let us know in the comments! 

Tags:  Drupal Planet Drupal Title:  How to Choose the Right Open Source CMS for Your Website (Infographic)
Categories: FLOSS Project Planets

Qt Creator 4.2.2 released

Planet KDE - Fri, 2017-04-21 06:57

The spring has arrived, so we took our brooms and swept the bugs from under the carpets and out of the door.

We meticulously noted down all the classes, orders, families, species, and other details of what we found and removed, which you can look up in our spring cleaning log.

Get Qt Creator 4.2.2

The opensource version is available on the Qt download page, and you find commercially licensed packages on the Qt Account Portal. Qt Creator 4.2.2 is also available through an update in the online installer. Please post issues in our bug tracker. You can also find us on IRC on #qt-creator on chat.freenode.net, and on the Qt Creator mailing list.

The post Qt Creator 4.2.2 released appeared first on Qt Blog.

Categories: FLOSS Project Planets

Five days and counting

Planet KDE - Fri, 2017-04-21 06:12

It is five days left until foss-north 2017, so it is high time to get your ticket! Please notice that tickets can be bought all the way until the night of the 25th (Tuesday), but catering is only included is you get your ticket on the 24th (Monday), so help a poor organizer and get your tickets as soon as possible!

And just to reiterate what’s on the menu. This is a full day conference with two tracks and many interesting speakers from many projects, including Debian, Wikimedia, KDE, systemd, PulseAudio, Nextcloud, PostgreSQL, OpenRISC, flatpak, AsteroidOS and more.

Last year, over 30% of the tickets was sold in the last 48h. This year, we cannot provide catering for the ones buying tickets the last 24h, so hopefully the peak will come earlier. For now, we have 93 persons on the visiting list, but there are chairs for 100 more!

Categories: FLOSS Project Planets

PyCon: Announcing the PyCon 2017 Keynote Speakers

Planet Python - Fri, 2017-04-21 06:09

Only one month from today, PyCon will be almost over! The conference will be on the third and final day of its program. The sponsor booths will all have been packed up the night before and the Expo Hall re-purposed for a morning full of Poster presentations and Job Fair tables. Only one quick afternoon of talks will stand between us and the closing ceremonies.

Here in the present, the hatches are nearly all battened down. The schedule is set. The conference is completely sold out of registrations. The sponsor lineup is nearly finished, with only a few booths still left to be claimed. Almost everything is now in place — though, we do still need more attendeees to sign up as volunteers, a topic about which we will blog in further detail next week.

Meanwhile, the time has come to announce this year’s keynote speakers, who will be addressing the conference during our plenary sessions! They are:

  • Kelsey Hightower
  • Katy Huff
  • Jake Vanderplas
  • Lisa Guo & Hui Ding

We look forward to hearing from each of them!

You might be wondering: where on this list is Python’s fearless leader and perpetual keynote favorite, Guido van Rossum? Don’t worry! Guido will definitely be on stage this year as part of a special Sunday morning plenary session — the details of which we will be announcing soon. Intrigued? Watch for our announcement next week!

Here are more details about 2017’s keynote speakers:

Kelsey Hightower

Kelsey Hightower is an open source advocate and recovering sysadmin who is currently serving the application container and distributed systems community as an educator and toolsmith. He is currently employed by Google.

Katy Huff

Dr. Kathryn D. Huff is an unapologetic advocate for open reproducible scientific computing and for emissions-free base-load nuclear energy. She is currently an Assistant Professor in the Department of Nuclear, Plasma, and Radiological Engineering at the University of Illinois at Urbana-Champaign where she leads the Advanced Reactors and Fuel Cycles Research Group. She holds an affiliate faculty position with the National Center for Supercomputing Applications and is one of the University of Illinois' most recent Blue Waters Professors.

Her current research focuses on modeling and simulation of advanced nuclear reactors and fuel cycles. She is currently the elected chair of the Fuel Cycle and Waste Management Division of the American Nuclear Society. Through leadership with the Hacker Within, Software Carpentry, SciPy, the Journal of Open Source Software, and other initiatives, she strives to advocate for best practices in open, reproducible scientific computing. With colleagues, collaborators, and friends, she has co-authored two books to help scientists with these practices: Effective Computation in Physics, O’Reilly, 2015 and The Practice of Reproducible Research, UC Press, 2017.

Jake Vanderplas

Jake VanderPlas is an astronomer by training, and a long-time user and developer of the scientific Python stack. He currently works as an interdisciplinary research director at the University of Washington, where he writes, teaches, collaborates on research, and spends time consulting with local scientists from a wide range of fields.

Lisa Guo

Lisa Guo is a networking, platform, and scalability software engineer with over 20 years experience. She has been working with the Instagram Infrastructure team since 2014, where she led efforts to expand from a single to multiple data centers and improve efficiency and cost-effectiveness.

Prior to joining Instagram, Lisa worked on Facebook’s Software Defined Networks strategy and deployment. She was also Director, Engineering at Juniper Networks in charge of software development for EX switching series. She joined Juniper through its acquisition of NetScreen, and held core infrastructure development roles at Shasta Networks, Tahoe Networks.

Hui Ding

Hui Ding is Head of Infrastructure org at Instagram, where he oversees the scaling of Instagram backend platform that supports hundreds of millions of concurrent users on a daily basis. Hui has been with Instagram since 2012, and has led the development of many Instagram product launches as well as all infrastructure efforts.

Before joining Instagram, Hui was a core member of the Facebook infrastructure team, building its distributed data store for the social graph. Hui holds a PhD in computer engineering from Northwestern University.

Categories: FLOSS Project Planets

Colm O hEigeartaigh: Securing Apache Hadoop Distributed File System (HDFS) - part III

Planet Apache - Fri, 2017-04-21 05:54
This is the third in a series of posts on securing HDFS. The first post described how to install Apache Hadoop, and how to use POSIX permissions and ACLs to restrict access to data stored in HDFS. The second post looked at how to use Apache Ranger to authorize access to data stored in HDFS. In this post we will look at how Apache Ranger can create "tag" based authorization policies for HDFS using Apache Atlas. For information on how to create tag-based authorization policies for Apache Kafka, see a post I wrote earlier this year.

The Apache Ranger admin console allows you to create security policies for HDFS by associating a user/group with some permissions (read/write/execute) and a resource, such as a directory or file. This is called a "Resource based policy" in Apache Ranger. An alternative is to use a "Tag based policy", which instead associates the user/group + permissions with a "tag". You can create and manage tags in Apache Atlas, and Apache Ranger supports the ability to imports tags from Apache Atlas via a tagsync service, something we will cover in this post.

1) Start Apache Atlas and create entities/tags for HDFS

First let's look at setting up Apache Atlas. Download the latest released version (0.8-incubating) and extract it. Build the distribution that contains an embedded HBase and Solr instance via:
  • mvn clean package -Pdist,embedded-hbase-solr -DskipTests
The distribution will then be available in 'distro/target/apache-atlas-0.8-incubating-bin'. To launch Atlas, we need to set some variables to tell it to use the local HBase and Solr instances:
  • export MANAGE_LOCAL_HBASE=true
  • export MANAGE_LOCAL_SOLR=true
Now let's start Apache Atlas with 'bin/atlas_start.py'. Open a browser and go to 'http://localhost:21000/', logging on with credentials 'admin/admin'. Click on "TAGS" and create a new tag called "Data".  Click on "Search" and the "Create new entity" link. Select an entity type of "hdfs_path" with the following values:
  • QualifiedName: data@cl1
  • Name: Data
  • Path: /data
Once the new entity has been created, then click on "+" beside "Tags" and associate the new entity with the "Data" tag.

2) Use the Apache Ranger TagSync service to import tags from Atlas into Ranger

To create tag based policies in Apache Ranger, we have to import the entity + tag we have created in Apache Atlas into Ranger via the Ranger TagSync service. First, start the Apache Ranger admin service and rename the HDFS service we created in the previous tutorial from "HDFSTest" to "cl1_hadoop". This is because the Tagsync service will sync tags into the Ranger service that corresponds to the suffix of the qualified name of the tag with "_hadoop". Also edit 'etc/hadoop/ranger-hdfs-security.xml' in your Hadoop distribution and change the "ranger.plugin.hdfs.service.name" to "cl1_hadoop". Also change the "ranger.plugin.hdfs.policy.cache.dir" along the same lines. Finally, make sure the directory '/etc/ranger/cl1_hadoop/policycache' exists and the user you are running Hadoop as can write and read from this directory.

After building Apache Ranger then extract the file called "target/ranger-<version>-tagsync.tar.gz". Edit 'install.properties' as follows:
  • Set TAG_SOURCE_ATLAS_ENABLED to "false"
  • Set TAG_SOURCE_ATLASREST_ENABLED to  "true"
  • Set TAG_SOURCE_ATLASREST_DOWNLOAD_INTERVAL_IN_MILLIS to "60000" (just for testing purposes)
  • Specify "admin" for both TAG_SOURCE_ATLASREST_USERNAME and TAG_SOURCE_ATLASREST_PASSWORD
Save 'install.properties' and install the tagsync service via "sudo ./setup.sh". It can now be started via "sudo ranger-tagsync-services.sh start".

3) Create Tag-based authorization policies in Apache Ranger

Now let's create a tag-based authorization policy in the Apache Ranger admin UI. Click on "Access Manager" and then "Tag based policies". Create a new Tag service called "HDFSTagService". Create a new policy for this service called "DataPolicy". In the "TAG" field enter a capital "D" and the "Data" tag should pop up, meaning that it was successfully synced in from Apache Atlas. Create an "Allow" condition for the user "bob" with component permission of "HDFS" and "read" and "execute":


The last thing we need to do is to go back to the Resource based policies and edit "cl1_hadoop" and select the tag service we have created above.

4) Testing authorization in HDFS using our tag based policy

Wait until the Ranger authorization plugin syncs the new authorization policies from the Ranger Admin service and then we can test authorization. In the previous tutorial we showed that the file owner and user "alice" can read the data stored in '/data', but "bob" could not. Now we should be able to successfully read the data as "bob" due to the tag based authorization policy we have created:
  • sudo -u bob bin/hadoop fs -cat /data/LICENSE.txt
Categories: FLOSS Project Planets

Rhonda D'Vine: Home

Planet Debian - Fri, 2017-04-21 04:01

A fair amount of things happened since I last blogged something else than music. First of all we did actually hold a Debian Diversity meeting. It was quite nice, less people around than hoped for, and I account that to some extend to the trolls and haters that defaced the titanpad page for the agenda and destroyed the doodle entry for settling on a date for the meeting. They even tried to troll my blog with comments, and while I did approve controversial responses in the past, those went over the line of being acceptable and didn't carry any relevant content.

One response that I didn't approve but kept in my mailbox is even giving me strength to carry on. There is one sentence in it that speaks to me: Think you can stop us? You can't you stupid b*tch. You have ruined the Debian community for us. The rest of the message is of no further relevance, but even though I can't take credit for being responsible for that, I'm glad to be a perceived part of ruining the Debian community for intolerant and hateful people.

A lot of other things happened since too. Mostly locally here in Vienna, several queer empowering groups were founding around me, some of them existed already, some formed with the help of myself. We now have several great regular meetings for non-binary people, for queer polyamory people about which we gave an interview, a queer playfight (I might explain that concept another time), a polyamory discussion group, two bi-/pansexual groups, a queer-feminist choir, and there will be an European Lesbian* Conference in October where I help with the organization …

… and on June 21st I'll finally receive the keys to my flat in Que[e]rbau Seestadt. I'm sooo looking forward to it. It will be part of the Let me come Home experience that I'm currently in. Another part of that experience is that I started changing my name (and gender marker) officially. I had my first appointment in the corresponding bureau, and I hope that it won't last too long because I have to get my papers in time for booking my flight to Montreal, and somewhen along the process my current passport won't contain correct data anymore. So for the people who have it in their signing policy to see government IDs this might be your chance to finally sign my key then.

I plan to do a diversity BoF at debconf where we can speak more directly on where we want to head with the project. I hope I'll find the time to do an IRC meeting beforehand. I'm just uncertain how to coordinate that one to make it accessible for interested parties while keeping the destructive trolls out. I'm open for ideas here.

/personal | permanent link | Comments: 3 | Flattr this

Categories: FLOSS Project Planets

Talk Python to Me: #108 MicroPython and Open Source Hardware at Adafruit

Planet Python - Fri, 2017-04-21 04:00
Want to learn how to build an Iron-man like arc reactor accessory or maybe a solar charging backpack? What if you could program these devices with Python? <br/> <br/> We'll be talking about a project and company making this possible. This week you'll meet Tony DiCola who works at Adafruit. A company making hardware programming accessible. We will also talk about micropython which lets you program these cool devices in Python! <br/> <br/> Links from the show: <br/> <div style="font-size: .85em;"> <br/> <b>Tony D</b>: <a href='https://twitter.com/tdicola' target='_blank'>@tdicola</a> <br/> <b>AdaFruit</b>: <a href='https://www.adafruit.com/' target='_blank'>adafruit.com</a> <br/> <b>Watch Tony D's Desk</b>: <a href='https://www.youtube.com/playlist?list=PLjF7R1fz_OOUmgpmVDTyX85Z8OO142eia' target='_blank'>youtube.com/playlist</a> <br/> <br/> <b>Projects <a href='https://learn.adafruit.com' target='_blank'>learn.adafruit.com</a></b>: <br/> <b>Onion Pi</b>: <a href='https://learn.adafruit.com/onion-pi' target='_blank'>learn.adafruit.com/onion-pi</a> <br/> <b>GPS Dog Collar</b>: <a href='https://learn.adafruit.com/gps-dog-collar' target='_blank'>learn.adafruit.com/gps-dog-collar</a> <br/> <b>LED Bicycle Handlebars</b>: <a href='https://learn.adafruit.com/led-bicycle-handlebars' target='_blank'>learn.adafruit.com/led-bicycle-handlebars</a> <br/> <b>DeLorean Time Circuit</b>: <a href='https://learn.adafruit.com/delorean-time-circuit' target='_blank'>learn.adafruit.com/delorean-time-circuit</a> <br/> <b>Raspberry Pi WiFi Radio</b>: <a href='https://learn.adafruit.com/pi-wifi-radio' target='_blank'>learn.adafruit.com/pi-wifi-radio</a> <br/> <b>Halloween Pumpkin</b>: <a href='https://learn.adafruit.com/halloween-pumpkin' target='_blank'>learn.adafruit.com/halloween-pumpkin</a> <br/> <b>Electronic Demon Costume</b>: <a href='https://learn.adafruit.com/electronic-demon-costume' target='_blank'>learn.adafruit.com/electronic-demon-costume</a> <br/> <b>Solar Charging Handbag</b>: <a href='https://learn.adafruit.com/solar-charging-handbag' target='_blank'>learn.adafruit.com/solar-charging-handbag</a> <br/> <br/> <strong>Sponsored links</strong> <br/> <b>Advance Digital Jobs</b>: <a href='http://python.advance.net' target='_blank'>python.advance.net</a> <br/> <b>StrangeLoop Conference</b>: <a href='https://talkpython.fm/strangeloop' target='_blank'>talkpython.fm/strangeloop</a> <br/> <b>Talk Python Courses</b>: <a href='https://training.talkpython.fm/' target='_blank'>training.talkpython.fm</a> <br/> </div>
Categories: FLOSS Project Planets

Kushal Das: Event report: FOSSASIA 2017

Planet Python - Fri, 2017-04-21 03:46

FOSSASIA 2017 reminded me of foss.in. After a long time, finally, a conference which has the similar aspects. Similar kind of tight organizing team, the presence of upstream communities from different locations. The participation from the local Singapore tech community along with Hackerspace Singapore is a serious boost. This was my 4th FOSSASIA conference, and also 3rd time in Singapore. I should thank Mario, Hong, and rest of the organizers to make this event a very pleasant experience.

This time Sayan booked an Airbnb for Anwesha and me. Saptak + Medo + Siddhesh + Praveen Patil, and Pooja Yadav, & Pravin Kumar were also staying in the same Airbnb in the Chinatown. The conference venue was the Singapore Science Center just like last year. Having the conference in the same place helps as the MRT route is very easy to reach there on time.

The day before the conference we had a speakers meetup in the Singapore Microsoft office. We also received a tour of the office, the person in-charge also explained about how are managing an office without permanent seating positions.

Day one

The conference started at 9:24AM (as Hong asked us to remember the time). I attended the talks from Harish Pillay and Chan Cheow Hoe. The idea of having the CIO of the country coming to the conference and giving a talk on Open Source is a great feeling. In 2015 we had Minister for Foreign Affairs, Mr. Vivian Balakrishnan giving a keynote (and talking about the NodeJS code he wrote). The way govt. is taking part in the local community events is something other countries should try to learn. Of course, Singapore has the benefit of being small in size.

Though the day was full of talks related to AI and machine learning, there were two talks I was waiting to attend. After lunch, the first one was from Bunnie Huang, where he spoke about making technology more inclusive. He talked about Chibitronics. Before I traveled to Singapore, I actually asked him to get a copy of his new book, The Hardware Hacker. I got my copy signed by him after his talk :) (I already finished the book while coming back to India, more on that later in a separate blog post). I also met Xobs and found a Chibtronics Love-to-Code board in his pocket :)

Later in the day, Frank Karlitschek gave his keynote titled Protecting privacy with free software. He brought up the original idea of the Internet being decentralized. The last talk of the day was a panel discussion on Artificial Intelligence.

Day 2 & Day 3

I spent most of the time in the Python track, and in between jumping around different floors of the venue meeting people. I personally had a two-hour workshop on MicroPython and NodeMCU. Anwesha was busy in the PyLadies table along with Pooja. I forgot to show the poster of PyCon APAC in the Python track :( But you can still submit talks and attend the conference. Sadly this will clash with another conference for me.

Anwesha had her talk on day 3, and her laptop’s display decided to crash just before the talk. But finally the slides came back :) I also attended the SELinux workshop from Jason Zaman. He and few BSD friends convinced me to try out ZFS, and then build a new home storage with FreeNAS.

Now I have to wait for the next edition of FOSSASIA. It is a great place where I can meet my friends from different parts of the world, and share ideas :)

Categories: FLOSS Project Planets

InternetDevels: The User Personas module in Drupal 8: perfect order in roles and permissions

Planet Drupal - Fri, 2017-04-21 02:53

Drupal 8 promised to make website management a piece of cake, and it is fulfilling its promise!

Read more
Categories: FLOSS Project Planets

Noah Meyerhans: Stretch images for Amazon EC2, round 2

Planet Debian - Fri, 2017-04-21 00:37

Following up on a previous post announcing the availability of a first round of AWS AMIs for stretch, I'm happy to announce the availability of a second round of images. These images address all the feedback we've received about the first round. The notable changes include:

  • Don't install a local MTA.
  • Don't install busybox.
  • Ensure that /etc/machine-id is recreated at launch.
  • Fix the security.debian.org sources.list entry.
  • Enable Enhanced Networking and ENA support.
  • Images are owned by the official debian.org AWS account, rather than my personal account.

AMI details are listed on the wiki. As usual, you're encouraged to submit feedback to the cloud team via the cloud.debian.org BTS pseudopackage, the debian-cloud mailing list, or #debian-cloud on irc.

Categories: FLOSS Project Planets

Vasudev Ram: Python callbacks using classes and methods

Planet Python - Fri, 2017-04-21 00:32
By Vasudev Ram

Hi readers,

I had written this post a few days ago:

Implementing and using callbacks in Python

In it, I had shown how to create simple callbacks using just plain Python functions, and said I would write a bit more about callbacks in my next post.

This is that next post. It discusses how to create callbacks in Python using classes and methods.

Here is an example program, callback_demo2.py, that shows how to do that:
'''
File: callback_demo2.py
To demonstrate implementation and use of callbacks in Python,
using classes with methods as the callbacks.
Author: Vasudev Ram
Copyright 2017 Vasudev Ram
Web site: https://vasudevram.github.io
Blog: https://jugad2.blogspot.com
Product store: https://gumroad.com/vasudevram
'''

from __future__ import print_function
import sys
from time import sleep

class FileReporter(object):

def __init__(self, filename):
self._filename = filename
try:
self._fil = open(self._filename, "w")
except IOError as ioe:
sys.stderr.write("While opening {}, caught IOError: {}\n".format(
self._filename, repr(ioe)))
sys.exit(1)

def report(self, message):
self._fil.write(message)

class ScreenReporter(object):

def __init__(self, dest):
self._dest = dest

def report(self, message):
self._dest.write(message)

def square(i):
return i * i

def cube(i):
return i * i * i

def processor(process, times, report_interval, reporter):
result = 0
for i in range(1, times + 1):
result += process(i)
sleep(0.1)
if i % report_interval == 0:
# This is the call to the callback method
# that was passed to this function.
reporter.report("Items processed: {}. Running result: {}.\n".format(i, result))

file_reporter = FileReporter("processor_report.txt")
processor(square, 20, 5, file_reporter)

stdout_reporter = ScreenReporter(sys.stdout)
processor(square, 20, 5, stdout_reporter)

stderr_reporter = ScreenReporter(sys.stderr)
processor(square, 20, 5, stderr_reporter)
I ran it with:
$ python callback_demo2.py >out 2>err
The above command creates 3 files, processor_report.txt, out and err.
Running fc /l on those 3 files, pairwise, shows that all three have the same content, but the output has gone to 3 different destinations (a specified file, standard output, and standard error output, based on which callback was passed to the processor function, in the 3 calls to it.

These two lines from the program:
file_reporter = FileReporter("processor_report.txt")
processor(square, 20, 5, file_reporter)
send output to the file processor_report.txt.

These two lines:
stdout_reporter = ScreenReporter(sys.stdout)
processor(square, 20, 5, stdout_reporter)
send output to the standard output (stdout), which is redirected to the file out.

These two lines:
stderr_reporter = ScreenReporter(sys.stderr)
processor(square, 20, 5, stderr_reporter)
send output to the standard error output (stderr), which is redirected to the file err.

The difference between this program (callback_demo2.py) and the one in the previous post (callback_demo.py), is that in this one, I pass an instance of some class to the processor function, as its last argument. This argument is the callback. And this time, rather than treating it as a function, processor treats it as an object, and invokes the report method on it, giving us much the same output as before (I just made minor cosmetic changes in the output). This same thing is done in all the 3 calls to processor. The difference is that different types of objects are passed each time, for the callback argument.

[ An interesting side note here is that in some other languages, for example, Java, at least in earlier Java versions (I'm not sure about recent ones), we cannot just pass different types of objects for the same callback parameter, unless they are all derived from some common base class (via inheritance). (Can it be done using Java interfaces? Need to check - been a while.) While that is also possible in Python, it is not necessary, as we see here, due to Python's more dynamic nature. Just passing any object that implements a report method is enough, as the program shows. The FileReporter and ScreenReporter classes do not (need to) descend from some common base class (other than object, but that is just the syntax for new-style classes; the object class not provide the report method). Python's smooth polymorphism, a.k.a. duck typing, takes care of it. ]

The first time, a FileReporter object is passed, so the output goes to a text file.

The second and third times, a ScreenReporter object is passed, initializing the _dest field to stdout and stderr respectively, so the output goes to those two destinations respectively. And the command line redirects those outputs to the files out and err.

Although I didn't say so in the previous post, the first argument to processor (square), is also a callback, since it, in turn, is called from processor, via the process parameter. So I can pass some other argument in place of it, like the cube function defined in the program, to get different computations done and hence different results.



Keen on creating and selling products online? Check out the free Product Creation Masterclass. It runs for about 4 weeks from 24 April 2017 (that's 3 days from now), with lots of emails, videos and expert interviews, all geared toward helping you create and sell your first product online. Check it out: The Product Creation Masterclass.

- Vasudev Ram - Online Python training and consulting

Get updates (via Gumroad) on my forthcoming apps and content.

Jump to posts: Python * DLang * xtopdf

Subscribe to my blog by email

My ActiveState Code recipes

Follow me on: LinkedIn * Twitter

Are you a blogger with some traffic? Get Convertkit:

Email marketing for professional bloggers

Share |



Vasudev Ram
Categories: FLOSS Project Planets

Kdenlive 17.04 released

Planet KDE - Thu, 2017-04-20 23:31

We release 17.04 with a redesigned profile selection dialog to make it easier to set screen size, framerate, and other parameters of your film. Now you can also play your video directly from the notification when rendering is finished. Some crashes that happened when moving clips around on the timeline have been corrected, and the DVD Wizard has been improved.

Please note that while this major release may seem to have few features development is at full throttle in the refactoring branch. You can monitor the progress here.

Categories: FLOSS Project Planets

Dries Buytaert: Thoughts as we head to DrupalCon Baltimore

Planet Drupal - Thu, 2017-04-20 22:20

The past weeks have been difficult. I'm well aware that the community is struggling, and it really pains me. I respect the various opinions expressed, including opinions different from my own. I want you to know that I'm listening and that I'm carefully considering the different aspects of this situation. I'm doing my best to progress through the issues and support the work that needs to happen to evolve our governance model. For those that are attending DrupalCon Baltimore and want to help, we just added a community discussions track.

There is a lot to figure out, and I know that it's difficult when there are unresolved questions. Leading up to DrupalCon Baltimore next week, it may be helpful for people to know that Larry Garfield and I are talking. As members of the Community Working Group reported this week, Larry remains a member of the community. While we figure out Larry's future roles, Larry is attending DrupalCon as a regular community member with the opportunity to participate in sessions, code sprints and issue queues.

As we are about to kick off DrupalCon Baltimore, please know that my wish for this conference is for it to be everything you've made it over the years; a time for bringing out the best in each other, for learning and sharing our knowledge, and for great minds to work together to move the project forward. We owe it to the 3,000 people who will be in attendance to make DrupalCon about Drupal. To that end, I ask for your patience towards me, so I can do my part in helping to achieve these goals. It can only happen with your help, support, patience and understanding. Please join me in making DrupalCon Baltimore an amazing time to connect, collaborate and learn, like the many DrupalCons before it.

(I have received a lot of comments and at this time I just want to respond with an update. I decided to close the comments on this post.)

Categories: FLOSS Project Planets

Third & Grove: Introducing the Drupal Extrafield Settings Module

Planet Drupal - Thu, 2017-04-20 22:20
Introducing the Drupal Extrafield Settings Module john Thu, 04/20/2017 - 22:20
Categories: FLOSS Project Planets
Syndicate content